The scientific publishing system is broken. Bring in peer-review 2.0

Zurich, Switzerland, 19th November 2017

In the last years, data from replication experiments have pointed towards a systemic reproducibility crisis of peer-reviewed research[1, 2]. The problem spans across all fields, from social to biological and hard sciences, albeit to different degrees. Additionally, the rising issues regarding fraudulence [3] demonstrate that the current peer-review system, put in place to ensure the quality of scientific publications, has proven to be insufficient and requires an update.

Scientific publishing currently relies on a peer-review system where papers are evaluated by academics from the same field considered equipped to make decisions on the value of the proposed research. The process consists of evaluating the soundness of the manuscript but it relies on an honor system that assumes that the results presented are correct to the best of the authors’ knowledge. Therefore the peer-review process does not aim to validate the results presented in a paper making the system susceptible to erroneous results being published accidentally or due to fraud [3]. Non-reproducible, and often fraudulent, published research can have impact within but also far beyond the academic community, despite the scientific consensus on a topic [4-10]. Published research that condemns vaccines[11-14] and GMO’s [15, 16], supports climate change denial [17, 18] or downplays the effects of cigarette smoking [19, 20] can lend undue credibility to questionable science, especially amongst the general public, harming the progress and well being of our society.

It is unfeasible for the current peer-reviewing system to ensure the reproducibility of studies and not surprisingly, depending on the field, an estimated 60-92% of published studies cannot be reproduced [1, 2]. Given these percentages, the average of 32 billion dollars invested in research every year by the NIH alone [21] represents, an excessive loss of tax-paying money. Such numbers are bound to raise skepticism amongst the general public regarding their return on the investments in science. This lack of public support encourages governmental policies to reduce research budgets [22]. However, progress in the world depends on good research, and simply cutting resources will not stimulate progress but rather further halt it. To change this, the scientific community needs to start by earning back public trust.

To assist in repairing the current peer-review system, we created a post-publication online reviewing platform called Bare-Science. This academically driven resource provides publicly available crowd-sourced information, from scientists around the world, about publications such as data reproducibility and availability. Bare-Science’s main goal is to shift the current incentives in academic publishing from producing high numbers of flashy articles, towards fewer, higher quality publications by providing a different dimension of information with which to judge papers.

The current pressure on researchers to publish high numbers of articles leading to fraudulent and subpar publications, make these trying times to restore trust in science. With Bare-Science, we extend the peer-review system post publication to shift away from the wrong incentives to reward researchers who focus on publishing high quality research. We created Bare-Science as an effort to update the peer-review system and highlight that real research should not simply be innovatively sexy albeit short-lived, but rather transparent, reproducible and ultimately, trustworthy.

References

  1. Estimating the reproducibility of psychological science. Science, 2015. 349(6251).
  2. Begley, C.G. and L.M. Ellis, Drug development: Raise standards for preclinical cancer research. Nature, 2012. 483(7391): p. 531-533.
  3. Fang, F.C., R.G. Steen, and A. Casadevall, Misconduct accounts for the majority of retracted scientific publications. Proceedings of the National Academy of Sciences, 2012. 109(42): p. 17028-17033.
  4. Tosar, J.P., et al., Mining of public sequencing databases supports a non-dietary origin for putative foreign miRNAs: underestimated effects of contamination in NGS. RNA, 2014. 20(6): p. 754-757.
  5. Lutz, B., et al., Degradation of Cry1Ab Protein from Genetically Modified Maize in the Bovine Gastrointestinal Tract. Journal of Agricultural and Food Chemistry, 2005. 53(5): p. 1453-1456.
  6. Goldstein, D.A., et al., Comment: Aris and Leblanc “Maternal and fetal exposure to pesticides associated to genetically modified foods in Eastern Townships of Quebec, Canada”. Reproductive Toxicology, 2012. 33(1): p. 120-121.
  7. Van Eenennaam, A.L. and A.E. Young, Prevalence and impacts of genetically engineered feedstuffs on livestock populations1. Journal of Animal Science, 2014. 92(10): p. 4255-4278.
  8. Sasco, A.J., M.B. Secretan, and K. Straif, Trends in Diagnosis and Therapy of Lung CancerTobacco smoking and cancer: a brief review of recent epidemiological evidence. Lung Cancer, 2004. 45: p. S3-S9.
  9. Doran, P.T. and M.K. Zimmerman, Examining the Scientific Consensus on Climate Change. Eos, Transactions American Geophysical Union, 2009. 90(3): p. 22-23.
  10. Anderegg, W.R.L., et al., Expert credibility in climate change. Proceedings of the National Academy of Sciences, 2010. 107(27): p. 12107-12109.
  11. Madsen , K.M., et al., A Population-Based Study of Measles, Mumps, and Rubella Vaccination and Autism. New England Journal of Medicine, 2002. 347(19): p. 1477-1482.
  12. Farrington, C.P., E. Miller, and B. Taylor, MMR and autism: further evidence against a causal association. Vaccine, 2001. 19(27): p. 3632-3635.
  13. Taylor, B., et al., Autism and measles, mumps, and rubella vaccine: no epidemiological evidence for a causal association. The Lancet, 1999. 353(9169): p. 2026-2029.
  14. Wakefield, A.J., et al., RETRACTED: Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. The Lancet. 351(9103): p. 637-641.
  15. Aris, A. and S. Leblanc, Maternal and fetal exposure to pesticides associated to genetically modified foods in Eastern Townships of Quebec, Canada. Reproductive Toxicology, 2011. 31(4): p. 528-533.
  16. Séralini, G.-E., et al., RETRACTED: Long term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize. Food and Chemical Toxicology, 2012. 50(11): p. 4221-4231.
  17. Dunlap, R.E., Climate Change Skepticism and Denial: An Introduction. American Behavioral Scientist, 2013. 57(6): p. 691-698.
  18. Khilyuk, L.F. and G.V. Chilingar, On global forces of nature driving the Earth’s climate. Are humans involved? Environmental Geology, 2006. 50(6): p. 899-910.
  19. Lee, P.N. and B.A. Forey, Trends in cigarette consumption cannot fully explain trends in British lung cancer rates. Journal of Epidemiology and Community Health, 1998. 52(2): p. 82-92.
  20. Nilsson, R., Environmental Tobacco Smoke and Lung Cancer: A Reappraisal. Ecotoxicology and Environmental Safety, 1996. 34(1): p. 2-17.
  21. National Institute of Health. http://www.nih.gov/about-nih/what-we-do/budget, 2016.
  22. Education; The National Academies of Sciences, Engineering, and Medicine., Trust and Confidence at the Interfaces of the Life Sciences and Society: Does the Public Trust Science? A Workshop Summary. National Academies Press (US). 2015, Washington (DC).