Reproducibility

The subject of reproducibility has been prominent in the scientific press over the past year. This has led many scientists to ask if there is a problem. The answer is no; science is being well conducted. Non-scientists point to studies with conflicting results. If certainty of findings were absolute, there would be no reason to reproduce studies. However, the nature of science is the investigation of uncertainty. Very few studies result in a proof and, as knowledge expands—especially in the biomedical sector—reaching a conclusion that is absolute and generalizable is ever more challenging.

Can science be improved? Certainly: very few things cannot be improved.

A prioritization of the elements that can be improved is challenging. One commonly identified and well agreed upon element is the description of methodologies and specification of reagents. This is a responsibility of the peer review literature. It is essential for editors to provide appropriate opportunity for authors to report methodology without penalty, and to demand that authors specify methods and reagents with sufficient detail to allow replication. Furthermore, editors must seek and demand peer review to ensure these are appropriately addressed.

I have previously written on the challenges of controls (Baskin and Hewitt 2014), emphasizing the minimum controls for reporting immunohistochemical studies in this journal (Hewitt et al. 2014); however, this remains one of the most challenging elements of managing a peer-reviewed journal. Authors are not trying to deceive, and reviewers are not failing; rather, authors and reviewers want to believe that their descriptions of methods and reagents are sufficient to reproduce the work, that is, until others attempt to replicate their findings.

Ultimately, this discourse leads to two topics: scientific method and the role of reproducibility of results. Scientists, spend too little time discussing scientific method. This is more than the context of teaching graduate students and post-doctoral fellows. There is no singular “scientific method” of hypothesis and discovery. The field of philosophy of science is complex and frustrating to many; albeit, enlightening to some. Regardless, we as scientists need to discuss the scientific method more.

Reproducibility of results remains the cornerstone of modern science. No doubt the more unexpected the results, the more exciting. Equally, the more ambiguous the results, the more vexing. But both call for further investigation. Were hypotheses wrong in the former? And what is the source of the ambiguity in the latter? Ultimately, discussions of reproducibility are about promoting better science.