Reply to Margalida and Colomer: Science should strive to prevent mistakes, not corrections

Margalida and Colomer (1) proffer a “mistake index” based on corrections published by scientific journals to gauge peer review quality. This is an interesting idea but has numerous theoretical and practical problems.

Even with recently slightly rising rates, only 2–3% of articles in Nature, PNAS, and Science issue corrections. Most are minute or trivial. We examined a sample of the last 100 corrections issued for research articles in Nature, PNAS, and Science in 2013. There were six retractions and one seemingly important revision. Beyond those 7 corrections, the other 93 corrections mostly involved small details that did not affect the research findings of the original article. Thirty corrections involved author names, affiliations, or acknowledgments, which have no bearing on the content of science.

Because most retractions involved author malfeasance, it is unfair to pin those errors on peer review. Further, the 30 corrections to names/acknowledgments had nothing to do with scientific quality control. Expecting perfection from peer reviewers is unrealistic, particularly with minutiae and unshared data. Peer reviewers might have done a brilliant job filtering out most mistakes and positively refining the article in other ways. Errors that make it into print are visible, unlike errors averted through quality control processes. As data and information become more accessible in the internet age, increased transparency enables greater scrutiny of research findings from larger and more diverse populations. Consequently, published corrections—or at least challenges to published articles—should continue to rise in science. This would likely be a positive development. Debate and refinement of previous work drives scientific progress.

In our sample, the lack of corrections dealing with moderate or severe errors was notable. This may be a sign of the effectiveness of peer review to filter out and/or revise such mistakes, as well as the skill of researchers. However, surely more than 2–3% of articles have flaws of some sort. Gelman (2) lamented that it is excessively difficult to publish criticisms and obtain data for published articles. Although criticisms during peer review must be handled carefully, criticisms after publication are held to a much higher bar. As a result, it is often difficult to directly refute mistakes, perhaps as shown by extremely low correction rates in scientific journals. Publishing mistakes go beyond empirical accuracy. Our article (3) suggested that gatekeepers reject many high-quality articles. Still, often in science and in life, imperfection does not necessarily imply a lack of merit.

It is worth noting that articles published in high-impact outlets are under intense scrutiny. This is in part how and why small—and usually inconsequential —errors are spotted and reported. A mistake index would also provide journals with incentives to not issue corrections or prefer less risky or complex articles.

Although even venial errors are lamentable, corrections are not, especially because scientific errors can affect lives beyond the ivory tower (4). Corrections help ensure that scientists conduct research based on accurate information, teach lessons regarding how and why mistakes occur, and reinforce norms of meticulousness and vigilance in science.