The majority of articles published in PNAS involve statistical analyses. Statistical analysis of data can be complicated and challenging, especially as scientific data have been increasing steadily in volume and complexity. While the prevalence of statistical errors in scientific publications, including PNAS, is unknown, it has been credibly claimed to be large (1). Errors in statistical analysis have been identified as a contributor to the reproducibility crisis in science (2). These errors arise even in the presence of rigorous peer review, whose form is currently under debate (3).
One remedy to this dilemma that has been found to improve the statistical quality of published scientific papers is expert statistical review (4).* Statistical review has been a regular feature of biomedical journals since the late 1970s (5–7) but has been less prominent in other fields. To this end, several general science journals have established groups of statistical editors in the past decade, including PLoS One, PLoS Biology, and Science (e.g., ref. 2).
Until recently, PNAS editors sought outside statistical reviewers when they saw the need. The success of this approach was mixed, as statisticians are already highly committed and are often not available for ad hoc statistical reviewing outside their own specialty.
In response, PNAS established a Statistical Review Committee in September 2022, consisting initially of seven statisticians. Manuscripts that are otherwise viewed as potentially acceptable, but the statistical soundness of which the original reviewers have been unable to assess, are flagged by the editor, the Editorial Board Member, or a reviewer and referred to the Statistical Review Committee for further evaluation. The committee formally reviews the manuscript: The committee may evaluate the statistical methods themselves or refer the manuscript to an additional statistical or methodological reviewer with appropriate expertise.
So far, the committee has been functioning well, and more than 56 manuscripts have been evaluated. Some of these have been improved by addressing the feedback in the statistical review. As well, some papers have been rejected due to concerns raised about the appropriateness of the statistical methods. It is anticipated that the committee will help raise the quality of the conduct, presentation, or interpretation of the statistical analyses in PNAS articles, as well as improve the reproducibility of articles in the journal.
Acknowledgments
Author contributions
A.E.R., A.L.C., M.J.D., C.G., S.N.G., A.H.H., and N.M.R. wrote the paper.
Competing interests
The authors declare no competing interest.
Footnotes
*The same study did not find an alternative approach, the use of methodological checklists or reporting guidelines, to have a significant effect on article quality.
References
- 1.Vaux D. L., Know when your numbers are significant. Nature 492, 180–181 (2012). [DOI] [PubMed] [Google Scholar]
- 2.McNutt M., Raising the bar. Science 345, 9 (2014), 10.1126/science.1257891. [DOI] [PubMed] [Google Scholar]
- 3.Berenbaum M. R., On peer review—then, now, and soon to be? Proc. Natl. Acad. Sci. U.S.A. 120, 1–3 (2023), 10.1073/pnas.2302593120. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Cobo E., et al. , Statistical reviewers improve reporting in biomedical articles: A randomized trial. PLoS One 3, e332 (2007). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Altman D. G., Statistics in medical journals. Stat. Med. 1, 59–71 (1982). [DOI] [PubMed] [Google Scholar]
- 6.Goodman S. N., Glaman D. G., George S. L., Statistical reviewers improve reporting in biomedical articles: A randomized trial? J. Gen. Intern. Med. 13, 753–756 (1998).9824521 [Google Scholar]
- 7.Hardicke T. E., Goodman S. N., How often do leading biomedical journals use statistical experts to evaluate statistical methods? the results of a survey. PLoS One 15, e0239598 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]