Publication Rate and Journal Review Time of COVID-19–Related Research

To The Editor:

The academic community has responded swiftly to the novel coronavirus 2019 (COVID-19) pandemic. Anecdotally, academic researchers have noticed a reduction in the amount of time journals require to review COVID-19 manuscripts. In this letter we describe the growth of this literature and the review time of COVID-19–related manuscripts.

Bibliographic data were extracted from all articles in the dedicated COVID-19 research sections of the preprint databases medRxiv and bioRxiv¹ (https://connect.medrxiv.org/relate/content/181) and the National Center for Biotechnology Information section LitCovid² (https://www.ncbi.nlm.nih.gov/research/coronavirus/). Bibliographic data were also extracted from all non–COVID-19 articles published by the journals listed in LitCovid. Articles were included if posted between November 1, 2019, and May 26, 2020. For published articles, the difference between the date of submission and the date of acceptance (“review time”) was compared using the independent samples Student t test. See Supplemental Material for detailed methodology (available at http://www.mayoclinicproceedings.org).

Our search identified 802 bioRxiv preprints, 3268 medRxiv preprints, and 16,081 LitCovid publications. COVID-19 publications per week have increased over time (Figure A ). Of 2427 journals in the LitCovid archive, 1294 (53.3%) listed bibliographic information. These journals published 7798 COVID-19 publications and 340,032 non–COVID-19 publications. The average time to review an article was significantly fewer days for COVID-19–related publications (11.3 days vs 106.3 days; P<.001) (Figure B). Given the COVID-19 crisis, these findings may be appropriate. Editors should be commended for eliminating unnecessary administrative barriers and reviewers should be lauded for rapidly reviewing COVID-19–related manuscripts. However, potential consequences of these data should be considered.

Figure.

Publication trends in coronavirus disease 2019 (COVID-19)–related preprints and published literature. A, Total COVID-19 articles per week, stratified by archive (n= 802 [bioRxiv], n=3268 [medRxiv], and n=16,081 [Pubmed]). B, Weekly average of review time from Pubmed listed publications in the LitCovid database and from non–COVID-19–related publications from the same journals. Review time calculated by subtracting submission date from acceptance date (n=7.798 [COVID-19] and n=340,032 [non–COVID-19]).

First, thousands of COVID-19 publications have been disseminated in preprint archives, which have not undergone peer review. Second, the volume of submissions might render it difficult to identify reporting of the same patients in different articles. This has already occurred in high-impact journals and makes it challenging to estimate the prevalence of disease manifestations or outcomes.³ Third, reporting errors have also occurred⁴ and might be more frequent in an expedited process. Finally, the reduction in review time may in part be attributable to an abridged process of peer review.

The recent experience with hydroxychloroquine in COVID-19 may illustrate the consequences of expedited or inadequate peer review. Initially, a small cohort study with substantive methodologic flaws was accepted after 1 day of peer review.⁵ Public acquisition, medication shortages, and widespread adoption in clinical practice followed. Subsequent large observational cohorts and randomized controlled trials have not verified these results. More recently, randomized controlled trials of hydroxychloroquine in COVID-19 halted enrollment after a study by Mehra et al⁶ suggested an association between hydroxychloroquine use and increased mortality. Substantive concerns about the validity of these data could not be addressed, and the study has been retracted.⁷

Our approach has limitations. The types of published articles could not be assessed and many journals did not list submission or acceptance dates in bibliographic metadata. However, our data highlight important threats to the validity of the evolving COVID-19 literature, which may be particularly acute in the current climate. While our response to this crisis should be swift, it must also be scientifically rigorous.