The peer review at high risk from COVID-19 – are we socially distancing from scientific quality control?

Editor

An exponential global spread of the disease and preliminary mortality reports, exceeding a dramatic 10 per cent mark, created a state of emergency during the coronavirus disease (COVID-19) pandemic^1,2. Given the novelty of the threat, evidence was scarce and the world faced an extraordinary situation of uncertainty^3,4.

Whereas social life all over the world ‘locked down’, the scientific world seemed to ‘loosen’ its restrictions. Renowned journals encouraged submissions of COVID-19 related articles and promoted fast-track processing. Given the overwhelming amount of evolving reports during the pandemic, the World Health Organization (WHO) raised the concern that we equally fight a misleading ‘infodemic’⁵. However, differing from the plethora of social media tweets and fast-breaking newsletters, the detailed critical appraisal of content is the backbone of scientific work and publication. Inevitably, the question arises:

Are we socially distancing from scientific quality control?

Within the first 3 months of the pandemic (1 January to 12 April 2020) the number of COVID-19 related articles increased exponentially, resulting in over 5000 PubMed listed communications (Fig. 1a). By the end of the observation period, 1454 COVID-19 related articles were published on PubMed within just 1 week. Of note, in that week COVID-19 communications constituted 5 per cent of all articles uploaded on PubMed. In that context, the ‘spread’ of COVID-19 related articles overtook the exponential kinetics of global COVID-19 infections documented by the WHO (Fig. 1a).

Fig. 1.

COVID-19 literature within the first 3 months of the pandemic

a Research responds to the exponential curve of COVID-19. Within the first 3 months of the pandemic exponential increase of information about COVID-19. *The red curve indicates the exponential kinetic global spread of COVID-19 within the observation period (increase from n = 584 to n = 1 847 796 detected cases, www.who.int). b The peer review at risk. Under non-COVID-19 circumstances median expectable duration from submission to decision for included journals was 56 days (IQR 42-84, black curve). The peer review process of COVID-19 articles was reduced to a median of 8 days (IQR 3-18, blue curve). Median time from submission to publication for COVID-19 articles was 14 days (IQR 6-28, red curve).

However, the extraordinary amount of articles did not reflect scientific quality. Whereas numerous opinion-based articles and case reports became the blueprint for treatment and drastic social measures, only 8 per cent (n = 381) of articles were original research. Even high impact factor journals, under normal circumstances characterized by a rigorous ‘natural barrier’ of peer reviewers, became susceptible for COVID-19. Renowned high impact factor journals dedicated up to 20 per cent of their capacity to COVID-19 related articles (Fig. S1). Again, the portion of original research was 11 per cent only.

During the pandemic an immediate intersection between scientific findings and current global affairs emerged: today's findings became recommendation of tomorrow⁶. Hence, editorial teams and peer reviewers faced an exceptional responsibility as gatekeepers of valid information.

A more detailed look at the 381 identified original articles revealed a full article history in 47 per cent (n = 177), including: date of submission, date of acceptance and date of publication. For included journals, according to the journals' homepages, the median duration of a peer review was 56 days ((IQR 42-84 days) under normal circumstances (Fig. 1b). On the contrary, for COVID-19 original articles the median time from submission to acceptance was as short as 8 days (IQR 3-18 days, Fig. 1b). To a worrying extent, the peer review was completely abandoned in 14 per cent (n = 25/177) of included COVID-19 original articles. Of note, the majority of articles declare ethical approval. Still, given the shortage of time it remains questionable if these approvals comply with today's standards³.

In conclusion, the COVID-19 pandemic led to an overwhelming amount of communications. Established measures of scientific quality control have been challenged or even disrupted. Concepts of scientific publication need to be reconsidered in order not to distance too far from our scientific quality standards.

Supplementary Material

bjs11785-sup-0001-FigureS1

Supplementary Figure 1 COVID-19 literature in high impact journals. Among leading journals of selected fields (infectious disease, intensive care, internal medicine and respiratory medicine), a high impact factor was associated with a high number of COVID-19 articles within the first three months of the pandemic. Up to >20% of published articles within the observation period were COVID-19 related in high impact factor journals. Abbreviations: IF: Impact Factor, JAMA: The Journal of the American Medical Association, NEJM: New England Journal of Medicine