Retracted papers on SARS-CoV-2 and COVID-19

Editor—Severe acute respiratory syndrome-related coronavirus-2 (SARS-CoV-2) has become the most important field of research in 2020. Scientific journals have been managing an unprecedented number of manuscript submissions comprised of research on this topic. Accelerated publication of papers containing data regarding SARS-CoV-2 has been facilitated, assuming that rapid circulation of important knowledge may save lives. Experts have raised concerns regarding the methodological quality of articles published after accelerated review processes.1, 2, 3 Access to preprint versions of scientific papers has also increased. Retraction is a mechanism for alerting readers that an article contains seriously flawed or erroneous content and is unreliable.⁴ The aim of this study was to systematically collect data on articles on SARS-CoV-2 and coronavirus disease 2019 (COVID-19) that have been retracted, temporarily retracted, or that have triggered expressions of concern.

We used the Retraction Watch list of retracted articles on SARS-CoV-2/COVID-19 (‘Retracted coronavirus papers’),⁵ and included all the articles listed in this source. The PubMed database was also searched for retracted articles regarding SARS-CoV-2/COVID-19. Both searches were performed on November 2, 2020. The data collected per article included title, type of article, subject area, reasons for retraction, journal, publisher, first and corresponding authors and their affiliations, article metrics, and source countries.

We identified 45 articles on SARS-CoV-2/COVID-19 that had been retracted (n=39, 87%), temporarily retracted (n=3, 6.7%) or that had generated an expression of concern (n=3, 6.7%). Among the 39 articles definitively retracted, the type of study was available in 34 cases. These articles included 20 clinical studies (20/39, 51%, five missing data), five preclinical studies (5/39, 12.8%, five missing data), and other types of articles (9/39, 23.1%, five missing data). None of the studies was an RCT. Half of the retracted articles were research articles (19/39, 49%, one missing data) and almost one-third were preprint research articles (11/39, 28%, one missing data). Of the three temporarily retracted articles, one was a clinical study and two were commentaries/editorials/letters. The three papers with an expression of concern were clinical studies. Table S1 shows the detailed characteristics of the articles. The articles were retracted after a median [inter-quartile range, IQR] (range) of 14 [3.5–52.5] (1–193) days from publication. The reasons for retraction varied, ranging from issues with results (18%) and data (14%) to ethical violations (10%), including lack of Institutional Review Board approval. The median [IQR] (range) h-index of first authors was 5 [2–15.2] (0–68) and that of corresponding authors was 14.5 [5–26] (1–68). For the temporarily retracted articles and those with expressions of concern, the median [IQR] (range) h-index of first authors was 11 [4.2–14.7] (1–35), and that of corresponding authors was 72 [19–127.2] (4–149). A quarter of the articles were retracted while available in preprint repositories, such as medRxiv (7/39, 18%) and bioRxiv (4/39, 10%). Elsevier and Cold Spring Harbor Laboratory Press published most of the retracted articles (12/39, 31% and 11/39, 28%, respectively). Of the publishing journals, 35 of the 39 (90%) were indexed in the National Library of Medicine (NLM) catalogue, 20/39 in Scopus (51%), 20/39 in Scimago (51%), 17/39 in Web of Science (WOS) (44%), and 4/39 in the Directory of Open Access Journals (DOAJ) (10%). The median [IQR] (range) Journal Impact Factor (JIF) of the journals at the time of data collection was 4.5 [2.6–21.4] (1.5–74.7). Half of the articles (18/36, five missing data) were published in journals with an Open Access policy. The six articles temporarily retracted or with expressions of concern were all published in journals indexed in NLM, Scopus, and Scimago, and five out of six also in WOS. Two of the five in journals with Open Access policy were published in journals indexed in DOAJ. Elsevier published five of these articles and Public Library of Science published one. The 39 retracted articles received a median [IQR] (range) of 6 [1–37] (0–304) citations, 321 [7–4316] (0–30 969) tweets, and had a median [IQR] (range) Altmetric score of 320 [11.7–3262.7] (1–24938) (Table S1). The six papers temporarily retracted or with expressions of concerns had a median [IQR] (range) of 10.5 citations [2–158.5] (1–1517), 738.5 [75.2–6075.7] (1–11 084) tweets, and Altmetric score of 687.5 [64.2–3215.7] (15–11 583). See the Supplementary Appendix for full Methods and Results.

The retracted articles had no common threads or characteristics. Well established journals were as exposed to retraction as were those with lower JIFs. Authors of the retracted articles had a moderately high h-index. Publication during a pandemic seems fraught with risk. It is tempting to publish articles about a topic receiving intense interest from the public. Compared with other topics, articles on COVID-19 have been shown to generate more citations (median [IQR], 45 [30–244] vs 2 [1–4] citations; P<0.001).¹

Lay persons are rarely interested in the source of publications, whereas researchers often are. Preprint publications may appear more trustworthy compared with social media content and ‘an unsuspecting public cannot differentiate between preprint postings and peer-reviewed, published, trusted evidence’.⁶ The question that arises is whether enabling early access to data of unclear quality to the few that are knowledgeable enough to save lives justifies exposure of such data to the many that are much less discerning.⁶ Heightened social media interest in pandemic-related content and the thirst for positive news among lay persons contributed to the broad dissemination of some retracted articles, especially those proposing a cure. By the time scholarly journals responded to the issues justifying retraction, many papers had already been disseminated. Indeed, at least two of the included articles7, 8, 9, 10 were used to justify policies chosen for pandemic management.

The high publication rate of papers on COVID-19 will quickly render our results rapidly outdated, but the concepts presented on how to study this phenomenon could be used for future studies, to heighten awareness and to design countermeasures. While it is easy to lay the blame for such publications on editors and reviewers, prevention is not simple and probably requires a systems-level approach. Supporting organisations of preprint servers must carefully consider shouldering responsibility for the risks of public exposure to prepublication. Access to prepublications may also be limited to academic organisations, and each article should be headed by a clear warning regarding the unknown validity/reliability of its contents. Journals may also demand that all authors undertake signed responsibility for, and understand the potential scientific and legal repercussions of, retraction.

Declarations of interest

The authors declare that they have no conflicts of interest.

Appendix A. Supplementary data

The following is the Supplementary data to this article: