Biased, wrong and counterfeited evidences published during the COVID-19 pandemic, a systematic review of retracted COVID-19 papers

Abstract

In 2020 COVID-19 led to an unprecedented stream of papers being submitted to journals. Scientists and physicians all around the globe were in need for information about this new disease. In this climate, many articles were accepted after extremely fast peer-reviews to provide the scientific community with the latest discoveries and knowledge. Unfortunately, this also led to articles retraction due to authors’ misconduct or errors in methodology and/or conclusions. The aim of this study is to investigate the number and characteristics of retracted papers, and to explore the main causes that led to retraction. We conducted a systematic review on retracted articles, using PubMed as data source. Our inclusion criteria were the following: English-language retracted articles that reported original data, results, opinions or hypotheses on COVID-19 and Sars-CoV-2. Twenty-seven retracted articles were identified, mainly reporting observational studies and opinion pieces. Many articles published during the first year of the pandemic have been retracted, mainly due to the authors' scientific misconduct. Duplications, plagiarism, frauds and absence of consent, were the main reasons for retractions. In modern medicine, researchers are required to publish frequently, and, especially during situations like the COVID-19 pandemic, when articles were rapidly published, gaps in peer-reviews system and in the path to scientific publication arose.

Introduction

In present times, the creation of sound scientific evidence in healthcare lies within the Evidence-Based Medicine (EBM) framework; scholars trust their peers to review their work, and, if it is found to be rigorous and unbiased, the process ends with publication (Boetto et al. 2020). The COVID-19 pandemic had a significant effect on scientific literature and practice in underlining their greatest strengths by promoting interprofessional collaborations and sharing ideas between multiple sites (Hynicka et al. 2022; Pawłowicz-Szlarska et al. 2022; Lincoln et al. 2022). These vivid collaborations sparked by the pandemic outbreak, united with globalization and easy online access, led to an enormous stream of research papers: to understand the scale of this phenomenon, Dimension Database estimates well over 870,000 research papers published about COVID-19, from 198 countries, in 2020 and 2021 alone, as compared to 109,000 papers regarding Seasonal Influenza published between 2015 and 2019 (Dimension Database 2022). In 2020 a great deal -around 33%- of academic papers about COVID-19 were disseminated as pre-prints, which could be published online without the need for peer review (Franser et al. 2021; Watson 2022), and this contributed to the infodemic which altered mass media information (Palayew et al. 2020; Pian et al. 2021; Corinti et al. 2022). These preprints were eventually screened by peers, but the vast amount of information needing peer-review led to decreased review times for COVID-19 papers, mainly at the expense of other topics (Else 2020).

The overwhelming need for quick peer-reviews, combined with the necessity for focused healthcare workers, made the peer-review and publication process more vulnerable to errors, and amplified the risk of lacking transparency and reproducibility in studies, and therefore of frauds and misconducts (Publications Office of the EU 2020). Publishing timeframes have also become increasingly tight during the pandemic, due to expedited review and fast-tracks specifically suited for COVID-19 papers (Bagdasarian et al. 2020; Benjamens et al. 2021; Schonhaut et al. 2022). As a result, as Yeo-Teh and al. described in their work (Yeo-Teh et al. 2019), the retraction rate spiked to alarming heights, having more than tripled from a pre-pandemic retraction rate regarding viruses and epidemics of 0.021% to a COVID-19 related retraction rate of 0.074%. This worrying pattern is reaffirmed in comparing present times retraction rates with past ones, as retractions were most commonly related to either Basic Life Sciences or Technology, rather than medicine, and, furthermore, the number of retractions per journal did not change since 1756–2019 (Vuong et al. 2020).

In this scenario, a noteworthy number of authors acted mistakenly or sometimes even maliciously to exploit this “opportunity” by submitting papers of dubious scientific value, duplicating their submission to various journals, plagiarizing others’ work, or plainly tampering data to fit their assumption. On the other hand, other authors simply made honest mistakes or suffered from unreliable data; at this date Retraction Watch has listed 253 COVID-19 related, retracted, papers (Retraction Watch 2022). Although peer review is designed to promptly assess methodology and conclusion flaws, it is not yet fully equipped to deal rapidly with shady behaviors, especially during times of high demand of information, such as the COVID-19 pandemic. All the aforementioned issues had serious consequences, reducing trust in science and medicine, fueling misinformation and contributing to the infodemic.

Aim of the study

We reviewed the COVID-19 related scientific literature to investigate the number and characteristics of retracted papers, and to explore the main causes or motivations that led to retraction.

Methods

Search strategy and selection criteria

We conducted a systematic review of the scientific literature, following the Preferred Reporting Items for Systematic Reviews (PRISMA) approach (Moher et al. 2009), to identify COVID-19-related quantitative and qualitative studies of different designs published during the pandemic and then retracted for any cause.

The initial search was implemented on March 1, 2022 and was limited to the timespan between December 1, 2019 and December 31, 2021. The search query consisted of terms considered pertinent by the authors to review the literature on retracted COVID-19 articles. We searched for publications on Pubmed using the following search string: “(retracted publication[pt]) AND ((covid*) OR (coronavirus) OR (Sars-c*))”. We included English-language retracted articles that reported original data, results, opinions or hypotheses on COVID-19 and Sars-CoV-2. We excluded studies that did not focus on COVID-19 as the primary subject of the study or considered it as a background factor only.

Data extraction

Data was extracted by four independent reviewers (AC, AS, FS, DGor), and disagreement on extracted data was discussed with one independent tiebreaker (DGol). Descriptive variables extracted from each article and journal were: publishing journal, journal’s 3-year impact factor, first author affiliation’s country (categorized in high-income, middle-high income, middle-low-income and low-income countries (The World Bank 2021)), study type, data availability, analyzed population, sample size, aim, results, retraction statement, who first requested retraction, subscription or open access, funding and timestamps of publication and retraction (i.e., time between date of publication and date of retraction). Articles’ lifespan and the time difference between the timestamps were extracted from the article information or full-text. Whenever the exact day was not provided and the article reported only the month, we referred to the 15th of the same month as the corresponding date.

The following categories were assigned to retraction statements: Process, Misconduct and Others. Each of these categories had specific reasons for retraction, “Process” included: “Approval issue”, which implied problems regarding the use of data, “Error”, which implied problems regarding the methodology of the study and “Not reproducible”, which implied the non-reproducibility of the study due to inaccessible data. “Misconduct” comprehended all mischievous behaviors by authors: fraud, plagiarism and duplication of the article. Finally, the label “Other” was used to categorize all those retraction statements which did not fall in the previous labels, in this category we found two specific reasons: “not first-hand account”, “premature conclusions”, “redundant”, “out of journal scope”, “publisher error” and “not specified”.

Results

Of the initial 69 records identified, 66 full-text articles matched our inclusion criteria and were assessed and included in the review (Fig. 1, Table 1). 38 studies were published in 2020 and 28 in 2021. Studies' characteristics are summarized in Table 1. Although we did not limit our search to specific study types, the majority of retracted articles were either observational studies (n = 34, 52%) or opinion papers (n = 11, 17%) (Fig. 2 and Table 1), followed by reviews articles (n = 9, 14%), experimental (n = 7, 11%), meta-analysis (n = 4, 6%), study protocol (n = 1, 1%). Three studies were excluded during the screening process (Zellmer 2021; Hill et al. 2021a, b; Liu et al. 2020) since COVID-19 was not the main focus of the paper. The included studies came from all regions of the world, with 46 (60%) of the articles drafted in high-income countries (Table 1).

Fig. 1.

PRISMA flowchart

Table 1.

Included articles’ summary

Title	Country	Journal	3-year IF	Study type	Retraction label	Specific reasons for retraction	OA article	Retraction requested/accepted by authors	Journal type	Δt retraction (days)
Racial disparity amongst stroke patients during the coronavirus disease 2019 pandemic	USA	Cureus	0.00	Observational	Process	Approval issue	Yes	No	Open	125
Effects of the covid-19 pandemic on stroke patients	USA	Cureus	0.00	Observational	Process	Approval issue	Yes	No	Open	142
Noteworthy neurological manifestations associated with covid-19 infection	USA	Cureus	0.00	Observational	Misconduct	Fraud	Yes	No	Open	245
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection mimicking as pulmonary tuberculosis in an inmate	USA	Cureus	0.00	Observational	Process	Error	Yes	Yes	Open	452
Severe dengue with multisystem inflammatory syndrome in children due to COVID-19: A Co-infection case series	Bangladesh	Cureus	0.00	Observational	Process	Error	Yes	No	Open	4
Smoker, former smoker and COVID-19: nicotine does not protect against SARS-CoV-2	Spain	Arch Bronconeumol	0.69	Observational	Other	NS	Yes	No	Subscription/open	4
Covid-19, suicide, and femicide: rapid research using google search phrases	New Zealand	J Gen Psychol	0.75	Review	Process	Error	No	Yes	Subscription/open	117
Basic demographic parameters help predict outcomes in patients hospitalized With COVID-19 during the first wave of infection in West Texas	USA	J Prim Care Community Health	0.96	Observational	Process	Approval issue	Yes	Yes	Open	256
A retrospective analysis and comparison of prisoners and community-based patients with COVID-19 requiring intensive care during the first phase of the pandemic in West Texas	USA	J Prim Care Community Health	0.96	Observational	Process	Approval issue	Yes	Yes	Open	321
Clinical characteristics and blood test results in covid-19 patients	China	Ann Clin Lab Sci	1.02	Observational	Process	Error	No	Yes	Subscription	61
Retracted: facemasks in the covid-19 era: a health hypothesis	USA	Med Hypotheses	1.46	Review	Other	Premature Conclusions	No	No	Subscription/open	171
Retracted: 5 g technology and induction of coronavirus in skin cells	Italy	J Biol Regul Homeost Agents	1.47	Opinion	Misconduct	Fraud	No	No	Subscription	37
Retracted: A study of potential SARS-CoV-2 antiviral drugs and preliminary research of their molecular mechanism, based on Anti-SARS-CoV drug screening and molecular dynamics simulation	China	J Comput Biol	1.48	Experimental	Other	Premature conclusions	Yes	Yes	Subscription/open	226
WITHDRAWN: health risk assessment and health management of urban residents facing epidemic pneumonia	China	Work	1.51	Observational	Misconduct	Fraud	Yes	No	Subscription/open	107
Convalescent plasma therapy in covid 19: every dark cloud has a silver lining	India	J Anaesthesiol Clin Pharmacol	1.74	Opinion	Misconduct	Duplication	Yes	No	Subscription/open	94
Noninvasive versus invasive ventilation: one modality cannot fit all during covid-19 outbreak	India	Korean J Anesthesiol	1.79	Review	Misconduct	Plagiarism	Yes	No	Open	68
Efficacy and safety of acupuncture therapy for asymptomatic infection of covid-19: a protocol for systematic review and meta-analysis	China	Medicine (Baltimore)	1.87	Study protocol	Misconduct	Duplication	Yes	No	Open	161
Human immunodeficiency virus (HIV) and outcomes from coronavirus disease 2019 (COVID-19) pneumonia: A Meta-Analysis and Meta-Regression	Indonesia	AIDS res hum retroviruses	1.91	Meta-analysis	Misconduct	Plagiarism	No	No	Subscription/open	125
Withdrawn: a mechanistic analysis placental intravascular thrombus formation in covid-19 patients	USA	Ann diagn pathol	1.92	In vitro	Misconduct	Duplication	Yes	No	Subscription/open	58
Impact of the COVID-19 pandemic on stroke epidemiology and clinical stroke practice in the US	USA	J Stroke cerebrovasc dis	1.95	Observational	Process	Approval Issue	Yes	No	Subscription/open	165
Mental health burden for the public affected by the covid-19 outbreak in china: who will be the high-risk group?	China	Psychol health med	2.00	Observational	Misconduct	Duplication	Yes	No	Subscription/open	192
Vitritis and outer retinal abnormalities in a patient with COVID-19	Brazil	Ocul immunol inflamm	2.02	Observational	Process	Error	No	No	Subscription/open	366
Implementation of a telemedicine service during COVID-19 pandemic in Pakistan	Pakistan	Int J Clin Pract	2.04	Experimental	Process	Error	Yes	Yes	Subscription/open	147
Clinical characteristics and outcomes of patients with COVID-19 pneumonia admitted to an intensive care unit in Faisalabad, Pakistan	Pakistan	Int J Clin Pract	2.04	Observational	Misconduct	Fraud	Yes	Yes	Subscription/open	201
Neutrophil/lymphocyte ratio-A marker of COVID-19 pneumonia severity	Pakistan	Int J Clin Pract	2.04	Observational	Misconduct	Fraud	Yes	Yes	Subscription/open	401
Obesity and mortality of covid-19. Meta-analysis	UK	Obes Res Clin Pract	2.17	Meta-analysis	Process	Error	Yes	Yes	Subscription/open	212
COVID-19 admissions calculators – revisited	Malta	Early Hum Dev	2.18	Observational	Other	Premature conclusions	Yes	Yes	Subscription/open	424
COVID-19 admissions calculators: general population and paediatric cohort	Malta	Early Hum Dev	2.18	Observational	Other	Premature conclusions	Yes	Yes	Subscription/open	433
Unknown unknowns—COVID-19 and potential global mortality	Malta	Early Hum Dev	2.18	Observational	Other	Premature conclusions	Yes	Yes	Subscription/open	437
Clinical sequelae of the novel coronavirus: does covid-19 infection predispose patients to cancer?	USA	Future Oncol	2.22	Review	Misconduct	Plagiarism	Yes	No	Subscription/open	189
RETRACTED: Biopsychosocial intersections of social/affective touch and psychiatry: Implications of 'touch hunger' during COVID-19	India	Int J Soc Psychiatry	2.24	Review	Misconduct	Fraud	Yes	No	Subscription/open	275
A deep learning model and machine learning methods for the classification of potential coronavirus treatments on a single human cell	Egypt	J Nanopart Res	2.25	Observational	Other	Out of journal scope	Yes	No	Subscription/open	303
Coronavirus disease-2019: a brief compilation of facts	India	J Oral Maxillofac Pathol	2.27	Opinion	Misconduct	Plagiarism	Yes	No	Subscription/open	53
Efficacy of favipiravir in COVID-19 treatment: a multi-center randomized study	Egypt	Arch Virol	2.44	Experimental	Process	Error	Yes	No	Subscription/open	301
Anal swab as a potentially optimal specimen for sars-cov-2 detection to evaluate hospital discharge of covid-19 patients	China	Future Microbiol	2.51	Observational	Misconduct	Absence of consent by patients	Yes	No	Subscription/open	242
Clinical and scientific rationale for the "MATH + " hospital treatment protocol for COVID-19	USA	J Intensive Care Med	2.51	Review	Process	Error	Yes	No	Subscription/open	329
Chinese mental health burden during the covid-19 pandemic	China	Asian J Psychiatr	2.53	Observational	Misconduct	Duplication	Yes	No	Subscription/open	217
Covid-19 and potential global mortality – revisited	Malta	Early Hum Dev	2.53	Opinion	Other	Premature Conclusions	Yes	Yes	Subscription/open	330
The mechanisms of action of ivermectin against SARS-CoV-2-an extensive review	Italy	J Antibiot (Tokyo)	2.69	Review	Process	Error	Yes	No	Subscription/open	104
SARS-CoV-2 vaccination and antibody testing in immunosuppressed populations: you can't tell the players without a scorecard [RETRACTED]	USA	Transplantation	2.83	Opinion	Misconduct	Duplication	Yes	No	Subscription/open	63
Meta-analysis of randomized trials of ivermectin to treat SARS-CoV-2 infection	UK	Open Forum Infect Dis	3.02	Meta-analysis	Process	Error	Yes	Yes	Open	214
Methylene blue photochemical treatment as a reliable SARS-CoV-2 plasma virus inactivation method for blood safety and convalescent plasma therapy for COVID-19	China	BMC Infect Dis	3.02	Experimental	Process	Error	Yes	No	Open	86
Lung ultrasound score in establishing the timing of intubation in covid-19 interstitial pneumonia: a preliminary retrospective observational study	China	PLoS One	3.21	Observational	Misconduct	Duplication	Yes	No	Open	119
Tracking COVID-19 vaccine hesitancy and logistical challenges: a machine learning approach	Canada	PLoS One	3.21	Observational	Process	Approval Issue	Yes	Yes	Open	50
Identify and measure the degree of over-prevention behaviors in the post-COVID-19 era in China	China	BMC Public Health	3.36	Observational	Misconduct	Fraud	Yes	No	Open	76
Comment on an article: "Osteoporosis in the age of COVID-19 patients"	Bosnia and Herzegovina	Osteoporos Int	3.59	Opinion	Other	Redundant	Yes	No	Subscription/open	174
A meta-analysis of granulocyte–macrophage colony-stimulating factor (GM-CSF) antibody treatment for COVID-19 patients	China	Ther Adv Chronic Dis	4.42	Meta-analysis	Process	Error	Yes	No	Open	93
Retracted: The safety of COVID-19 vaccinations-we should rethink the policy	Poland	Vaccines	4.42	Observational	Process	Error	Yes	No	Open	8
Phytotherapeutic options for the treatment of covid-19: a concise viewpoint	Pakistan	Phytother Res	4.55	Opinion	Misconduct	Plagiarism	Yes	Yes	Subscription/open	132
Safety and efficacy of favipiravir versus hydroxychloroquine in management of COVID-19: A randomised controlled trial	Egypt	Sci Rep	4.60	Experimental	Misconduct	Fraud	No	No	Subscription/open	171
Exploring the potential effect of COVID-19 on an endangered great ape	Denmark	Sci Rep	4.60	Observational	Process	Error	Yes	Yes	Subscription/open	96
GraphCovidNet: A graph neural network based model for detecting COVID-19 from CT scans and X-rays of chest	India	Sci Rep	4.60	Observational	Process	Error	Yes	No	Subscription/open	229
Stay-at-home policy is a case of exception fallacy: an internet-based ecological study	Brazil	Sci Rep	4.60	Observational	Process	Error	Yes	No	Subscription/open	284
Retracted: no deleterious effect of lockdown due to covid-19 pandemic on glycaemic control, measured by glucose monitoring, in adults with type 1 diabetes	Spain	Diabetes Technol Ther	4.80	Observational	Process	Approval issue	Yes	No	Subscription/open	76
Effects of a single dose of ivermectin on viral and clinical outcomes in asymptomatic SARS-CoV-2 infected subjects: a pilot clinical trial in Lebanon	Lebanon	Viruses	5.02	Experimental	Process	Error	Yes	Yes	Open	153
Rationale and criteria for a COVID-19 model framework	Italy	Viruses	5.02	Review	Misconduct	Plagiarism	Yes	No	Open	23
Intersectionality and inequalities in medical risk for severe COVID-19 in the Canadian longitudinal study on aging	Canada	Gerontologist	5.27	Observational	Process	Approval issue	Yes	Yes	Subscription/open	120
Analyzing pre-pandemic patterns of contacts is partly inappropriate to explain the current COVID-19 situation in Germany	Germany	Lancet Reg Health Eur	6.37	Review	Other	Publisher error	Yes	No	Open	28
Retracted article: sars-cov-2 infects t lymphocytes through its spike protein-mediated membrane fusion	China	Cell Mol Immunol	6.76	In vitro	Process	Error	Yes	Yes	Subscription/open	94
Will the extraction of COVID-19 from wastewater help flatten the curve?	South Africa	Chemosphere	7.18	Observational	Misconduct	Duplication	Yes	No	Subscription/open	231
Effectiveness of surgical and cotton masks in blocking sars-cov-2: a controlled comparison in 4 patients	South Korea	Ann Intern Med	9.79	Opinion	Process	Error	Yes	No	Subscription/open	57
Chinese medical staff request international medical assistance in fighting against covid-19	China	Lancet Glob Health	22.28	Opinion	Other	Not first hand account	Yes	No	Subscription/open	3
Family planning in COVID-19 times: access for all	Kenya	Lancet Glob Health	22.28	Opinion	Process	Error	Yes	Yes	Subscription/open	38
Cardiovascular disease, drug therapy, and mortality in covid-19	USA	N Engl J Med	40.27	Observational	Process	Not reproducible	Yes	No	Subscription/open	34
Chloroquine or hydroxychloroquine for covid-19: why might they be hazardous?	France	Lancet	47.90	Opinion	Process	Not reproducible	Yes	Yes	Subscription/open	48
Retracted: hydroxychloroquine or chloroquine with or without a macrolide for treatment of covid-19: a multinational registry analysis	USA	Lancet	47.90	Observational	Process	Not reproducible	Yes	No	Subscription/open	14

Fig. 2.

Number of publications by study type

The most frequent label attributed to the articles’ retraction was “process” (32, 48%), followed by “misconduct” (23, 35%). As for retraction’s specific reasons, “Authors’ Error” (21, 32%), “Approval Issue” (8, 12%), “Fraud” (8, 12%) and “Duplication” (8, 12%) were the most frequent ones (Fig. 3). The articles’ retraction followed authors’ self-report in 35% (23) of the included records. Most of the retracted articles (45, 68%) were submitted to hybrid journals, whereas 19 (29%) were submitted to open-access journals and 2 (3%) to subscription-only journals. Interestingly, 91% (60) of the included articles were published with the open access model. The median journals’ impact factor was 2.352 (Range: 0—47.9).

Fig. 3.

Number of publications by retraction’s specific reason

The median number of days that occurred between publication and retraction was 137 (Range: 3—452). This timespan differed among retractions’ specific reasons: articles with the “premature conclusions” label were the ones retracted after the longest period, with a mean of 337 days. The fastest retraction occurred for the study labeled as “Not first hand account” (3 days). As for study type, the 34 meta-Observational studies were those with the longest mean lifespan by study type (mean = 191). On the other hand, the 11 opinion papers were the ones with the shorter timespan (mean = 94 days) (Fig. 4).

Fig. 4.

Time to retraction by A study type and B retraction’s specific reason

For reference, the entirety of the selected papers can be found in Table 1, and their citations are as follows: Ghanchi et al. 2020a, b; Elkhouly et al. 2020; An et al. 2020; Mulvey et al. 2020; Huang et al. 2021; Sun et al. 2020; Huang et al. 2020; Lu et al. 2020; Beato-Víbora 2020; Mehra et al. 2020a, b; Mehra et al. 2020a, b; Walach et al. 2021; Dutta et al. 2021; Friedlich et al. 2021; Ali et al. 2020a, b; Lin 2021; Ali et al. 2020a, b; Khalifa et al. 2020; Gul et al. 2021; Victor 2020a, b; Victor 2020a, b; Colchero et al. 2021; Saha et al. 2021; Ma et al. 2021; Imran et al. 2021; Akbar et al. 2020; Ferdous et al. 2021; Jiménez-Ruiz et al. 2021; Savaris et al. 2021; Grech 2020a, b; Zago Filho et al. 2020; Atangana et al. 2021; Mao et al. 2021; Fioranelli et al. 2020; Deokar et al. 2020; Temmerman 2021; Ibrahimagić et al. 2021; Saxena 2020; Grech 2020a, b; Din et al. 2020; Bae et al. 2020; Zeng et al. 2020; Funck-Brentano et al. 2020; Woodle et al. 2021; Standish 2021; Vainshelboim 2021; Singh 2020; Hays 2020; Kampf 2022; Kory et al. 2021; Messina et al. 2021; Banerjee et al. 2021; Zaidi et al. 2022; Wang et al. 2020; Jin et al. 2021; Zhao et al. 2020; Samaha et al. 2021; Dabbous et al. 2021; Nagra et al. 2021; Dabbous et al. 2021; Hussain et al. 2020; Hariyanto et al. 2019; Hill et al. 2021a, b; Guan et al. 2021; Huang et al. 2020.

Discussion

In this systematic literature review, we sought and described the characteristics of the COVID-19 related retracted papers that were accepted in impacted journals after the peer review process during the first two years of the COVID-19 pandemic (2020–2021).

We found that observational studies were the most retracted ones, followed by opinion papers and reviews; this could be a consequence of the epidemic outbreak, since in a brief period of time it was not possible to properly design and publish results from Randomized Controlled Trials (RCTs) or methodologically sound prospective studies, while observational retrospective studies can be set up and conducted more rapidly, being, nonetheless, at higher risk of several types of bias and pitfalls (Viswanathan et al. 2013; Grimes et al. 2002).

We found misconduct and process-related issues as the most prominent cause of paper retraction. As an example of observational study that got retracted due to misconduct reasons, Sun M, et al. (Sun et al. 2020), in their article “Anal swab as a potentially optimal specimen for SARS-CoV-2 detection to evaluate hospital discharge of COVID-19 patients” did not acquire patients’ consent prior to publication in order to conduct their study. Even though these issues have always been there for the scientific path to publication, a stress test such as COVID-19 amplified their detrimental effects.

An interesting peer-review process’ pitfall can be described recalling one attempt at meta-analysis by Hussain et Al. (Hussain et al. 2020). The retraction statement goes: “This article has been retracted at the request of the authors due to inadvertent errors (such as in the calculation of odds ratio for age and naming patients’ groups in the plots) that unfortunately passed unnoticed during the extremely rapid review and publication process at the peak of the COVID-19 pandemic”. This meta-analysis wanted to assess how mortality by COVID-19 was influenced by obesity, and it was based on data from the first four months of the pandemic. This retracted study demonstrates the importance of relying on sound scientific methods and rigorous peer-reviews. Furthermore, one study supported the non-effectiveness of both surgical and cotton masks in preventing the dissemination of SARS-CoV-2 from patients with COVID-19 (Bae et al. 2020). It is possible that this type of publications can have a negative indirect effect on health policy decisions and therefore on the health of the population.

Notably, Impact-Factor (IF) was extremely heterogeneous, and some articles found their way to high IF journals, as was the case with the two articles published in the New England Journal of Medicine and The Lancet, by Dr. Mandeep R. Mehra et Al., who utilized an unfit database taking part in the “Surgisphere Scandal” (Baker et al. 2020). Articles published in high IF journals were checked and retracted relatively quickly, while articles published in journals with lower IFs took much more time to be community checked and retracted. A possible explanation for this is that journals with higher IF have more visibility and are therefore read by a higher number of experts. As a result, poor quality articles are more easily spotted and forced to retraction. Overall, journals’ IF was found to be relatively high and with high variance. This means IF cannot always be considered as a guarantee for peer-review quality, as various authors have stated in their work (Vrabel 2019; Paulus et al. 2018; Juyal et al. 2019).

As for retracted papers’ geographic distribution, we found that the Asian continent was the most afflicted. This represents a direct consequence of one key factor: the urgent need for information within the epicenter of the pandemic, which required observational studies to be preferred during the first months of the pandemic, even though these are more easily tampered in their data and afflicted with errors in their method (Viswanathan et al. 2013; Grimes et al. 2002).

In order to address these issues, publishers, editors and other stakeholders should focus on several aspects of the path to scientific publication: improving the process by granting adequate peer-review times (Barnabic et al. 2022; COVID-19 Report 2022; Improving pandemic preparedness and management 2020); using open data when possible or making it always available upon reasonable requests (Hynicka et al. 2022). It could also be beneficial to use distributed ledger technologies (eg, blockchains) in order to tamper proof data and its analysis or machine learning to double check data (Boetto et al. 2020).

Study limitations

This review presents various limitations. First, we may have missed other studies since we used only one database. Secondly, information about specific reasons for article retraction are only available in the article retraction statement, therefore the comprehensiveness of retraction motivations could not be as complete as it should.

Although these limitations exist, to the best of our knowledge no other studies on the subject have been published yet. This study therefore starts a much-needed discussion on the topic of peer reviewed scientific literature during times in which the need for accurate new information is in high demand. Finally, in reading this review, a healthy dose of discretion is advised, since we do not suggest the academic situation was much better or worse before the pandemic, but only analyze the effects COVID-19 has had on the scientific path to publication.

Conclusions

In a modern scientific community in which “publish or perish” is still mandatory and h-index is considered a sound metric to value scholars, quantity is frequently prioritized over quality. Although peer-reviewers usually defend the scientific community's integrity, the publication process is not perfect and it can miss its target, as reported here about the COVID-19 pandemic. Open access and fast track paths to publication are useful tools but might represent a menace if left unchecked since the former can be an incentive to accept manuscripts, and the latter can lead to lower quality threshold and reduced attention by peer-reviewers.

It is reasonable to assume the retracted articles found in this review are just the tip of an iceberg of dubious-value publications pre- and during the pandemic. During times of great stress, such as the SARS-CoV-2 pandemic, this should be considered a menace for the integrity of modern science as well as for the well-being of the global community since unchecked facts and fake evidence are the ground on which fake news thrive.

Author contributions

AC had the idea, contributed to data extraction and to drafting the manuscript; AS contributed to data extraction and to drafting the manuscript; FS data extraction and analyzed the data; DG contributed to data extraction and to drafting the manuscript; DG contributed to drafting the manuscript and coordinated the project. All authors read and approved the final manuscript.

Funding

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Declarations

Competing interests

The authors have no relevant financial or non-financial interests to disclose. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Italian or the United States government, nor of the institutions the authors are affiliated with.