Retraction in the era of COVID-19 and its influence on evidence-based medicine: is science in jeopardy?

In order to keep the scientific community well informed about the SARS-CoV-2 virus, a great number of articles have been published.¹ Up to July 14, 2020 a total of 31,360 documents were indexed on Pubmed, according to the LitCovid- NCBI.2, 3 The USA is the country with most articles published (5,033), followed by China (3,511) and Italy (2,590). The journals with most publications were: BMJ (BMJ Publishing Group Ltd.) (574), Journal of Medical Virology (John Wiley & Sons, Inc.) (317) and The Lancet (Elsevier) (230).

In this context, a search was carried out using the PubMed-Medline database on October 12, 2020 and using the following descriptors “coronavirus disease-19 OR coronavirus disease OR corona virus OR COVID-19 OR COVID19 OR SARS-CoV-2” and the following filters were applied in the data search: “Retracted Publication, Retraction of Publication”. Retraction Watch was used to identify retracted articles in preprint services.

Table 1 shows the data related to retracted articles in Pubmed database and pre-print services [Bioxviv and medRxiv – preprint server operated by Cold Harbor Laboratory Spring].4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25

Table 1.

Descriptive analysis of retracted studies in the PubMed-Medline® database.

Paper	First author	First author's Country	Reason for the retraction	Journal	SJR	Study type	Study area	Published online	Final publication	Retraction note	Objective	Main findings	Conclusions
Noninvasive versus invasive ventilation: one modality cannot fit all during COVID-19 outbreak4	Singh A	India	Plagiarism	Korean Journal of Anesthesiology	0.44	Letter to the editor	COVID-19 and noninvasive ventilation	Jul 08	Aug	Sep 14	To discuss the noninvasive ventilation in patients with COVID-19.	(i) To avoid intubation and, in this case, to reduce the risk of the mortality; (ii) The potential to generate aerosol and to transmit the SARS-CoV-2 virus using non-invasive ventilation has not been confirmed yet; (iii) There is a poor lung retractability in patients with COVID-19 because a massive alveolar damage was evident due to the release of inflammatory exudates in the alveoli and infiltrates in the interstitial, thus, leading to the development of acute respiratory distress syndrome; (iv) Non-invasive ventilation avoids the disadvantages associated with invasive ventilation.	A certain demographic profile of patients with COVID-19 and with acute respiratory distress syndrome (those with lesser comorbidities and younger) may benefit from noninvasive ventilation instead of intubation.
No deleterious effect of lockdown due to COVID-19 pandemic on glycemic control, measured by glucose monitoring, in adults with type 1 diabetes5	Beato-Víbora PI	Spain	Ethical issues	Diabetes Technology & Therapeutics	1.82	Observational	COVID-19 and diabetes mellitus	May 12	Ahead of print article	Aug	To evaluate the effect of COVD-19 lockdown on glycemia measures of patients with type 1 diabetes mellitus.	(i) Improvement was observed in 37% (n = 55) and deterioration in 16% (n = 23) of the patients for glycated hemoglobin during the lockdown period; (ii) During the lockdown, the continuous glucose monitoring (n = 68) increased in time from 70 to 180, from 61.2 ± 16.7 mg/dL to 64.1 ± 17.2 mg/dL and their estimated glycated hemoglobin decreased from 57 ± 12 mmol/L to 55 ± 12 mmol/L; (iii) Fast glucose monitoring users (n = 79) increased in time from 70 to 180, from 59.5 ± 15.4 mmol/L to 62.4 ± 15.7 mmol/L and estimated glycated hemoglobin from 57 ± 11 mmol/L to 54 ± 11 mmol/L.	COVID-19 lockdown was not negatively associated with altered glucose, by remote analysis of sensor data in patients diagnosed with type 1 diabetes mellitus.
5G Technology and induction of coronavirus in skin cells6	Fioranelli M	Italy	Peer review manipulation	Journal of Biological Regulators and Homeostatic Agents.	0.4	Experimental (Hypothesis)	5G as an application in the construction of virus-like structures	Jul 16	Ahead of print article	Jul 16	To evaluate whether 5G millimeter waves may act favoring the production of Coronaviruses in biological cells.	The article described that, by decreasing the wavelength, waves emitted from towers in 5G could be more effective in evolutions of DNAs within cells.	It was proposed that a new generation mobile technology could play the main role in constructing several types of viruses, such as Coronaviruses within a cell.
Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis7	Mehra M	United States of America	The authors were unable to validate the veracity of the data described in the study.	Lancet	14.55	Observational	COVID-19 and Hydroxychloroquine	May 22	Ahead of print article	Jun 5	To evaluate chloroquine or hydroxychloroquine alone or associated with a macrolide for treatment of patients with COVID-19 using as main outcomes the occurrence of de-novo clinically significant ventricular arrhythmias and in-hospital mortality.	(i) Ventricular arrhythmias were more frequent in the treatment groups when compared with the control population; (ii) The treatment group presented a higher mortality when compared to the control population; (iii) Increase in hospital death was associated with age, body mass index, black race or Hispanic ethnicity, coronary artery disease, congestive heart failure, history of arrhythmia, diabetes, hypertension, hyperlipidemia, chronic pulmonary obstructive disease, being a current smoker, and immunosuppressed condition; (iv) Lower in-hospital mortality risk was associated with use of statin, female sex, ethnicity of Asian origin, use of angiotensin-converting enzyme inhibitors (but not angiotensin receptor blockers).	There were no benefits in-hospital mortality for the treatment of COVID-19 with hydroxychloroquine or chloroquine (with or without a macrolide), instead, a higher risk of ventricular arrhythmias and greater hazard for in hospital death with COVID-19 was found.
SARS-CoV-2 infects T lymphocytes through its spike protein-mediated membrane fusion8	Wang X	China	Methodological issues	Cellular and Molecular Immunology	2.57	Experimental	Susceptibility of T lymphocytes to SARS-CoV-2 virus infection	Apr 7	Ahead of print article	Aug	To test the susceptibility of T lymphocytes to SARS-CoV-2 virus infection.	(i) SARS-CoV-2 virus may infect T cells through S protein-mediated membrane fusion; (ii) EK1 could inhibited the infection; (iii) Perhaps a different receptor may mediated the infection of T cells by SARS-CoV-2 virus, due to lower expression of angiotensin-converting enzyme 2 in T cells.	SARS-CoV-2 virus can infect T cells through S protein-mediated membrane fusion, and perhaps, through a different receptor, due to lower expression of angiotensin-converting enzyme 2 in T cells.
A mechanistic analysis placental intravascular thrombus formation in COVID-19 patients9	Mulvey JJ	United States of America	Duplicate	Annals of Diagnostic Pathology	0.7	Observational	COVID-19 and gestations	Apr 25	Jun	Jun 22	To evaluate the placental pathology of full-term births to patients with COVID-19.	(i) The 5 cases enrolled showed fetal vascular malperfusion and thrombosis found within the chorionic plate and stem villi in larger vessels in the fetal circulation; (ii) A deposition might be present within the villi and perivillous areas and decidua similar to the normal placental controls; (iii) Frank thrombosis of fetal chorionic plate vessels occurred in 3 cases; also, in 2 cases larger vessel thrombosis was confined to the stem villi; (iv) Distal lesions in villi associated with the fetal malperfusion occurred in 2 cases (one case as foci of avascular villi and in another case villous stromal-vascular karyorrhexis); (v) The effects of thrombosis might result from the systematic effects of the virus, considering the rare identification of viral spike protein and viral RNA staining within the COVID-19 placentas.	It concluded that the vascular thrombosis without complement deposition can be a characteristic of the systemic nature of COVID-19's procoagulant effects unrelated to systemic complement activation.
Clinical characteristics and blood test results in COVID-19 patients10	An XS	China	The authors identified errors in the laboratory data from patients with COVID-19. Also, the data was imputed in the statistical software.	Annals of Clinical & Laboratory Science	0.36	Observational	Epidemiology of the COVID-19	Apr	May	Jul	To evaluate blood test results and the clinical data from patients with COVID-19.	(i) 47 (73.4%) study participants were exposed to a confirmed source of COVID-19 transmission; (ii) The most common symptoms were fever (75%) and cough (76.6%); (iii) 28 (43.8%) patients with COVID-19 presented leukopenia, 10 (15.6%) lymphopenia, 47 (73.4%) elevated high-sensitivity C-reactive protein, 41 (64.1%) elevated erythrocyte sedimentation rate, and 30 (46.9%) had increased fibrinogen concentration; (iv) The counts of white blood cells and platelets, and the level of prealbumin increased significantly after treatment while aspartate aminotransferase, lactate dehydrogenase, and high-sensitivity C-reactive protein decreased; (v) Patients with COVID-19 who stayed more than 12 days in hospital presented higher body mass index and increased levels of aspartate aminotransferase, lactate dehydrogenase, fibrinogen, high-sensitivity C-reactive protein, and erythrocyte sedimentation rate.	Blood test results were associated with the clinical data and with the disease evolution in patients with COVID-19.
Cardiovascular disease, drug therapy, and mortality in COVID-1911	Mehra MR	United States of America	The authors were not granted access to the raw data to validate the findings.	New England Journal of Medicine	18.29	Observational	Cardiovascular disease and drug therapy in patients with COVID-19	May 1	Jun 18	Jun 25	To evaluate cardiovascular risk and drug therapy among hospitalized patients, as well as hospital deaths.	(i) Factors associated with in hospital death included age over 65 years, coronary artery disease, heart failure, cardiac arrhythmia, chronic obstructive pulmonary disease, current smoking; (ii) Angiotensin-converting-enzyme inhibitors and angiotensin-receptor blockers were not associated with in hospital death.	Patients with COVID-19 and cardiovascular disease had an increased risk of in hospital death; however, no association between in hospital death and use of Angiotensin-converting-enzyme inhibitors and angiotensin-receptor blockers was found.
Effectiveness of surgical and cotton masks in blocking SARS-CoV-212	Bae S	South of Korea	The authors did not recognize the concept of limit of detection for in-house reverse transcriptase polymerase chain reaction, which made some of the data unreliable and uninterpretable.	Annals of Internal Medicine	4.74	Experimental	Personal protective equipment	Apr 6	Jul 7	Jun 2	To associate the effectiveness of two types of masks (surgical and cotton masks) to filter the SARS-CoV-2 virus.	The median viral loads (log copies/mL) for SARS-CoV-2 virus were described for nasopharyngeal (5.66) samples, saliva (4.00) samples, after coughs without a mask (2.56), after coughs with a surgical mask (2.42), and coughs with a cotton mask (1.85).	Neither of the masks (surgical and cotton) was able to prevent the dissemination of SARS-CoV-2 virus to the environment and external mask surface.
Chinese medical staff request international medical assistance in fighting against COVID-1913	Zeng Y and Zeng Y	China	The account described therein was not a first-hand account.	Lancet Global Health	8.06	Correspondence	Health professional and medical support for COVID-19	Feb 24	Ahead of print article	Aug	To describe the urgent need for medical assistance to deal with COVID-19 in Wuhan, China.	The article described a physical and psychological demand to deal with COVID-19 in Wuhan, China. In addition, it reported that 1,716 Chinese medical workers were infected with SARS-CoV-2 virus and nine of them died.	There is an urgent need of medical staff support to deal with COVID-19 pandemic in Wuhan, China.
Chloroquine or hydroxychloroquine for COVID-19: why might they be hazardous14	Funck-Brentano and Salem JE	France	The article is a comment for a previous retracted study, and it was republicated as "Retraction and republication: Cardiac toxicity of hydroxychloroquine in COVID-19".	Lancet	14.55	Comment	COVID-19 and treatment	May 22	Ahead of print article	Jul 18	To discuss the findings and limitations of the “Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis” study.	The comment highlighted the main findings of the original study and discussed its limitations.	The comment demonstrated that more studies should be carried out to understand the relationship between death and ventricular tachycardia regarding the drug therapy to treat patients with COVID-19. The causes of deaths (i.e., arrhythmic vs non-arrhythmic) were not adjudicated in the study and should be better evaluated.
Uncanny similarity of unique inserts in the 2019-nCoV spike protein to HIV-1 gp120 and Gag15	Pradhan P	India	Withdrawn by the authors to revise its results	Bioxriv	-	Experimental	COVID-19 and HIV	Jan 31	Not applied	Feb 2	To compare the structure of HIV and SARS-CoV-2 virus	(i) 4 insertions are unique to SARS-CoV-2, and other coronaviruses do not present it; (ii) These 4 insertions are aligned with short segments with HIV-1 gp120 and Gag; (iii) S protein may have evolved from SARS-GZ02.	The authors evaluated a new evolutionary origin from SARS-CoV-2 virus, also, observed a similarity in the structure of HIV-1 and SARS-CoV-2 virus.
Epidemiological and clinical features of the 2019 novel coronavirus outbreak in China16	Yang Y	China	The number of confirmed COVID-19 patients was 18 times higher than they predicted.	Medrxiv	-	Observational	Epidemiology of COVID-19 in China	Feb 11	Not applied	Feb 21	Data was not available	Data was not available	Data was not available
Hydroxychloroquine plus azithromycin: a potential interest in reducing in-hospital morbidity due to COVID-19 pneumonia (HI-ZY-COVID)?17	Davido B	France	Controversy about Hydroxychloroquine and because the study was retrospective	Medrxiv	-	Data was not available	Treatment for COVID-19	May 11	Not applied	May 20	Data was not available	The data was not available	The data was not available
From SARS-CoV to Wuhan 2019-nCoV outbreak: similarity of early epidemic and prediction of future trends18	Chen Z	China	Submitted without the full consent of all authors	Bioxriv	-	Observational	Epidemiology and disease progression	Jan 25	Not applied	Jan 28	Comparison of COVID-19 pandemic and SARS	(i) Super spreader emerged early; (ii) SARS-COV-2 virus has a highly capability of human-to-human transmission; (iii) Medical staff was highly infected (iv) Discovery of human-to-human transmission in SARS-CoV-2 virus was late, compared do SARS-CoV virus; (v) The daily counts of COVID-19 cases were higher than the daily counts of SARS virus cases in 2003; (vi) Large-scale migration made the spread of disease favorable; (vii) The authors predicted that cumulative cases of SARS-CoV-2 virus might be 2 to 3 times the total of SARS; (viii) The infection peak will be in February.	The COVID-19 pandemic and SARS-CoV outbreaks were very similar, even though the Chinese government is taking very efficient decisions, the lack of awareness of the human-to-human transmission by the SARS-CoV-2 earlier, a super spreader may exist, contributes to the pandemic.
Analysis of ten microsecond simulation data of SARS-CoV-2 dimeric main protease19	Parves R	Bangladesh	Ethics violation	Biorxiv	-	Experimental	Bioinformatics	Apr 12	Not applied	Apr 16	The study carried out basic structural analysis, advanced flexibility and conformational analysis, for revealing out the regions and residues, which are mostly flexible and likely to be responsible for conformation of protease protein.	The authors were unable to understand the study findings and the techniques performed.	The authors were unable to understand the study's findings and the techniques performed.
Computational analysis suggests putative intermediate animal hosts of the SARS-CoV-220	Chu P	China	The authors want to perform an additional experiment to validate its results	Biorxiv	-	Experimental	Bioinformatics	Apr 05	Not applied	Apr 15	To evaluate the viral receptors binding with the host receptors	(i) SARS-CoV-2 virus showed the best binding with ACE2, when compared to SARS-CoV virus, RaTG13-CoV and Bat-CoV; (ii) RaTG13-CoV and Bat-CoV cannot bind efficiently into ACE2, compared with and SARS-CoV virus, which implicates in the existence of an intermediate host; (iii) Pangolin may not be the intermediate host for SARS-CoV-2 virus, in fact, tree shrew and ferret may be the intermediate hosts.	Tree shrew and ferret may be the intermediate hosts for SARS-CoV-2 virus, and not pangolins.
Mental health status and coping strategy of medical workers in China during the COVID-19 outbreak21	Siyu C	China	Authors withdrew it because they are performing more experiments to support their conclusion	Medrxiv	-	Data was not available	Clinical psychology and COVID-19	Feb 25	Not applied	Mar 07	Data was not available	Data was not available	Data was not available
Lung disease severity, coronary artery calcium, coronary inflammation and mortality in Coronavirus Disease 201922	Gaibazzi N	Italy	The objectives of the study were not approved by the institutional review board	Medrxiv	-	Data was not available	Comorbidities and COVID-19	May 06	Not applied	Jun 20	Data was not available	Data was not available	Data was not available
Smoking prevalence is low in symptomatic patients admitted for COVID-1923	Gaibazzi N	Italy	The objectives of the study were not approved by the institutional review board	Medrxiv	-	Data was not available	Smoking and COVID-19	May 10	Not applied	Jun 13	Data was not available	Data was not available	Data was not available
Psychiatric predictors of COVID-19 outcomes in a skilled nursing facility cohort24	Cercy ST	USA	Privacy issues when conducting the retrospective chart review	Medrxiv	-	Data was not available	Psychiatry and COVID-19	May 26	Not applied	June 21	Data was not available	Data was not available	Data was not available
Treatment response to hydroxychloroquine, lopinavir/ritonavir, and antibiotics for moderate COVID 19: a first report on the pharmacological outcomes from South Korea25	Kim SM	Republic of Korea	Controversy about Hydroxychloroquine and potential changes in results after peer review	Medrxiv	-	Data was not available	Treatment for COVID-19	May 18	Not applied	Jun 14	Data was not available	Data was not available	Data was not available

Bioxviv, pronounced "bio-archive" is a preprint server for biology and operated by Cold Harbor Laboratory Spring; medRxiv, pronounced "med-archive" is a preprint server for biology and operated by Cold Harbor Laboratory Spring; 2019-nCoV, new Coronavirus 2019; SARS-CoV, severe acute respiratory syndrome coronavirus; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; SARS-Gz02, SARS coronavirus GZ02; COVID-19, coronavirus disease 2019; PubMed-Medline®, PubMed Medical Literature Analysis and Retrieval System Online; HI-ZY-COVID, Hydroxychloroquine plus azithromycin for COVID-19 treatment; HIV-1, Human immunodeficiency virus 1; Gag, Gag protein; Gp120, glycoprotein 120; SCImago Journal Rank indicator; SARS, severe acute respiratory syndrome; RATG-13-CoV, Bat Coronavirus RaTG13; Bat-Cov, Bat Coronavirus; ACE2, angiotensin-converting enzyme 2; USA, United States of America.

The first author’s countries with most retraction were the USA and China with 3 articles, perhaps due to the large amount of publications from these countries. Also, a huge variety of SJR indicators was observed, ranging from low SJR indicator, such as Annals of Clinical & Laboratory Science (0.36) to those with the highest SJR indicator among medicine journals, such as New England Journal of Medicine (18.29) and Lancet (14.55). There were countless reasons for retraction, from duplicates and plagiarism to methodological issues and data misinterpretation. Duplication, ethical issues and plagiarism were more frequent in journals with low SJR indicator, whereas journals with high SJR indicator mostly reported methodological issues as the reason for retraction. The majority of the studies retracted were observational⁴ followed by experimental.³ A great variation was found in the study area, which included epidemiology, treatment, experimental and analysis.

The pre-peer-review databases presented the same amount of retractions as the journals, totaling 11 studies.15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25 However, most of the reasons for retractions were related to ethical issues, including objectives of the studies not being approved by the institutional review board (IRB), absence of consent from all the authors and lack of experiments to confirm the results. Most of the reasons for retractions in these databases might have been prevented by the reviewers’ careful analysis, which could have contributed to a more accurate paper.

Retraction should be avoided by using the maximum number of tools available, such as plagiarism identification by computational software and by improving the efficacy of the peer-review process.²⁶ Also, researchers should be more cautious when submitting data for publication, in order to avoid the problems related to data analysis or ethical issues, such as lack of authorization by the IRB. However, in several studies, it was not possible to determine the tenuous threshold between honest mistake and bad faith due to the author’s desire to publish in high impact factor journals. Perhaps, retractions in high impact factor journals are more noticeable due to the greater number of readers, contrasting with low impact factor journals, where retraction is not as evident and does not cause as much “fuss” as in high impact journals.

Misconduct in science can cause serious consequences for society, health policy and other matters. During the COVID-19 pandemic the best example was the publication of the article entitled “Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis”.⁷ The article demonsrated no benefits on in-hospital mortality for COVID-19 treatment with hydroxychloroquine or chloroquine (with or without a macrolide), instead, a higher risk of ventricular arrhythmias and greater hazard for in-hospital death was found. The study was based on observational data from an analytics company known as “Surgisphere”. No author evaluated the data included in the article and they were unable to access the full data to perform the statistical analysis. Following the publication, several groups identified database errors and the study was retracted. WHO denied that hydroxychloroquine or chloroquine had value in the treatment of COVID-19, based on its findings. The consequence of this retraction was discontinuation by the WHO and hydroxychloroquine, or chloroquine, were reallocated as drugs in test. Moreover, the Lancet changed its publication protocol and policy following this tragic episode.

To avoid publication of articles like the one in the Lancet, it is crucial to identify problematic articles, and it should not be totally the responsibility of the journal editorial staff and/or reviewers and/or tools; it is mainly up to the integrity and the ethics of the researcher who conducted the study. For example, the Brazilian government advocated in favor of the drug use and several governmental attitudes during the COVID-19 pandemic were contrary to the WHO recommendation. Following the retraction of that paper,⁷ the Government and some citizens openly criticized the WHO as to the credibility of their recommendations.

The COVID-19 pandemic was associated with a high index of publication “paperdemic”²⁷ and it favored the high level of retractions, including journals with the highest SJR and credibility in health science. Retractions can have consequences for health policies, mainly public ones, and can result in the rejection of evidence-based medicine by the government, like the Brazilian government and its hydroxychloroquine or chloroquine passion.

Conflicts of interest

The authors have no conflicts of interest to declare.

All authors have approved the manuscript and agreed with its submission to the journal. Also, all authors wrote and revised the manuscript.