Redundancy and methodological issues in articles on COVID‐19

Despite the seriousness of the current pandemic, logical and critical thinking, common sense and method remain the mainstay of biomedicine. Unfortunately, the panic caused by the disease has led many to abandon those principles. Some manuscripts with suboptimal methodology, that would never get published in normal times, are published as journals publish quickly (and gain citations), without proper review and level of criticism. This situation has been used by medical equipment manufacturers and pharmaceutical industry as well, to promote publication of biased‐sponsored articles. It seems that anything related to COVID‐19 goes during this pandemic. In the aftermath, months from now, the same authors and journals are likely to publish corrections and retrospectives recognizing the mentioned issues and justifying them with the need for rapid spread of information that was necessary to fight the virus.

In a recent comment on the article by Dr Ioannidis, ¹ we have discussed the issue of redundancy in scientific research and reporting. ² Redundant research usually denoted duplicate studies that present already published data. In a broader sense, any study or review can be considered redundant (not required, unnecessary) if it is without clinically significant or useful result, and such outcome could have been foreseen prior to carrying out the study. Redundant research is published just for the sake of publishing. Although this is not a new problem, ³ this pandemic has created a very rich soil for such studies.

There are several reasons, intentional and unintentional, why redundant publications occur. Intentionally, redundant studies are mostly done to further careers (because of publication pressure), to favour industry or simply because funds are available to do them. The overgrowing publishing industry is certainly encouraging it which is obvious from, in example, the geometric growth in number of redundant reviews and meta‐analyses. ⁴

One of the common unintentional reasons why redundant studies are conducted is the lack of understanding of the concept of clinical significance, its relation to statistical significance and how clinical significance is defined. Good examples are randomized controlled trials (RCTs) that evaluated the effect of anti‐fibrinolytic substances on blood loss during various surgical procedures which are usually powered to detect statistically significant difference in blood loss in millilitres, but not powered enough to detect whether this difference was clinically significant (increase in blood transfusion rate).

In this research, we aimed to evaluate the trend in the number of publication on COVID‐19, quantify publication redundancy and determine methodological issues in publications on COVID‐19. In order to do this, we searched LitCovid ⁵ (a curated literature hub), identified and collected information about publications on COVID‐19 indexed in PubMed. The information on publication type and country of origin was obtained from MEDLINE records. All articles found in LitCovid were included.

On May 10, there were 10 899 publications with valid PMID indexed in PubMed, with around 25% originating from China (Table 1). The weekly rise in the number of publications continued to approximate an exponential function. ² By reviewing the published material, one can determine that the majority of publications are at high risk of bias (opinions, case reports and series, studies with design that does not support reporting of an outcome), redundant or methodologically flawed studies. Although the problem of hyperpublication has been recognized, ⁶ , ⁷ , ⁸ specific issues have not yet been addressed.

TABLE 1.

Publication types of 10,899 scientific publications on COVID‐19, as reported by the MEDLINE database

Publication type	N	%
Primary studies
Case reports	309	2.84
Comparative study	10	0.09
Randomized controlled trial or clinical trial	6	0.06
Evaluation or validation study	6	0.06
Secondary studies
Review	761	6.98
Guideline/practice guideline	59	0.54
Systematic review/systematic review with meta‐analysis/meta‐analysis	20	0.18
Historical article/review/biography	12	0.11
News/interviews
News	171	1.57
Interview	6	0.06
Erratums/retractions
Published erratum	34	0.31
Retraction of publication	2	0.02
Other	27	0.25
Journal Article a	6185	56.7
Letter	2135	19.6
Editorial	1156	10.6

For the rest of retrieved publications, the PMID identifier cited in LitCovid was not recognized. Majority of publications (57%) fall into the category 'Journal Article' because their design was not specified. Although most of the remaining publications are editorials, news, letters and opinions, there are at least 20 systematic reviews and over 50 guidelines.

^a Nonspecified design. Wherever a MEDLINE record assigned a publication to both Journal Article and specific design type such as review, the publication was counted as a specific design type.

We have determined three main issues in COVID‐19 publications:

1. Nonevidence‐based information/recommendation. For example, the risk of spreading COVID‐19 during laparoscopic surgery has been given much significance, although it is in fact negligible. A very unrealistic laboratory model of HIV virus spreading in surgically created vapour ⁹ has been accepted as fact and used as an argument ¹⁰ without noticing that it had been refuted. ¹¹ When assessing the risk of disease spread through surgical vapour, splashed blood and evacuated artificial pneumoperitoneum, the presence of viable viral particles is constantly being equalized with infection risk ¹⁰ , ¹² without considering the number of particles that are actually needed to cause infection, and whether there are any viral particles in tissues outside the respiratory tract at all.

2. Redundancy and over‐publication. Our search retrieved 407 publications in PubMed on pulmonary CT findings in COVID‐19 until May 10. CT findings are mostly nonspecific and cannot be considered pathognomonic of COVID‐19 infection because they substantially overlap with other entities. ² There are two studies reporting good results in differentiating CT scan findings in COVID‐19 from non‐COVID pneumonias with the aid of AI, but the majority of the non‐COVID pneumonias were bacterial, and such CT findings are much easier to differentiate. ¹³ Suggestions that chest CT scan should be used for screening or even added to standard preoperative workup are irresponsible, have no scientific basis and risk exposing numerous patients to unnecessary radiation. ¹² , ¹⁴ , ¹⁵

3. Methodological errors and bias. Sponsored and highly biased clinical trials that have almost equal number of authors and patients, are being published without control group, under the excuse of compassion and urgency, ¹⁶ although properly controlled clinical trials on COVID‐19 had already been published in the same journal. ¹⁷ The results are declared promising based on lower mortality in comparison with some other series, but series that reported equal mortality were not cited. ¹⁸

This and future pandemics are different from the previous ones because they happen in the age of information. Social media have enabled various individuals and groups to spread their beliefs, no matter how harmful they may be. Easy access to information has created a gap between a large quantity of information on one side, and an insufficient percentage of general public who are able to critically assess them on the other. Similarly to containing infectious diseases by vaccinating a critical percentage of the population, false information and sensationalism will fall on barren ground and get marginalized only when a sufficient proportion of population is taught logical reasoning. Unlike information in the social media that cannot be controlled, information released in scientific publications can and should be controlled.

Besides measures mentioned previously, ² there are several other ways redundant publications could be reduced:

1. RCT protocols, besides being registered, should be published and critically assessed by international readership to reduce bias and avoid incomparable outcomes. This way RCTs protocols could be modified/unified to obtain results that could be meta‐analysed;

2. RCTs should provide a supplemental file that would contain a summary of relevant data, properly formatted, to enable faster systematic reviews and meta‐analyses;

3. both statistical and clinical significance criteria should be clearly defined in RCTs to avoid statistically significant results that are useless in clinical practice;

4. through development of a special journal devoted to publishing RCT protocols that would also serve as a forum to discuss methodologically insufficiencies.

Currently, a vast quantity of useless and biased data is being intentionally and unintentionally poured into the pool of evidence‐based medicine, diluting it to such extent that finding relevant and true information will soon become very difficult, if at all possible.

Often, low‐quality or false information is obscured by being embedded in lengthy texts and supplementary material, or wrapped in technical terms (machine learning, neural network, Bayesian statistics) which hinder potential criticism, since most physicians are probably not well acquainted with them. Good examples are various useless predictive models for COVID‐19 that have been hastily developed by advanced methods and will, of course, never be used. ¹⁹

To account for this, future medical students should be formally trained in mathematics/logic, computer science and statistics so they could critically asses each information and claim they are being served, do their own research and analysis, and improve the methods of data retrieval and synthesis in their favour.

Of course, there have been and there will be bigger problems than redundancy in publications, but at some point, this issue requires to be addressed.

CONFLICT OF INTEREST

All authors state that they have no conflict of interest.