Meta-analysis of statin and outcomes of coronavirus disease 2019 (COVID-19): Reconsideration is needed

Dear editor,

A meta-analysis is a popular approach to introduce novel findings by doing quantitative and qualitative analysis of established studies. However, a meta-analysis is subject to biases and mistakes [1]. Self-plagiarism or plagiarism and salami publication can be less apparent in meta-analyses, unlike other research, and may be easily missed.

In this letter, we would like to discuss an issue of methodological fallacies, potential self-plagiarism and text-recycling by Hariyanto et al. study entitled “Statin and outcomes of coronavirus disease 2019: A systematic review, meta-analysis, and meta-regression” recently published in Nutrition, Metabolism, and Cardiovascular Disease journal [2] and their similar paper entitled “Statin therapy did not improve the in-hospital outcome of coronavirus disease 2019 infection” published in Diabetes & Metabolic Syndrome: Clinical Research and Reviews [3].

There are several points deserved to be pointed out. First of all, we would like to address the self-plagiarism, text-recycling, and salami publication which is an unfair practice.

Both mentioned articles [2,3] have the same conclusion on the same topic. Surprisingly, the authors did not cite or mention that the current study is an updated meta-analysis, which may affect editors and reviewers' decision, considering that the results are the same. According to International Prospective Register of Systematic Reviews (PROSPERO) guidelines, reviewers have to mention similar publication to the existing one “Details of any existing review of the same topic by the same authors”, in other words, the authors have to mention if this review is an updated version of the previous work. Failure to mention this matter is against systematic review publication ethics and will result in retraction of PROSPERO registration number (if registered). Moreover, the review is not in line with PRISMA guidelines as there is no PROSPERO or other registration number. Registration of protocol not only provides clear insight into the methodology of each systematic review, but also decreases the authors' biases [4].

We were surprised that the authors self-cited some of their irrelevant publications such as anemia, thyroid, and dementia papers but did not cite the most important one [3]. It seems reasonable to update the meta-analysis in case you may face with adequate number of studies which subsequently may affect the conclusion. In addition, the search date between the first and second publications was only 3 months, Thus, there is inadequate justification for updating the review.

In the latest article [2], the authors included risk of COVID-19 on 8 studies. However, It could be done in the previous one [3]. If the novelty of the current study is meta-regression, it could actually be done in the previous publication [3] with or without addition of “Risk of COVID-19”. Thus omitting the need for conducting another meta-analysis.

Reusing data, methodological approach, and ideas to answer the same question with same answer without referring/citing and providing adequate explanation can be considered as text-recycling and self-plagiarism. Omitting analysis so that it can be written as another paper is a salami publication [5].

As another major point, we would also like to ask for clarification in the methodological fallacies presented by the authors.

Findings from previous literature has indicated that the search strategy must be carried out in different databases depend on the review question, including Scopus, PubMed/Medline, Web of Science, and Embase to capture all potential literature that satisfy the inclusion criteria [6,7]. For instance, Bramer et al. concluded that at least Embase, Medline, Web of Science, and Google Scholar should be searched as the minimum requirement to guarantee complete and efficient coverage [8]. Therefore, it is highly recommended not to restrict the search strategy to just two databases [9], which is happened in this study and raised our concerns. In addition, the authors declared that they included a wide range of study designs including: “randomized control trial, cohort, clinical trial, case-cohort, and cross-over design” in their review (Eligibility criteria, MATERIALS AND METHODs). Although, no clinical trial studies were included in the final analysis, the authors pooled the effect sizes from retrospective cohort studies, prospective cohort studies and case–control studies together. This analytical approach may result in biased and misleading conclusions. We wonder how is it possible to combine the effect sizes of different study designs together?

Moreover, the authors considered “risk of COVID-19” as a composite outcome meaning that risk of COVID-19 is in the same league as severity or mortality, we think this approach is erroneous which may affect the meta-regression results. Combining risk of COVID-19 with mortality is far-fetched, considering that the mortality rate in COVID-19 is approximately 2%. Also, studies on “risk of COVID-19” usually has greater number of participants because of non-COVID-19 control. The authors indicated that finally 11.930.583 patients were included in the analysis. The three largest studies were Hippisley-Cox et al. [10], Ho et al. [11], and Holman et al. [12], with 8.275.949, 285.817, and 3.138.410 patients, respectively. The first two studies were performed to assess the risk of COVID-19 while the third study did not entirely include COVID-19 patients. The authors failed to mention the number of COVID-19 patients in Holman et al. study [12]. Holman et al. study [12] was conducted in individuals with type 1 and 2 diabetes, irrespective of COVID-19 status. The authors did not mention the number of COVID-19 positive patients, which may be well below the number.

In addition, the authors included Huh et al. study with 65.149 patients [13] and Vila-Corcoles et al. study with 34.926 patients [14]. All of the above-mentioned studies reported risk of COVID-19 and did not present severity or mortality of COVID-19, except Holamn et al. [12] study. This leaves 130.332 patients. So, 11.930.583 patients were an overinflated estimate. With most of the patients were included for a “risk of COVID-19” analysis, participants with COVID-19 for the real “outcome” analysis was only 2% of this meta-analysis. Which may affect the pooled estimate and meta-regression analysis. Risk of COVID-19 analysis is usually derived from cohorts and we wonder, is it possible to combine their effect sizes together with the hospitalized patients?

In addition, the numbers on Figure 1 (PRISMA flow chart) do not add up, the authors provided explanation for exclusion of 10 studies (Study Selection and Characteristics, RESULTS), but did not provide for the other 10? 35 plus 10 is 45, not 55.

Our next concern relates to the extent to which the risk of bias assessment was explained. Elaborating the justifications behind the judgments of risk of bias not only provides clear transparency, but also helps reviewers/readers to decide whether they agree with the judgements or not [15], according to the Cochrane Collaboration. This matter is of great importance, considering that risk of bias often threatens the validity of meta-analyses results.

Based on the above mentioned points. We would like to ask the authors to provide clarification on these issues. We also urge editors to re-evaluate the papers for fallacy, mistakes, and unethical practices. If academic misconduct or flaws are found, necessary action(s) should be taken to state the commitment towards academic integrity and honesty. We are afraid that, if these things keep on going and disciplinary actions are not taken, there will be more biases, fallacies, and unethical practices done by other researchers in the future because such things can be done without consequences.

We hope that this letter could be taken into consideration and bring more clear future towards research in the field of medicine.

Declaration of competing interest

Neither of the authors declared a conflict of interest.

Handling Editor: J. Bella