Ivermectin for Prevention and Treatment of COVID-19 Infection: A Systematic Review, Meta-analysis, and Trial Sequential Analysis to Inform Clinical Guidelines. American Journal of Therapeutics, 28, e434–e460, July 2021

To the Editor:

Our meta-analysis of trial data on ivermectin for prevention and treatment of COVID-19¹ is potentially affected by a recent newspaper article² making 2 significant claims. First, the preprint manuscript by Elgazzar et al³ has been withdrawn; second, “if you get rid of just this research, most meta-analyses that have found positive results would have their conclusions entirely reversed.” The first claim seems untrue at this writing and the second unsustainable.

The preprint server Research Square makes explicit³ that the preprint was not withdrawn by the authors but removed on receipt of a complaint, and Prof. Elgazzar has confirmed⁴ that this was without any opportunity of reply. The complaints have been denied as defamatory, and his manuscript is said to remain under review for publication elsewhere. Currently, these claims and counterclaims have appeared only in news and social media and have had no independent adjudication.

Our inclusion of Elgazzar³ was in full conformity with the PRISMA guidelines⁵ which encourage the use of unpublished data, supported where necessary by direct author inquiries, to ameliorate tendencies to publication bias. Such inquiries were indeed made during the review process. Where satisfactory clarification was received, the data were included. We had no basis for excluding a trial that met the inclusion criteria of our review protocol. This applied equally to the study by Lopez-Medina et al⁶ that has also received postpublication criticism⁷ for its trial protocol violations (with different consequences to the analysis) but likewise met our review protocol inclusion criteria. The mechanism in systematic reviews for noting doubts over reliability lies in the risk of bias assessments, made appropriately in both cases.

Pending clarification of data reliability in the study by Elgazzar et al,³ we turn to the second claim made in the press² that conclusions are reversed if this study is removed. If Figure 3 in the study by Bryant et al¹ is reanalyzed to exclude the study by Elgazzar,³ there is still a clear result, showing a 49% reduction in mortality in favor of ivermectin (aRR = 0.51, 95% confidence interval 0.27–0.95) (Figure 1). Similarly, if we conduct the same sensitivity analysis in Figure 15 (prophylaxis outcome), there was an 87% reduction in COVID-19 infection in favor of ivermectin (aRR = 0.13, 95% confidence interval 0.08–0.21) (Figure 2), virtually unchanged from the previous analysis. Revised Figures are shown below. Hence, the leading outcome conclusions (ie, for mortality and prophylaxis) are robust to the removal of the study by Elgazzar,³ contrary to the press claims.²

FIGURE 1.

Mortality outcome analysis as in Figure 3 of the study by Bryant et al¹ but with the study by Elgazzar³ removed.

FIGURE 2.

Prophylaxis outcome analysis as in Figure 15 of the study by Bryant et al¹ but with the study by Elgazzar³ removed.

Removal of a single study is part of the exercise of a “leave one out” sensitivity analysis. This has already been performed by others,⁸ finding similarly that the evidence for ivermectin efficacy is robust. Other meta-analyses on selected subsets of the known randomized trials are already available. The WHO “Living Guideline”⁹ noted in our Discussion¹ already excluded the study by Elgazzar³ in March yet reported a mortality odds ratio reduction of 81%, with narrow 95% confidence intervals. Roman et al¹⁰ have recently offered a reduced data set, including the study by Elgazzar.³ Their conclusion that ivermectin shows no mortality advantage has been elegantly refuted by Neil and Fenton¹¹ in a Bayesian analysis of the same trial subset. This approach is likewise amenable to a sensitivity analysis removing the study by Elgazzar.³ Preliminary findings¹² still show positive benefit in mortality. The Bayesian approach is of course applicable to other trial data selections, including our own. These will be reported in due course.

In conclusion, accusations of fraud are of course very serious. Prof. Elgazzar and colleagues must however be given appropriate opportunity to respond. A comprehensive correction to our meta-analyses will be issued if the data are found to be unreliable, but this would be premature while accusations remain disputed.

Pending resolution of the conflicting claims and counterclaims, we simply point out that while quantitative measures of effect do of course change on removal of any study, the overall findings of a significant mortality advantage in ivermectin treatment, and in prophylaxis, remain robust to removal of the disputed data. The claim that conclusions are “entirely reversed”² cannot be sustained on the evidence.