Re: Expression of Concern for Bryant A, Lawrie TA, Dowswell T, Fordham EJ, Mitchell S, Hill SR, Tham TC. Ivermectin for Prevention and Treatment of COVID-19 Infection. Am J Ther. 2022;29:e232.

To the Editor:

The Editorial “Expression of Concern”¹ further to the “Commentary”² by Rothrock et al regarding our systematic review³ of clinical trials of ivermectin in COVID-19, raises issues to which we responded⁴ rapidly, following the allegations of unreliability made against one large clinical trial.⁵ The Editorial “Erratum” now posted with our original article³ contains hyperlinks not corresponding with the citations in the PDF version¹; these point in error to irrelevant citations.

We have already responded to the criticisms made by Rothrock et al² in a separate letter.⁶ Concerning the disputed study,⁵ the preprint server Research Square posted a notice that concerns expressed by undisclosed complainants were “under formal investigation.” Enquiry of the Egyptian Ministry of Education has been acknowledged but without substantive report; at this time, we have no further information.

The imbalance of covariates in the control group of Niaee⁷ would not normally be a ground for exclusion of a trial from a systematic review. If such imbalance was detected, then it would justify placing the trial at unclear or high risk of bias, but not for exclusion. As with Elgazzar,⁵ the Niaee⁷ study was included because the trial met the inclusion criteria of our review protocol.⁸ The substantive criticism is that the control group had fewer diagnoses confirmed by the polymerase chain reaction (PCR) test (inclusions being laboratory-confirmed by either PCR or computed tomography (CT) scan). However, all participants had severity attributed from CT, and severity is if anything biased toward slightly fewer severe cases in the control group, corresponding to a bias conservatively against ivermectin. Suspicion that lack of PCR confirmation might imply inclusion of non–COVID-19 cases in controls would likely produce a similarly conservative bias. The slightly lower SpO₂ in the controls appears typical of what randomization in a small group might be expected to deliver, and the body mass index (BMI) distribution appears remarkably uniform across study arms, contrary to the complaint.² Vital sign data do appear anomalous, but these are not critical to the mortality assessment, only to the bias assessment.

Subsequent to our article,³ further clinical trials of ivermectin in COVID-19 have (as expected) also been published. We offer 3 illustrations of mortality results derived with the addition of several new clinical trials, by Vallejos et al⁹ and Abd-Elsalam et al,¹⁰ the TOGETHER clinical trial¹¹ and the recently published I-TECH study.¹² The mortality data for the latter are, unusually, not reported in the main article but are available in the Supplementary Materials.¹³ The TOGETHER trial¹¹ has been formally reported even more recently, and many questions have been raised by multiple parties. Although the Data Sharing Statement promises a complete deidentified patient data set “immediately on publication,” the individual patient data as demanded by Rothrock et al² (among others) are not yet available. Here, we take data at face value from the article, but using all-cause mortality results from Table S6 of the Supplementary Materials (the mortality reported for our meta-analysis) as published on March 30, 2022 and advised by one Principal Investigator on April 3, 2022. From the Niaee⁷ study, we now use mortality stratified by severity, derived from raw data provided to Karale et al,¹⁴ but not available to us at the time of our original article.³

Figure 1 shows the results for the mortality outcome, subgrouped by disease severity, including all qualifying randomized trials. Figure 2 shows the consequence of deleting the disputed Elgazzar trial.⁵ We have no adequate basis for excluding Niaee⁷ but exhibit the results of doing so in Figure 3, which excludes both Elgazzar⁵ and Niaee.⁷

FIGURE 1.

Updated mortality meta-analysis including all qualifying trials.

FIGURE 2.

Updated mortality meta-analysis excluding Elgazzar.

FIGURE 3.

Updated mortality analysis with exclusions of both Elgazzar and Niaee.

The overall point estimates of mortality risk ratio vary from 0.51 (all studies) to 0.65 (Elgazzar excluded) to 0.75 (both Elgazzar and Niaee excluded). Only in the last case do orthodox 95% confidence intervals stray above unity, and by a very small amount. Above 95% CIs in Figures 2 and 3 are 0.96 and 1.03, respectively, in practical terms a minor difference. It is obvious that removal of study data will decrease the overall statistical power of any meta-analysis. However, even when significantly diluted by exclusion of disputed trials, meta-analysis continues to show an improved mortality outcome. Moreover, additional data from the later trials are broadly consistent with the original findings.

New data could justify full revision¹⁵ of a systematic review.³ As previously commented,⁶ there are 3 leading options for updating systematic reviews: (1) a living review continuously updated, (2) periodic review subject to criteria⁹ (typical intervals for specialist groups are 2–3 years), and (3) a Trial Sequential Analysis. We opted for the latter approach.

Subsequent to our article,³ Neil and Fenton¹⁶ confirmed by Bayesian hypothesis testing robust evidence of a mortality advantage under treatments including ivermectin. Their sensitivity analysis covered the exclusion of Elgazzar,⁵ the Niaee⁷ trial having been already disregarded, not on grounds of reliability, but simply because mortalities were not, at that time, reported⁷ by disease severity. Severity being a key component of the hypothesis of Neil and Fenton,¹⁶ data not stratified by severity were of no value. They showed explicitly that the removal of the disputed study⁵ (Niaee⁷ excluded by design) simply reduced the probability of a favorable risk reduction to around 0.77 or odds of 77:23 that ivermectin treatment offers a mortality benefit. Even under this significant reduction in participants, the conclusion of mortality benefit continued to hold.¹⁶

The Trial Sequential Analysis³ and the independent corroboration of Neil and Fenton¹⁶ by different methods provided good evidence that the conclusion of mortality advantage is robust. On the criteria of Garner et al¹⁵ and their decision flowchart (their Figure 1), the question “will the new studies change findings or credibility” could arguably be answered No, however given the controversies raised and the speed with which new data have arisen an updated review may be justified. This will be offered for publication in due course and include outcomes other than mortality.

Finally, we remark that we know of no clinicians using ivermectin in COVID-19 who would regard it as the sole therapeutic to be used in severe cases. In particular, corticosteroids are now recognized¹⁷ as critically important in late-stage disease. For seriously ill patients, it should be obvious that their survival probability will depend on many details of their management, not simply the use or nonuse of ivermectin.