Does hydroxychloroquine reduce mortality in patients with COVID‐19? A meta‐analysis with trial sequential analysis

DISCLOSURE

The authors declared no conflict of interest.

AUTHORS' CONTRIBUTIONS

PHC and CHL were involved in conceptualisation; PHC and HJJ in methodology and writing—original draft preparation; PHC in formal analysis; HJJ and LJOY in investigation; and LJOY and CHL in writing—review and editing.

The COVID‐19 outbreak has caused an unprecedented global health and financial crisis. As of August 2020, more than 20 million people worldwide have been infected; however, specific treatments remain investigational. Hydroxychloroquine, a classic drug derived from chloroquine for rheumatological diseases, has shown activity against the novel coronavirus in vitro and been authorised in some national regulatory agencies to treat patients with COVID‐19. ¹ , ² However, some studies reported no effect on the intubation rate or mortality. ³ We therefore performed a meta‐analysis to evaluate the effects of hydroxychloroquine on overall mortality in patients with COVID‐19. Furthermore, we employed trial sequential analysis (TSA) to verify whether the results of the meta‐analysis were conclusive.

Comprehensive literature searches using PubMed, Google Scholar, MedRxiv and the preprints literature were undertaken for studies published up to February 2021, using the keywords “COVID‐19”, “hydroxychloroquine” and “mortality” with related MeSH terms. Two reviewers (PHC and HJJ) independently screened the titles and abstracts and extracted the data. Any discrepancy was solved by group discussion. A random‐effects DerSimonian–Laird model was used to estimate the odds ratio (OR) with 95% confidence interval (CI). Heterogeneity across the studies was detected by I ² > 50% and Cochran Q‐test P < .1. Subgroup analyses were performed toward the study designs, the therapeutic regimens and the COVID‐19 severities to detect clinical and statistical heterogeneity. All statistical analyses were performed using the “metafor” and “meta” packages of R software (version 3.6.1., R Foundation for Statistical Computing, Vienna, Austria) We constructed TSA boundaries according to the O'Brien‐Fleming alpha‐spending function with two‐sided α = 5% and 1 – β = 80% power. We assumed a relative risk reduction by calculating from the mean of the event proportions for both the intervention and control arms. Random‐effect TSA was performed using TSA software (version 0.9.5.10 Beta; Copenhagen Trial Unit, Copenhagen, Denmark).

A total of 26 studies (n = 30 167, Table 1) were selected in meta‐analysis. The use of hydroxychloroquine with or without azithromycin did not reduce the mortality rates as compared with standard care (random‐effects OR, 1.01; 95% CI, 0.81‐1.25; I ² = 82%; Cochran P‐value < .01). TSA revealed an intervention event proportion of 16%, a control event proportion of 19% and a diversity of 88%. The cumulative Z‐curve did not cross the conventional boundary, and the required information size of 14 064 has been reached, demonstrating no difference in overall mortality in patients who received hydroxychloroquine with or without azithromycin compared with standard of care (TSA‐adjusted OR, 1.01; 95% CI, 0.79‐1.28; I ² = 77%; Cochran Q P‐value < .05, Figure 1). TSA results confirmed that our meta‐analysis was robust and authentic. The subgroup analyses yielded similar results that no difference exists in overall mortality in the different study designs (randomised control trials and non‐randomised control trials), the different therapeutic regimens (hydroxychloroquine with and without azithromycin) and the different COVID‐19 severities (all severity hospitalised, mild‐to‐moderate hospitalised, moderate‐to‐severe hospitalised and non‐hospitalised patients).

TABLE 1.

Included studies of the use of hydroxychloroquine regarding mortality

First author	Journal	Type	Patient characteristics	Treatment
Randomised control trial
Chen et al PMID: 32391667	J Zhejiang Univ (Med Sci)	Randomised control trial	Hospitalised patients (exclude severe symptoms)	HCQ 400 mg/d for 5 d
Skipper et al PMID: 32673060	Annals of Internal Medicine	Randomised control trial	Non‐hospitalised adults with early COVID‐19	HCQ (800 mg once, followed by 60 mg in 6‐8 h, then 600 mg daily for 4 more days)
Cavalcanti et al PMID: 32 706 953	NEJM	Randomised control trial	Mild‐to‐moderate hospitalised patients	HCQ 800 mg/d for 7 d with/without AZI
RECOVERY Collaborative Group PMID: 33031652	N Engl J Med	Randomised control trial	Hospitalised patients	HCQ 1600 mg on day 1, followed by 800 mg/d for 9 d
Abd‐Elsalam et al PMID: 32828135	The American Journal of Tropical Medicine and Hygiene	Randomised control trial	Hospitalised patients	HCQ 800 mg on day 1, followed by 400 mg/d for 15 d
Mitjà et al PMID: 32674126	Clinical Infectious Diseases	Randomised control trial	Non‐hospitalised adults with mild COVID‐19	HCQ 800 mg on day 1, followed by 400 mg/d for 6 d
Tang et al PMID: 32409561	BMJ	Randomised control trial	Mild‐to‐moderate hospitalised patients	HCQ 1200 mg/d for 3 d, followed by 800 mg/d for 2‐3 wk
WHO Solidarity Trial Consortium PMID: 33264556	N Engl J Med	Randomised control trial	Hospitalised patients	HCQ 2400 mg on day 1, followed by 400 mg/d for 9 d
Non‐randomised clinical trial
Gautret et al PMID: 32205204	Int J Antimicrob Agents	Non‐randomised clinical trial	Mild symptoms hospitalised patients	HCQ 600 mg/d for 10 d with AZI
Rosenberg et al PMID: 32392282	JAMA	Cohort/observational	Hospitalised patients	HCQ with/without AZI
Mahevas et al PMID: 32554525	BMJ	Cohort/observational	Mild‐to‐moderate hospitalised patients	HCQ 600 mg/d
Geleris et al PMID: 32379955	N Engl J Med	Cohort/observational	Moderate‐to‐severe respiratory illness hospitalised patients	HCQ 1200 mg on day 1, followed by 400 mg/d for 4 additional days with/without AZI
Yu et al PMID: 32418114	Sci. China Life Sci.	Cohort/observational	Critically ill patients with COVID‐19	HCQ 400 mg/d for 7‐10 d
Lagier et al PMID: 32593867	Travel Med Infect Dis	Cohort/observational	Hospitalised patients	HCQ 600 mg/d for 10 d with AZI
Magagnoli et al PMC7274588	Med (N Y).	Cohort/observational	Hospitalised patients	HCQ with/without AZI
Paccoud et al PMID: 32556143	Clin Infect Dis.	Cohort/observational	Mild‐to‐severe hospitalised patients	HCQ 600 mg/d for 10 d
Arshad et al PMID: 32623082	International Journal of Infectious Diseases	Cohort/observational	Hospitalised patients	HCQ with/without AZI
Huang et al PMC7313782	National Science Review	Cohort/observational	Non‐critical hospitalised patients	CQ 300‐600 mg/d for no more than 10 d
Grimaldi et al PMID: 33025225	Annals of Intensive Care	Cohort/observational	Moderate‐to‐severe hospitalised patients	HCQ 400‐800 mg/d for 5‐10 d
Joshua Barbosa (preprint)	Unpublished	Cohort/observational	Moderate‐to‐severe symptoms hospitalised patients	HCQ 800 mg on day 1, followed by 200‐400 mg/d for 4 additional days
RB Esper (preprint)	Unpublished	Cohort/observational	Outpatients	HCQ 800 mg on day 1, followed by 400 mg/d for 6 additional days with AZI
Ip et al PMID: 32790733	PLoS One.	Cohort/observational	Hospitalised patients	HCQ 800 mg on day 1, followed by 400‐600 mg/d for 4 additional days with/without AZI
FJM de Novales (preprint)	Preprints (www.preprints.org)	Cohort/observational	Hospitalised patients	HCQ 800 mg on day 1, followed by 400 mg/d
Mallat et al PMID: 33350752	Medicine (Baltimore)	Cohort/observational	Hospitalised patients	HCQ 800 mg on day 1, followed by 400 mg/d for 10 d
Shailendra Singh (preprint)	MedRxiv	Cohort/observational	Hospitalised patients	HCQ with/without AZI
Emilie Sbidian (preprint)	MedRxiv	Cohort/observational	Hospitalised patients	HCQ with/without AZI

Abbreviations: AZI, azithromycin; HCQ, hydroxychloroquine.

FIGURE 1.

Trial sequential analysis in meta‐analysis of overall mortality. In this figure, TSA showed that the Z‐curve has crossed the required information size but has not crossed the conventional boundary, suggesting that the hydroxychloroquine treatment is no different from conventional therapy in reducing overall mortality. The result is conclusive and robust based on TSA. X‐axis, accrued information size; Y‐axis, cumulative Z‐score; blue line, cumulative Z‐value; solid red lines, trial sequential monitoring boundaries and the futility boundaries (threshold for statistical significance in TSA); horizontal dark red line, conventional boundaries (threshold for significance in conventional meta‐analysis); and vertical solid red line, required information size. TSA, trial sequential analysis

As the COVID‐19 pandemic has reached critical new levels, there is an urgent need for effective treatments of the disease. A previous meta‐analysis evaluating the effects of hydroxychloroquine in the treatment of COVID‐19 was characterised by insufficient and often conflicting evidence. ⁴ However, with numerous studies emerging since its publication, the previous meta‐analysis, which included limited studies, does not reflect the current understanding of the literature. Thus, we performed an updated meta‐analysis that showed no evidence of benefit for hydroxychloroquine in reducing mortality. ⁴

Even if the meta‐analysis alone concluded that the intervention has no effect, it still could have insufficient statistical power to investigate the true effects. ⁵ TSA is a realistic and reliable tool to test whether the meta‐analysis is adequately powered or reports spurious results because of systematic bias or random errors. The advantages of TSA include re‐estimating the sample size needed or stopping further trials when the benefits of intervention are not existent. ⁶ We performed TSA in the present meta‐analysis and demonstrated that it might be futile to conduct future trials. Regarding the opportunity costs, further hydroxychloroquine trials aiming for reducing mortality should not be a top priority in the war against the COVID‐19 pandemic.

In conclusion, our meta‐analysis with TSA suggests that the use of hydroxychloroquine in patients with COVID‐19 has no benefit in reducing overall mortality.

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request.