Systematic review and meta-analysis of randomized trials of hydroxychloroquine for the prevention of COVID-19

Abstract

Background

Recruitment into randomized trials of hydroxychloroquine (HCQ) for prevention of COVID-19 has been adversely affected by a widespread conviction that HCQ is not effective for prevention. In the absence of an updated systematic review, we conducted a meta-analysis of randomized trials that study the effectiveness of HCQ to prevent COVID-19.

Methods

A search of PubMed, medRxiv, and clinicaltrials.gov combined with expert consultation found 11 completed randomized trials: 7 pre-exposure prophylaxis trials and 4 post-exposure prophylaxis trials. We obtained or calculated the risk ratio of COVID-19 diagnosis for assignment to HCQ versus no HCQ (either placebo or usual care) for each trial, and then pooled the risk ratio estimates.

Results

The pooled risk ratio estimate of the pre-exposure prophylaxis trials was 0.72 (95% CI: 0.58–0.90) when using either a fixed effect or a standard random effects approach, and 0.72 (95% CI: 0.55–0.95) when using a conservative modification of the Hartung-Knapp random effects approach. The corresponding estimates for the post-exposure prophylaxis trials were 0.91 (95% CI: 0.72–1.16) and 0.91 (95% CI: 0.62–1.35). All trials found a similar rate of serious adverse effects in the HCQ and no HCQ groups.

Discussion

A benefit of HCQ as prophylaxis for COVID-19 cannot be ruled out based on the available evidence from randomized trials. However, the “not statistically significant” findings from early prophylaxis trials were widely interpreted as definite evidence of lack of effectiveness of HCQ. This interpretation disrupted the timely completion of the remaining trials and thus the generation of precise estimates for pandemic management before the development of vaccines.

Background

Hydroxychloroquine (HCQ) is not an effective treatment for established coronavirus disease 2019 (COVID-19) [1, 2], but it is unclear whether HCQ can prevent symptomatic COVID-19. Early in the SARS-CoV-2 pandemic, about 30 randomized trials were designed to study HCQ as prophylaxis for COVID-19 [3]. After the findings from two of these trials were reported in the Summer of 2020 [4, 5], HCQ was generally viewed by the medical community as ineffective for COVID-19 prophylaxis. The emergence of that consensus was surprising because both trials found a lower risk of COVID-19 in the HCQ group, though they were too small to rule out either benefit or harm of HCQ.

A timely completion of the remaining trials would have generated precise estimates of the potential effectiveness of HCQ to prevent COVID-19 among those at high risk of infection or complications. However, the widespread conviction about HCQ’s lack of effectiveness dramatically slowed down the recruitment into ongoing trials of HCQ prophylaxis (one of them carried out by the authors of this report) [6]. As a result, key decisions were made based on insufficient evidence during the pre-vaccine period of the pandemic.

We conducted a systematic review and meta-analysis of randomized trials that study the effectiveness of HCQ to prevent COVID-19 either before known exposure to an infected individual (pre-exposure prophylaxis) or after known exposure to an infected individual (post-exposure prophylaxis).

Methods

Studies were eligible for inclusion if they were randomized clinical trials comparing hydroxychloroquine as prophylaxis (pre-exposure or post-exposure) for COVID-19 with a non-active control, included individuals who had a polymerase chain reaction (PCR) negative test for SARS-CoV-2 at the time of treatment assignment, and had the full text published in a peer-reviewed journal or as a pre-print. Given the relatively small number of trials, we identified candidate studies through expert consultation and confirmed the findings with a search of PubMed, medRxiv and Clinicaltrials.gov as of March 20, 2022, using the search strategies described in the Appendix (first reported as part of a preprint on September 29, 2020 [7]). Two authors (XGA and MAH) independently reviewed the full text of the identified studies and extracted the data. Disagreements were resolved by consulting other co-authors (JdA, RP). Incomplete or unpublished data were requested from the investigators of each trial. The risk of bias of the included studies was assessed independently by 2 authors (XGA, MAH) using the “Rob 2” tool by the Cochrane Bias Methods Group [8].

For each of the identified trials, we obtained or calculated the risk ratio of COVID-19 for assignment to HCQ versus no HCQ (either placebo or usual care) among test-negative individuals at baseline. Our primary analysis is based on the definition of COVID-19 reported in the primary analysis of each study. In addition, we tried to harmonize the definition of COVID-19 across studies by conducting a separate meta-analysis for laboratory-confirmed symptomatic COVID-19.

We calculated, separately for pre- and for post-exposure prophylaxis studies, the pooled risk ratio estimate and its 95% confidence or compatibility interval (CI) using a fixed (or common) effect approach and two types of random effects approaches [9, 10]. Because the standard random effects method may yield an anticonservative (that is, too narrow) 95% CI [9], we also used the Hartung-Knapp random effects method [10], which has been shown to generally outperform the standard random effects method [11]. However, this latter method results in an even more anticonservative 95% CI (narrower than the 95% CI from the standard method) when, as in our meta-analysis, the estimated between-study heterogeneity is small (tau-squared near zero) [12, 13]. We therefore used the Hartung-Knapp method with an ad hoc modification [14] designed to ensure that its 95% CI remains wider than that of the standard method, even though the resulting 95% CI is expected to be conservative (too wide) when, as in our meta-analysis, the number of studies is small [15]. We also computed prediction intervals [16]. The protocol of the review was not registered. All analyses were conducted in Stata (version 16.1; Stata Corporation, College Station, Texas, USA).

Results

Eighty-eight records were identified, of which 16 were duplicates. The screening excluded fifty-nine records because of lack of randomization, use of HCQ as treatment for COVID-19 or, in the case of records from clinicaltrials.gov, the absence of published results. Of the remaining 13 reports, two were excluded because they found no incident events in the HCQ arm [17] or at all [18] (Supplementary Fig. 1 contains the PRISMA 2020 flow diagram). Eleven completed randomized trials were included in the meta-analysis: seven trials studied hydroxychloroquine as pre-exposure prophylaxis [19–25] and four as post-exposure prophylaxis [4, 5, 26, 27].

Table 1 summarizes the key design characteristics and effect estimates reported by each study. The primary outcome definition varied across trials, with some trials using the presence of symptoms with or without laboratory confirmation [4, 19, 23], others using laboratory-confirmed symptomatic COVID-19 [5, 17, 18, 24, 25, 27] and others laboratory-confirmed SARS-CoV-2 infection with or without symptoms [20, 21, 26]. Our risk of bias assessment identified the handling of incomplete ascertainment of the outcome and the exclusion of patients after randomization as possible sources of moderate bias (see Supplementary Table 1).

Table 1.

Summary of randomized trials studying hydroxychloroquine as prophylaxis for COVID-19

Trial	Publication date	Sample size	HCQ dose	Primary Outcome Definition	Effect estimate (95% CI)	Percentage of patients fully adherent to treatment
Pre-exposure prophylaxis trials
Rajasingham et al. (19)	September 2020*	494 HCQ (arm 1) 495 HCQ (arm 2) 494 Placebo	Arm 1: 400 mg loading dose twice, then 400 mg once weekly for 12 weeks Arm 2: 400 mg loading dose twice, then 400 mg twice weekly for 12 weeks	PCR-confirmeda COVID-19 or probable compatible illness	HR: 0.73 (0.48, 1.09), both HCQ arms combined	Not reportedb
Abella et al. (20)	September 2020	66 HCQ 66 Placebo	600 mg daily for 56 days	PCR-confirmed SARS-CoV-2 infection, regardless of symptoms	RD: -0.3 (-8.9, 8.3) cases per 100	97% HCQ 98% placebo
Seet et al. (21)	April 2021	432 HCQ 619 ascorbic acid	400 mg loading dose, then 200 mg daily for 42 days	PCR-confirmed SARS-CoV-2 infection, regardless of symptoms	RR: 0.70 (0.44, 0.97)c	71.4% HCQd 80.0% ascorbic acidd
Rojas-Serrano et al. (25)	May 2021*	62 HCQ 65 Placebo	200 mg daily for 60 days	PCR-confirmed symptomatic COVID-19	HR: 0.18 (0.21, 1.59)	Not reportede
Syed et al. (22)	May 2021*	48 HCQ arm 1 51 HCQ arm 2 55 HCQ arm 3 46 Placebo	Arm 1: 400 mg loading dose twice on day 1, then 400 mg weekly for 12 weeks Arm 2: 400 mg once every 3 weeks Arm 3: 200 mg once every 3 weeks	Unclear: “COVID-19-free survival”	Not reported Calculated RRg: 0.70 (0.19, 2.59)	Not reported
Naggie et al. (23)	August 2021*	683 HCQ 676 Placebo	600 mg loading dose twice on day 1, then 400 mg daily for 29 days	Symptomatic COVID-19, PCR-confirmed or not	RD: -1.98 (-4.6, 0.9) cases per 100	94.4% HCQf 95.7% placebof
Grau-Pujol et al.h**. (17)	November 2021	142 HCQ 127 Placebo	400 mg daily x 4, then 400 mg weekly for 30 daysi	Laboratory-confirmed (seroconversion or positive PCR) symptomatic COVID-19	Not reportedj	98% HCQ 97% Placebo
Polo et al. (24)	March 2022	231 HCQ 223 Placebo	200 mg daily for 12 weeks	PCR-confirmed symptomatic COVID-19	RR: 0.49 (0.00, 2.29)	85% HCQ 86% placebo
Post-exposure prophylaxis trials
Boulware et al. (4)	June 2020	414 HCQ 407 Placebo	800 mg on day 1, then 600 mg daily days 1–5	Symptomatic COVID-19, PCR-confirmeda or not, by day 14	RD: -2.4 (-7.0, 2.2) cases per 100	75.4% HCQ 82.6% placebo
Mitja et al. (5)	July 2020*	1225 HCQ 1300 Usual care	800 mg day 1, then 400 mg daily days 1–6	PCR-confirmed, symptomatic COVID-19 by day 14	RRk: 0.68 (0.34, 1.34)	95.1% HCQ 97.5% usual care
Barnabas et al. (26)	December 2020	353 HCQ 336 ascorbic acid	400 mg daily for 3 days, then 200 mg daily for 11 days	PRC-confirmed SARS-CoV-2 infection, regardless of symptoms	HR: 1.10 (0.73, 1.66)	88.0% HCQf,,l 87% ascorbic acidf,l
Liu et al** (18)	May 2021	32 HCQ 19 Observation	200 mg twice a day for 10 days	PCR-confirmed symptomatic COVID-19 by day 14	There were no incident outcomes	Not reported
McKinnon et al. (27)m	March 2022	199 HCQ arm 1 188 HCQ arm 2 191 Placebo	Arm 1: 400 mg weekly for 8 weeks Arm 2: 400 mg loading dose, then 200 mg daily	PCR- or IgM/IgG serology-confirmed symptomatic COVID-19	Not reported Calculated RR: 0.99 (0.09, 10.82)	Not reported

* Date of posting to MedRxiv

** Not included in meta-analysis

^a The result of the PCR was self-reported

^b Overall adherence was not provided. Week-specific adherence ranged from 73–93% in the placebo group, from 76–93% in the HCQ Arm 1 and from 69–93% in the HCQ Arm 2, but a non-adherent participant during the first week can be counted as adherent in subsequent weeks

^c 98.75% confidence interval

^d Percentage of patients reporting > 70% adherence to trial intervention

^e 39% of patients were lost to follow-up. Among those who were not, 36.4% reported missing at least one daily dose

^f Adherence was self-reported

^g Risk ratio calculated for positive PCR at 12 weeks. The three HCQ groups were aggregated. Patients without a PCR test (4 in the HCQ groups, 1 in the control group) were assumed to have a negative PCR

^h Labelled as “pre-exposure prophylaxis” study, but 25.7% of participants were in “close contact with a confirmed COVID-19 case without using personal protective equipment” in the 20 days before screening

ⁱTreatment was planned for 6 months, but the study was halted after a month because of futility

^j There were no cases in the HCQ group and one case in the placebo group

^k Among participants who were PCR-negative at baseline

^l Percentage of patients receiving any dose

^m The authors do not label the study as post-exposure prophylaxis, but report that “60% [of study participants] reported contact with a COVID-19 positive patient before study entry

HCQ: hydroxychloroquine; RR: risk ratio; RD: risk difference; HR: hazard ratio

The seven pre-exposure prophylaxis trials were double-blinded, placebo-controlled trials that included healthcare workers with ongoing exposure to patients with COVID-19 [19–25]. The occurrence of COVID-19 was ascertained during a period between 4 and 12 weeks. Figure 1 shows the risk ratio estimates from the seven pre-exposure prophylaxis trials. The pooled risk ratio of COVID-19 for hydroxychloroquine vs. no hydroxychloroquine was 0.72 (95% CI: 0.58, 0.90) when using either a fixed effect or a standard random effects approach, and 0.72 (95% CI: 0.55, 0.95) when using the ad hoc modification of the Hartung-Knapp approach. The corresponding pooled risk difference was − 2.1 cases per 100 individuals (95% CI -3.7, -0.6) when using a fixed effect or a standard random effects approach and − 2.1 cases per 100 individuals (95% CI -4.0, -0.2) when using the ad hoc modification of the Hartung-Knapp approach (Supplementary Fig. 1). The pooled risk ratio estimate ranged from 0.71 to 0.74 under the alternative outcome definition and in a sensitivity analysis that used only studies published in peer-reviewed journals [19–22, 25], but the 95% CIs were wider (Supplementary Figs. 2–3). Supplementary Table 2 contains the prediction intervals.

Fig. 1.

Risk ratio estimates of COVID-19 for hydroxychloroquine vs. no hydroxychloroquine in randomized trials of pre-exposure prophylaxis, pooled and by study. These estimates are based on the definition of COVID-19 reported in the primary analysis of each study

Four post-exposure prophylaxis trials included asymptomatic contacts of confirmed COVID-19 cases. The time from exposure to initiation of prophylaxis was relatively long: in one trial, about a third of participants were enrolled 4 days after exposure (none was enrolled later) [4]; in the other, about 10% of participants were enrolled 7 or more days after exposure [5]; in the third study, 8% of participants received a first dose 5 days or later after exposure [26]. For comparison, post-exposure prophylaxis for HIV is recommended in the first 6–72 h after the exposure [28, 29]. The occurrence of COVID-19 was ascertained during a period of 2 weeks. A fourth post-exposure prophylaxis trial was not labelled as such by the authors, but 60% of the participants reported contact with a COVID-19 positive patients before study entry (time from contact to study entry not reported) [27]. The pooled risk ratio of COVID-19 for assignment to hydroxychloroquine vs. no hydroxychloroquine was 0.91 (95% CI: 0.72, 1.16) when using either a fixed effect or a standard random effects approach, and 0.91 (95% CI: 0.62, 1.35) when using the ad hoc modification of the Hartung-Knapp approach (Fig. 2). Supplementary Table 2 contains the prediction intervals.

Fig. 2.

Risk ratio estimates of COVID-19 for hydroxychloroquine vs. no hydroxychloroquine in randomized trials of post-exposure prophylaxis, pooled and by study. These estimates are based on the definition of COVID-19 reported in the primary analysis of each study

All trials found a similarly low rate of serious adverse effects in the HCQ and no HCQ groups. As expected, the proportion of mild gastrointestinal side effects was greater in the HCQ group. Adherence to treatment was heterogeneous across trials (Table 1). A funnel plot of the included studies is presented in Supplementary Fig. 4.

Discussion

When considered together, the available pre-exposure prophylaxis randomized trials yield a point estimate of an approximately 28% lower risk of COVID-19 for assignment to HCQ compared with no HCQ among PCR-negative individuals at randomization. Any effect between approximately 45% and a 5% reduction in risk is highly compatible with the data from these trials. The pooled effect estimate for the post-exposure randomized trial was closer to the null and both substantial reduction and moderate increase in risk were highly compatible with the data from the trials.

The choice of between common effect and random effects approaches for meta-analysis had little impact on the point estimates because the statistical heterogeneity across studies, as measured by the I², was close to zero. In the pre-exposure prophylaxis trials, the period over which the outcome was ascertained varied by a maximum of only 4 weeks and, given the long half-life of HCQ [30], variations in the order of weeks are not expected to substantially alter the effect estimates.

Our pooled estimates are based on the design and analytic choices made by the investigators of each trial. Ideally, these investigators could coordinate analyses of individual-level data with standardized outcome definitions; corrections for differences in length of follow-up and treatment dose; adjustment for losses to follow-up and other deviations from protocol [31]; and adoption of a more causally-interpretable meta-analytic approach [32]. All but two pre-exposure prophylaxis trials used laboratory-confirmed infection as the primary outcome; the remaining trials [19, 23], and one of the four post-exposure prophylaxis trials [4], did not require laboratory confirmation and included some self-reported test results. Excluding them resulted in a similar point estimate (Supplement Fig. 2).

As the results of pre- and post-exposure prophylaxis trials appeared between June and September of 2020, the pooled point estimate hovered around a 20% reduction with increasingly narrower compatibility intervals (Fig. 3). Yet throughout this entire period the opinion of many medical researchers was that HCQ was ineffective for prevention and that additional trials were unnecessary. The point estimate moved to around 10% reduction in December 2020 after the publication of a post-exposure prophylaxis cluster-randomized study [26] with unusual findings (no outcomes were recorded among individuals with unavailable PCR at baseline, different proportion of individuals with negative PCR at baseline between randomized groups) that may explain the investigators’ decision to deviate from protocol to adjust for baseline variables [33]. The final pooled risk ratio estimate, including all pre- and post-exposure trials, was 0.80 (95% CI 0.68, 0.95) when using the standard random effects approach and 0.80 (95% CI 0.67, 0.97) when using the ad hoc modification of the Hartung-Knapp approach.

Fig. 3.

Cumulative estimate of the pooled risk ratio of COVID-19 for hydroxychloroquine vs. no hydroxychloroquine in randomized trials of pre- and post-exposure prophylaxis. Confidence intervals are computed using the standard random effects method

The pooled estimates in Fig. 3 combine pre- and post-exposure prophylaxis trials, under the assumption of a similar mechanism of action for both preventive approaches [30]. However, the long time from exposure to treatment initiation in the post-exposure trials (several days as opposed to the 48 h recommended for other viruses like HIV) weakens the case for pooling studies of pre- and post-exposure prophylaxis because the latter can resemble studies of early treatment. Therefore, as the number of completed randomized trials increased in 2021 and 2022, meta-analyses like ours need to present separate estimates for pre-exposure and post-exposure prophylaxis. Such distinction is not always easy: for example, one of the pre-exposure prophylaxis trials identified and isolated 346 PCR positive cases at baseline in the premises where the study was run [21].

This systematic review offers an important lesson for future research on drug repurposing: Recruitment for most trials of HCQ prophylaxis was severely impeded by incorrect interpretations of the evidence from the early, mostly post-exposure prophylaxis, trials. The findings from the first reported trials were widely (and incorrectly) portrayed as definite evidence of the lack of effectiveness of HCQ, simply because they were not “statistically significant” when taken individually. Thus, the common confusion between the concepts “no effect” and “no statistical significance” led many to prematurely conclude that HCQ had no prophylactic effect when the correct conclusion was that the effect estimate was too imprecise [34, 35].

Concerns about potentially deleterious effects of treatment with HCQ [36] and backslash against unsubstantiated claims in the lay media [37] may have also affected the timely completion of the prophylaxis trials. In one study [24], conversations with investigators in participating hospitals confirmed that recruitment was difficult because of toxicity concerns. Of four eligible trials labelled as “completed” or “terminated” in clinicaltrials.gov but without posted results (NCT04414241, NCT04374942, NCT04372017 and NCT04344379), at least one had to be terminated because of poor recruitment, and another was closed after recruitment of one patient because researchers considered that the risk/benefit balance of HCQ was unfavorable (personal communications from the investigators). Yet these safety concerns were unwarranted as demonstrated by the findings of the randomized trials included in our meta-analysis. We encourage the authors of living systematic reviews [38] to update their findings frequently when, as in this case, public opinion interferes with the generation of the very evidence that will feed their systematic review.

In summary, throughout much of the pre-vaccine period of the pandemic, the available evidence was compatible with HCQ being viable as prophylaxis. Yet an incorrect interpretation of early inconclusive studies interfered with recruitment into additional trials. Though the availability of effective COVID-19 vaccines reduces the need for pharmacological prophylaxis, it is important to improve the process by which the medical community generates and interprets evidence before the next public health emergency arrives. To avoid he proliferation of small studies with different methodologies, national regulators and international health organizations can play a key role in the coordination and harmonization of the design of the randomized trials that they approve or endorse.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Author contribution

XGA and MAH analysed the data and drafted the manuscript. All authors made substantial contributions to the design of the study, interpreted the data, revised the manuscript critically, approved the version to be published and agree to be accountable for all aspects of the work. Code is available upon request from Xabier Garcia-Albeniz (xgarcia@rti.org).

Funding

No funding was received to support the development of this manuscript.

Declarations

Competing Interests

The authors have no relevant financial or non-financial interests to disclose.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10654-022-00891-4.