Adverse effects of remdesivir, hydroxychloroquine and lopinavir/ritonavir when used for COVID-19: systematic review and meta-analysis of randomised trials

Abstract

Background

To summarise specific adverse effects of remdesivir, hydroxychloroquine and lopinavir/ritonavir in patients with COVID-19.

Methods

We searched 32 databases through 27 October 2020. We included randomised trials comparing any of the drugs of interest to placebo or standard care, or against each other. We conducted fixed-effects pairwise meta-analysis and assessed the certainty of evidence using the grading of recommendations assessment, development and evaluation approach.

Results

We included 16 randomised trials which enrolled 8152 patients. For most interventions and outcomes the certainty of the evidence was very low to low except for gastrointestinal adverse effects from hydroxychloroquine, which was moderate certainty. Compared with standard care or placebo, low certainty evidence suggests that remdesivir may not have an important effect on acute kidney injury (risk difference (RD) 8 fewer per 1000, 95% CI 27 fewer to 21 more) or cognitive dysfunction/delirium (RD 3 more per 1000, 95% CI 12 fewer to 19 more). Low certainty evidence suggests that hydroxychloroquine may increase the risk of cardiac toxicity (RD 10 more per 1000, 95% CI 0 more to 30 more) and cognitive dysfunction/delirium (RD 33 more per 1000, 95% CI 18 fewer to 84 more), whereas moderate certainty evidence suggests hydroxychloroquine probably increases the risk of diarrhoea (RD 106 more per 1000, 95% CI 48 more to 175 more) and nausea and/or vomiting (RD 62 more per 1000, 95% CI 23 more to 110 more) compared with standard care or placebo. Low certainty evidence suggests lopinavir/ritonavir may increase the risk of diarrhoea (RD 168 more per 1000, 95% CI 58 more to 330 more) and nausea and/or vomiting (RD 160 more per 1000, 95% CI 100 more to 210 more) compared with standard care or placebo.

Discussion

Hydroxychloroquine probably increases the risk of diarrhoea and nausea and/or vomiting and may increase the risk of cardiac toxicity and cognitive dysfunction/delirium. Lopinavir/ritonavir may increase the risk of diarrhoea and nausea and/or vomiting. Remdesivir may have no important effect on risk of acute kidney injury or cognitive dysfunction/delirium. These findings provide important information to support the development of evidence-based management strategies for patients with COVID-19.

Strengths and limitations of this study.

• The search strategy was comprehensive with explicit eligibility criteria, and no restrictions on language or publication status.

• The review team was composed of clinical and methods experts who have undergone training and calibration exercises for all stages of the review process.

• We assessed the certainty of the evidence using the grading of recommendations assessment, development and evaluation approach and interpreted the results considering absolute, rather than relative, effects.

• We evaluated only a limited number of adverse effects and interventions.

• So far there is limited evidence for the harms associated with most drugs as adverse effects were only reported by a limited number of studies.

Introduction

As of 16 November 2020, there are 54.6 million cumulative cases of COVID-19 worldwide, and at least 1.3 million deaths.1 Several drugs have been used for the treatment of patients with COVID-19, often without high-quality evidence demonstrating efficacy. Three drugs that have been used for COVID-19 include remdesivir, hydroxychloroquine with or without azithromycin, and lopinavir/ritonavir. None of these drugs have high certainty evidence evaluating their effectiveness for key patient-important outcomes such as mortality, need for mechanical ventilation, duration of hospital stay or time to clinical improvement.2

We are conducting a living systematic review and network meta-analysis to provide a summary of the evidence for all drugs used in the treatment of COVID-19.2 Until now, we have not found that any one of these drugs increases the risk of adverse effects leading to drug continuation when compared with standard care or another drug treatment. However, we have not evaluated drug-specific adverse effects, which patients might consider to be important when making decisions about whether to use or not use a drug, particularly in the face of considerable uncertainty regarding their desirable effects.

Building on the work of the living systematic review, the aim of this paper is to summarise the best available evidence addressing drug-specific adverse effects in COVID-19. This evidence synthesis is part of the BMJ-Rapid Recommendations project,3 to inform WHO Living Guidelines on drugs for treatment of COVID-19.4 5

Methods

We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines for reporting.6

Eligibility criteria

As selected by the linked guideline panel we included randomised clinical trials (RCTs) that included people with suspected, probable, or confirmed COVID-19 comparing remdesivir, hydroxychloroquine and lopinavir/ritonavir, alone or in combination with other drugs, for treatment against one another or against no intervention, placebo, or standard care, and reported on drug-specific adverse effects of interest (see outcome identification below). We included trials regardless of publication status (peer reviewed, in press or preprint) or language. No restrictions were applied based on severity of COVID-19 illness, setting in which the trial was conducted (outpatient, hospital, ICU, etc), dose administered or length of treatment. We excluded studies in which remdesivir, hydroxychloroquine and lopinavir/ritonavir were used for prophylaxis and studies in which different doses of the same intervention were compared.

Information sources

We performed daily searches from Monday to Friday using the WHO COVID-19 database for eligible studies, which is a comprehensive multilingual source of global literature on COVID-19.7 Prior to its merge with the WHO COVID-19 database on 9 October 2020, we also performed daily searches for eligible studies from Monday to Friday in the US Centers for Disease Control and Prevention (CDC) COVID-19 Research Articles Downloadable Database.8 To identify RCTs, we filtered the results from the CDC’s database through a validated and highly sensitive machine learning model.9 In addition, we searched six Chinese databases. We adapted the search terms for COVID-19 developed by the CDC to the Chinese language. For the Chinese literature search, we also included search terms for randomised trials.

We also used living evidence retrieval services to identify any trials that might have been missed with traditional search methods. These included the Living Overview of the Evidence COVID-19 Repository by the Epistemonikos Foundation10 and the Systematic and Living Map on COVID-19 Evidence by the Norwegian Institute of Public Health, in collaboration with the Cochrane Canada Centre at McMaster University.11 We searched all English information sources from 1 December 2019 to 27 October 2020, and the Chinese literature from inception of the databases to 16 October 2020. A complete list of information sources and search strategies is available in online supplemental text 1.

Study selection

Using systematic review software, Covidence,12 following training and calibration exercises, pairs of reviewers independently screened all titles and abstracts, followed by full texts of trials that were identified as potentially eligible. A third reviewer adjudicated conflicts.

Data collection

For each eligible trial, pairs of reviewers extracted data independently using a standardised, pilot-tested data extraction form. Reviewers collected information on trial characteristics (trial registration, publication status, study status, design), participant characteristics (country, age, sex, smoking habits, comorbidities) and outcomes of interest. Reviewers resolved discrepancies by discussion and, when necessary, with adjudication by a third party.

Outcome identification

A linked WHO-BMJ Rapid Recommendations guideline panel4 13 14 consisting of patients, clinicians and research methodologists with representation from all WHO geographic regions provided input on potentially important adverse effects of the medications. If any of the panellists believed a specific adverse effect was possible and might influence the decision to use or not use each drug, it was included in this systematic review as an outcome of interest. Panellists were asked to focus on adverse effects important to patients, rather than surrogate measures. For example, we considered clinically important cardiac toxicity including arrhythmias important, but did not consider changes to the QT interval important. A detailed description of outcome ratings is included in the linked guideline.14 At the beginning of the guideline development process, the panel identified adverse effects that were common to most drugs and thus relevant for decision making. In addition, when deciding to focus on some specific interventions, the panel requested evidence regarding adverse effects that were specific to such interventions (eg, acute kidney injury when addressing remdesivir).

The panel identified specific adverse effects for each drug. For remdesivir, we included acute kidney injury. For hydroxychloroquine and hydroxychloroquine with azithromycin, we included cardiac toxicity, diarrhoea and nausea and/or vomiting. For lopinavir/ritonavir, we included acute kidney injury, diarrhoea, and nausea and/or vomiting. For all of the drugs, we included cognitive dysfunction/delirium and fatigue. We included studies in which researchers used any definitions of these outcomes. In cases in which the definitions did not appropriately reflect what is important to patients, we rated down the certainty of the evidence for indirectness (see certainty of the evidence below). For acute kidney injury definition, we used change in serum creatinine as reported by all included studies. However, the panel judged change in serum creatinine as not relevant to patients and a surrogate of severe kidney injury (ie, need for renal replacement therapy) which is relevant to patients.

Risk of bias within individual studies

For each eligible trial and outcome, following training and calibration exercises, reviewers used a revision of the Cochrane tool for assessing risk of bias in RCTs (RoB 2.0)15 to rate trials as either at (1) low risk of bias, (2) some concerns—probably low risk of bias, (3) some concerns—probably high risk of bias or (4) high risk of bias, across the following domains: bias arising from the randomisation process; bias due to departures from the intended intervention; bias due to missing outcome data; bias in measurement of the outcome; bias in selection of the reported results, including deviations from the registered protocol; and bias arising from early termination for benefit. We rated trials at high risk of bias overall if one or more domains were rated as ‘some concerns—probably high risk of bias’ or as ‘igh risk of bias’, and as low risk of bias overall if all domains were rated as ‘some concerns—probably low risk of bias’ or ‘low risk of bias’. Reviewers resolved discrepancies by discussion and, when not possible, with adjudication by a third party.

Data synthesis

Measures of effect and statistical analysis

We summarised the effect of interventions on selected outcomes using ORs and corresponding 95% CIs. We conducted frequentist fixed-effects pairwise meta-analyses using the R package ‘meta’ in RStudio V.1.3.1093,16 using the Mantel-Haenszel method with a continuity of 0.5 for studies in which there were 0 events in one arm of the trial. We used fixed rather than random effects for the primary analysis because for many of the interventions, the evidence consisted of two or fewer trials. For outcomes in which there were more than one trial with no events in both groups, we meta-analysed the data using risk differences (RD) to avoid continuity correction. For these outcomes, we report the pooled estimate of effect obtained using the RD. Pooled ORs can be found in online supplemental figure 11–22.

Certainty of the evidence

We assessed the certainty of evidence using the grading of recommendations assessment, development and evaluation (GRADE) approach.17 Two methodologists with experience in using GRADE rated each domain for each comparison separately and resolved discrepancies by consensus. We rated the certainty for each comparison and outcome as high, moderate, low, or very low, based on considerations of risk of bias, inconsistency, indirectness, publication bias and imprecision. We made judgements of imprecision using a minimally contextualised approach with the null effect as a threshold. This minimally contextualised approach considers whether the CI includes the null effect, or, when the point estimate is close to the null effect, whether the CI lies within the boundaries of small but important benefit and harm.18 To define severe or very severe imprecision we considered if the CI included not only the null effect, but important benefits and harms. Additionally we analysed if the total number of patients included in the meta-analysis was less than the required number of patients generated by a conventional sample size calculation for a single adequately powered trial to define if optimal information size (OIS) was met. For some of the interventions, extensively implemented in other clinical scenarios, we used indirect evidence to complement the certainty of evidence judgements. We created GRADE evidence summaries (Summary of Findings tables) using the MAGIC Authoring and Publication Platform (www.magicapp.org) to provide user friendly formats for clinicians and patients and to allow re-use in the context of clinical practice guidelines for COVID-19.4 5 We calculated the absolute risks and RD from the ORs (and their CIs) and the mean risk in the control groups across all of the included trials. In cases where no events were reported in the control arm of any of the included studies, we used baseline risks calculated for other comparisons on the same outcome.

To communicate our findings and conclusions using statements we followed published guidance.19

Subgroup and sensitivity analyses

We performed Bayesian random-effects meta-analysis using the bayesmeta package.20 We used a plausible prior for the variance parameter and a uniform prior for the effect parameter, as suggested in an empirical study using prespecified empiric priors as a sensitivity analysis for all comparisons.21 We also conducted frequentist fixed-effects pairwise meta-analyses using the R package ‘meta’ in RStudio V.1.3.1093,16 using the Peto’s method. We did not conduct any subgroup analyses.

Patient and public involvement

No patient involved.

Results

Study identification

After screening 14 806 titles and abstracts and 300 full texts, we included 16 unique RCTs with 8152 patients that informed on drug-specific adverse effects (figure 1).22–37 We did not identify any additional eligible RCTs through the living evidence retrieval services. Two studies reported adverse effects for remdesivir,22 36 10 for hydroxychloroquine,24–30 32–35 1 for hydroxychloroquine plus azithromycin24 and 4 for lopinavir/ritonavir.23 30 31 37 Of the 16 eligible RCTs, 13 have been published in peer reviewed journals, and 3 only as preprints.25 27 29 All of the trials were registered, published in English and most evaluated treatment in patients admitted to hospital with COVID-19 (15/16; 93.7%). Most of the trials were conducted in China (10/16; 62.5%). Table 1 presents the characteristics of the included studies. Additional study characteristics, outcome data and risk of bias assessments for each study are available in online supplemental file.

Figure 1.

Study selection.

Table 1.

Characteristics of the included trials

Study	Publication status, registration no	No of participants	Country	Mean age (years)	Men (%)	Type of care, comorbidities	Severity (according to study authors)	Mechanical ventilation at baseline (%)	Treatments (dose and duration)	Outcomes
Beigel 2020; ACTT-122	Published, NCT04280705	1062	USA, Denmark, UK, Greece, Germany, Korea, Mexico, Spain, Japan, Singapore	58.9	64.4	Inpatient; coronary artery disease (11.9%); congestive heart failure (5.6%); diabetes (30.6%); hypertension (50.7%); asthma (11.4%); chronic oxygen requirement (2.2%); chronic respiratory disease (7.6%)	Severe (90.1%)	45.0	Remdesivir (200 mg/day for 1 day, then 100 mg/day for 9 days); placebo	Acute kidney injury; cognitive dysfunction/delirium
Cao 2020; LOTUS China23	Published, ChiCTR2000029308	199	China	58.0	60.3	Inpatient; cerebrovascular disease (6.5%); diabetes (11.6%)	Severe (100%)	16.1	Lopinavir-ritonavir (400 mg and 100 mg two times daily for 14 days); standard care	Acute kidney injury; diarrhoea; nausea and/or vomiting; fatigue
Cavalcanti, 202024	Published, NCT04322123	667	Brazil	50.3	58.4	Inpatient; intensive care (13.8%); heart failure (1.5%); diabetes (19.1%); hypertension (38.3%); asthma (6.0%); chronic obstructive pulmonary disease (1.8%)	Mild/Moderate (100%)	0	Hydroxychloroquine (400 mg two times daily for 7 days); hydroxychloroquine (400 mg twice daily for 7 days) and azithromycin (500 mg/day for 7 days); standard care	Cardiac toxicity; nausea and/or vomiting
Chen 202025	Preprint, ChiCTR2000029559	62	China	44.7	46.8	Inpatient; NR	Mild/moderate (100%)	NR	Hydroxychloroquine (200 mg two times daily for 5 days); standard care	Cardiac toxicity
Chen 202026	Published, NCT04261517	30	China	48.6	70.0	Inpatient; diabetes (6.7%); hypertension (26.7%); chronic obstructive pulmonary disease (3.3%)	Mild/moderate (100%)	NR	Hydroxychloroquine (400 mg/day for 5 days); standard care	Diarrhoea; nausea /vomiting
Chen 202027	Preprint, ChiCTR2000030054	48	China	46.9	45.8	Inpatient; diabetes (18.8%); hypertension (16.7%)	Mild/moderate (100%)	NR	Chloroquine (1000 /day for 1 day, then 500 mg/day for 9 days); hydroxychloroquine (200 mg two times daily for 10 days); standard care	Cardiac toxicity; diarrhoea; nausea and/or vomiting
Chen 202028	Preprint, NCT04384380	33	Taiwan	32.9	57.6	Inpatient	Mild/Moderate (100%)	0	Hydroxychloroquine (400 mg two times daily for 1 day, then 200 mg two times daily for 6 days); standard care	Diarrhoea; nausea and/or vomiting
Horby 2020; RECOVERY29	Published, NCT04381936	4716	UK	65.3	62.2	Inpatient; heart disease (25.7%); diabetes (27.2%); chronic lung disease (22.2%); tuberculosis (0.3%)	NR	16.8	Hydroxychloroquine (800 mg at zero and 6 hours, then 400 mg two times daily for 9 days); standard care	Cardiac toxicity
Huang 202030	Published, ChiCTR2000029387	101	China	42.5	45.5	Inpatient	Mild/moderate (100%)	NR	Ribavirin (400–600 mg three times daily for 14 days) and interferon-alfa (5 mg two times daily for 14 days); lopinavir-ritonavir (400 mg and 100 mg two times daily for 14 days) and interferon-alfa (5 mg two times daily for 14 days); ribavirin (400–600 mg three times daily for 14 days) and lopinavir-ritonavir (400 mg and 100 mg two times daily for 14 days) and interferon-alfa (5 mg two times daily for 14 days)	Acute Kidney injury; diarrhoea; nausea and/or vomiting
Li 2020; ELACOI31	Published, NCT04252885	86	China	49.4	46.5	Inpatient; cardiovascular disease (2.3%); diabetes (2.3%); hypertension (10.5%)	Mild/moderate (100%)	0	Lopinavir-ritonavir (400 mg and 100 mg two times daily for 7 to 14 days); umifenovir (200 mg three times daily for 7–14 days); standard care	Diarrhoea; nausea and/or vomiting
Lyngbakken 202032	Published, NCT04316377	53	Norway	62.0	66.0	Inpatient; coronary heart disease (9.4%); diabetes (17.0%); hypertension (32.1%); chronic obstructive pulmonary disease or asthma (26.4%)	Mild/moderate (0%)	0	Hydroxychloroquine (400 mg two times daily for 7 days); standard care	Diarrhoea; nausea and/or vomiting
Skipper 202033	Published, NCT04308668	491	USA, Canada	40.0	45.8	Outpatient; cardiovascular disease (1.2%); diabetes (3.9%); hypertension (11.0%); asthma (10.4%); chronic lung disease (0.4%)	Mild/moderate (100%)	0	Hydroxychloroquine (800 mg at zero hours, then 600 mg 6–8 hours later, then 600 mg/day for 4 days); placebo	Cardiac toxicity; diarrhoea; nausea /vomiting; cognitive dysfunction/delirium
Tang 202034	Published, ChiCTR2000029868	150	China	46.1	55.0	Inpatient; diabetes (14.0%); hypertension (6.0%)	Mild/moderate (99.0%); severe (1.0%)	NR	Hydroxychloroquine (1200 mg/day for 3 days, then 800 mg/day until 14 to 21 days of total treatment); standard care	Cardiac toxicity; diarrhoea; nausea /vomiting; cognitive
Ulrich 2020; TEACH35	Published, NCT04369742	128	USA	66.2	59.4	Inpatient; non-hypertensive cardiovascular disease (25.6%); diabetes (32.0%); hypertension (57.8%); asthma (15.6%); chronic obstructive pulmonary disease (7.0%)	Mild/moderate (0%)	0.78	Hydroxychloroquine (400 mg two times daily for 1 day, then 200 mg two times daily for 4 days); placebo	Cardiac toxicity
Wang 202036	Published, NCT04257656	237	China	65.0	59.3	Inpatient; cardiovascular disease (7.2%); diabetes (23.7%); hypertension (43.2%)	Severe (100%)	16.1	Remdesivir (200 mg/day for 1 day, then 100 mg/day for 9 days); placebo	Acute kidney injury
Zheng 202037	Published, ChiCTR2000029496	89	China	46.7	47.2	Inpatient; chronic bronchitis (2.0%)	Mild/moderate (94.4%); severe (5.6%)	NR	Novaferon (20 µg two times daily for 7–10 days); novaferon and lopinavir-ritonavir (400 mg and 100 mg two times daily for 7–10 days); lopinavir-ritonavir (400 mg and 100 mg twice daily for 7–10 days)	Diarrhoea; nausea and/or vomiting; fatigue

NR, not reported.

Risk of bias in included studies

Online supplemental figure 1 presents the risk of bias assessment of the 16 included studies for each outcome. Overall and domain specific risk of bias judgements did not differ between the outcomes reported in each individual study, and most of the studies (13/16, 81.2%) presented important methodological limitations.

Adverse effects of the interventions

Remdesivir

Two studies22 36 including 1281 patients reported on remdesivir specific adverse effects. Both studies reported on acute kidney injury and one study22 including 1048 patients reported on cognitive dysfunction/delirium. No studies reported on fatigue (table 2).

Table 2.

Summary of findings table

Outcome time frame	Study results and measurements	Absolute effect estimates		Certainty of the evidence (quality of evidence)	Plain text summary
		Standard care	Intervention
Remdesivir
Acute kidney injury	OR: 0.85 (95% CI 0.51 to 1.41) Based on data from 1281 patients in two studies	56 per 1000	48 per 1000	Low Due to serious imprecision and serious indirectness*	Remdesivir may have little or no effect on acute kidney injury.
		Difference: 8 fewer per 1000 (95% CI 27 fewer to 21 more)
Cognitive dysfunction/delirium	OR: 1.22 (95% CI 0.48 to 3.11) Based on data from 1048 patients in one study	16 per 1000	19 per 1000	Low Due to serious imprecision and serious indirectness†	Remdesivir may have little or no effect on cognitive dysfunction/delirium.
		Difference: 3 more per 1000 (95% CI eight fewer to 32 more)
Fatigue	NR	NR		NA	NA
		NR
Hydroxychloroquine
Cardiac toxicity	Based on data from 3287 patients in seven studies	46 per 1000	56 per 1000	Low Due to serious imprecision and risk of bias‡	Hydroxychloroquine may increase the risk of cardiac toxicity, including serious arrhythmias.
		Difference: 10 more per 1000 (95% CI 0 more to 30 more)
Diarrhoea	OR: 1.95 (95% CI 1.40 to 2.73) Based on data from 979 patients in six studies	149 per 1000	255 per 1000	Moderate Due to serious imprecision§	Hydroxychloroquine probably increases the risk of diarrhoea.
		Difference: 106 more per 1000 (95% CI 48 more to 175 more)
Nausea and/or vomiting	OR: 1.74 (95% CI 1.26 to 2.41) Based on data from 1429 patients in seven studies	99 per 1000	161 per 1000	Moderate Due to serious imprecision§	Hydroxychloroquine probably increases the risk of nausea and vomiting.
		Difference: 62 more per 1000 (95% CI 23 more to 110 more)
Cognitive dysfunction/delirium	OR: 1.59 (95% CI 0.77 to 3.28) Based on data from 423 patients in one study	62 per 1000	95 per 1000	Low Due to serious imprecision and serious indirectness†	Hydroxychloroquine may increase cognitive dysfunction/delirium
		Difference: 33 more per 1000 (95% CI 18 fewer to 84 more)
Fatigue	OR: 2.75 (95% CI 0.28 to 27.28) Based on data from 180 patients in two studies	54 per 1000¶	136 per 1000	Very Low Due to very serious imprecision and serious risk of bias**	The effect of Hydroxychloroquine on fatigue is uncertain
		Difference: 82 more per 1000 (95% CI 38 fewer to 555 more)
Hydroxychloroquine with azithromycin
Cardiac toxicity	Based on data from 667 patients in one study	6 per 1000**	16 per 1000	Very Low Due to very serious imprecision and serious risk of bias**	The effect of Hydroxychloroquine with azithromycin on cardiac toxicity is uncertain
		Difference: 10 more per 1000 (95% CI 10 fewer to 20 more)
Nausea and/or vomiting	OR: 1.49 (95% CI 0.37 to 6.06) Based on data from 667 patients in one study	17 per 1000	25 per 1000	Very Low Due to very serious imprecision and serious risk of bias**	The effect of Hydroxychloroquine with azithromycin on nausea and/or vomiting is uncertain
		Difference: 8 more per 1000 (95% CI 11 fewer to 78 more)
Diarrhoea	NR	NR		NA	NA
		NR
Cognitive dysfunction/delirium	NR	NR		NA	NA
		NR
Fatigue	NR	NR		NA	NA
		NR
Lopinavir/ritonavir
Acute kidney injury	Based on data from 259 patients in two studies	45 per 1000	25 per 1000	Very Low Due to very serious imprecision and serious risk of bias**	The effect of lopinavir/ritonavir on acute kidney injury is uncertain.
		Difference: 20 fewer per 1000 (95% CI 70 fewer to 20 more)
Diarrhoea	OR: 4.28 (95% CI 1.99 to 9.18) Based on data from 370 patients in four studies	67 per 1000	235 per 1000	Low Due to very serious imprecision††	Lopinavir/ritonavir may increase the risk of diarrhoea.
		Difference: 168 more per 1000 (95% CI 58 more to 330 more)
Nausea and/or vomiting	Based on data from 370 patients in four studies	17 per 1000	177 per 1000	Low Due to very serious imprecision††	Lopinavir/ritonavir may increase the risk of nausea and vomiting.
		Difference: 160 more per 1000 (95% CI 100 more to 210 more)
Fatigue	OR: 1.56 (95% CI 0.53 to 4.58) Based on data from 254 patients in two studies	54 per 1000	82 per 1000	Very Low Due to very serious imprecision and serious risk of bias**	The effect of lopinavir/ritonavir on fatigue is uncertain.
		Difference: 28 more per 1000 (95% CI 25 fewer to 154 more)
Cognitive dysfunction/delirium	NR	NR		NA	NA
		NR

*Risk of bias: not serious. Indirectness: serious as studies used change in serum creatinine rather than patient-important measures of acute kidney injury (ie, renal replacement therapy requirement). Imprecision: Serious. Using a threshold of 15 per 1000, CIs include important risk increase.

†Risk of bias: Not serious. Indirectness: Serious as this outcome was not collected systematically, and the definition of cognitive dysfunction/delirium was not specified. Imprecision: Serious. Using a threshold of 15 per 1000, confidence intervals include important risk increase.

‡Risk of bias: Data primarily from unblinded studies, but we would expect that patients would be more closely monitored for cardiac toxicity in trials than in usual clinical practice. Therefore, we expect the risk of cardiac toxicity to be higher in usual clinical practice. Indirectness: Not serious. Trials measured cardiac toxicity differently in different trials. Imprecision: Serious. CIs include no effect.

§Risk of bias: Serious. Most of the evidence is from unblinded trials, we did not downgrade for RoB as our concerns were mitigated by a large effect size and indirect evidence showing consistent results. Imprecision: OIS not met.

¶As there were no events in the control arms of included studies, we used the baseline risk estimated for Lopinavir/ritonavir versus SOC comparison for the same outcome.

**Risk of bias: Serious. Most of the evidence is from unblinded trials. Imprecision: Very serious. Very small number of events.

††Risk of bias: Serious. Most of the evidence is from unblinded trials; we did not downgrade for RoB as our concerns were mitigated by a large effect size and indirect evidence showing consistent results; Imprecision: Very serious. Very small number of events.

NA, not applicable; NR, not reported; OIS, optimal information size; RoB, risk of bias; SOC, standard of care.

Acute kidney injury

Remdesivir may have little or no effect on acute kidney injury when compared with placebo (OR 0.85, 95% CI 0.51 to 1.41; RD 8 fewer per 1000 participants, 95% CI 27 fewer to 21 more) (online supplemental figure 2). The certainty of the evidence was low because of serious imprecision and serious indirectness (studies used change in serum creatinine rather than patient-important measures of acute kidney injury like renal replacement therapy requirement).

Cognitive dysfunction/delirium

Remdesivir may have little or no effect on cognitive dysfunction/delirium when compared with placebo (OR 1.22, 95% CI 0.48 to 3.11; RD 3 more per 1000 participants, 95% CI 8 fewer to 32 more). The certainty of the evidence was low because of serious imprecision and serious indirectness (this outcome was not collected systematically, and the definition of cognitive dysfunction/delirium was not specified).

Hydroxychloroquine

Ten studies24–29 32–35 including 3663 patients reported on hydroxychloroquine specific adverse effects. Seven studies including 3287 patients reported cardiac toxicity,24 25 27 30 33–35 6 trials including 979 patients reported diarrhoea,26–28 32–34 7 studies including 1429 patients26–29 32–34 reported nausea and/or vomiting, 1 study33 including 423 patients reported on cognitive dysfunction/delirium and 2 studies27 34 including 180 patients reported on fatigue.

Cardiac toxicity

Definitions of cardiac toxicity varied between trials: RECOVERY defined the outcome as new major arrhythmias (supraventricular tachycardia, ventricular tachycardia or fibrillation or atrioventricular block requiring intervention),29 two studies as new arrhythmias,24 33 and one study as new arrhythmias or cardiac arrest.35 The remaining studies did not provide details about cardiac toxicity definition. Hydroxychloroquine may increase the risk of cardiac toxicity when compared with standard care or placebo (RD 10 more per 1000 participants, 95% CI 0 more to 30 more) (online supplemental figure 3). The certainty of the evidence was low because of serious imprecision and risk of bias (unblinded studies with possible detection bias).

Diarrhoea

Hydroxychloroquine probably increases the risk of diarrhoea when compared with standard care or placebo (OR 1.95, 95% CI 1.40 to 2.73; RD 106 more per 1000 participants, 95% CI 48 more to 175 more) (online supplemental figure 4). The certainty of the evidence was moderate because of imprecision as the OIS was not met. Although most studies presented methodological limitations, we did not rate down for risk of bias (RoB) as our concerns were mitigated by a large effect size and indirect evidence showing consistent results.38

Nausea and/or vomiting

Hydroxychloroquine probably increases nausea and vomiting (OR 1.74, 95% CI 1.26 to 2.41; RD 62 more per 1000 participants, 95% CI 23 more to 110 more) (online supplemental figure 5). The certainty of the evidence was moderate because of imprecision as OIS was not met. Although most studies presented methodological limitations, we did not rate down for RoB as our concerns were mitigated by a large effect size and indirect evidence showing consistent results.38

Cognitive dysfunction/delirium

Hydroxychloroquine may increase cognitive dysfunction/delirium when compared with standard care or placebo (OR 1.59, 95% CI 0.77 to 3.28; RD 33 more per 1000 participants, 95% CI 18 fewer to 84 more). The certainty of the evidence was low because of serious imprecision and serious indirectness (this outcome was not collected systematically, and the definition of cognitive dysfunction/delirium was not specified).

Fatigue

The effect of hydroxychloroquine on fatigue is uncertain when compared with standard care or placebo (OR 2.75, 95% CI 0.28 to 27.28; RD 82 more per 1000 participants, 95% CI 38 fewer to 555 more) (online supplemental figure 6). The certainty of the evidence was very low because of very serious imprecision and serious risk of bias.

Hydroxychloroquine with azithromycin

Only one study24 including 667 patients reported drug-specific adverse effects for hydroxychloroquine with azithromycin. The study compared hydroxychloroquine with azithromycin, hydroxychloroquine alone and standard care and reported on cardiac toxicity and nausea and/or vomiting. Other outcomes, including diarrhoea, cognitive dysfunction/delirium or fatigue were not reported.

Cardiac toxicity

The effect of hydroxychloroquine with azithromycin on cardiac toxicity is uncertain when compared with standard care or placebo (RD 10 more per 1000 participants, 95% CI 10 fewer to 20 more), or hydroxychloroquine alone (RD 0 more per 1000 participants, 95% CI 20 fewer to 20 more). The certainty of the evidence was very low because of very serious imprecision and serious risk of bias.

Nausea and/or vomiting

The effect of hydroxychloroquine with azithromycin on nausea and vomiting in uncertain when compared with standard care or placebo (OR 1.49, 95% CI 0.37 to 6.06; RD 8 more per 1000 participants, 95% CI 11 fewer to 78 more) or hydroxychloroquine alone (OR 0.54, 95% CI 0.18 to 1.57; RD 20 fewer per 1000 participants, 95% CI 37 fewer to 24 more). The certainty of the evidence was very low because of very serious imprecision and serious risk of bias.

Lopinavir/ritonavir

Four studies23 30 31 37 including 370 patients reported adverse effects of lopinavir/ritonavir. All four studies reported diarrhoea and nausea and/or vomiting. Two studies including 259 patients reported acute kidney injury23 30 and two studies including 254 patients reported fatigue.30 37 No studies reported on cognitive dysfunction/delirium.

Acute kidney injury

The effect of lopinavir/ritonavir on acute kidney injury is uncertain when compared with standard care or placebo (20 fewer per 1000 participants, 95% CI 70 fewer to 20 more) (online supplemental figure 7). The certainty of the evidence was very low because of very serious imprecision and serious risk of bias.

Diarrhoea

Lopinavir/ritonavir may increase the risk of diarrhoea when compared with standard care or placebo (OR 4.28, 95% CI 1.99 to 9.18; RD 168 more per 1000 participants, 95% CI 58 more to 330 more) (online supplemental figure 8). The certainty of the evidence was low because of very serious imprecision. Although most studies presented methodological limitations, we did not rate down for RoB as our concerns were mitigated by a large effect size and indirect evidence showing consistent results.39

Nausea and/or vomiting

Lopinavir/ritonavir may increase the risk of nausea and/or vomiting when compared with standard care or placebo (RD 160 more per 1000 participants, 95% CI 100 more to 210 more) (online supplemental figure 9). The certainty of the evidence was low because of very serious imprecision. Although most studies presented methodologic limitations, we did not rate down for RoB as our concerns were mitigated by a large effect size and indirect evidence showing consistent results.39

Fatigue

The effect of lopinavir/ritonavir on fatigue is uncertain when compared with standard care or placebo (OR 1.56, 95% CI 0.53 to 4.58; 28 more per 1000 participants, 95% CI 25 fewer to 154 more) (online supplemental figure 10). The certainty of the evidence was very low because of very serious imprecision and serious risk of bias.

Sensitivity analyses

Our interpretation of the results did not substantially change when using a Bayesian random effects model rather than frequentist fixed effects, when pooling relative estimates rather than absolute estimates or when using Peto’s method (online supplemental figure 11–31 and online supplemental table 1).

Discussion

This systematic review and meta-analysis—directly informing the living WHO guideline for COVID-19 therapeutics—provides a comprehensive overview of the evidence for drug-specific adverse effects of interest for three commonly used drugs for treatment of COVID-19. From 40 interventions included in our living network meta-analysis,2 we only included studies reporting on drug specific adverse effects for remdesivir, hydroxychloroquine, hydroxychloroquine with azithromycin and lopinavir/ritonavir in this review as these drugs received a high degree of interest, particularly in the early stages of the pandemic. None of these interventions may increase the risk of adverse effects leading to discontinuation, however, the certainty of the evidence was low for hydroxychloroquine and moderate for remdesivir, while no information was available for hydroxychloroquine with azithromycin, or lopinavir-ritonavir.2 In this review, we found moderate certainty evidence that hydroxychloroquine increases the risk of diarrhoea and nausea and/or vomiting and low certainty evidence that it increases the risk of cardiac toxicity and cognitive dysfunction/delirium. For lopinavir/ritonavir, we found low certainty evidence that it increases the risk of diarrhoea, and nausea and/or vomiting. Based on low or very low certainty evidence, we did not find evidence that remdesivir or lopinavir/ritonavir increase the risk of acute kidney injury or cognitive dysfunction/delirium.

Strengths and limitations of this review

The search strategy was comprehensive with explicit eligibility criteria, and no restrictions on language or publication status. To ensure expertise in all areas, the review team was composed of clinical and methods experts who have undergone training and calibration exercises for all stages of the review process. We assessed the certainty of the evidence using the GRADE approach and interpreted the results considering absolute, rather than relative, effects.

We evaluated only a limited number of adverse effects and interventions, as selected by the linked guideline panel. We included an adverse effect if any panel member believed it might be important to patients when deciding whether to use or not to use a drug. However, there may be other patient-important adverse drug effects that were not prespecified by the panel. Further, some may perceive that excluding surrogate outcomes, such as an increase in liver enzymes or ECG changes may lead to underappreciation of potential harms, especially for surrogates that are more closely linked on the causal pathway to patient important harms.

So far there is limited evidence for the harms associated with most drugs as adverse effects were only reported by a limited number of studies. For comparisons with sufficient data, the primary limitation of the evidence was lack of blinding, which might introduce bias through differences in cointerventions or outcome assessment between randomisation groups. In addition, as observed in other scenarios,38–40 adverse effects were seldom reported which also represents a potential source of bias (selective reporting). However, the large magnitude of effects observed resulted in moderate certainty that hydroxychloroquine causes specific adverse effects.

Some patients may be at higher or lower risk of adverse events. For example, patients with an underlying heart disease may be at higher risk of cardiac toxicity from hydroxychloroquine. However, we were unable to determine which patients may be more or less likely to experience drug-specific adverse effects.

These findings are consistent with ‘The Living Project’ (https://covid-nma.com/), which found an increase in any adverse effects with hydroxychloroquine (RR 2.16, 95% CI 1.21 to 3.86) and lopinavir/ritonavir (RR 2.39, 95% CI 0.21 to 27.57), but not with remdesivir (RR 1.00, 95% CI 0.87 to 1.15). However, they did not report on specific adverse effects. Other systematic reviews found an increase in the risk of diarrhoea and nausea and/or vomiting with lopinavir-ritonavir41 42 and hydroxychloroquine,42–44 increase in arrhythmias and QTc interval prolongation with hydroxychloroquine alone,44–46 or combined with a macrolide,47 48 and no important increase in renal failure with remdesivir.49

Conclusion

Hydroxychloroquine probably increases the risk of diarrhoea and nausea and/or vomiting and may increase the risk of cardiac toxicity and cognitive dysfunction/delirium. Lopinavir/ritonavir may increase the risk of diarrhoea and nausea and/or vomiting. Remdesivir may have no important effect on risk of acute kidney injury or cognitive dysfunction/delirium. These findings provide important information to support the development of evidence-based management strategies for patients with COVID-19.

Data availability statement

All data relevant to the study are included in the article or uploaded as online supplemental information. No additional data available.

Ethics statements

Patient consent for publication

Not applicable.