An observational cohort study of hydroxychloroquine and azithromycin for COVID-19: (Can’t Get No) Satisfaction

Introduction

It is clear that SARS-CoV-2 caught the world by surprise. In large part due to globalization, the virus quickly evolved from a serious regional concern to a worldwide pandemic, the likes of which are unprecedented in the last century. In a matter of weeks, COVID-19 became a leading cause of death in 2020, with a staggering potential death toll. Due to a heavy burden of illness, and in the absence of proven therapies, several experimental treatments have been, and continue to be, prescribed outside of clinical trial settings. Of the potential therapeutic options that showed early promise, few have generated as much controversy, or been subject to such politicization, as hydroxychloroquine. In this issue of the International Journal of Infectious Diseases, Arshad et al. (2020) have heightened the controversy.

The authors conducted a retrospective cohort study of 2541 consecutive patients admitted to their health system in Michigan, USA. Patients were separated into four groups: no treatment (n = 409), azithromycin alone (n = 147), hydroxychloroquine alone (n = 1202), and hydroxychloroquine plus azithromycin (n = 783). The primary outcome was mortality. Using a multivariable Cox regression model they adjusted for a number of demographics (e.g. age and sex), comorbidities (e.g. cardiovascular disease and BMI), as well as some markers of disease severity on admission (e.g. oxygen saturation). The mSOFA score was missing in 25% of participants, and so while it was present in the demographics, it was not included in the final model. The authors’ analysis suggested that hydroxychloroquine, with or without azithromycin, was associated with a reduced hazard ratio for death when compared with receipt of neither medication. Using a propensity score matched analysis, with the same variables, they reached a similar conclusion.

Strengths and weaknesses

The main strength of this retrospective cohort study was the inclusion of consecutive patients with laboratory-confirmed COVID-19 from a large healthcare system that included a representative population of socially vulnerable, ethnically diverse individuals. Statistical methods included efforts to adjust for possible confounders through multivariable Cox regression and via propensity score matching.

The limitations of this study are important to consider. First, the precision of the results was impacted by immortal time bias, because several time-dependent covariates were not modelled in this manner. Fortunately, since the average time to receipt of treatment was only 1 day, this bias may be small; nonetheless, it favored treatment and so should be taken into consideration.

Second, there was an important potential for residual confounding because of a number of prognostic factors (e.g. frailty, residence in long term care, or ‘do not resuscitate’ orders), potentially important markers of disease severity (e.g. ferritin, C-reactive protein (Zeng et al., 2020), troponins (Vrsalovic and Vrsalovic Presecki, 2020), and D-dimer (Zhang et al., 2020)), and co-administration of potentially beneficial therapies (e.g. anticoagulants (Paranjpe et al., 2020)) that were not included in the analysis.

Third, confounding by severity or indication (Kyriacou and Lewis, 2016) was likely. While there was a hospital treatment protocol in place, unmeasured clinical factors likely influenced the decision not to treat 16.1% of patients, in a center where 78% received treatment. These factors are often difficult to capture in an observational study. Were the decision to withhold treatment related to poor prognosis (e.g. palliative intent), it stands to reason that patients receiving neither hydroxychloroquine nor azithromycin would have the highest mortality. Indeed, the non-treated group had an overall mortality that was higher than the rate of admission to the ICU (26.4% vs 15.2%), suggesting that many patients were not considered appropriate for critical care. Such being the case, their care may have differed in other substantive ways that were also associated with death (e.g. terminal illness or advanced directives limiting invasive care). In the hydroxychloroquine treatment groups, the inverse was true, with mortality lower than the rate of admission to the ICU (16.1% vs 26.9%). While a propensity score analysis might further account for some differences between treatment groups, this approach was still limited to the information available in the dataset.

Fourth, the chronological time point over the course of the pandemic at which patients were managed was not included in the study. As the Henry Ford Health System became more experienced in treating patients with COVID-19, survival may have improved, regardless of the use of specific therapies. Hospital-specific guidelines regarding COVID-19 screening eligibility as well as the availability of COVID-19 testing may have also changed over time, introducing additional chronological bias.

Finally, concomitant steroid use in patients receiving hydroxychloroquine was more than double the non-treated group. This is relevant if we consider the recent RECOVERY trial, which showed a mortality benefit with dexamethasone (Horby et al., 2020) among individuals requiring supplemental oxygen or mechanical ventilation, potentially biasing this study’s results in favor of hydroxychloroquine.

Implications

In a global context, this study is thought-provoking, with results that contradict those from other large US cohorts (Geleris et al., 2020, Rosenberg et al., 2020). It remains fundamentally limited by its observational nature and is subject to residual confounding. The published results of the UK RECOVERY and WHO Solidarity trials are not available at the time of writing, but both studies are expected to conclude that hydroxychloroquine does not decrease mortality in hospitalized COVID-19 patients when compared with standard of care (RECOVERY, 2020, World Health Organization, 2020). Given the current polarized opinions surrounding hydroxychloroquine, there will likely be energetic discussion following their eventual publication.

Overall, the authors should be commended for rapidly compiling and analyzing data from a large cohort of COVID-19 patients. Clinicians worldwide should be acknowledged for their best efforts in caring for patients in uncertain times and in the absence of proven therapies. It is, however, very sobering to note that the number of patients in this single observational study would have made a substantive contribution to any randomized controlled trial. While all healthcare providers feel a clinical imperative to offer patients treatment, there was little evidence to justify a hydroxychloroquine protocol at the outset of the pandemic. It is a failing of healthcare systems and the research infrastructure that the protocolization of unproven therapies is exponentially easier to execute than participation in pragmatic randomized controlled trials. Moving forward, we encourage academic centers to commit to participating in the necessary clinical trials that will establish high-quality evidence for safe and effective therapies in the shortest possible time.

Contributions

All authors contributed equally to this editorial, and are listed in alphabetical order. The initial draft was written by TCL and then jointly revised by all co-authors.