We would like to thank all our colleagues for taking the opportunity to start a scientific discussion on the matter of hydroxychloroquine in COVID-19. We feel that it is important in the current climate, where politics and economics interfere with the scientific domain, to have a calm and scientific discussion on this topic. Below you will find our response to the letters.
Authors’ reply:
The first letter “Comparison of hospital treatment strategy or of treatment actually received in COVID-19?” asks us about our methodology. We understand the nature of the question quite well, since we have discussed this issue before deciding which analysis strategy to choose. Clearly, this is an observational study that might be prone to potential bias. To avoid this, we used a two-step procedure. The first step encompasses the inclusion of patients. The second step involves the comparison of treated and untreated patients after their hospitalization.
The inclusion of patients for our study was based on hospital treatment strategy. Patients were admitted to a HCQ, CQ or non-treatment hospital randomly; the hospital policy regarding the use of HCQ/CQ was not communicated to the patient beforehand, neither was it part of a selection procedure for the study. Thus, the fact that a patient was admitted to a HCQ, CQ or non-treatment hospital was a random event and based on a random assignment, based on geographical loction only. That is the reason why we believe that our study design and patient inclusion approaches the intention-to-treat strategy used in randomized-controlled trials.
The assignment to receive HCQ or CQ treatment occurs after hospitalization and is influenced by patient characteristics, such as co-morbidity. This means that the actual assignment of a treatment may lead to differences between treated patients and untreated patients. Thus, to make accurate inferences, it is crucial to control for confounding by indication and for the differences between treated and untreated patients. Since the assignment of treatment occurs after hospitalization, it is independent of the actual hospital treatment strategy. Therefore, we believe that it is justified in this second step to pool the data and compare all treated and untreated patients directly. Subsequently, we used the propensity score (PS) matching as a balancing score between treated and untreated patients, and to control for confounding by indication. PS matching is a proven tool to adjust a treatment effect for measured confounders in non-randomized studies, such as ours. At each value of the PS, the distribution of the considered covariates is the same in the treated and the untreated or control group. The balance that a randomized experiment is expected to create by design, is in this way established through statistical matching. In our study, after PS matching no significant statistical differences were found between the baseline characteristics of treated and untreated patients, including the differences between hospital strategies (as is shown in Table 1). Furthermore, the type of hospital treatment was not a confounder in the definite weighted regression model.
In conclusion, we believe that with our two-step procedure (hospital treatment strategy at inclusion and actual treatment received in the analysis), our study limits the influence of potential bias and enables us to make correct inferences based on the available data.
The second letter, by Asselbergs et al raises several questions on the methodology as well.
The authors stated: “in the presence of ICU restriction for medical reasons or patient preference, analyses on the outcome ‘transfer to ICU’ are limited to patients without ICU restriction, a particularly selected group. This might impact the generalizability of the results considerably”.
Since patients with an ICU restriction cannot reach the endpoint “Transfer to ICU”, we started by computing the outcome as Composite Adverse Endpoint; taking the two endpoints (Mortality and Transfer to ICU) together. This eliminates the bias of competing risk between the two endpoints. We did not consider “Discharge” as a competing risk, since it was defined as discharge for cure only, discharge with the expectation of the patient to die for example in a hospice facility was considered “Death”. To adjust for potential bias by ICU restriction, we performed stratified analysis for this “composite adverse endpoint” to reflect underlying potential differences between the two groups with regard to adverse incidences and risk factor prevalence. In addition, multivariable competing risk analysis was conducted; including for ICU restriction. The hazard ratio (HR) changed from 0.49 without, to 0.47 with ICU restriction. This corresponds with a 4% change, which is less than the 10% threshold, which is commonly used, if it were a confounder. With the results of PS matching, competing multivariable risk analyses and stratified analyses, we do not believe that ICU restriction influenced the effect of HCQ/CQ that we have reported on ICU admission. Subsequently, we decided to report the two outcomes separately, after performing additional competing risk analysis corrections. Here, we did not include patients with an ICU restriction, therefore our data indeed only describe a possible protective effect of HCQ in patients without an ICU restriction. As we described in our discussion, prospective intervention studies are needed to confirm our results. However, even in randomized controlled trials generalizability can still be a problem to extrapolate from the results, although a causal effect has been shown.
Furthermore, the authors mention that the ICU-population changed in the course of the pandemic. We did not include patients on the ICU in our study, for the patients on the ward we corrected for age (which was not different between the HCQ, CQ or no-treatment groups).
It is suggested that regional differences in infection rates might have caused different patient populations. This is unlikely, since 14 Dutch hospitals participated (from Goes to Groningen, both rural hospitals as centers in large cities). Fortunately, we did not reach “code black” in the Netherlands, making large differences in ICU admission rates unlikely. Obviously, all observational research is at risk for bias, including selection bias by leaving patients out of the analysis that were transferred between hospitals.
For an answer on the question on the methodology of inclusion by treatment center or treatment exposure, please see our response to the letter by Peters et al. Patients that were directly admitted to the ICU or patients that died within the first 24 h after admission, were excluded from the analysis. In addition, we have used skimming as a routine procedure in our analyses. Separate sensitivity analyses were performed to investigate whether immortal bias or informative censoring could influence our results. First, there were no events before the start of the therapy. Second, time-dependent analysis shows no difference in our reported results (HRHCQ = 0.47; 95% = 0.27–0.81). Third, in a logistic regression analysis we found the same effect of HCQ on ICU admission (ORHCQ = 0.40; 0.21–0.77). Thus, neither potential immortal bias nor informative censoring had a significant influence on the reported results.
Finally, we would like to thank the authors of the letter by Burger et al, ‘More gastro-intestinal adverse events in non-ICU hospitalized COVID-19 patients treated with chloroquine versus hydroxychloroquine’ for sharing the results of their interesting study. The authors show that HCQ has a better safety profile than CQ, with less gastro-intestinal adverse events, and suggest that this may partially explain the difference in the risk of transfer to the ICU that we found.
We deliberately choose to refrain from collecting data on adverse effects, since symptoms of COVID-19 are possibly difficult to distinguish from treatment side-effects, especially in the elderly population, so these results add valuable information on the use of both drugs.
In their study, 1.3% of the patients discontinued treatment in the HCQ group due to gastro-intestinal adverse events, as compared to 17% in the CQ group (a total of 2 patients on HCQ versus 13 patients on CQ).
We completely agree with the authors’ suggestion that since the use of CQ is associated with a higher risk of discontinuation due to toxicity, more patients in the HCQ are able to complete their therapy, possibly making it more effective.
Conflict of interest
We state on behalf of all the authors that there are no known competing financial interests, or personal relationships that could have appeared to influence the work reported in this paper.
Funding source
There were no Funding Sources or Study Sponsors involved in this study.s
Ethical approval
The Medical Ethics Review Committee waived ethical Approval for this study.