Response to commentary on the comparative efficacy of esketamine vs. ketamine meta-analysis: Putting the cart before the horse?

We thank Drevets and colleagues for their comments on our recent meta-analysis of the comparative efficacy of racemic ketamine and esketamine for depression (Bahji et al., 2020c). Preliminary evidence from our meta-analysis appeared to suggest that racemic ketamine was more efficacious than esketamine for the treatment of major depressive disorder. Given the recent publication of additional esketamine trials for suicidality and major depression (Fu et al., 2020; Ionescu et al., 2020), we are currently working on an updated meta-analysis. Because research investigating glutamatergic modulators for the management of mood disorders is rapidly advancing, we must all continue to work together to find effective treatments for an ailment that afflicts more than 300 million individuals globally. In the interim, we wish to address individual comments made by Drevets and colleagues and provide responses where appropriate.

Broadly, Drevets and colleagues asserted that our conclusions were invalid because we chose to compare ketamine and esketamine data from trials that were not sufficiently similar, thereby violating meta-analytic assumptions. In the absence of head-to-head trials, a meta-analytic approach is needed to determine which of the three available ketamine formulations (racemic ketamine, esketamine, and arketamine) performs best—and for whom. However, what Drevets and colleagues failed to appreciate is that the scope of our meta-analysis was much broader because we summarized the extant literature on ketamine for the treatment of depression. The focus of their criticism was on a single subgroup analysis rather than on the meta-analysis as a whole. Towards that end, we have drafted a detailed, itemized response to their specific concerns, and provided updated analyses in some cases where appropriate.

Background References.

In our meta-analysis, the efficacy of racemic ketamine was statistically significant over esketamine for treating major depressive disorder. We examined the literature to identify potential reasons for this finding. One possibility relates to the differential bioavailability of alternative ketamine formulations. Both ketamine and esketamine can be delivered intravenously, intramuscularly, or intranasally. However, the intravenous delivery route has maximal bioavailability (~100%) and is an effective option for the dose’s absorption, accuracy, and efficacy (Zanos et al., 2018). In contrast, the bioavailability of intranasal esketamine is approximately 54% (Perez-Ruixo et al., 2020). Furthermore, emerging evidence suggests that arketamine is a viable and promising candidate for depression; evidence is drawn from several preclinical studies (Chaki, 2018; Hashimoto, 2020; Zanos et al., 2019), a recent positive open-label clinical study (Leal et al., 2020), and several reviews (Hashimoto, 2020, 2019; Jelen et al., 2020; Zanos et al., 2019; Zhang and Hashimoto, 2019). If both arketamine and esketamine have antidepressant properties, then our finding that racemic ketamine outperformed esketamine is not unexpected.

Coverage and Cost Issues.

Although ketamine has had FDA approval as an anesthetic for decades, it remains off-label for treating depression. Furthermore, because racemic ketamine is a generic drug, it cannot be patented, branded, or profited from, and currently lacks FDA approval. In contrast, esketamine has gained FDA approval for treatment-resistant depression (TRD) based on evidence from several recent clinical trials (Davoudian and Wilkinson, 2020; Salahudeen et al., 2020; Turner, 2019). Intranasal esketamine is only available at select facilities per FDA regulations to mitigate this drug’s misuse, and because of its side effects, the FDA requires that patients be monitored in a health care setting for two hours after each dose to monitor for dissociation and hypertension (Kim et al., 2019). Nevertheless, cost and insurance company coverage for esketamine remains an issue (Ross and Soeteman, 2020). In addition, the recommended treatment protocols for esketamine are inconvenient, requiring twice weekly, supervised administration for the first four weeks of treatment. Hence, research must discern between the clinical role of intranasal esketamine and its economic value (Dadiomov, 2020).

Lack of head-to-head trials.

A key component of comparative treatment effectiveness research is head-to-head trials. Unfortunately, few head-to-head trials exist for most psychiatric disorders, including ketamine for depression. However, patients and clinicians are increasingly enquiring about the potential advantages of intranasal esketamine, which appears to be more costly than intravenous racemic ketamine despite having FDA approval (Dadiomov, 2020; Ross and Soeteman, 2020). Until more head-to-head trials emerge (Correia-Melo et al., 2018), research to address this critical question could serve the public’s best interest.

Consequently, researchers turn to alternative methods, such as Phase IV, real-world effectiveness studies, pooled meta-analyses, and network meta-analyses. Our meta-analysis included indirect comparisons of racemic ketamine and esketamine using a random-effects meta-analysis model. Several previous studies have used similar methods (Bahji et al., 2020a, 2020b; Nelson and Papakostas, 2009; Papadimitropoulou et al., 2017; Papakostas et al., 2020; Strawbridge et al., 2019; Zhou et al., 2015). While meta-analyses cannot substitute for head-to-head trials, the findings can still provide valuable insights into the comparative performance of different ketamine formulations for major depressive disorder.

Statistical validity.

In the absence of head-to-head trials, indirect comparisons can be used when the trials are similar in terms of design elements, comparator conditions, study populations, and outcome measures. However, as no arbitrary definitions for indirect comparisons exist, the decision to pool findings across studies is not an exact science. Ideally, the compared trials should mostly resemble one another except for the intervention of interest. We primarily adhered to this principle in our meta-analysis. For example, most studies were double-blind, randomized, placebo-controlled trials of adults with severe major depressive disorder. While perfect congruence across trials cannot exist given the lack of standardization, several statistical methods fortunately allow researchers to manage deviations from the ideal scenario (Dold et al., 2020; McIntyre et al., 2020; Nelson and Papakostas, 2009; Witt et al., 2020; Xu et al., 2016). In this context, our meta-analysis accounted for potential effect modifications from several vital variables, using subgroup and meta-regression analyses for severity of baseline depression, age, sex, trial design (crossover vs. parallel), route of delivery, dose, timepoint, severity of depressive symptoms, and medication model (adjuvant vs. monotherapy). Most of these subgroup analyses, however, were non-significant, supporting our decision to pool outcomes.

Relationship between sample size and effect size.

Drevets and colleagues noted that sample size affects effect size. Because trial sample size increases by expanding to multiple centers and geographic sites, there is a compensatory attenuation of treatment-specific effect sizes. For this reason, we opted to use standardized mean differences to measure effect sizes across studies of varying sizes. The standardized mean difference is the ratio of the mean difference to the pooled standard deviation. This measure helps adjust for the relatively larger effect sizes seen in pilot studies, which have wider confidence intervals due to their smaller sample sizes.

Treatment-Resistant depression.

We are receptive to Drevets and colleagues’ suggestions regarding additional potential effect modifiers on our findings. They aptly noted that most esketamine-treated patients were required to meet a more rigorous definition of TRD, which in most cases mandated documented failure to respond to at least two oral antidepressants in the current episode. While we had already included subgroup analyses by TRD and for bipolar depression, we also conducted a post-hoc meta-regression analysis by the minimum number of failed trials. However, the number of previous failed attempts had no significant effect on the relative efficacy of ketamine and esketamine in our meta-analysis.

Baseline severity.

Drevets and colleagues also pointed out the potential influence of severity of baseline depression on effect sizes. However, in our meta-analysis, only two of the trials did not require a minimum severity of baseline symptoms, making it difficult to assess the impact of this variable. We therefore conducted post-hoc meta-regression analyses for each outcome against the baseline severity of the group receiving ketamine. However, the severity of baseline symptoms had no significant effect on the results of our meta-analysis.

Age.

Drevets and colleagues criticized our inclusion of one trial of older adults receiving esketamine for depression (mean study participant age= 70.0) (Ochs-Ross et al., 2020). The basis for their criticism is the apparent diminished response to esketamine among older adults, as esketamine did not separate from placebo. Our review’s eligibility criteria were inclusive concerning the ages of study participants, although we identified no studies of children, adolescents, or older adults receiving racemic ketamine. Post-hoc meta-regression analyses for age were not significant. Similarly, sensitivity analyses that excluded individual studies to determine the impact on the pooled estimate did not flag the trial by Ochs-Ross and colleagues as an outlier. Hence, the decision to include this trial involving older adults likely did not influence the validity of the meta-analysis to an appreciable extent.

Blinding.

As in all clinical trials, blinding participants and researchers to treatment allocation is a critical component of the trial’s intrinsic validity. However, psychoactive substances, such as ketamine and other psychedelics, are prone to “functional unblinding” because participants may be able to recognize the dissociative properties of active treatment. Drevets and colleagues suggest that using remote (telephone-based) raters, as done in the esketamine trials, might minimize the risk of functional unblinding. We believe that the effectiveness of this approach on functional unblinding remains unclear due to the lack of systematic testing.

Errors.

We thank Drevets and colleagues for identifying a labelling mistake; specifically, an IV esketamine trial (Singh et al., 2016) was misplaced as a ketamine trial, and an intranasal racemic ketamine trial (Lapidus et al., 2014) was misplaced as an esketamine trial. After correcting these errors, some minor changes emerged in our subgroup analyses by ketamine type (racemic vs. esketamine). For example, the initial subgroup analysis showed that the rate ratio for depression response to racemic ketamine and esketamine were 3.1166 and 1.3005, respectively; the revised estimates are 2.9158 and 1.3013, respectively. The subgroup analysis still statistically significantly favours ketamine (p=0.0011), which is slightly less significant than the original (p=0.004). The conserved estimates point to the robustness of our analyses.

Conclusions.

The only goal of our meta-analysis was to provide patients, mental health care providers, stakeholders, and the general public with the most up-to-date and accurate information possible to appraise the efficacy of ketamine and esketamine for treating depression. Selective interpretation of our findings fails to consider the whole picture, which is contrary to the goals of a meta-analysis used in the absence of randomized head-to-head trials. Without enough data to support their claim against our findings, racemic ketamine appears to demonstrate greater efficacy than esketamine for depression. While it is likely that esketamine will also be useful, there is a current lack of real-world clinical experience regarding actual dosing, effectiveness, side effects, or treatment protocols outside of the FDA-approval clinical trials that had stringent selection criteria for patient participation. In this context, we invite industry and academicians to sponsor a head-to-head comparison of IV ketamine and intranasal esketamine; such a comparison could alleviate both patients’ and clinicians’ concerns.