Efficacy of intranasal esketamine versus rTMS for treatment-resistant depression: analysis of individual participant data from two clinical trials

Summary

Background

Repetitive transcranial magnetic stimulation (rTMS) and intranasal esketamine are effective treatment options for treatment-resistant depression (TRD) that have not been directly compared. We aimed to compare the effectiveness of rTMS, intranasal esketamine, and pharmacotherapy for TRD using data from two randomised clinical trials (RCTs).

Methods

This secondary analysis of individual patient data (IPD) included data from two large clinical trials for rTMS (THREE-D; non-inferiority RCT, four sites in Canada from Sept 2013 to Oct 2016, n = 388 eligible) and intranasal esketamine (TRANSFORM-2; superiority RCT, 39 sites across five countries from Aug 2015 to Nov 2017, n = 227 eligible) to compare efficacy for TRD (defined as non-response to at least one evidence-based treatment). A third group included participants who received standard of care (new antidepressant medications) plus placebo nasal spray (TRANSFORM-2). rTMS was delivered in a once daily format, while intranasal esketamine was delivered twice weekly. Restriction, propensity-score matching, and regression adjustments were used to minimise confounding factors between studies. The primary outcome for this analysis was the severity of depressive symptoms using the 17-item Hamilton Depression Rating scale (HDRS), assessed as a continuous measure, after 4 weeks of treatment. THREE-D and TRANSFORM-2 are registered with ClinicalTrials.gov as NCT01887782 and NCT02418585, respectively.

Findings

After restriction and propensity-score matching, the analytic sample consisted of 282 participants in three arms: rTMS (n = 94), intranasal esketamine (n = 94), and new antidepressant medication (n = 94). Both rTMS (β: −5.35 [95% CI, −8.77, −1.93]) and intranasal esketamine (−2.89 [−5.38, −0.40]) were superior at reducing depression severity when compared with initiating a new antidepressant medication. rTMS resulted in a non-statistically significant greater symptom reduction compared with intranasal esketamine (−2.46 [−5.82, 0.89]). The upper confidence limit interval for the rTMS and intranasal esketamine comparison provide preliminary evidence that intranasal esketamine is not superior to rTMS by a minimal clinically important difference.

Interpretation

This analysis showed rTMS and intranasal esketamine are superior to initiation of a new antidepressant medication, which is consistent with prior randomised clinical trials. Acknowledging its limitations and exploratory nature, our work indicates that rTMS may be superior, or at least similarly effective, to intranasal esketamine and highlights the need for large, prospective, comparative effectiveness trials directly comparing these interventions.

Funding

Support for this work was provided by the Delaney Family Foundation. THREE-D was funded by the Canadian Institutes of Health Research and TRANSFORM-2 was funded by Johnson and Johnson Pharmaceuticals.

Research in context.

Evidence before this study

Both repetitive transcranial magnetic stimulation (rTMS) and intranasal esketamine are evidence-based treatment options for treatment-resistant depression (TRD); however, their relative effectiveness compared with one another is unclear. We conducted a Medline search on September 25, 2025 using the OVID interface using the terms “transcranial magnetic stimulation”, “ketamine” and “depression” searching database inception to September 24, 2025 in all languages. This search result yielded 20 studies, of which there was a single study comparing rTMS with intranasal esketamine but this study used a naturalistic design and an experimental rTMS protocol such that it was not suitable to inform clinical care.

Added value of this study

To the best of our knowledge, this is the first analysis directly comparing once daily rTMS treatment with intranasal esketamine. Using individual patient data (IPD) from two large clinical trials of evidence-based rTMS (THREE-D; non-inferiority RCT, four sites in Canada, n = 388 eligible) and intranasal esketamine (TRANSFORM-2; superiority RCT, 39 sites across five countries, n = 227 eligible) protocols, the present analysis shows that both rTMS and intranasal esketamine were superior to pharmacotherapy for TRD. We found preliminary evidence that intranasal esketamine is not superior to rTMS by a minimal clinically important difference, but that rTMS is either similarly effective or superior to intranasal esketamine.

Implications of all the available evidence

Though there are limitations to consider with such exploratory work, the findings of this analysis of IPD from two clinical trials suggest that rTMS and intranasal esketamine are both superior to standard pharmacotherapy for TRD. Intranasal esketamine does not appear to be more effective than rTMS; however, rTMS is either similarly effective, or potentially even superior, to intranasal esketamine. Large-scale clinical trials should be conducted to definitively determine the relative effectiveness of both these interventions in TRD.

Introduction

Depression is the second leading cause of disability worldwide and an illness whose burden has significantly increased since 2010.¹ Though medications and psychotherapy are the mainstay of initial treatment, many individuals do not respond to one or more adequate evidence-based treatments and are diagnosed with treatment-resistant depression (TRD).^2,3 Only a minority of individuals with TRD achieve remission of their symptoms as demonstrated by the seminal STAR∗D trial published in 2006, which found that with a third medication trial, fewer than 15% of individuals achieved remission of their symptoms.³ Though the STAR∗D findings have informed TRD management for the past two decades, this period has seen a surge in treatment options for those suffering from TRD, many of which extend beyond the standard pharmacotherapy and psychotherapy.4, 5, 6, 7

Two prominent novel treatment options for TRD of the last two decades are repetitive transcranial magnetic stimulation (rTMS) and intranasal esketamine.^5,6 Both treatments have been approved by the FDA, with rTMS approved in 2008⁶ and intranasal esketamine in 2019.⁵ Though both are evidence-based treatments, they are markedly different in their mechanism of action and logistical aspects of treatment. rTMS is thought to work through the stimulation of cortical targets that modulate brain networks implicated in the pathophysiology of depression,⁸ while esketamine is thought to exert its mechanism of action through antagonism of the N-methyl-d-aspartate (NMDA) glutamate receptor.⁷ They also differ in their administration with a modern form of rTMS known as intermittent theta-burst stimulation (iTBS) being delivered once daily for 6 weeks with each treatment lasting about 3 min⁶ while intranasal esketamine is delivered twice weekly for 4 weeks requiring 2 h of post-treatment medical monitoring.⁵

Despite differences in mechanism of action and administration, both rTMS and esketamine have demonstrated evidence of efficacy in TRD in large clinical trials. In the study by Blumberger et al., rTMS demonstrated a response and remission rate of approximately 50% and 30%, respectively.⁶ Similarly in the study by Popova et al., intranasal esketamine—when delivered in conjunction with a new antidepressant—demonstrated response and remission rates of approximately 70% and 50%, respectively.⁵ Though intranasal esketamine demonstrated numerically superior treatment outcomes compared with rTMS, these results cannot be directly compared due to differences in eligibility and participant characteristics that confound treatment outcomes. The lack of comparative effectiveness studies comparing rTMS and esketamine is a key knowledge gap because a direct comparison of these interventions is necessary for patients and their providers to make informed decisions about which treatment to pursue. These informed decisions are particularly important with interventional psychiatry treatments (such as rTMS and esketamine) given their significant financial costs and logistical burdens. At present, there is a single naturalistic study comparing an experimental rTMS protocol with intranasal esketamine.⁹ However, this study used an experimental rTMS protocol and did not account for differences between treatment arms such that it cannot be used to inform clinical care. Prior studies comparing evidence-based rTMS protocols to other interventions have found that rTMS is more effective than atypical antipsychotic augmentation,¹⁰ and antidepressant switching.⁴ A direct comparison between rTMS and intranasal esketamine using evidence-based protocols would therefore be a meaningful advance in the management of patients with TRD.

To address this knowledge gap, we conducted a comparative effectiveness analysis between rTMS and intranasal esketamine using individual patient data from two large clinical trials leveraging modern statistical techniques to address differences between study populations.^5,6 The objective of this work was to determine in a sample of individuals with TRD whether rTMS or intranasal esketamine was superior at reducing depressive symptoms at the end of an acute treatment course for individuals with TRD.

Methods

Data source and study sample

This study was a secondary analysis of individual patient-level data from two randomised clinical trials (RCTs). The first was the investigator-initiated THREE-D study (n = 388 eligible), which was a two-arm non-inferiority RCT comparing once daily high-frequency left rTMS treatment with the novel iTBS stimulation (ClinicalTrials.gov, NCT01887782).⁶ This study was conducted at 3 academic sites in Canada from Sept 2013 to Oct 2016 with rTMS being delivered once daily over the course of 4–6 weeks according to standard practices.⁶ This study demonstrated non-inferiority for both treatment arms, and therefore for the present work we have pooled these arms together as a single “rTMS” arm. The second study was the TRANSFORM-2 study (n = 227) funded by Janssen Pharmaceuticals that was conducted across 39 sites in United States, Czech Republic, Germany, Poland, and Spain from Aug 2015 to Nov 2017 (ClinicalTrials.Gov, NCT02418585). This was a two-armed superiority RCT comparing flexibly-dosed intranasal esketamine against placebo nasal spray. Both arms were initiated on a new antidepressant and intranasal esketamine treatment was administered twice weekly for 4 weeks.⁵ As a result of this design, we have referred to the esketamine arm as the “esketamine” arm and the placebo nasal spray as “medication” arm throughout this work. Further details of study eligibility criteria are available in the Supplemental Materials.^5,6

Ethics

For THREE-D, local research ethics board approval was obtained for all three study sites. For TRANSFORM-2, institutional review boards (in the United States) or independent ethics committees (in Europe) at each study site approved the study protocol and amendments. For both studies, all participants provided written informed consent before study entry. The specific names of the authorities that granted ethical approval for each trial, along with reference numbers are available in the original publications of each trial.^5,6

Outcomes

The primary outcome measure for THREE-D was the 17-item Hamilton Depression Rating Scale (HDRS) at the end of the treatment course, while for TRANSFORM-2 it was the Montgomery-Asberg Depression Rating Scale (MADRS). We used the HDRS as our primary outcome measure because it is the mostly widely used psychometric scale and has established thresholds for clinically important difference.¹¹ We used a published conversion table to convert the MADRS scores in TRANSFORM-2 into HDRS scores.¹² The primary outcome for this study was the severity of depressive symptoms using HDRS as a continuous measure to maximise statistical power.¹³ Secondary outcomes were response (≥50% reduction from baseline) and remission (<8 HDRS or <10 MADRS) rates.

Confounding adjustment

Due to differences in trial eligibility criteria, which have the potential to confound the association between treatment and outcome, we performed several procedures to minimise confounding. First, we restricted all participants to having failed 1-3 adequate trials of antidepressants to ensure comparability between trials. We then used propensity scores, which are the probability of treatment assignment conditional on observed baseline characteristics,¹⁴ to minimise differences in baseline depression severity, which based on a priori knowledge was identified as the most important confounder.⁶ While typically used in large administrative health studies, they can also be carefully used in studies with smaller sample sizes.¹⁴ To estimate the propensity of belonging to THREE-D or TRANSFORM-2 we used a logistic regression model and included as independent variables the three measures of depression severity measured at baseline including HDRS, MADRS, and clinician-rated Inventory of Depressive Symptoms (IDS). The dependent variable in this model was participation in either THREE-D or TRANSFORM-2.

Once propensity scores were estimated for each trial participant, we then performed 1:1:1 matching on the propensity score between THREE-D (rTMS):TRANSFORM-2 (esketamine): TRANSFORM-2 (medication) arms using the MatchIt package (version 4.5.5).¹⁵ We used a greedy nearest neighbor matching algorithm and conducted matching without replacement. To conduct the three-arm matching we first matched 1:1 TRANSFORM-2 (esketamine) and TRANSFORM-2 (medication). We then conducted a second iteration of matching using the matched participants from TRANSFORM-2 (esketamine) to match 1:1 with THREE-D (rTMS).

To assess the success of the matching procedure we then assessed for statistically significant differences before and after matching on all baseline characteristics using ANOVA or chi-squared test as appropriate with p < 0.05 indicating a significant covariate imbalance between groups.¹⁶

Statistical analysis

Baseline socio-demographic and clinical characteristics were compared between the three different arms before and after matching. Our primary analysis used the propensity-score matched cohort to conduct a linear regression with depression severity quantified by the HDRS as a continuous measure at 4 weeks after treatment start as the outcome of interest. 4 weeks was selected as the primary analytic time point because it would provide the most relevant comparison with the TRANSFORM-2 study (which went to 4 weeks) and the design of the THREE-D study was such that weeks 5 and 6 of treatment were missing not at random (Supplemental Material). In the regression model, we included a categorical variable denoting treatment arm consisting of three levels: rTMS, esketamine, and medication (reference group). This variable was the primary effect of interest within the regression model as it quantifies the magnitude of effect of each treatment arm on depressive symptoms. To further minimise the effect of confounding, we also adjusted for the following potential confounders in the propensity-score matched sample at baseline: age, biological sex, education level, employment status, age of depression onset, current depressive episode duration, anxiety, benzodiazepine use, and number of failed treatments.

We also conducted several sensitivity analyses. First, to compare the magnitude of effect on depressive symptoms between each treatment arm, we calculated the estimated marginal means (EMM) and estimated marginal mean differences (EMMD) in HDRS at 4 weeks between each treatment arm using the emmeans package (v.1.8.9). The EMMD of HDRS between treatment arms was compared with the threshold of 3 points on the HDRS which is the threshold for minimal clinically important difference (MCID).⁶ Second, to compare different outcome definitions, we changed our primary analytic model to be a logistic regression model with the primary outcome being response or remission on the HDRS at 4 weeks. Third, to assess the robustness of our matching procedure to residual covariate imbalance, we also used the estimated propensity scores to generate overlap weights (OW).¹⁷ We then applied these overlap weights to the sample to generate a pseudo-population with balanced covariates.¹⁷ In this weighted sample, we then applied our primary analytic model examining HDRS at 4 weeks. We also conducted two post hoc analyses. First, to assess the impact of converting the MADRS to the HDRS, we repeated the analysis converting all scores to the MADRS. Second, to account for the fact that some definitions of TRD require ≥2 failed trials¹⁸ and also to account for the prospective screening phase of TRANSFORM-2, we repeated the analysis but restricted to participants with ≥2 adequate antidepressant trials at study entry.

For all regression models, we reported point estimates as unstandardised beta-coefficient (β) and odds ratio (OR) as appropriate along with a 95% confidence interval (CI). We did not adjust for multiple comparisons given the exploratory nature of this analysis.¹⁹ Results were reported in accordance with the STROBE guidelines given that this study is best conceptualized as an observational study (Supplemental Material).²⁰ All analyses were conducted in R version 4.3.0. This study was registered with the Yale University Open Access Project (registration ID, #2024-0540).

Role of the funding source

Support for this manuscript was provided by the Delaney Family Foundation. All funders—including those of the original trials (THREE-D and TRANSFORM-2)—did not have any role in study design, data collection, data analyses, data interpretation, or the writing of the report. The corresponding author (TSK) had final responsibility to submit for publication.

Results

Characteristics of study sample

615 participants were included in the analytic cohorts of THREE-D (n = 388) and TRANSFORM-2 (n = 227). After antidepressant treatment failure restriction and propensity score-matching, the final analytic cohort for this study was 282 participants with 94 participants in each treatment arm (Fig. 1). Baseline characteristics of included participants before and after propensity-score matching are reported in Table 1. After propensity-score matching (Supplemental Material), all participants were balanced on baseline depression severity; however, there were baseline imbalances between studies on characteristics including age, employment status, age of onset, anxiety comorbidities, benzodiazepine use, and antidepressant treatment history. Baseline characteristics were similar when OW was used though the sample size was smaller (n = 239; Supplemental Material).

Fig. 1.

Participant flow and analytic cohort creation for the TRANSFORM-2 and THREE-D trials. Note: Data are from the TRANSFORM-2 (ClinicalTrials.gov, NCT02418585) and THREE-D (ClinicalTrials.gov, NCT01887782) randomised controlled trials. The study samples were matched using a 1:1:1 propensity score matching technique based on the baseline Montgomery–Åsberg Depression Rating Scale (MADRS), Hamilton Depression Rating Scale (HDRS), and Inventory for Depressive Symptomatology (IDS) scores. Acronym: repetitive transcranial magnetic stimulation (rTMS); Good Clinical Practice (GCP).

Table 1.

Selected characteristics (column %) of the study samples at baseline by three comparison groups (rTMS, esketamine and medication) after treatment-failure restriction before and after 1:1:1 propensity score matching.

Before matching						After matching
	rTMS	Esketamine	Medication	Total	P-value		rTMS	Esketamine	Medication	Total	P-value
Sample size	358	102	94	554		Sample size	94	94	94	282
Age, mean ± standard deviation (SD)	42.3 ± 11.5	45.6 ± 12.6	47.1 ± 10.5	43.7 ± 11.7	<0.001	Age, mean ± standard deviation (SD)	42.3 ± 10.8	45.9 ± 12.8	47.1 ± 10.5	45.1 ± 11.5	0.012
Sex						Sex
Male	145 (40.5%)	33 (32.4%)	41 (43.6%)	219 (39.5%)	0.224	Male	41 (43.6%)	32 (34.0%)	41 (43.6%)	114 (40.4%)	0.303
Female	213 (59.5%)	69 (67.6%)	53 (56.4%)	335 (60.5%)		Female	53 (56.4%)	62 (66.0%)	53 (56.4%)	168 (59.6%)
Educational attainment						Educational attainment
Not/Partially complete high school/Unknown	8 (2.2%)	8 (7.8%)	13 (13.8%)	29 (5.2%)	<0.001	Not/Partially complete high school/Unknown	2 (2.1%)	8 (8.5%)	13 (13.8%)	23 (8.2%)	0.058
High school/Technical degree/Part college	134 (37.4%)	41 (40.2%)	39 (41.5%)	214 (38.6%)		High school/Technical degree/Part college	40 (42.6%)	37 (39.4%)	39 (41.5%)	116 (41.1%)
College/Graduate Education	216 (60.3%)	53 (52.0%)	42 (44.7%)	311 (56.1%)		College/Graduate Education	52 (55.3%)	49 (52.1%)	42 (44.7%)	143 (50.7%)
Current employment						Current employment
Yes	136 (38.0%)	62 (60.8%)	58 (61.7%)	256 (46.2%)	<0.001	Yes	29 (30.9%)	57 (60.6%)	58 (61.7%)	144 (51.1%)	<0.001
No	222 (62.0%)	40 (39.2%)	36 (38.3%)	298 (53.8%)		No	65 (69.1%)	37 (39.4%)	36 (38.3%)	138 (48.9%)
Age of onset, mean ± SD	20.8 ± 10.8	32.4 ± 12.5	36.5 ± 11.9	25.6 ± 13.1	<0.001	Age of onset, mean ± SD	21.0 ± 12.0	32.5 ± 12.4	36.5 ± 11.9	30.0 ± 13.7	<0.001
Depressive episode duration (in months), mean ± SD	23.9 ± 28.2	25.5 ± 30.8	24.2 ± 38.7	24.2 ± 30.7	0.888	Depressive episode duration, months	23.1 ± 28.2	25.2 ± 30.7	24.2 ± 38.7	24.2 ± 32.7	0.916
Anxiety comorbidity						Anxiety comorbidity
Yes	195 (54.5%)	16 (15.7%)	13 (13.8%)	224 (40.4%)	<0.001	Yes	60 (63.8%)	16 (17.0%)	13 (13.8%)	89 (31.6%)	<0.001
No	163 (45.5%)	86 (84.3%)	81 (86.2%)	330 (59.6%)		No	34 (36.2%)	78 (83.0%)	81 (86.2%)	193 (68.4%)
Receiving benzodiazepine during treatment						Receiving benzodiazepine during treatment
Yes	112 (31.3%)	12 (11.8%)	11 (11.7%)	135 (24.4%)	<0.001	Yes	35 (37.2%)	12 (12.8%)	11 (11.7%)	58 (20.6%)	<0.001
No	246 (68.7%)	90 (88.2%)	83 (88.3%)	419 (75.6%)		No	59 (62.8%)	82 (87.2%)	83 (88.3%)	224 (79.4%)
Previous treatment historya						Previous treatment historya
Unable to tolerate two trials	0 (0.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)	<0.001	Unable to tolerate two trials	0 (0.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)	<0.001
One failed antidepressant	173 (48.3%)	9 (8.8%)	18 (19.1%)	200 (36.1%)		One failed antidepressant	52 (55.3%)	9 (9.6%)	18 (19.1%)	79 (28.0%)
Two failed antidepressants	111 (31.0%)	69 (67.6%)	54 (57.4%)	234 (42.2%)		Two failed antidepressants	21 (22.3%)	65 (69.1%)	54 (57.4%)	140 (49.6%)
Three failed antidepressants	74 (20.7%)	24 (23.5%)	22 (23.4%)	120 (21.7%)		Three failed antidepressants	21 (22.3%)	20 (21.3%)	22 (23.4%)	63 (22.3%)
≥4 failed antidepressants	0 (0.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)		≥4 failed antidepressants	0 (0.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
Baseline MADRS score, mean ± SD	30.3 ± 5.6	36.9 ± 5.7	37.2 ± 5.7	32.7 ± 6.5	<0.001	Baseline MADRS score	36.5 ± 4.6	36.5 ± 5.6	37.2 ± 5.7	36.8 ± 5.3	0.553
Baseline HDRS score, mean ± SD	23.5 ± 4.3	28.7 ± 4.6	28.9 ± 4.6	25.4 ± 5.1	<0.001	Baseline HDRS score	28.3 ± 3.7	28.4 ± 4.5	28.9 ± 4.6	28.5 ± 4.3	0.565
Baseline IDS-30 score, mean ± SD	39.2 ± 9.9	45.9 ± 6.2	45.6 ± 5.6	41.5 ± 9.3	<0.001	Baseline IDS-30 score	45.7 ± 8.6	46.4 ± 6.3	45.6 ± 5.6	45.9 ± 6.9	0.72

Note: Data are from TRANSFORM-2 (NCT: # NCT02418585) and THREE-D (NCT: # NCT01887782) trials. The 1:1:1 propensity score matching was performed based on the baseline MADRS, HDRS, and IDS scores. Standardised mean differences are reported in eFig. 2.

Acronym: MADRS, Montgomery–Åsberg Depression Rating Scale; HDRS, Hamilton Depression Rating Scale; IDS, Inventory for Depressive Symptomatology; rTMS, repetitive Transcranial Magnetic Stimulation.

^aTreatment history prior to study entry.

Depressive symptoms

Depressive symptoms of each treatment arm of the propensity-score matched cohort are depicted in Fig. 2 with crude values in Fig. 2A and the EMM values of depressive symptoms that were adjusted for confounders in Fig. 2B. In our primary analysis adjusting for baseline covariates, we found that, compared with 4 weeks of medication treatment, both rTMS (β = −5.35 [95% CI: −8.77, −1.93]) and esketamine (−2.89 [−5.38, −0.40]) resulted in a statistically significant reduction in HDRS (Table 2). For the EMMD between rTMS and esketamine we found that rTMS resulted in numerically larger symptom reduction compared with esketamine (−2.46 [−5.82, 0.89]). Compared with the MCID of 3 points, only rTMS versus medication had a point estimate greater than the MCID; however, the upper limit of 95% CI did include the MCID. Of note, the rTMS versus esketamine comparison 95% CI upper limit did not include the MCID.

Fig. 2.

Depressive symptoms measured over four weeks of treatment. Shaded region is 95% confidence interval. Panel 2 A. Changes in raw scores of Hamilton Depression Rating Scale (HDRS). Panel 2 B. Estimated marginal mean (EMM) of HDRS over time. Acronym: repetitive transcranial magnetic stimulation (rTMS).

Table 2.

Multivariable-adjusted linear regression model for depression severity at week 4.

Reference group in parenthesis	Coefficient (95% CI; P)
Comparison group (medication)
Esketamine	−2.89 (−5.38 to −0.4; 0.023)
rTMS	−5.35 (−8.77 to −1.93; 0.002)
Age	0.12 (0.01–0.23; 0.026)
Sex (male)
Female	0.38 (−1.71 to 2.47; 0.719)
Education (<high school/unknown)
High school or some college	−0.13 (−3.99 to 3.72; 0.947)
≥Bachelor's	0.48 (−3.35 to 4.3; 0.807)
Employment (unemployed)
Employed	−1.07 (−3.2 to 1.06; 0.322)
Age of onset	−0.11 (−0.21 to −0.01; 0.034)
Episode duration	0.01 (−0.02 to 0.05; 0.408)
Anxiety (no)
Yes	4.94 (2.36–7.51; <0.001)
Benzodiazepine use (no)
Yes	3.01 (0.33–5.69; 0.028)
Treatment History (≤1 failed antidepressant trial)
2 failed antidepressant trials	−0.89 (−3.63 to 1.86; 0.525)
3 failed antidepressant trials	1.54 (−1.59 to 4.67; 0.333)
Baseline MADRS score	2.7 (0.04–5.37; 0.047)
Baseline HDRS score	−3.41 (−6.73 to −0.09; 0.044)
Baseline IDS score	0.10 (−0.06 to 0.26; 0.240)

Note: Data are from TRANSFORM-2 (NCT: # NCT02418585) and THREE-D (NCT: # NCT01887782) trials. The study samples were matched using a 1:1:1 propensity score matching technique based on the baseline MADRS, HDRS, and IDS scores. Acronym: MADRS, Montgomery–Åsberg Depression Rating Scale; HDRS, Hamilton Depression Rating Scale; IDS, Inventory for Depressive Symptomatology; rTMS, repetitive Transcranial Magnetic Stimulation.

The secondary analyses for the HDRS EMM at 4 weeks yielded estimates for each treatment arm of: medication 19.7 [17.4, 22.1]; esketamine 16.9 [14.4, 19.3]; and rTMS 14.4 [11.9, 16.9]. Sensitivity analyses using OW yielded similar results to our primary analyses (Supplemental Material). Conversion to MADRS rather than HDRS yielded similar results in which both rTMS and esketamine resulted in significantly greater reduction in symptoms compared with medication treatment at end of treatment, while the rTMS and esketamine comparison was not statistically significant (Supplemental Material). Remission rates assessed using MADRS were qualitatively similar to HDRS (Supplemental Material). Analyses restricted to participants with ≥2 failed antidepressant trials yielded similar results as the primary analysis, though balance on depressive symptom scales after propensity score matching was notably worse (Supplemental Material). In the cohort with ≥2 failed medication trials, both esketamine (−3.71 [−6.93, −0.50]) and rTMS (−5.28 [−9.52, −1.04]) were superior to medication for reducing depressive symptoms at treatment completion. rTMS was also numerically superior to esketamine (−1.56 [−5.55, 2.42]), and consistent with our primary analyses, the upper limit of the 95% CI upper limit did not include the MCID.

Response and remission rates

Crude response and remission rates of the propensity-score matched cohort are depicted in Fig. 3. In secondary analyses using logistic regression models adjusting for baseline covariates to assess response rates we found that, compared with 4 weeks of medication treatment, both rTMS (OR = 3.41; [1.41, 8.57]) and esketamine (2.08 [1.09, 4.04]) had significantly higher response rates (Supplemental Material). In contrast, both rTMS and esketamine did not demonstrate statistically significant increased odds of remission compared with medication treatment (Supplemental Material). Sensitivity analyses using the OW cohort yielded similar findings in which both esketamine and rTMS were superior to medication at achieving response; however, neither were superior to medication at achieving remission (Supplemental Material).

Fig. 3.

Unadjusted response and remission rates measured by the Hamilton Depression Rating Scale (HDRS) at week 4. Whisker bars indicate 95% confidence intervals. Acronym: repetitive transcranial magnetic stimulation (rTMS).

Discussion

In this secondary analysis of two large clinical trials for rTMS and intranasal esketamine that enrolled more than 600 study participants, we found that, after adjustment for participant differences, both rTMS and esketamine were significantly more effective than medication treatment at reducing depressive symptoms after 4 weeks of treatment. Response outcomes followed a similar pattern, though suffered from insufficient power due to loss of information in context of dichotomisation of a continuous outcome. While we did not find a statistically significant difference between rTMS and esketamine, the confidence interval limits indicate that our findings reject the hypothesis that esketamine is superior to rTMS by a clinically meaningful amount. In contrast, the possibility that rTMS is superior to esketamine remains possible though could not be confirmed due to limited sample size.

These findings are particularly important in the context of prior work of comparative effectiveness studies in which rTMS has been compared with other interventions. The ASCERTAIN-TRD study found that rTMS was significantly more effective than switching antidepressants at alleviating depressive symptoms.⁴ Similarly, a separate group from Europe found that rTMS yielded significantly higher response and remission rates compared with medication optimisation.¹⁰ In this regard, the findings of the current study are consistent with prior work as we found rTMS was significantly more likely to result in response compared with initiation of a new antidepressant. Regarding intranasal esketamine, there have been no similar comparative effectives studies to draw comparisons with. However, the current work—which used a subset of the original TRANSFORM-2 dataset and differing analytic model—yielded a similar finding as the original publication (i.e., intranasal esketamine with a new antidepressant is superior to a new antidepressant alone).⁵ The superiority of both rTMS and esketamine to a new antidepressant is clinically relevant because it provides additional evidence regarding the emerging role of “interventional psychiatry” treatments as being superior to additional pharmacotherapy treatment trials and suggests the need to develop new treatment algorithms for depression.²¹

The comparison between rTMS and intranasal esketamine conducted in the current work improves our understanding regarding the relative effectiveness of intranasal esketamine compared with rTMS for TRD. Our current findings yielded a point estimate favouring rTMS over intranasal esketamine at reducing depressive symptoms after 4 weeks. Though this finding was not statistically significant, our confidence interval provides meaningful information. The upper limit of the confidence interval (0.89) was below the MCID of 3 points. Therefore, our results reject the hypothesis that intranasal esketamine is superior to rTMS by a clinically meaningful amount. In fact, the confidence interval indicates that, allowing for the conventional 5% type 1 error, our results suggest intranasal esketamine is unlikely to reduce depressive symptoms more than 0.89 points on the HDRS—an amount that is not clinically meaningful by most thresholds.¹¹ In contrast, the lower 95% confidence interval does include the MCID value such that our results indicate that rTMS is likely either superior or similarly effective (i.e., non-inferior) compared with intranasal esketamine. Similar findings were identified in secondary analyses restricting the analytic cohort to ≥2 failed antidepressant trials, which demonstrates the robustness of the present findings. Ultimately; however, properly designed prospective trials are required to definitively determine the effectiveness of rTMS compared to intranasal esketamine.

This finding—that rTMS is either similarly or more effective than intranasal esketamine—is particularly important when taken in the context of two additional considerations not directly examined in this work: cost and side effects. With respect to cost, multiple cost-effectiveness analyses in different healthcare settings, using different methods and perspectives have consistently found that rTMS is cost-effective amongst those with TRD.22, 23, 24, 25, 26, 27 In contrast, most cost-effectiveness studies of intranasal esketamine have found that it is not cost-effective in TRD,28, 29, 30 with only one study finding it cost-effective from a societal perspective but not funder perspective.³¹ This lack of cost-effectiveness has been cited as one of the reasons that intranasal esketamine is not funded by the UK National Institute for Health and Care Excellence.³² With respect to side effects, intranasal esketamine is known to be associated with transient psychiatric, hemodynamic, and neurologic side effects in addition to potentially long-term hepatobiliary/genitourinary side effects and potential for misuse.³³ In contrast, rTMS is typically only associated with transient pain at the site of stimulation, headache, and in extremely rare cases seizure.^6,34 Comparing the cost-effectiveness and side effect profile of both these interventions will be an important area of further work, particularly when developing new treatment algorithms for TRD.

While this study had several clinically important findings, there are important limitations to acknowledge. The first and most notable is that this is not a randomised controlled trial, which by its design, can achieve balance on observed and unobserved characteristics. The current study was a secondary analysis, is exploratory in nature, and though we used several analytic techniques (restriction, matching, regression) to minimise the effects of confounding the possibility of residual confounding continues to remain. However, given the balance in observed characteristics, the results of the sensitivity analyses, and the consistency of our results with prior published work, the impact of residual confounding is unlikely to meaningfully alter our findings. A second limitation is that our current results only considered outcomes at week 4, while most standard rTMS treatment courses last at least 6 weeks.^4,10 This decision was made to ensure comparability of observation period between intranasal esketamine (data only available until 4 weeks) with rTMS that was missing at random (weeks 5 and 6 of THREE-D was missing not at random).^5,6 However, given that recent work has demonstrated prolonged treatment courses of both interventions may more successfully alleviate depressive symptoms, analyses examining longer treatment durations will be necessary to understand the most clinically relevant treatment effect.^35,36 A third limitation of this study is the difference in study designs between TRANSFORM-2 and THREE-D. TRANSFORM-2 was an industry-funded multi-site trial using a placebo saline spray with a new antidepressant medication that may have overestimated the effect of initiating a new antidepressant.³⁷ In contrast, THREE-D was an academic investigator-initiated study that did not include a sham arm, which may have resulted in larger estimated treatment effects compared with sham-controlled trials due to the expectancy effect.³⁸ A fourth limitation of this study was that it did not use a formal non-inferiority testing framework or a pre-registered statistical analysis plan (though it was pre-registered with the YODA platform). This lack of pre-specified analyses means the results from this work should be taken in the appropriate context. A fifth limitation is the degree of treatment-resistance in this study that may not reflect more refractory definitions as all individuals had only failed 1–3 treatment trials, and so our results may not apply to more refractory cases. Sixth, we applied regression adjustments to the propensity-score matched sample, which is often recommended to improve robustness of treatment effect estimates,³⁹ though others have cautioned that it may increase bias.⁴⁰ The final limitation of this work is that we did not consider the other indications for intranasal esketamine (i.e., depression with suicidality) or changing administration (i.e., esketamine without combined antidepressant).

In conclusion, the findings from this study indicate that both intranasal esketamine when combined with a new antidepressant and rTMS are more effective at reducing depressive symptoms than initiation of a new antidepressant medication alone. Importantly our results provide preliminary evidence that rTMS is either similarly effective or superior to intranasal esketamine, while intranasal esketamine is not superior to rTMS by a meaningful amount. While randomised trials should be conducted to compare rTMS with intranasal esketamine, this hypothetical trial will take several years to conduct. Patients and providers in the interim must continue to make treatment decisions, and despite its limitations, the current study serves as the best source of evidence regarding effectiveness of rTMS compared with intranasal esketamine. These results can therefore support the decision-making process between patients with TRD and their providers when deciding between treatments, which hopefully will improve the lives of those suffering from severe depression.

Contributors

Study concept and design: TSK, YD, DMB, and TGR; Data acquisition and statistical analyses: TSK, DMB, and TGR; Interpretation of data: All authors; Drafting of manuscript: TSK, YD, & TGR; Critical revision of manuscript for important intellectual content: All authors. YD and TGR accessed and verified the underlying data.

Data sharing statement

THREE-D: deidentified participant data along with data dictionaries is available and can be shared with researchers who provide a methodologically sound proposal that includes a protocol and a statistical analysis plan, and is not in conflict with the investigators’ publication plan. Proposals should be directed to daniel.blumberger@camh.ca. To gain access, data requestors will need to sign a data access agreement. TRANSFORM-2: deidentified participant data is available through the Yale University Open Data Access (YODA) Project. Data requestors must submit a request through this platform for review and approval.

Declaration of interests

TSK receives research support from the Canadian Institutes of Health Research (CIHR), Patient-Centered Outcomes Research Institute (PCORI), and the AFP Innovation Fund. FVR has received research support from CIHR, Brain Canada, Michael Smith Foundation for Health Research, Vancouver Coastal Health Research Institute, and Weston Brain Institute for investigator-initiated research. Philanthropic support from Seedlings Foundation. In-kind equipment support for investigator-initiated trial from MagVenture. He has received honoraria for participation in an advisory board for Allergan. FVR is a volunteer director on the board of directors of the British Columbia Schizophrenia Society. He is a member of the Educational Committee of the Clinical TMS Society (unpaid). YD declares no competing interests. JD has received research support from NIH, CIHR, Brain Canada, Ontario Brain Institute, the Klarman Family Foundation, the Arrell Family Foundation, and the Buchan Family Foundation, in-kind equipment support for investigator-initiated trials from MagVenture, is an advisor for Arc Health Partners and Salience Neuro Health, and is a co-founder of Ampa Health. ZJD has received research and in-kind equipment support for an investigator-initiated study through BrainsWay and MagVenture and industry-initiated trials through Magnus; he currently serves on the scientific advisory board for BrainsWay; his work has been supported by Brain Canada, CIHR, NIMH, and the Grant, Kreutzcamp, and Temerty Family Foundations. DMB receives research support from CIHR, NIMH, Wellcome Trust, Patient Centered Outcomes Research Institute, Brain Canada and the Temerty Family through the CAMH Foundation and the Campbell Family Research Institute. He received research support and in-kind equipment support for an investigator-initiated study from BrainsWay Ltd. He was the site principal investigator for three sponsor-initiated studies for BrainsWay Ltd. He also received in-kind equipment support from Magventure for investigator-initiated studies. He received medication supplies for an investigator-initiated trial from Indivior. He is a scientific advisor for Sooma Medical. He is the Co-Chair of the Clinical Standards Committee of the Clinical TMS Society (unpaid). TGR is supported in part by the National Institute on Aging (#R21AG070666; R21AG078972; R01AG088647), National Institute of Mental Health (#R01MH131528), National Institute on Drug Abuse (#R21DA057540), and Health Resources and Services Administration (#R42MC53154-01-00). Dr. Rhee serves as a review committee member for National Institutes of Health (NIH), Patient-Centered Outcomes Research Institute (PCORI) and Substance Abuse and Mental Health Services Administration (SAMHSA) and has received honoraria payments from NIH, PCORI and SAMHSA. Dr. Rhee has also served as a stakeholder/consultant for PCORI and received consulting fees from PCORI. Dr. Rhee serves as an advisory committee member for International Alliance of Mental Health Research Funders (IAMHRF).