Comparative efficacy and safety of intravenous racemic ketamine, repetitive transcranial magnetic stimulation and electroconvulsive therapy for Stage 2 or higher treatment‐resistant depression: A systematic review and network meta‐analysis

Abstract

Although electroconvulsive therapy (ECT) is the standard treatment for Stage 2 or higher treatment‐resistant depression, it has several drawbacks. Repetitive transcranial magnetic stimulation (rTMS) and intravenous racemic ketamine (IV ketamine) have emerged as potential alternatives, but their comparative effectiveness and safety remain unclear due to limited evidence. We systematically searched the Cochrane Central Register of Controlled Trials, MEDLINE, EMBASE, and trial registries for randomized controlled trials (RCTs) up to 18 December 2023. Random‐effects network meta‐analyses were performed to compare the antidepressant efficacy, tolerability, and acceptability of ECT, IV ketamine, and rTMS. Thirty‐five RCTs (n = 2109) were included. Most of the included studies had small sample sizes. There were no significant differences in response rate, remission rate, or tolerability of IV ketamine, rTMS, and ECT, while IV ketamine had significantly higher acceptability compared to rTMS and ECT. The confidence in the evidence for efficacy, tolerability, and acceptability was very low. IV ketamine and rTMS did not demonstrate apparent inferiority to ECT in terms of antidepressant efficacy and safety. Considering the various drawbacks of ECT, both treatments may have the potential to serve as viable alternatives. However, the confidence of the evidence in this study is rated as very low, possibly due to small‐study bias, highlighting the need for further large‐scale studies to strengthen the findings.

Considering the drawbacks of electroconvulsive therapy (ECT), intravenous racemic ketamine (IV ketamine) and repetitive transcranial magnetic stimulation (rTMS) may be preferable alternative treatments as they demonstrate no apparent inferiority in antidepressant efficacy. In addition, IV ketamine may be superior to rTMS in terms of acceptability.

INTRODUCTION

There is an urgent need to establish alternative treatments to antidepressants for treatment‐resistant depression (TRD). TRD is characterized as a major depressive episode with no response or remission after at least two antidepressant trials of adequate dose and duration, referred to as Stage 2 TRD. ¹ TRD has a worse prognosis, including approximately two‐times higher rates of hospitalization and seven‐times higher rates of suicide compared with treatment‐responsive depression. ² Of note, only 19% of patients in interventional trials for TRD met this definition of Stage 2 or higher, suggesting a lack of evidence for this condition. ³

Electroconvulsive therapy (ECT) remains the gold standard treatment for TRD because of its rapid and substantial antidepressant effect. ⁴ However, ECT has the following drawbacks: ineffectiveness in some patients, cognitive dysfunction, the need for the use of muscle relaxants, limited implementation facilities, and social stigma. ⁵ , ⁶ Therefore, there is a need for alternative treatments to ECT.

Repetitive transcranial magnetic stimulation (rTMS) has attracted attention as an alternative treatment to ECT for TRD for more than 20 years. ⁷ It showed significantly better antidepressant efficacy for Stage 2 or higher TRD than sham control ⁸ , ⁹ , ¹⁰ and improved cognitive function in specific domains. ¹¹ In addition, rTMS was not inferior to ECT when comparing the antidepressant effects of rTMS and ECT on Stage 2 or higher TRD. ¹²

Intravenous racemic ketamine (IV ketamine) has recently attracted attention. ¹⁰ It improved cognitive function of patients with TRD. ¹³ Our systematic review ¹⁴ showed that IV ketamine has significantly better antidepressant effects than placebo as well as conventional treatments, such as intranasal esketamine and aripiprazole for TRD; however, only a few small RCTs could be encompassed. In comparison with ECT, two large RCTs have examined the relative efficacy of IV ketamine on MDD. ¹⁵ , ¹⁶ Anand et al. ¹⁵ focused on Stage 2 or higher TRD, where they showed noninferiority of IV ketamine to ECT.

In summary, rTMS and IV ketamine are likely to be noninferior to ECT in the treatment of TRD Stage 2 or higher. Rather, these modalities mitigate the disadvantages of ECT, such as cognitive dysfunction, psychological hurdles, and limited delivery facilities, making them alternative treatment options to ECT. Although no head‐to‐head studies comparing IV ketamine and rTMS have been conducted at this time, Papadimitropoulou et al. ¹⁷ conducted a systematic review and network meta‐analysis (NMA) for Stage 2 or higher TRD. They showed that IV ketamine achieved higher response rates than rTMS at 2 weeks, with no significant difference between ECT and IV ketamine or rTMS. However, they systematically searched over a limited period of time from 2003 to 2014 and were unable to include subsequent important RCTs. ¹⁵ In addition, they did not report comparative tolerability and acceptability of IV ketamine, rTMS, and ECT.

Therefore, we conducted an updated systematic review and NMA with the hypothesis that IV ketamine and rTMS are not inferior to ECT in terms of response rate, remission rate, tolerability, and acceptability for treatment of Stage 2 or higher depression. In addition, we explored whether IV ketamine or rTMS is better in terms of these outcomes.

METHODS

The full protocol was preregistered on the OSF website (an open‐source platform for promoting transparency and reproducibility in research). ¹⁸ The amendments and the reasons for them were also added. The study was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta‐analyses (PRISMA)—network meta‐analysis guideline. ¹⁹

Inclusion criteria for articles in the review

Type of studies

We included RCTs assessing the relative antidepressant efficacy of intravenous racemic ketamine, rTMS, and ECT with placebo/sham control or head‐to‐head comparisons between the three active interventions for the treatment of TRD.

Study participants

Adult patients (aged 18 years and older) diagnosed with a major depressive episode according to standard diagnostic criteria (i.e. Diagnostic and Statistical Manual of Mental Disorders [DSM]‐III, DSM‐III‐R, DSM‐IV, DSM‐IV‐TR, DSM‐5, International Classification of Disease [ICD]‐10, ICD‐11) and resistant to treatment, defined as not responding to treatment with two or more antidepressants of sufficient dose and duration for the current major depressive episode.

Intervention

IV ketamine, rTMS, and ECT both as monotherapy and in combination with psychotropic drugs other than these interventions were eligible. Controls were placebo and sham stimulation. Anesthetics, such as midazolam, were not considered eligible as placebo.

Primary outcomes

Response and remission rates were defined by each study. For outcome measurement scales, we used the following predefined hierarchy: The Montgomery–Åsberg Depression Rating Scale (MADRS) was used first, then the Hamilton Rating Scale for Depression (HRSD), and finally the Beck Depression Inventory (BDI). In addition, safety outcomes were defined as tolerability, defined as the discontinuation rate due to adverse events, and acceptability, defined as the discontinuation rate due to any reason.

Secondary outcomes

Cognitive function, change scores between baseline and endpoints in cognitive function, relapse rates at approximately 6 months postintervention, and continuous change in depressive symptoms from pre‐ to post‐treatment were to be assessed. However, relapse rate and change in cognitive function could not be assessed due to a lack of trials that assessed these outcomes, resulting in no more than three intervention or control groups being included in the network.

Search strategies

We searched the MEDLINE, Cochrane Central Register of Controlled Trials (CENTRAL), EMBASE, International Clinical Trials Registry Platform (ICTRP) and ClinicalTrials.gov databases up to December 18, 2023, with no language or country restrictions (Supplementary Text 1). We also reviewed the reference lists of previous relevant international guidelines (i.e., World Federation of Societies of Biological Psychiatry [WFSBP] guidelines and Canadian Network for Mood and Anxiety Treatments [CANMAT] 2016 clinical guidelines ²⁰ , ²¹ ). Two of the three reviewers (I. T., T. T., and T. S.) screened the titles and abstracts and then conducted full‐text reviews.

Data extraction and quality assessment

Using the predefined form, two of the four reviewers (I. T., T. T., K. E., and T. S.) extracted the data from full text articles independently. The form included information on demographic and clinical characteristics, including primary or secondary outcomes and potential effect modifiers. To reduce extraction time, reviewers referred the extracted data using the GPT‐4‐turbo 2024‐02‐15‐preview ²² via Microsoft Azure. The source code is available (https://github.com/youkiti/ARE/blob/main/various%20projects/Terao_Data_extraction_from_fulltext_github.ipynb). Disagreements were resolved through discussion. We contacted authors for missing data for analysis.

Measures of treatment effects

Pooled odds ratios (ORs) were calculated for binary variables, while pooled standard mean differences (SMDs) were calculated for continuous variables.

Assessment of risk of bias in included studies

Two of the four independent reviewers (I. T., T. T., K. E., and T. S.) independently assessed the risk of bias using Risk of Bias 2 for each primary outcome. ²³ Disagreements were resolved through discussion.

Assessment of reporting bias

We compared the outcomes defined in the trial protocols with the outcomes reported in the publications. Potential publication bias was assessed visually using the funnel plot and statistically using the Egger test. ²⁴

Network meta‐analysis

The transitivity assumption was assessed by comparing clinical and methodological characteristics extracted from the data extraction. Differences in these potential effect modifiers between interventions were explored. A frequentist random effects NMA was performed using MetaInsight, ²⁵ where inconsistency was assessed as well. Heterogeneity was also assessed using Confidence in Network Meta‐Analysis (CINeMA). ²⁶ Where inconsistency or heterogeneity was statistically significant, we explored the reason by assessing the distribution of potential effect modifiers across studies.

Subgroup analysis

Given the rapid onset of the antidepressant effect of intravenous racemic ketamine, ²⁷ subgroup analyses were planned to be conducted separately for periods of ≤1 week and for longer periods, a setting similar to our previous study. ¹⁴ However, the subgroup analysis could not be performed due to an insufficient number of evaluable studies.

Sensitivity analysis

The following sensitivity analyses for the primary outcomes were performed to confirm the robustness:

• 1.Trials with participants with unipolar depression only.
• 2.Trials including only participants without psychotic symptoms.
• 3.Trials including only repeated administrative interventions.
• 4.Trials including only sham stimulation as the control group.
• 5.Trials excluding bilateral and low‐frequency stimulation rTMS protocols.

The sensitivity analyses that were not performed due to insufficient numbers of participants are shown in Supplementary Text 2.

Assessment of the confidence of the evidence

To assess the confidence of each relative treatment effect of the primary outcomes, we used CINeMA approach, ²⁶ focusing on “within‐study bias,” “reporting bias,” “indirectness,” “imprecision,” “heterogeneity,” and “incoherence.” ²⁸ , ²⁹ , ³⁰

RESULTS

Identification of relevant studies

The literature search identified 8417 candidate records, and 35 studies were included after screening and full‐text review (Figure S1; Tables S1, S2). Details of the 48 trials excluded from the full‐text review are described in Table S1. Lack of response to the author's enquiries was the main reason for not being able to retrieve references. The number of trials and participants for the included treatments were as follows: ECT, 263 in four studies; rTMS, 820 in 28 studies; and IV ketamine, 376 in seven studies (Supplementary Text 3). The majority of the included studies had small sample sizes. The mean age and percentage of females in the intervention groups were as follows: 35.6–56.3 years and 45.5%–80% for ECT; 26.9–65.97 years and 17.3%–90.9% for rTMS; and 36.1–48.5 years and 44.4%–89.0% for IV ketamine. Study duration ranged from a single dose to 6 weeks. Most participants were not hospitalized and had no psychotic symptoms or bipolar depression. As potential effect modifiers, there were inevitably differences in intervention methods, although there were no obvious differences in patients' medical conditions, such as diagnosis and severity. Cognitive symptoms and relapse rates were not assessed in most of the included trials.

Risk of bias assessment

The risk of bias was of “some concern” for most studies, mainly due to the lack of protocols (Figure S2). The study by Anand et al., ¹⁵ which had the largest number of participants of the included studies, was rated as high risk because of a possible disproportionate drop‐out before the intervention potentially due to the lack of allocation blinding for patients, which is an unavoidable feature of the distinctly different ECT and IV ketamine interventions.

Results of network meta‐analyses

Primary analyses

The results of the primary analyses are shown in Tables 1 and S3, and Figure S3. IV ketamine, rTMS, and ECT were superior to control for response rate (OR 95% confidence interval [CI], 0.16 [0.08, 0.34], 0.22 [0.14, 0.35], and 0.23 [0.09, 0.62], respectively) (Table 1a) and IV ketamine and rTMS were also superior to control for remission rate (OR 95% CI, 0. 11 [0.02, 0.57] and 0.24 [0.10, 0.55]) (Table 1b). IV ketamine was significantly more acceptable than ECT, rTMS, and control (OR 95% CI, 0.12 [0.05, 0.28], 0.26 [0.11, 0.62] and 0.24 [0.10, 0.55]) (Table 1d). There were no significant differences in the other comparisons (Table 1c).

Table 1.

Results of a primary analysis of IV ketamine, rTMS, and ECT.

Control	.	.	.
(a) Response rates		.	.
0.23 [0.09, 0.62]	ECT
0.22 [0.14, 0.35]	0.93 [0.36, 2.44]	rTMS	.
0.16 [0.08, 0.34]	0.68 [0.28, 1.67]	0.73 [0.32, 1.69]	Ketamine
(b) Remission rates	.	.	.
0.24 [0.10, 0.55]	rTMS	.	.
0.20 [0.03, 1.13]	0.83 [0.14, 4.74]	ECT	.
0.11 [0.02, 0.57]	0.48 [0.09, 2.56]	0.58 [0.15, 2.18]	Ketamine
(c) Tolerability	.	.	.
0.30 [0.08, 1.19]	rTMS	.	.
0.29 [0.06, 1.44]	0.97 [0.12, 8.05]	Ketamine	.
0.24 [0.03, 1.67]	0.79 [0.20, 3.16]	0.82 [0.07, 10.25]	ECT
(d) Acceptability (ketamine)	.	.	.
0.26 [0.11, 0.62]	rTMS	.	.
0.24 [0.10, 0.55]	0.91 [0.60, 1.38]	Control	.
0.12 [0.05, 0.28]	0.47 [0.20, 1.06]	0.51 [0.22, 1.21]	ECT

Note: Bold font indicates statistical significance. (a, b) The number below 1 indicates that the intervention in the row is superior to the other intervention in the column. (c, d) The number below 1 indicates that the intervention in the column is superior to the other intervention in the row. Odds ratios are used for the values in the table.

Abbreviations: ECT, electroconvulsive therapy; IV ketamine, intravenous racemic ketamine; rTMS, repetitive transcranial magnetic stimulation.

Secondary analysis

IV ketamine and rTMS were significantly more effective than the control (SMD 95% CI, −1.66 [−2.61, −0.70] for IV ketamine and −0.55 [−0.96, −0.14] for rTMS) on continuous change in depressive symptoms from pre‐ to post‐treatment. Furthermore, IV ketamine was significantly more effective than rTMS (SMD 95% CI, −1.11 [−2.12, −0.09]) (Tables 2, S4; Figure S4). There were no significant differences in the other comparisons.

Table 2.

Results of a secondary analysis on continuous change of depressive symptoms pre‐ and post‐treatment.

Ketamine	.	.	.
−1.11 [−2.12, −0.09]	rTMS	.	.
−1.27 [−2.74, 0.20]	−0.16 [−1.64, 1.32]	ECT	.
−1.66 [−2.61, −0.70]	−0.55 [−0.96, −0.14]	−0.39 [−1.87, 1.10]	Control

Note: Bold font indicates statistical significance. The number below 0 indicates that the intervention in the column is superior to the other intervention in the row. Standard mean differences are used for the values in the table.

Abbreviations: ECT, electroconvulsive therapy; rTMS, repetitive transcranial magnetic stimulation.

Sensitivity analyses

The results of the sensitivity analyses are shown in Tables S5 and S6. All results of the sensitivity analyses were similar to the results of the primary or secondary analyses.

Confidence of evidence

The confidence of evidence for each outcome was very low to high (Table S7). The most common factors downgrading the confidence of evidence were “within‐study bias” and “imprecision.” Due to the lack of direct comparison trials, “indirectness” was considered a major concern for the comparison of IV ketamine versus ECT or rTMS. Two major concerns in heterogeneity were detected, but the causes could not be identified. In reporting bias, Egger's test of rTMS versus control was significant, but visual inspection of the funnel plot showed that there appeared to be no small study effect, so it was judged to be of “no concern” (Figure S5).

DISCUSSION

This is the updated NMA comparing the antidepressant efficacy of IV ketamine, rTMS, and ECT for the treatment of Stage 2 or higher TRD. In addition, this is the first NMA comparing the tolerability and acceptability of IV ketamine and rTMS. The number of RCTs in IV ketamine increased from one to seven and the number of participants treated with IV ketamine increased from 35 to 376. The number of RCTs in rTMS increased from eight to 28 and the number of participants treated increased from 242 to 820. The number of RCTs in ECT increased from one to four and the number of participants treated increased from 15 to 320. ¹⁷

There was no significant difference in response or remission rates between IV ketamine, rTMS, and ECT, which was robust in the sensitivity analyses. The confidence of the evidence for IV ketamine versus ECT on response rate and remission rate was very low, suggesting that it is not yet possible to conclude which IV ketamine and ECT is better in terms of efficacy. Therefore, IV ketamine and rTMS showed no apparent inferiority in response or remission rates over ECT. The significant superior response rates of IV ketamine over rTMS reported by Papadimitropoulou et al. (OR 0.21, 95% CI [0.04, 0.91]) disappeared. However, IV ketamine significantly outperformed rTMS for the continuous change in depressive symptoms with a large effect size (SMD 95% CI, −1.11 [−2.12, −0.09]), which was also robust in the sensitivity analysis excluding single administration studies. This implies a possible superiority of IV ketamine over rTMS for continuous change in depressive symptoms but this cannot be concluded because the upper limit of its 95% CI is close to zero and this is a secondary analysis.

There was no significant difference between IV ketamine, rTMS, and ECT in tolerability. IV ketamine was more acceptable than ECT, rTMS, and control, which was robust in the sensitivity analyses, particularly those restricted to repeated‐dose trials. Taken together, IV ketamine and rTMS are alternative treatments whose antidepressant effects and safety are comparable to ECT.

Most participants in the study were middle‐aged outpatients with unipolar TRD without psychosis. Therefore, the results of this study cannot be extrapolated to patients who are younger or older, severe enough to require hospitalization, or who have bipolar or psychotic depression. In addition, the clinical feasibility of IV ketamine, rTMS, and ECT is influenced by multiple factors, including insurance coverage, treatment costs, and the healthcare setting. IV ketamine, not the US FDA‐approved for depression, is often not covered by insurance, leading to significant out‐of‐pocket expenses for patients seeking treatment in specialty clinics. However, its ability to provide rapid antidepressant effects after a single administration may render it economically advantageous in certain contexts. While rTMS is covered by some insurance plans, the high cost associated with multiple treatment sessions may still present substantial financial barriers. ECT, typically covered by insurance, necessitates hospitalization, thereby limiting its availability to certain healthcare facilities.

LIMITATIONS

This study has several limitations: (1) There were some unavoidable violations of the transitivity assumption, such as different routes of administration and study durations. In addition, there were no head‐to‐head RCTs between IV ketamine and rTMS, and their relative relationships depend only on indirect comparisons, although their indirect comparisons are doubly obtained via control or ECT. Direct‐comparison RCTs are needed. (2) The RCTs of IV ketamine included single‐dose studies. ³¹ , ³² , ³³ Although this introduces heterogeneity, the inclusion of single‐dose trials in this meta‐analysis is a disadvantage for IV ketamine given the results of repeated‐dose trials of IV ketamine showing that the effect tends to increase with increasing number of doses. ¹⁵ , ³⁴ , ³⁵ Nevertheless, it is noteworthy that IV ketamine did not differ significantly from ECT in antidepressant efficacy and significantly outperformed rTMS in continuous values. Furthermore, the results of the sensitivity analysis excluding single‐dose trials were similar to those of the primary and secondary analyses. (3) The significant superiority of the antidepressant effect of ECT over controls was not confirmed for remission rates and continuous values. This was also similar to the study by Papadimitropoulou et al. ¹⁷ The lack of sham‐controlled RCTs of ECT for TRD Stage 2 or higher may lead to a lack of power, which may be due to ethical reasons, as patients for whom ECT is recommended are usually critically ill, including those with imminent suicidal ideation, and sham stimulation would result in missed treatment opportunities, as well as the administration of muscle relaxants and anesthetics with the risk of adverse effects. ³⁶ (4) Ketamine raises concerns regarding misuse, abuse, dependence, and tolerance, with its involvement in μ‐opioid receptor activation. ³⁷ This meta‐analysis does not include an analysis of such adverse effects. However, a systematic review of 2174 patients with major depressive disorder who received ketamine identified only four cases of tolerance or dependence‐related phenomena, suggesting a low prevalence. ³⁸ This may be associated with the use of subanesthetic ketamine doses for the treatment of depression, which are typically lower than those linked to ketamine use disorder. ³⁸ Dissociation, nausea, headache, elevated heart rate, and blood pressure are transient and do not persist long‐term, whereas prolonged high‐dose use has been reported to be associated with potential cognitive impairment. ³⁹ Therefore, continuous monitoring are crucial to ensuring its safe use. (5) A large proportion of the included studies had small sample sizes, potentially introducing small‐study bias that may have impacted the confidence of the evidence, underscoring the need for further large‐scale studies. (6) In several studies, concerns were raised regarding the risk of bias due to the lack of protocols, which could lead to potential inconsistencies in study design and reporting, ultimately affecting the reliability and comparability of findings. Moreover, the high‐risk‐of‐bias studies may have influenced the direction of the conclusions. In particular, a large‐scale direct comparison study, ¹⁵ which demonstrated a stronger antidepressant effect of ketamine over ECT, exhibited a high risk of bias, requiring careful interpretation of the results of the primary analysis. Therefore, we conducted a post‐hoc sensitivity analysis excluding studies with a high risk of bias and confirmed that the results were similar to those of the primary analysis (Table S8). (7) The confidence of the evidence for the primary outcome was rated as very low, and no firm conclusions can be drawn. Further accumulation of high‐quality RCTs are warranted.

CONCLUSION

This systematic review and NMA assessed the clinical benefits with greater reliability and more comprehensiveness by substantially increasing the number of studies and subjects included, as well as including tolerability and acceptability, compared to the previous NMA. ¹⁷ IV ketamine and rTMS may be preferable alternative treatments to ECT for middle‐aged outpatients with unipolar TRD Stage 2 or higher without psychosis. However, due to the very low confidence of the evidence, possibly attributable to small‐study bias, definitive conclusions cannot be drawn, emphasizing the need for further large‐scale studies.

AUTHOR CONTRIBUTIONS

Conception and design of the study: Itsuki Terao, Takahiro Tsuge, Tomoo Sato, Kaori Endo, Kazumasa Kotake, Masahiro Banno. Acquisition and analysis of data: Itsuki Terao, Takahiro Tsuge, Tomoo Sato, Kaori Endo, Kazumasa Kotake, Masahiro Banno, Yuki Kataoka. Drafting the manuscript or figures: Itsuki Terao, Takahiro Tsuge, Tomoo Sato, Kaori Endo, Kazumasa Kotake, Masahiro Banno, Yuki Kataoka. Others: Itsuki Terao, Takahiro Tsuge, Tomoo Sato, Kaori Endo, Kazumasa Kotake, Masahiro Banno, Yuki Kataoka.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflicts of interest.

ETHICS APPROVAL STATEMENT

As this study is a systematic review and network meta‐analysis, it does not involve human subjects, and therefore, ethics approval and patient consent statements are not applicable.

PATIENT CONSENT STATEMENT

N/A.

CLINICAL TRIAL REGISTRATION

N/A.

Supporting information

Figure 1.

Figure 2.

Figure 3. Figure 4. Figure 5.

Text 1.

Text 2.

Text 3.

Table 1.

Table 2.

Table 3.Table 4. Table 5. Table 6.

Table 7.

Table 8.

PRISMAchecklist.

Terao I, Tsuge T, Sato T, Endo K, Kotake K, Banno M, et al. Comparative efficacy and safety of intravenous racemic ketamine, repetitive transcranial magnetic stimulation and electroconvulsive therapy for Stage 2 or higher treatment‐resistant depression: A systematic review and network meta‐analysis. Psychiatry Clin Neurosci Rep. 2025;4:e70136. 10.1002/pcn5.70136

DATA AVAILABILITY STATEMENT

The data sets analyzed in the current study are available from the corresponding author upon reasonable request.