Machine learning in the prediction of treatment response for emotional disorders: A systematic review and meta-analysis

Abstract

Background:

Emotional disorders such as depression and anxiety affect millions globally and pose a significant burden on public health. Personalized treatment approaches using machine learning (ML) to predict treatment response could revolutionize treatment strategies. However, there is limited evidence as to whether ML is successful in predicting treatment outcomes. This meta-analysis aims to evaluate the accuracy of ML algorithms in predicting binary treatment response (responder vs. non-responder) to evidence-based psychotherapies, pharmacotherapies, and other treatments for emotional disorders, and to examine moderators of prediction accuracy.

Methods:

Following PRISMA guidelines, a comprehensive literature search was conducted across PubMed and PsycINFO from January 1st, 2010 to March 27th, 2025. Studies were included if they used ML methods to predict treatment response in patients with emotional disorders. Data were extracted on sample size, type of treatment, predictors used, ML methods, and prediction accuracy. Meta-analytic techniques were used to synthesize findings and identify moderators of prediction accuracy.

Results:

Out of 3816 non-duplicate records, 155 studies met inclusion criteria. The overall mean prediction accuracy was 0.76 (95 % CI: 0.74–0.78), and the mean area under the curve was 0.80 indicating good discrimination. The average sensitivity and specificity were 0.73 and 0.75, respectively. Moderator analyses indicated that studies using more robust cross-validation procedures exhibited higher prediction accuracy. Neuroimaging data as predictors were associated with higher accuracy compared to clinical and demographic data. Moreover, results indicated that studies with larger responder rates, as well as those that did not correct for imbalances in outcome rates, were associated with higher prediction accuracy.

Conclusions:

ML methods show promise in predicting treatment response for emotional disorders, with varying degrees of accuracy depending on the type of predictors used and the rigor of methodological procedures implemented. Future research should focus on improving methodological integrity and exploring the integration of multimodal data to enhance prediction accuracy.

1. Introduction

Emotional disorders, such as depression and anxiety, are among the most prevalent and disabling conditions worldwide. These disorders not only eventuate in significant personal suffering and impairment, but also impose a considerable economic and systemic burden on healthcare systems. Traditional evidence-based treatment approaches– including evidence-based psychotherapies (e.g., cognitive behavioral therapy (CBT)), pharmacotherapies, neuromodulation, etc.– have proven efficacious for many patients. That notwithstanding, a substantial proportion of individuals fail to achieve adequate response to these interventions, underscoring the need for more personalized approaches to treatment planning (e.g., identifying what intervention or treatment strategy a patient should receive at a given time on the basis of individual differences in symptom, environmental, lifestyle, genetic, or other characteristics). Response to “gold-standard” evidence-based therapy is modest at best (response rates = 49.5 % – 65.2 %), and patients often continue to suffer significant disease and economic burden (Lemmens et al., 2019; Loerinc et al., 2015).

A primary reason for the unsatisfactory progress in improving treatment for emotional disorders is our inability to accurately predict treatment response. The advent and adoption of machine learning (ML) in medicine may prove profitable for addressing this need. Recent proposals for improving personalized predictions in psychology have illustrated how ML is particularly suited to optimizing prediction, as opposed to traditional statistical frameworks that focus on explanation and mechanistic knowledge regarding potentially causal relationships between psychological constructs (Yarkoni and Westfall, 2017). A salient advantage of certain ML algorithms, such as support vector machines or neural networks, consists in their ability to better capture nonlinear and complex patterns underlying a given data structure (Kuhn and Johnson, 2013; Pigoni et al., 2024; Winter et al., 2024). This is especially germane to psychological and mental health data, which has been posited to exhibit substantial nonlinear, dynamic structures (Hofmann and Curtiss, 2018; Hofmann, Curtiss and Hayes, 2020). In the context of emotional disorders, predictive modeling procedures such as ML can better facilitate precision medicine predictions of individual responses to various treatments, thereby enabling more targeted and effective interventions (Curtiss et al., 2024). ML’s capability to model individual differences is particularly apposite given the heterogeneous nature of emotional disorders, for which factors influencing treatment response can vary widely among individuals.

Recent studies have explored the use of ML to predict treatment outcomes in emotional disorders, utilizing diverse types of data, including clinical, demographic, genetic, and neuroimaging information. Notwithstanding the relative proliferation of such applied ML studies, several challenges remain. Extant studies have demonstrated varying degrees of success, with some reporting high prediction accuracies while others have yielded more modest results (Curtiss et al., 2024; Sajjadian et al., 2021). The variability in these findings highlights the importance of understanding the conditions under which ML models perform best and identifying the key moderating factors that contribute to accurate treatment response predictions. One principal issue is the relative diversity of ML methodological decisions represented in the applied literature, ranging from disparities in pre-processing procedures, management of class imbalances in the outcome data (i.e., responders versus non-responders), modalities of the features, cross-validation procedures, and innumerable other factors (Lee et al., 2018; Sajjadian et al., 2021). In light of the lack of standardization of methodologies across studies, which is essential for comparing results and drawing generalizable conclusions, a comprehensive survey of current practices and their impact on prediction accuracy is imperative.

The current meta-analysis constitutes the first comprehensive review and quantitative synthesis of existing research on the use of ML to predict treatment response in emotional disorders with a specific focus on identifying moderating factors that influence prediction accuracy. Although prior reviews have examined the potential for ML to improve personalized treatment strategies (Lee et al., 2018; Sajjadian et al., 2021), relatively little attention has been devoted toward understanding how disparate ML methodologies and study characteristics affect prediction performance across the spectrum of emotional disorders. This, accordingly, can furnish crucial insights into precision medicine treatment of mental health disorders and facilitate the adoption of better practices in applied ML research in psychiatry.

2. Methods

2.1. Literature search

A comprehensive literature search was conducted across PubMed and PsycINFO databases from January 1st, 2010 to March 27th, 2025. The search terms included combinations of “machine learning,” “deep learning,” “depression,” “anxiety,”, “social anxiety disorder”, “panic disorder”, “phobia”, “agoraphobia”, “social phobia”, “specific phobia”, “obsessive compulsive disorder”, “post traumatic stress disorder”, “body dysmorphic disorder”, “eating disorder”, “anorexia”, “bulimia”, “treatment,”, “therapy”, “psychotherapy”, “cognitive behavior therapy”, “mindfulness”, “dialectical behavior therapy”, “acceptance and commitment therapy”, “exposure therapy”, “medication”, “pharmacotherapy”, and “biofeedback”. The search strategy aimed to capture all relevant studies that used ML methods to predict treatment response in patients with emotional disorders. For full search terms with inclusion of logical operators refer to the Supplementary Material. This analysis was preregistered with the International Prospective Register of Systematic Reviews (PROSPERO) on October 10th, 2023, and was last updated on October 3rd, 2024 (registration number CRD42023469216).

2.2. Inclusion and exclusion criteria

Studies were included if they satisfied the following criteria: 1) Participants had a diagnosis of an emotional disorder (anxiety, depression, or related disorders). 2) The study used ML methods to predict binary treatment response (responder vs. non-responder) or remission (remitter vs. non-remitter). Reports examining continuous outcomes were not considered, given that binary performance metrics (e.g., accuracy) could not be derived. 3) The study reported binary prediction accuracy metrics (accuracy or area under the curve; AUC). 4) The study included adequate data partitioning methods for ML, dividing the dataset into appropriate training (and potentially validation) and test sets (e.g., simple split, external validation, cross-validation, nested cross-validation). 5) The study was written in English and published in a peer-review journal.

Studies were excluded if they did not meet these criteria or if they focused on non-human subjects or non-ML predictive methods. The principal requirement for binary data is necessitated by the fact that binary metrics such as accuracy and AUC are able to be meta-analyzed given that they are in a standardized metric. Furthermore, treatment outcome constructs such as response and remission convey widely recognized information to clinicians, thereby being readily interpretable. Conversely, meta-analyzing continuous performance metrics are beset with problems given that the vast majority of such metrics quantify error in the unstandardized units of the original scales (e.g., mean absolute error, root mean square error, etc.), which precludes meaningful synthesis into composite estimates given heterogeneity in symptom instruments.

2.3. Data extraction

Data were extracted independently by two reviewers using a standardized procedure, involving collating data according to whether it was a ML performance metric or a potential moderator variable. Discrepancies in whether certain data should be extracted were resolved by a third reviewer. Extracted data included study characteristics (e.g., year of publication, sample size, type of treatment, response type), ML methods (e.g., type of algorithm, validation method, feature selection), predictors used (e.g., clinical, demographic, neuroimaging, genetic), and prediction accuracy metrics.

Data Analysis.

All meta-analytic analyses were conducted in R using the package metafor (Viechtbauer, 2010) and mada (Doebler, 2015). Multilevel random-effects models were employed to account for heterogeneity between studies. Because some studies reported multiple outcomes (e.g., prediction accuracy for more than one ML algorithm, etc.), the data was inherently nested, thereby necessitating the use of a multilevel modeling meta-analytic approach (Cheung, 2014). That is, the assumption of statistical independence, one of the core assumptions of a traditional meta-analysis, is violated when dependency exists between effect sizes (Harrer et al., 2021). Such statistical dependence, when modeled improperly, can eventuate in reduced heterogeneity, which may prompt overly optimistic estimations of positive results (Harrer et al., 2021). Thus, a multilevel modeling approach was adopted to address dependency that would result from nesting and dependency in outcomes within some of the studies. Consistent with recommendations and default specifications (Viechtbauer, 2010), the error covariance structure was specified as compound symmetric to account for the correlated data structure.

Classification accuracy was the primary outcome metric. Prior to the meta-analysis, accuracy needed to be transformed into two components: (1) the absolute number of correct classification events (i.e., number of correctly classified responders and non-responders) and (2) the total number of events (i.e., the sample size) (Viechtbauer, 2010). Meta-analyzing results of percentages by themselves proves problematic, and accuracy is expressed as a percentage in individual studies, requiring the need for such transformation into whole events. Consistent with a prior meta-analysis (Lee et al., 2018), the number of correct classification events was calculated for each study by multiplying the reported classification accuracy proportion and sample size (n) and rounding the product to the nearest whole number. The two components—number of correctly classified events and total number of events—were treated as the numerator and denominator, respectively, to convert to logit transformed proportions (i.e., log-odds) and, thereby, perform a binomial-normal model for the meta-analysis of proportions (Viechtbauer, 2010). The estimated average log-odds from the meta-analysis were back-transformed using the inverse logit to produce results readily interpretable as a proportion reflecting the overall pooled accuracy (Viechtbauer, 2010). As a complement to the aforementioned procedures, a robust-variance estimation (i.e., Satterthwaite approximation through the clubSandwich package) was also conducted using the robust() argument in metafor to account for heteroscedasticity and unmodeled dependence between the errors (Pustejovsky and Tipton, 2018).

Furthermore, AUC, as well as sensitivity and specificity, was calculated and meta-analyzed for studies for which confusion matrix values could be derived (i.e., true positive, false positive, true negative, and false negative values). Consistent with the gold-standard procedures codified in the mada package (Doebler, 2015), a bivariate normal model was estimated for the logit-transformed pairs of sensitivities and false positive rates to produce a meta-analyzed AUC value (Arends et al., 2008; Reitsma et al., 2005). In accordance with proposed guidelines (Hosmer and Lemeshow, 2000), AUC values approaching 0.5 suggest no discrimination, whereas values between 0.7 and 0.8 suggest acceptable discrimination, values between 0.8 and 0.9 are considered excellent, and values greater than 0.9 reflect outstanding discrimination.

Meta-regressions models were conducted to examine potential moderators of prediction accuracy, including proportion of responders, age, sample size, treatment duration, preprocessing for imbalanced outcomes, feature type, feature selection method, validation method, missing data method, ML algorithm method, use of an ensemble of algorithms, type of treatment, type of outcome, sample race/ethnicity, and percentage of females in sample. For most models a random intercept model was used, but random slopes were included when appropriate should a moderator genuinely vary as a Level 1 variable. Because proportion of responders and, consequently, imbalance correction variables are closely related to sample size, the sample size variable was controlled for in those two models to avoid confounds.

Meta-analytic diagnostics are difficult to obtain from study data organized with a nested format. To calculate meta-analytic diagnostics, a separate meta-analytic model was pursued by conducting within-study averaging for a non-nested equal effects model, which involves extracting the marginal variance-covariance matrix of the estimates based on the multilevel meta-analysis model to obtain comparable results to the original multilevel model. To examine the presence of publication bias, we used the fail-safe N method to determine the number of additional studies with a null result needed to reduce the overall effect size to non-significance (Rosenthal, 1991). Furthermore, the rank correlation test for funnel plot asymmetry was employed to assess the correlation between study accuracy estimates and their variances (Begg and Mazumdar, 1994). Finally, a leave-one-out sensitivity analysis was performed to determine the robustness of the effects.

3. Results

3.1. Study characteristics

The initial search yielded 3816 non-duplicate records, of which 155 studies met the inclusion criteria (Fig. 1). The included studies varied in sample size, ranging from 16 to 77,371 participants (mean = 1865, median = 175). The majority of studies focused on pharmacotherapy (n = 90), followed by psychotherapies (n = 27), bio-stimulation (n = 24) and other treatments (e.g., psychedelics, combination therapy and medicine, etc.) (n = 14). In terms of different types of imbalance preprocessing procedures, most studies performed no imbalance correction (n = 125), and those that did used SMOTE (n = 7), down sampling (n = 6), up sampling (n = 5), weighting (n = 5), and other (n = 7). Regarding type of cross-validation procedure, the most frequently reported were k-fold cross-validation (n = 83), simple split (n = 55), and nested cross-validation (n = 16). For one study, it was undisclosed (Bi et al., 2021), but likely out-of-bag error estimate which is a default of the particular random forest algorithm used in the study. For a fuller understanding of study characteristics, please refer to Table 1.

Fig. 1.

Flow chart.

Table 1.

Study characteristics.

Study ID	Author	Year	Accuracy	AUC	N	Diagnosis	Treatment	ML Algorithm	Feature Type	Validation Type
1	Pei et al.	2020	0.86		98	MDD	Medicine	SVM	Combined	cross validation
2	Aderka et al.	2021		0.50	1514	MDD	Psychotherapy	SVM	Behavioral	simple split
3	Hilbert et al.	2021	0.65	0.72	458	OCD	Psychotherapy	Random Forest	Behavioral	simple split
4	Benoit et al.	2022	0.69		3776	MDD	Medicine	SVM	Combined	simple split
5	Kautzky et al.	2021	0.69		504	MDD	Medicine	Random Forest	Behavioral	cross validation
5	Kautzky et al.	2021	0.62		504	MDD	Medicine	Random Forest	Behavioral	cross validation
5	Kautzky et al.	2021	0.69		204	MDD	Medicine	Random Forest	Behavioral	cross validation
5	Kautzky et al.	2021	0.69		131	MDD	Medicine	Random Forest	Behavioral	cross validation
5	Kautzky et al.	2021	0.82		121	MDD	Medicine	Random Forest	Behavioral	cross validation
5	Kautzky et al.	2021	0.81		127	MDD	Medicine	Random Forest	Behavioral	cross validation
6	Rosellini et al.	2023		0.61	1210	MEmD	Psychotherapy	Super Learner	Behavioral	simple split
6	Rosellini et al.	2023		0.71	1210	MEmD	Psychotherapy	Super Learner	Behavioral	simple split
6	Rosellini et al.	2023		0.73	1210	MEmD	Psychotherapy	Super Learner	Behavioral	simple split
7	Grassi et al.	2022	0.72	0.78	287	OCD	Combined (Unspecified)	GB	Behavioral	simple split
8	Maciukiewicz et al.	2018	0.66		186	MDD	Medicine	SVM	Combined	nested cross validation
8	Maciukiewicz et al.	2018	0.52		186	MDD	Medicine	SVM	Combined	nested cross validation
9	Mumtaz et al.	2017	0.92		34	MDD	Medicine	Logistic Regression	Psychophys	cross validation
10	Hoogendoorn et al.	2016		0.78	69	SAD	Psychotherapy	Random Forest	Behavioral	cross validation
11	Benedetti et al.	2021	0.63	0.75	108	TRMDD	Medicine	Elastic Net	Combined	nested cross validation
12	Wallert et al.	2022	0.66		894	MDD	Psychotherapy	Random Forest	Combined	simple split
13	Athreya et al.	2019	0.69		326	MDD	Medicine	Random Forest	Combined	simple split
13	Athreya et al.	2019	0.66		539	MDD	Medicine	Random Forest	Combined	simple split
13	Athreya et al.	2019	0.77		206	MDD	Medicine	Random Forest	Combined	simple split
13	Athreya et al.	2019	0.75		357	MDD	Medicine	Random Forest	Combined	simple split
13	Athreya et al.	2019	0.75		326	MDD	Medicine	Random Forest	Combined	simple split
13	Athreya et al.	2019	0.66		539	MDD	Medicine	Random Forest	Combined	simple split
13	Athreya et al.	2019	0.76		206	MDD	Medicine	Random Forest	Combined	simple split
13	Athreya et al.	2019	0.74		357	MDD	Medicine	Random Forest	Combined	simple split
14	Zhou et al.	2021	0.74	0.73	400	MDD	Medicine	SVM	Behavioral	cross validation
15	Held et al.	2022		0.47	432	PTSD	Psychotherapy	GB	Behavioral	nested cross validation
16	Oakley et al.	2022	0.84	0.84	228	MDD	Medicine	SVM	Psychophys	cross validation
16	Oakley et al.	2022	0.83	0.82	228	MDD	Medicine	SVM	Psychophys	cross validation
17	Yun et al.	2015	0.89		56	OCD	Medicine	SVM	Neuroimaging	simple split
18	Pettorruso et al.	2023	0.66		149	TRMDD	Psychedelic	Random Forest	Behavioral	cross validation
18	Pettorruso et al.	2023	0.69		149	TRMDD	Psychedelic	Random Forest	Behavioral	cross validation
19	Rost et al.	2023	0.60		1102	MDD	Medicine	Elastic Net	Behavioral	simple split
19	Rost et al.	2023	0.41		1102	MDD	Medicine	Elastic Net	Combined	simple split
19	Rost et al.	2023	0.54		1102	MDD	Medicine	Elastic Net	Combined	simple split
20	Zhutovsky et al.	2021	0.76	0.82	40	PTSD	Psychotherapy	SVM	Neuroimaging	cross validation
21	Zhdanov et al.	2020	0.82		115	MDD	Medicine	SVM	Psychophys	cross validation
22	Nie et al.	2018	0.86	0.73	2679	MDD	Medicine	Random Forest	Behavioral	simple split
22	Nie et al.	2018	0.86		3007	MDD	Medicine	Random Forest	Behavioral	simple split
23	Nakajima et al.	2022	0.71		177	MDD_BD	BS	GB	Behavioral	cross validation
24	Harris et al.	2022	0.70		144	MDD	Medicine	Random Forest	Neuroimaging	cross validation
25	Bailey et al.	2019	0.87		42	MDD	BS	SVM	Psychophys	cross validation
26	Metin et al.	2020	0.80		50	OCD	BS	Neural Network	Psychophys	cross validation
27	Taliaz et al.	2021	0.60		1829	MDD	Medicine	SVM	Combined	simple split
27	Taliaz et al.	2021	0.75		1829	MDD	Medicine	SVM	Combined	simple split
27	Taliaz et al.	2021	0.75		1829	MDD	Medicine	SVM	Combined	simple split
27	Taliaz et al.	2021	0.68		1829	MDD	Medicine	SVM	Combined	simple split
28	Joyce et al.	2021	0.78	0.86	348	MDD	Medicine	Penalized Regression	Combined	simple split
29	Flygare et al.	2020	0.78	0.78	88	BDD	Psychotherapy	Random Forest	Behavioral	cross validation
30	Perlis	2013		0.72	2555	MDD	Medicine	Logistic Regression	Behavioral	simple split
31	Kong et al.	2021	0.90		82	MDD	Medicine	CNN	Neuroimaging	cross validation
32	Bartlett et al.	2018	0.64	0.59	184	MDD	Medicine	Random Forest	Combined	cross validation
33	Cao et al.	2018	0.83	0.90	24	MDD	BS	Support Vector Regression (SVR)	Neuroimaging	cross validation
34	Lenhard et al.	2018	0.83		61	OCD	Psychotherapy	Linear Model	Behavioral	cross validation
34	Lenhard et al.	2018	0.83		61	OCD	Psychotherapy	Linear Model	Behavioral	simple split
35	Rost et al.	2022	0.71		1022	MDD	Medicine	Elastic Net	Behavioral	simple split
36	Chen et al.	2023	0.70		291	MDD	Medicine	Random Forest	Combined	cross validation
37	Salem et al.	2023	0.66	0.73	1486	MDD	Medicine	Random Forest	Behavioral	cross validation
37	Salem et al.	2023	0.71	0.78	534	MDD	Medicine	Random Forest	Behavioral	cross validation
37	Salem et al.	2023	0.85	0.84	104	MDD	Medicine	Random Forest	Behavioral	cross validation
38	Nunez et al.	2021	0.65	0.69	3024	MDD	Medicine	Random Forest	Behavioral	simple split
38	Nunez et al.	2021	0.77	0.80	3024	MDD	Medicine	Logistic Regression	Behavioral	simple split
39	Hopman et al.	2021	0.89	1.00	61	TRMDD	BS	SVM	Neuroimaging	simple split
40	Leehr et al.	2021	0.60		171	SP	Psychotherapy	Random Forest	Behavioral	cross validation
41	Rangaprakash et al.	2021	0.86		44	OCD	Psychotherapy	SVM	Neuroimaging	cross validation
42	Hasanzadeh et al.	2019	0.91		46	MDD	BS	KNN	Psychophys	cross validation
43	Shahabi et al.	2023	0.82	0.83	170	MDD	BS	CNN	Psychophys	simple split
44	Tymofiyeva et al.	2019	0.83		30	MDD	Psychotherapy	J48 pruned tree classifier	Combined	cross validation
45	Redlich et al.	2016	0.78		23	MDD	BS	SVM	Neuroimaging	cross validation
46	Nobakhsh et al.	2023	0.90		34	MDD	BS	SVM	Psychophys	cross validation
47	Iniesta et al.	2016		0.72	793	MDD	Medicine	Elastic Net	Behavioral	cross validation
47	Iniesta et al.	2016		0.72	465	MDD	Medicine	Elastic Net	Behavioral	cross validation
47	Iniesta et al.	2016		0.70	328	MDD	Medicine	Elastic Net	Behavioral	cross validation
48	Zandvakili et al.	2019		0.83	35	MDD_PTSD	BS	LASSO	Psychophys	cross validation
48	Zandvakili et al.	2019		0.71	35	MDD_PTSD	BS	LASSO	Psychophys	cross validation
49	Tian et al.	2020	0.79		106	MDD	Medicine	SVM	Neuroimaging	cross validation
50	Khodayari-Rostamabad et al.	2013	0.88		22	MDD	Medicine	SVM	Psychophys	cross validation
51	Gao et al.	2022	0.79	0.83	98	MDD	Medicine	Logistic Regression	Combined	cross validation
52	Bailey et al.	2018	0.91		39	MDD	BS	SVM	Combined	cross validation
53	Gärtner et al.	2021	0.69		71	MAD	BS	SVM	Neuroimaging	cross validation
54	Bone et al.	2021		0.82	77,371	MEmD	Psychotherapy	Logistic Regression	Behavioral	simple split
55	Månsson et al.	2015	0.92	0.91	26	SAD	Psychotherapy	SVM	Neuroimaging	cross validation
56	Shumake et al.	2021	0.62	0.66	1663	MDD	Medicine	Meta Learner	Behavioral	cross validation
56	Shumake et al.	2021	0.63	0.66	1663	MDD	Medicine	Meta Learner	Combined	cross validation
56	Shumake et al.	2021	0.61	0.66	1663	MDD	Medicine	Meta Learner	Combined	cross validation
57	Chang et al.	2019	0.85		121	MDD	Medicine	Neural Network	Combined	simple split
58	Wang et al.	2023		0.71	110	MDD	Medicine	Random Forest	Molecular	cross validation
59	Patel et al.	2015	0.89		33	MDD	Medicine	ADTree	Behavioral	nested cross validation
60	Nguyen et al.	2022		0.62	106	MDD	Medicine	Neural Network	Combined	nested cross validation
60	Nguyen et al.	2022		0.60	106	MDD	Medicine	Neural Network	Combined	nested cross validation
60	Nguyen et al.	2022		0.67	116	MDD	Medicine	Neural Network	Combined	nested cross validation
60	Nguyen et al.	2022		0.65	116	MDD	Medicine	Neural Network	Combined	nested cross validation
60	Nguyen et al.	2022		0.57	37	MDD	Medicine	Neural Network	Combined	nested cross validation
60	Nguyen et al.	2022		0.71	37	MDD	Medicine	Neural Network	Combined	nested cross validation
61	Carrillo et al.	2018	0.85		17	TRMDD	Psychedelic	GNBC	Behavioral	cross validation
62	Bossarte et al.	2023		0.66	658	MDD	Medicine + Psychotherapy	Super Learner	Behavioral	simple split
63	Vetter et al.	2022		0.83	239	MDD	Psychotherapy	GB	Behavioral	simple split
64	Kannampallil et al.	2022	0.58	0.67	235	MDD	Psychotherapy	SVM	Behavioral	simple split
64	Kannampallil et al.	2022	0.62	0.71	235	MDD	Psychotherapy	SVM	Behavioral	simple split
65	Kim et al.	2023	0.94		24	MDD	Medicine	Neural Network	Smart Phone	cross validation
66	Lin et al.	2020		0.83	421	MDD	Medicine	GB	Combined	cross validation
66	Lin et al.	2020		0.81	421	MDD	Medicine	GB	Combined	cross validation
67	Athreya et al.	2018	0.64	0.68	99	MDD	Medicine	Linear Model	Combined	nested cross validation
67	Athreya et al.	2018	0.68	0.78	99	MDD	Medicine	SVM	Combined	nested cross validation
67	Athreya et al.	2018	0.72	0.74	191	MDD	Medicine	SVM	Combined	nested cross validation
67	Athreya et al.	2018	0.54	0.53	191	MDD	Medicine	SVM	Combined	nested cross validation
68	Ebrahimzadeh et al.	2023	0.94		88	MDD	BS	SVM	Psychophys	cross validation
69	Tsai et al.	2022	0.69	0.83	70	MDD	Medicine	Neural Network	Combined	simple split
70	Browning et al.	2019	0.60		296	MDD	Medicine	SVM	Behavioral	simple split
71	Hornstein et al.	2021	0.71	0.60	1236	MEmD	Psychotherapy	Random Forest	Behavioral	simple split
72	Lissemore et al.	2022	0.75		76	MDD	BS + Medicine	SVM	Combined	cross validation
73	Goyal et al.	2022	0.72		189	BED	Medicine	GNBC	Behavioral	simple split
74	van der Does et al.	2023	0.73	0.69	292	MSMI	BS	Logistic Regression	Behavioral	simple split
75	Díaz-Zuluaga et al.	2023		0.89	172	BD	Medicine	CART	Combined	simple split
76	Kim et al.	2022	0.82	0.93	48	PTSD	BS	SVM	Psychophys	cross validation
77	Squires et al.	2023	0.94		61	MDD	BS	Neural Network	Neuroimaging	simple split
77	Squires et al.	2023	0.63		133	MDD	BS	Neural Network	Behavioral	simple split
78	Grzenda et al.	2021		0.79	67	MDD	Medicine + Psychotherapy	Random Forest	Behavioral	cross validation
78	Grzenda et al.	2021		0.81	67	MDD	Medicine + Psychotherapy	SVM	Neuroimaging	cross validation
78	Grzenda et al.	2021		0.84	67	MDD	Medicine + Psychotherapy	Random Forest	Combined	cross validation
79	Coley et al.	2021		0.61	5554	MDD	Psychotherapy	Random Forest	Behavioral	simple split
80	Dougherty et al.	2023	0.85	0.88	101	TRMDD	Psychedelic	Logistic Regression	Behavioral	cross validation
80	Dougherty et al.	2023	0.88	0.85	90	TRMDD	Psychedelic	Logistic Regression	Behavioral	cross validation
81	Li et al.	2023a	0.82	0.86	40	TRMDD	BS	SVM	Neuroimaging	simple split
82	Li et al.	2023	0.61		163	MDD	BS	SVM	Psychophys	simple split
82	Li et al.	2023	0.46		163	MDD	BS	SVM	Psychophys	simple split
83	Kambeitz et al.	2020	0.65	0.65	94	MDD	BS + Medicine	GB	Behavioral	cross validation
83	Kambeitz et al.	2020	0.60	0.64	91	MDD	BS + Medicine	GB	Behavioral	cross validation
83	Kambeitz et al.	2020	0.43	0.47	60	MDD	BS + Medicine	GB	Behavioral	cross validation
84	Delgadillo et al.	2020		0.59	1435	MDD	Psychotherapy	Elastic Net	Behavioral	simple split
84	Delgadillo et al.	2020		0.65	1435	MDD	Psychotherapy	Elastic Net	Behavioral	simple split
85	Chekroud et al.	2016	0.60		2410	MDD	Medicine	GB	Behavioral	simple split
85	Chekroud et al.	2016	0.60		2410	MDD	Medicine	GB	Behavioral	simple split
85	Chekroud et al.	2016	0.51		2410	MDD	Medicine	GB	Behavioral	simple split
86	Curtiss et al.	2023	0.73	0.77	97	BDD	Medicine	SVM	Behavioral	cross validation
86	Curtiss et al.	2023	0.70	0.75	97	BDD	Medicine	SVM	Behavioral	cross validation
86	Curtiss et al.	2023	0.76	0.79	97	BDD	Medicine	SVM	Behavioral	cross validation
87	Reggente et al.	2018	0.70		42	OCD	Psychotherapy	SVM	Neuroimaging	cross validation
87	Reggente et al.	2018	0.68		42	OCD	Psychotherapy	SVM	Neuroimaging	cross validation
88	Forrest et al.	2023		0.59	191	BED	Psychotherapy	Elastic Net	Combined	cross validation
89	Zhang et al.	2020	0.76		606	MDD	Medicine	SVM	Behavioral	simple split
89	Zhang et al.	2020	0.88		606	MDD	Medicine	SVM	Combined	simple split
90	Kautzky et al.	2017	0.71		552	MDD	Medicine	Random Forest	Behavioral	simple split
91	Bruin et al.	2024	0.64	0.70	189	MDD	BS	SVM	Combined	cross validation
91	Bruin et al.	2024	0.77	0.80	109	MDD	BS	SVM	Combined	cross validation
92	Ma et al.	2023	0.65	0.78	85	MDD	Medicine	SVM	Neuroimaging	cross validation
92	Ma et al.	2023	0.73	0.80	90	MDD	Medicine	SVM	Neuroimaging	cross validation
92	Ma et al.	2023	0.79	0.80	121	MDD	Medicine	SVM	Neuroimaging	cross validation
92	Ma et al.	2023	0.89	0.95	85	MDD	Medicine	SVM	Neuroimaging	cross validation
92	Ma et al.	2023	0.87	0.94	90	MDD	Medicine	SVM	Neuroimaging	cross validation
92	Ma et al.	2023	0.85	0.89	121	MDD	Medicine	SVM	Neuroimaging	cross validation
93	Tsai et al.	2023	0.91	0.95	58	MDD	BS	KNN	Psychophys	simple split
93	Tsai et al.	2023	0.89	0.92	55	MDD	Medicine	KNN	Psychophys	simple split
94	Sajjadian et al.	2023	0.62	0.64	192	MDD	Medicine	SVM	Combined	nested cross validation
94	Sajjadian et al.	2023	0.61	0.61	192	MDD	Medicine	Random Forest	Molecular	nested cross validation
94	Sajjadian et al.	2023	0.59	0.63	192	MDD	Medicine	Random Forest	Combined	nested cross validation
95	Cearns et al.	2022	0.63	0.66	1034	BD	Medicine	Random Forest	Behavioral	simple split
95	Cearns et al.	2022	0.70	0.70	1034	BD	Medicine	Random Forest	Combined	simple split
96	Puac-Polanco et al.	2023		0.66	660	MDD	Medicine	Super Learner	Behavioral	simple split
97	Ziobrowski et al.	2023		0.65	807	MDD	Psychotherapy	LASSO	Behavioral	simple split
98	Solomonov et al.	2021	0.80		221	MDD	Psychotherapy	Random Forest	Behavioral	simple split
99	Bao et al.	2021	0.63		83	MDD	Psychedelic	Logistic Regression	Combined	nested cross validation
100	Bi et al.	2021		0.77	534	MDD	Medicine	Random Forest	Combined	undisclosed*
100	Bi et al.	2021		0.75	534	MDD	Medicine	Random Forest	Combined	udisclosed*
101	Takamiya et al.	2020	0.70		27	MDD	BS	SVM	Behavioral	cross validation
101	Takamiya et al.	2020	0.93		27	MDD	BS	SVM	Combined	cross validation
102	Brown et al.	2020	0.80		20	MDD_BD	BS	GNBC	Neuroimaging	cross validation
103	Qi et al.	2020		0.57	138	MDD	Medicine	GB	Combined	simple split
104	Corlier et al.	2019	0.62	0.53	109	MDD	BS	Elastic Net	Neuroimaging	cross validation
104	Corlier et al.	2019	0.65	0.58	109	MDD	BS	Elastic Net	Neuroimaging	cross validation
104	Corlier et al.	2019	0.69	0.66	109	MDD	BS	Elastic Net	Neuroimaging	cross validation
104	Corlier et al.	2019	0.77	0.76	68	MDD	BS	Elastic Net	Neuroimaging	cross validation
105	Bartova et al.	2019	0.85		2762	MDD	Medicine	Random Forest	Combined	simple split
105	Bartova et al.	2019	0.75		2762	MDD	Medicine	Random Forest	Combined	simple split
106	Jaworska et al.	2019	0.88	0.90	51	MDD	Medicine	Random Forest	Combined	cross validation
107	Lin et al.	2018		0.82	421	MDD	Medicine	Neural Network	Combined	cross validation
107	Lin et al.	2018		0.81	421	MDD	Medicine	Neural Network	Combined	cross validation
108	Leaver et al.	2018	0.68		46	MDD	BS	SVM	Neuroimaging	cross validation
109	Ball et al.	2014	0.69		48	MEmD	Psychotherapy	Random Forest	Behavioral	cross validation
109	Ball et al.	2014	0.79		48	MEmD	Psychotherapy	Random Forest	Neuroimaging	cross validation
109	Ball et al.	2014	0.73		48	MEmD	Psychotherapy	Random Forest	Combined	cross validation
110	Sundermann et al.	2017	0.54		59	PD	Psychotherapy	SVM	Neuroimaging	cross validation
110	Sundermann et al.	2017	0.67		54	PD	Psychotherapy	SVM	Neuroimaging	cross validation
111	Guilloux et al.	2015	0.76		34	MDD	Medicine	SVM	Molecular	simple split
111	Guilloux et al.	2015	0.97		34	MDD	Medicine	SVM	Combined	simple split
112	Hahn et al.	2015	0.82		49	PD	Psychotherapy	GPC	Neuroimaging	nested cross validation
113	Perlman et al.	2024	0.66	0.70	5032	MDD	Medicine	Neural Network	Behavioral	simple split
114	Sun et al.	2024	0.77	0.86	86	MDD	BS	Logistic Regression	Neuroimaging	cross validation
115	Mathai et al.	2024	0.80	0.81	11,441	MEmD	Psychedelic	Random Forest	Behavioral	nested cross validation
116	Hilbert et al.	2024	0.51		220	MEmD	Psychotherapy	Meta Learner	Combined	simple split
116	Hilbert et al.	2024	0.56		190	SP	Psychotherapy	Meta Learner	Combined	nested cross validation
117	Wang et al.	2024b	0.68	0.65	225	MDD	Medicine	Meta Learner	Combined	simple split
118	Curtiss et al.	2024	0.76	0.70	239	MDD	Medicine	Super Learner	Behavioral	nested cross validation
118	Curtiss et al.	2024	0.59	0.55	279	MDD	Medicine	Super Learner	Behavioral	nested cross validation
118	Curtiss et al.	2024	0.66	0.53	286	MDD	Medicine	Super Learner	Behavioral	nested cross validation
118	Curtiss et al.	2024	0.71	0.65	85	MDD	Medicine + Psychotherapy	Super Learner	Behavioral	nested cross validation
118	Curtiss et al.	2024	0.80	0.72	62	MDD	Psychotherapy	Super Learner	Behavioral	nested cross validation
118	Curtiss et al.	2024	0.70	0.59	238	MDD	Medicine	Super Learner	Behavioral	nested cross validation
118	Curtiss et al.	2024	0.71	0.60	250	MDD	Medicine	Super Learner	Behavioral	nested cross validation
119	Bian et al.	2024	0.75	0.76	69	MDD	Medicine	GNBC	Combined	nested cross validation
120	Lee et al.	2024a	0.80	0.79	247	MDD	Medicine	GB	Combined	simple split
121	Copa et al.	2024		0.95	16	TRMDD	Psychedelic	GB	Neuroimaging	cross validation
122	Mirjebreili et al.	2024	0.98		30	MDD	Medicine	CNN	Neuroimaging	cross validation
123	Poirot et al.	2024	0.52	0.56	109	MDD	Medicine	GB	Neuroimaging	simple split
123	Poirot et al.	2024	0.64	0.68	109	MDD	Medicine	GB	Neuroimaging	simple split
124	Ramasubbu et al.	2024	0.84		19	MDD_BD	BS	GNBC	Combined	cross validation
125	Hammelrath et al.	2024	0.70	0.77	1591	MDD	Psychotherapy	Random Forest	Behavioral	nested cross validation
126	Xu et al.	2023		0.77	808	MDD	Medicine	GB	Behavioral	nested cross validation
127	Lu et al.	2023	0.99		17	MDD	Medicine	CNN	Psychophys	cross validation
128	Ebrahimzadeh et al.	2024	0.95		106	MDD	BS	SVM	Psychophys	cross validation
129	Prasad et al.	2023	0.78		45,352	MDD	Psychotherapy	Neural Network	Behavioral	simple split
129	Prasad et al.	2023	0.77		45,352	MDD	Psychotherapy	Neural Network	Behavioral	simple split
129	Prasad et al.	2023	0.77		45,352	MDD	Psychotherapy	Neural Network	Behavioral	simple split
129	Prasad et al.	2023	0.86		45,756	Anxiety	Psychotherapy	Neural Network	Behavioral	simple split
129	Prasad et al.	2023	0.83		45,756	Anxiety	Psychotherapy	Neural Network	Behavioral	simple split
129	Prasad et al.	2023	0.82		45,756	Anxiety	Psychotherapy	Neural Network	Behavioral	simple split
130	Duan et al.	2023	0.74	0.74	198	MDD_BD	Medicine	Neural Network	Neuroimaging	cross validation
131	Wang et al.	2024c	0.96	1.00	98	MDD	Medicine	SVM	Combined	cross validation
132	Christ et al.	2023		0.64	235	PTSD	Psychotherapy	GB	Behavioral	nested cross validation
132	Christ et al.	2023		0.57	465	PTSD	Psychotherapy	SVM	Behavioral	nested cross validation
133	Li et al.	2023b	0.74		62	MDD	BS	Random Forest	Psychophys	cross validation
133	Li et al.	2023	0.71		69	MDD	BS	Random Forest	Psychophys	cross validation
134	Mizrahi et al.	2023	0.96		24	BD	Medicine	SVM	Genetic	nested cross validation
135	Ju et al.	2023	0.80		110	MDD	Medicine	SVM	Neuroimaging	cross validation
136	Carr et al.	2025		0.818	214	MDD	Medicine	Logistic Regression	Combined	cross validation
136	Carr et al.	2025		0.834	196	MDD	Medicine	Logistic Regression	Combined	cross validation
136	Carr et al.	2025		0.844	714	MDD	Medicine	Logistic Regression	Combined	cross validation
137	Ravan et al.	2024	0.91	0.91	105	MDD	Medicine	CNN	Psychophys	cross validation
137	Ravan et al.	2024	0.95	0.95	119	MDD	Medicine	CNN	Psychophys	cross validation
137	Ravan et al.	2024	0.87	0.90	25	MDD	Medicine	CNN	Psychophys	cross validation
138	Bertie et al.	2024		0.69	2114	Anxiety	Psychotherapy	Neural Network	Behavioral	cross validation
138	Bertie et al.	2024		0.68	2114	Anxiety	Psychotherapy	Neural Network	Behavioral	cross validation
139	Xiao et al.	2025	0.69		20	BD	BS	Neural Network	Psychophys	cross validation
140	LoParo et al.	2025	0.61		248	MDD	Medicine	Linear Model	Behavioral	cross validation
140	LoParo et al.	2025	0.81		248	MDD	Medicine	Linear Model	Behavioral	cross validation
140	LoParo et al.	2025	0.73		248	MDD	Psychotherapy	Linear Model	Behavioral	cross validation
141	Chai et al.	2024	0.70	0.76	4067	Anxiety	Psychotherapy	GB	Behavioral	simple split
142	Zhao et al.	2025	0.97	0.99	117	MDD	BS	SVM	Combined	cross validation
143	Pae et al.	2024	0.85	0.84	53	PD	Medicine	SVM	Neuroimaging	cross validation
144	Metin et al.	2025	0.73	0.78	120	MDD	Medicine	CNN	Psychophys	simple split
145	Semple et al.	2024	0.61	0.65	2074	MDD	BS	SVM	Behavioral	cross validation
146	Zhao et al.	2024	0.97	0.96	117	MDD	BS	Adaboost	Psychophys	cross validation
147	Kim et al.	2024	0.91	0.91	98	MDD	Medicine	CNN	Blood	cross validation
148	Calabrò et al.	2025	0.65		820	MDD	Medicine	Tree Classifier	Combined	cross validation
149	Lee et al.	2024b	0.94		52	MDD	Medicine	SVM	Combined	cross validation
150	Marrero-Polanco et al.	2025		0.70	505	BD	Medicine	GB	Behavioral	simple split
150	Marrero-Polanco et al.	2025		0.75	505	BD	Medicine	Penalized Regression	Behavioral	simple split
150	Marrero-Polanco et al.	2025		0.73	505	BD	Medicine	GB	Behavioral	simple split
151	Choi et al.	2024	0.87	0.83	31	MDD	Medicine	Linear Model	Psychophys	cross validation
152	Ho et al.	2025	0.73	0.77	64	MDD	Medicine	GNBC	Neuroimaging	cross validation
153	Zhukovsky et al.	2025	0.63	0.62	363	MDD	Medicine	Elastic Net	Combined	simple split
153	Zhukovsky et al.	2025	0.65	0.67	363	MDD	Medicine	Elastic Net	Combined	simple split
153	Zhukovsky et al.	2025	0.64	0.66	363	MDD	Medicine	Elastic Net	Combined	simple split
154	Wang et al.	2025	0.88		30	MDD	Medicine	SVM	Psychophys	simple split
155	Poirot et al.	2025	0.56		262	MDD	Medicine	SVM	Neuroimaging	cross validation
155	Poirot et al.	2025	0.54		262	MDD	Medicine	GB	Neuroimaging	cross validation
155	Poirot et al.	2025	0.51		262	MDD	Medicine	Neural Network	Neuroimaging	cross validation

Note: *For one study (Bi et al., 2021), the cross-validation procedure was not explicitly detailed, but was likely out-of-bag error estimation, which is a default of the particular random forest algorithm used in that study. The list of abbreviations provided represents various mental health conditions, where MDD stands for Major Depressive Disorder, MEmD indicates Mixed Emotional Disorders, SAD refers to Social Anxiety Disorder, TRMDD denotes Treatment Resistant Major Depressive Disorder, BDD represents Body Dysmorphic Disorder, PD signifies Panic Disorder, SP stands for Specific Phobia, MAD indicates Major Affective Disorder, MDD_BD represents Major Depressive Disorder and Bipolar Disorder, MDD_PTSD denotes Major Depressive Disorder and Post Traumatic Stress Disorder, BED refers to Binge Eating Disorder, MSMI indicates Mixed Severe Mental Illness, BDD also represents Bipolar Disorder, and GAD_PD signifies Generalized Anxiety Disorder or Panic Disorder. Regarding the treatment type, BS denotes brain stimulation.

3.2. Prediction accuracy

The overall mean prediction accuracy across studies was 0.76 (95 % CI: 0.74–0.78). Moreover, the test for heterogeneity was statistically significant (Q(df = 208) = 3919.62, p < 0.0001), indicating the presence of potentially meaningful individual differences between study accuracy rates. Refer to Fig. 2 for the full forest plot.

Fig. 2.

Forest plot.

Notes: These effects report one pooled effect per study, as determined by the multilevel random-effects model meta-analysis. Please note there might be slight discrepancies in the accuracy values in the Forest Plot and Table 1 due to rounding error. The meta-analyzed results presented in the current figure represent the accuracies calculated from inverse logit back-transformation.

The mean AUC was 0.80 with a range of 0.47 to 0.99 indicating adequate to good discrimination, and the sensitivity and specificity were 0.73 (95 % CI: 0.69–0.77) and 0.75 (95 % CI: 0.72–0.78), respectively.

3.3. Moderators of prediction accuracy

Moderator analyses revealed several factors influencing prediction accuracy (all presented as exponentiated beta coefficients). For a complete description, please refer to Table 2 and Fig. 3 for exponentiated beta coefficients of all meta-regression models. Salient moderators of interest include certain ML methodological properties and clinical characteristics. Larger percentage of responders (β = 1.02, p < 0.001) was significantly associated with higher prediction accuracy. The use of imbalance preprocessing (β = 0.59, p < 0.05) and addressing missing data (β = 0.58, p < 0.001) were associated with more conservative accuracy estimates. Regression-based ML algorithms (e.g., elastic net) performed comparably to most other sophisticated algorithms (p’s > 0.05). Of note, a supplementary analysis indicated that there may be a significant interaction effect between sample size and the comparison between neural networks and regression (b = 1.0008, p < 0.001). That is, neural network algorithms are associated with greater prediction accuracy under conditions of increased sample sizes (see Supplementary Section). Cross-validation procedures such as k-fold and nested cross-validation were associated with greater accuracy than simple split procedures (β = 1.37, p < 0.01).

Table 2.

Moderator Meta-Regression Analyses.

Term	b	SE	t	p	95 % CI	Model
Percentage Responders	1.02	0.00	18.46	< 0.001***	[1.02, 1.03]	Model 1
Age	1.00	0.01	−0.76	0.450	[0.98, 1.01]	Model 2
N	1.00	0.00	−0.01	0.995	[1.00, 1.00]	Model 3
Tx Duration (Weeks)	1.01	0.00	2.83	< 0.01**	[1.00, 1.02]	Model 4
Imbalance Preprocessing (Yes)	0.59	0.21	−2.52	<0.05*	[0.40, 0.89]	Model 5
Feature Type (Combined)	0.75	0.10	−3.00	< 0.01**	[0.62, 0.90]	Model 6
Feature Type (Behavior)	0.70	0.10	−3.62	< 0.001***	[0.58, 0.85]	Model 6
Feature Selection (None)	0.95	0.06	−1.00	0.318	[0.85, 1.05]	Model 7
Validation Type (CV/Nested CV)	1.37	0.10	3.10	< 0.01 **	[1.12, 1.67]	Model 8
Missing Data Solution (Yes)	0.58	0.10	−5.22	< 0.001***	[0.48, 0.71]	Model 9
Algorithm (GB)	0.86	0.18	−0.88	0.380	[0.61, 1.21]	Model 10
Algorithm (Neural Net)	0.99	0.17	−0.09	0.932	[0.71, 1.37]	Model 10
Algorithm (Other)	0.94	0.21	−0.31	0.758	[0.62, 1.41]	Model 10
Algorithm (SVM)	1.04	0.13	0.33	0.744	[0.81, 1.34]	Model 10
Algorithm (Tree)	0.80	0.14	−1.67	0.095	[0.61, 1.04]	Model 10
Ensemble Approach (Yes)	0.80	0.08	−2.83	< 0.01**	[0.69, 0.93]	Model 11
Diagnosis (Depression)	0.70	0.02	−21.54	< 0.001***	[0.67, 0.72]	Model 12
Tx Type (Medicine)	0.77	0.14	−1.88	0.060	[0.58, 1.01]	Model 13
Tx Type (Other)	0.83	0.28	−0.66	0.512	[0.48, 1.44]	Model 13
Tx Type (Psychotherapy)	0.81	0.16	−1.27	0.205	[0.59, 1.12]	Model 13
Outcome (Response)	1.07	0.05	1.27	0.202	[0.96, 1.19]	Model 14
Ethnicity (Western)	0.62	0.12	−3.96	< 0.001***	[0.49, 0.79]	Model 15
Sex (% Female)	0.997	0.00	−1.83	0.068	[0.995, 1.0002]	Model 16

Notes: Definitions of abbreviations are as follows: b = Beta Value; SE = Standard Error; CI = Confidence Interval;Tx = Treatment; CV = Cross Validation; GB = Gradient Boosting; SVM = Support Vector Machine. All coefficients are exponentiated values to represent odds ratios.

Reference variables for models with categorical variables: Model 5 = no; model 6 = biological; model 7 = yes; model 8 = simple Split; model 9 = no; model 10 = regression-based algorithms; model 11 = no; model 12 = non-depression disorders (e.g., anxiety); model 13 = bio-stimulation; model 14 = remission; model 15 = non-Western

Model 3 included both a random intercept and coefficient term, as the number of subjects varied across models presented within the same studies. Models 1 and 5 included sample size (N) as a covariate to control for its confounding influence, though for ease of presentation those coefficients are omitted from the table.

^*≤0.05.

^**≤0.01.

^***≤0.001.

Fig. 3.

Moderator analyses.

Notes: These represent exponentiated coefficients to be interpreted as odds ratios, along with their 95 % confidence intervals.

Treatment type did not significantly impact accuracy (p’s > 0.05), suggesting that the quality of the data and validation methods are more critical determinants. That notwithstanding, studies with depressed patient samples exhibited lower prediction accuracy relative to those with non-depressed patient samples (β = 0.70, p < 0.001). Length of treatment was modestly and significantly associated with higher prediction accuracies (β = 1.01, p < 0.01). Also, in terms of ML features, studies using behavioral predictors (β = 0.75, p < 0.01) or combinations of behavioral and biological predictors (β = 0.70, p < 0.001) exhibited lower prediction accuracies than those using solely biological features (e.g., neuroimaging data).

3.4. Meta-analysis diagnostics and robust estimation

The fail-safe N estimate using Viechtbauer’s General Approach was 12,596, which is larger than the 5 k + 10 recommendation indicating that the observed studies are robust to publication bias due to the file drawer problem (Viechtbauer, 2024). That notwithstanding, results of the rank correlation test for funnel plot asymmetry were significant (Kendall’s τ = 0.49, p < 0.001), indicating potential publication bias such that studies with smaller sample sizes may be more likely to have larger accuracy estimates. As part of a sensitivity analysis to confirm the robustness of the findings, a leave-one-out analysis was conducted to determine whether the results were influenced by any particular study result. The results indicated a range in accuracies between 0.755 and 0.76, suggesting that the leave-one-out range very closely fits the meta-analysis results and, thus, supports the robustness of the estimate.

Finally, to obtain cluster-robust tests and confidence intervals, a robust variance estimation procedure was estimated. The results remained unchanged with an accuracy estimate of 0.76 (95 % CI: 0.74–0.78), which was significant per the Satterthwaite approximation (t = 19.70 (df = 114.75), p < 0.0001).

4. Discussion

The current meta-analysis demonstrates that ML methods can predict treatment response in emotional disorders with moderate accuracy. The findings underscore the potential of ML to augment personalized medicine by enabling the identification of individuals who are most likely to benefit from specific treatments. This capability is particularly relevant in the context of emotional disorders, where treatment responses are highly variable and often unpredictable. Namely, predictive modeling may refine personalized medicine by addressing several hitherto unresolved problems such as: 1) providing prognostic information about the probability of success in response to different treatment options, 2) facilitating shared decision making between patients and clinicians (e.g., among those treatments identified as being potentially successful, which one may a patient select on the basis of other medical and life circumstantial factors, etc.), 3) identifying the most salient predictors from large feature sets to develop more parsimonious and easily implementable treatment outcome calculators, 4) optimizing the order and timing of treatments (e.g., guiding JITAI strategies, etc.), and 5) optimizing the dosing of treatments (Chekroud et al., 2021; Wang, Ouyang, et al., 2024).

Although this meta-analysis provides support for overall optimism regarding the utility of ML in predicting treatment response, results of the moderator analyses reveal potential complicating factors that underscore the need for circumspection. Notably, increases in performance accuracy were significantly predicted by higher responder rates and a lack of addressing outcome class imbalance. This suggests that successful instances of ML prediction merely may be a consequence of imbalances in responder status. When imbalances in binary outcomes occur, ML can exploit this trivial feature of a dataset, and high levels of accuracy can be obtained just by being biased in favor of over-representing the majority class in its predictions (i.e., being a responder instead of a non-responder) (Kuhn and Johnson, 2013). Furthermore, studies that did not account for class imbalances with corrective preprocessing strategies such as SMOTE, down sampling, or up sampling were more likely to achieve higher prediction accuracies, which again may reflect a bias for over-representing the majority class. ML studies that did leverage data sampling preprocessing procedures yielded a more conservative prediction accuracy. Thus, the success of ML in predicting treatment response needs to be construed with caution, given the potential consequences of class imbalances.

The higher accuracy observed in studies utilizing neuroimaging and biological data as predictors suggests that such data may provide valuable insights into the neural mechanisms underlying treatment response (Doehrmann et al., 2013; Gabrieli et al., 2015). Neuroimaging especially can capture complex brain patterns and structural abnormalities that may be associated with how individuals respond to various treatments. Moreover, neuroimaging techniques can elucidate potential biomarkers, which can be employed to define novel subtypes of disorders that might be valuable in determining which individuals are optimal candidates for targeted therapies (Drysdale et al., 2017). However, the integration of neuroimaging data into clinical practice remains a challenge due to high costs and the need for specialized equipment and expertise (Gabrieli et al., 2015). Another challenge of models validated on neuroimaging and biological data is the limited extent to which they generalize to new samples (i.e., the bias-variance tradeoff). Given the considerable heterogeneity in neuroimaging preprocessing procedures and types of neuroimaging data (e.g., region of interest vs. whole brain, task activation vs. resting state, etc.), such diversity in methodologies may impede generalization of predictions in new contexts.

Another important finding is the influence of ML methods (validation type, algorithm type, ensemble usage, and missing data) on prediction accuracy. Studies employing robust validation methods, such as k-fold cross-validation and nested cross-validation rather than simple split, tended to report higher accuracies. Additionally, in terms of algorithm type, the use of simpler regression-based algorithms (e.g., elastic net, regularized regression, etc.) were associated with comparable accuracies as studies employing more sophisticated algorithms (e.g., neural net, SVM, tree-based approaches, etc.). That notwithstanding, neural network algorithms are associated with greater prediction accuracy under conditions of increased sample sizes (see Supplementary Materials). This is perhaps not entirely surprising, given that neural networks and deep learning algorithms typically require large sample sizes (Kuhn and Johnson, 2013). Moreover, lower prediction accuracy was also associated with studies that used ensemble approaches or addressed missing data appropriately (e.g., imputation approaches, etc.). This may indicate that certain methodological decisions may yield more conservative ML prediction estimates, highlighting the importance of methodological rigor in ML research. Future studies should prioritize the use of large, well-characterized datasets and robust validation techniques to ensure the reliability and generalizability of ML models.

Certain clinical and demographic characteristics were also important in influencing ML prediction accuracy of treatment response. Specifically, studies using Western samples, as well as those including patients with depression, evidenced lower prediction accuracy. Although there is no obvious explanation for these findings, it has been acknowledged that prediction of treatment outcomes in depression has generally demonstrated small effects with little success in generalizing clinical prediction models in external populations (Gillett et al., 2020).

4.1. Challenges and future directions

Despite these promising findings, there is a need for further research to address existing gaps and challenges. One key area is the standardization of addressing class imbalances in studies predicting treatment response with machine learning. It is of paramount importance for researchers to consider the insidious influence of class imbalance, which may bias ML algorithms to always predict the majority class and, thereby, obtain good classification accuracy without adequately distinguishing between the responder and non-responder outcome classes. Corrective data sampling procedures (e.g., SMOTE) may be advisable to mitigate this concern and facilitate a more balanced assessment of how well ML can actually predict treatment outcomes (Abd Elrahman and Abraham, 2013). Relatedly, future studies should consider more rigorous cross-validation procedures for assessing model performance in test data, such as the implementation of nested cross-validation, which involves two layers of cross-validation for both validating hyperparameters and testing the final model on unseen data. Another challenge is related to the standardization of outcome measures and definitions of treatment response across studies. The variability in these measures makes it difficult to compare results and draw definitive conclusions. Establishing standardized criteria for treatment response and outcome assessment would facilitate more meaningful comparisons and meta-analyses.

Another critical area for future research is the exploration of multimodal data integration. Combining clinical, demographic, genetic, and neuroimaging data could enhance the predictive power of ML models by providing a more comprehensive understanding of the factors influencing treatment response. Multimodal data integration would furthermore profit from judicious feature engineering pipelines. Feature engineering, particularly variable filtering, can enhance ML performance by eliminating irrelevant or redundant features that can introduce noise into the model. This process allows the model to concentrate on the most impactful variables, improving accuracy, reducing overfitting, and yielding a more parsimonious feature set for predicting outcomes (Kuhn and Johnson, 2019), although such procedures should only be undertaken on the non-testing subsets (i.e., training/validation) of the data to preclude data leakage. Additionally, advancements in ML algorithms and computational techniques, such as time-series forecasting, hold promise for further improving prediction accuracy and clinical applicability at the individual level by exploiting personalized symptom dynamics that unfold over time.

A final limitation meriting discussion is that the meta-analysis diagnostics may indicate potential publication bias, such that studies with smaller sample sizes may be more likely to have larger accuracy estimates. This attests to the need for future research to leverage larger samples to provide more reliable and valid appraisals of ML prediction performance.

Future research should aim to standardize methodologies for ML-based prediction of treatment response, including the use of consistent corrective sampling procedures for imbalanced data and cross-validation methods. Additionally, the integration of multimodal data (e.g., combining clinical, genetic, and neuroimaging data) may further improve prediction accuracy. Large-scale collaborative studies are needed to validate and refine ML models in diverse patient populations.

5. Conclusion

Machine learning holds promise for predicting treatment response in emotional disorders, with potential applications in personalized medicine. While current prediction accuracies are optimistically moderate, advancements in data integration and methodological rigor may enhance the utility of ML in clinical practice. This meta-analysis provides a comprehensive overview of the current state of ML-based prediction in emotional disorders and highlights key areas for future research.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.cpr.2025.102593.