Generalizable structure–function covariation predictive of antidepressant response revealed by target-oriented multimodal fusion

Abstract

Major depressive disorder (MDD) is a prevalent condition that profoundly impairs quality of life across diverse populations. Despite widespread use, current antidepressant and psychotherapeutic treatments exhibit limited efficacy and unsatisfactory response rates. Progress in developing effective therapies is hampered by the insufficiently understood heterogeneity of MDD and its elusive underlying mechanisms. Here, to address these challenges, we develop a novel machine learning framework that identifies structure–function covariation through target-oriented fusion of structural and functional connectivity, which robustly predicts individual-level antidepressant response (sertraline, R² = 0.31; placebo, R² = 0.22). Validation in an independent escitalopram-medicated MDD cohort confirms the biomarker’s generalizability (P = 0.01) and suggests an overlap of psychopharmacological signatures across selective serotonin reuptake inhibitors. Our models highlight the right precuneus as a common key region for both sertraline and placebo responses, with the right middle frontal gyrus and left fusiform gyrus specific to sertraline and the left inferior and middle frontal gyri to placebo. We also find that structural connectivity is more predictive of sertraline response, while functional connectivity better predicts placebo response. The framework further decomposes the overall predictive patterns into three constitutive network constellations (default-mode regulatory, affective and sensory processing), which exhibit distinct generalizable structure–function covariation and treatment-specific association with personality traits and behavioral/cognitive profiles. These findings provide unique insights to the structure–function covariation in patients with MDD, its association to the heterogeneity in antidepressant response and the dissection of the intricate MDD neuropsychopharmacology, paving the way for precision medicine and development of more targeted antidepressant therapeutics. Clinicaltrials.gov registration: Establishing Moderators and Biosignatures of Antidepressant Response for Clinical Care for Depression (EMBARC), NCT01407094.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Major depressive disorder (MDD) is a prevalent condition that profoundly impairs quality of life for individuals across ages and demographic groups. Unfortunately, current depression treatments are marked by limited effectiveness and unsatisfactory response rates. The development of effective treatments for MDD is hampered by its insufficiently understood heterogeneity and elusive pathophysiology. Encouragingly, neuroimaging studies over the past decade have successfully identified promising biomarkers for diagnosing MDD^1,2, delineating symptom profiles³ and predicting treatment responsiveness^4–9. Recent research has further delved into individual-level antidepressant response^10,11, shedding light on the heterogeneity among patients with MDD. With the ability to measure physiological neural activity and potentially apply these measures to inform treatment of real patients, neuroimaging studies hold great potential in translating knowledge into precision medicine, thus advancing clinical practice for patients with MDD.

Nonetheless, challenges persist in parsing the heterogeneity in MDD treatment outcomes. First, many patients relapse after discontinuing initially successful treatments^12–14, suggesting that antidepressants primarily induce transient neural changes. This raises a critical question: how can these transient changes be converted into lasting neural alterations to sustain remission? Recent advances in antidepressant response biomarkers, largely based on functional connectivity (FC)^10,11, highlight regional brain connectivity but may inadequately capture their enduringness¹⁵. Alternatively, structural connectivity (SC) provides insights into the anatomical basis of neural connections, complementary to the understanding of regional synchronization captured by FC. Integrating SC and FC is thus crucial for developing clinically applicable biomarkers for antidepressant response.

Structure–function coupling has recently showed promise with its significant associations with both subclinical and psychiatric variability across individuals¹⁶. However, disruptions in structure–function coupling do not reveal their underlying causes, particularly since a single anatomical connection may influence multiple functionally connected regions. Alternatively, structure–function covariation describes SC–FC interactions with an emphasis on associated structural and functional abnormalities, which potentially involve distinct pairs of brain regions. This measure opens new avenues to explore SC–FC interplay across diverse regions, and therefore may offer valuable insights into how transient neural changes evolve into persistent alterations and inform strategies for sustained symptom improvement. Recent studies have also pioneered the application of multimodal fusion techniques in neuroimaging studies^17–20. However, they achieved multimodal fusion in an unsupervised manner, yielding task-agnostic features that may include irrelevant information, increasing the risk of overfitting and reducing interpretability. By contrast, supervised multimodal fusion guided by prediction targets offers greater potential for improving both prediction accuracy and interpretability.

Furthermore, the mechanisms underlying antidepressant effects remain poorly understood. Deciphering the intricate psychopharmacology of MDD requires dissecting the overall predictive pattern into constitutive components. In addition, the placebo effect in antidepressant response remains inadequately explored^21,22. A recent study has suggested a significant contribution of the placebo effect to rapid antidepressant effect²³, underscoring the need for careful interpretation of antidepressant biomarkers and further research to differentiate placebo and drug effects.

To address these challenges, we aim to identify structure–function covariation in patients with MDD that predicts antidepressant and placebo responses. By integrating SC and FC with emphasis on their consensus information, our goal is to develop robust pre-treatment biomarkers for stratifying patients with MDD and improving the predictability of antidepressant responses. FC features were derived from resting-state functional magnetic resonance imaging (fMRI) for its generalizability across clinical settings, compatibility with SC in multimodal fusion and capability of detecting distributed network-level disruptions. We have developed an innovative target-oriented multimodal fusion (TOMMF) framework that combines multimodal fusion with the treatment response prediction task, thus ensuring the clinical relevance of identified structure–function covariation (Fig. 1a). Leveraging an advanced machine learning technique²⁴, our framework minimizes overlap between the contributions of individual SC/FC across multiple dimensions, yielding dimension compositions that isolate distinct and interpretable components of antidepressant response biomarkers. With these distinct components in the multimodal biomarkers for antidepressant response, which we named as network constellations, we further investigate the treatment-specific associations of structure–function covariation with behavioral/cognitive profiles and personality traits (Fig. 1b). Our study aims to unravel the distinct roles of SC and FC in predicting antidepressant response, their interplay and how their treatment-specific patterns can differentiate placebo and drug effects. Importantly, we also assess the generalizability of biomarker findings on an independent cohort. These efforts aim to offer reliable novel insights into the intricate neuropsychopharmacology of antidepressant treatment and lay the groundwork for improved clinical practice in MDD treatment.

Fig. 1 |. Characterization of structure–function covariation predicting antidepressant and placebo responses.

a, Framework design. SC and FC features are extracted from DTI and resting-state fMRI. SC and FC dimensions are jointly identified to maximize their association, producing structure–function covariation features as weighted sums of SC and FC dimensions that encapsulate their consensus information. These features are used to predict individual treatment responses, with reciprocal feedback from prediction target ensuring clinical relevance of the identified structure–function covariation. The bidirectional propagation yields robust covariation features representing SC–FC interplay and predicting treatment outcomes. b, Dissection of MDD psychopharmacology. Distinct brain dimensions are identified to elucidate MDD psychopharmacology, which we name as network constellations. Pre-treatment SC and FC scores of these network constellations show significant associations, reflecting generalizable dimensions of structure–function covariation. To differentiate antidepressant and placebo effects, network constellations are respectively masked using antidepressant or placebo response-predictive biomarker, yielding components specific to each treatment. Associations of these treatment-specific components of network constellations with behavioral/cognitive task performance (A-not-B task, CRT, reaction time and accuracy of flanker test (Flk_RT and Flk_acc), WF and PRT) and NEO personality traits (neuroticism (N), extraversion (E), openness (O), agreeableness (A) and conscientiousness (C)) are further characterized.

Results

Multimodal neuroimaging biomarkers for sertraline response

We first applied the TOMMF framework to 103 patients with MDD receiving sertraline treatment from the Establishing Moderators and Biosignatures of Antidepressant Response in Clinical Care (EMBARC) dataset²⁵, developing a prediction model for individual antidepressant response based on SC and FC extracted for 100 cortical²⁶ and 35 subcortical^27,28 regions of interest (ROIs). Antidepressant response was quantified as the pre- to post-treatment change in 17-item Hamilton Depression Rating Scale (HAMD₁₇) scores. The model exhibited strong cross-validation (10× 10-fold) performance in predicting sertraline response (R² = 0.3149, r = 0.5689, P = 3.63 × 10⁻¹⁰; Fig. 2a). Its significance was confirmed by permutation testing (P_perm < 0.001; Supplementary Fig. 1a), and the prediction was not explained with clinical-irrelevant demographics (Supplementary Fig. 2). Notably, the sertraline response prediction model demonstrated no predictability to placebo response (R² < 0, r = −0.0312, P = 0.7352; Fig. 2b), suggesting its specificity to sertraline. Furthermore, the TOMMF framework outperformed various baseline methods, including regression models trained with concatenated SC and FC features, sparse canonical correlation analysis (sCCA)-based fused features, partial least squares regression, single data modalities (one-tailed Wilcoxon signed-rank test, all P_FDR ≤ 0.0020; Fig. 2c) and clinical regressors (Wilcoxon rank-sum test, P = 9.13 × 10⁻⁵; Supplementary Fig. 3a). The sertraline response did not differ between sexes (P = 0.7889; Supplementary Fig. 4a), and the model predicted treatment outcomes with high and comparable accuracy in both female (R² = 0.3136, r = 0.5701, P = 3.16 × 10⁻⁷; Supplementary Fig. 5a) and male patients (R² = 0.3165, r = 0.5757, P = 3.68 × 10⁻⁴, Supplementary Fig. 5b; comparison between sexes, P = 0.3223, Supplementary Fig. 5c). Collectively, these findings underscore the potential of structure–function covariation in providing enhanced predictive power for sertraline response on the individual patient level and identifying neural phenotypic characteristics that can be exploited for treatment stratification.

Fig. 2 |. Multimodal neuroimaging biomarkers for sertraline-induced antidepressant response.

a, Multimodal-based sertraline response prediction model. The model demonstrated robust prediction performance for sertraline response evaluated by 10× 10-fold cross-validation. b, Specificity of sertraline response prediction model to treatment. The model for sertraline response showed no prediction capability for placebo response. Error bars indicated 95% CIs. c, Comparison of TOMMF framework with baseline methods. The TOMMF framework showed significantly higher prediction performance than baseline methods, including partial least squares regression with concatenated SC and FC features (Concat_PLS), LASSO regression with only SC, only FC, sCCA-based fused features, or concatenated SC and FC features (Concat). The statistical significance is evaluated by one-tailed Wilcoxon signed-rank test (P = 0.0024) with 100× 10-fold cross-validation. The error bars indicate 95% CIs. d, Important FC and SC features for sertraline response. The predictive pattern exhibited a multivariate foundation encompassing every functional module of the brain. Specifically, the connection between left fusiform gyrus and left middle frontal gyrus showed the strongest contribution among SCs, while the connection between right precuneus and left amygdala showed the strongest contribution among FCs. The top 20 connectivity features are shown for each data modality for clarity. e, Important brain regions in the multimodal predictive pattern for sertraline response. The right orbital part of the middle frontal gyrus (R-oMFG) and the left fusiform gyrus (L-FG) showed the most substantial contribution in abnormal SC. The bilateral striata and the right precuneus played crucial roles in abnormal FC. Within the left striatum, the region with rich connections to the limbic network (L-LIM-Striatum) demonstrated the highest importance. Within the right striatum, the region functionally associated with the VAN (R-VAN-Striatum) held particular importance. For visualization purpose, the striata are projected to the nearest surface. The top 10 brain regions are shown for each data modality for clarity. f, Multimodal network connectivity importance for sertraline response prediction. The network connectivity importance is calculated as the average absolute value of non-zero ROI-level connectivity weights of the prediction model. Overall, the SC features show greater importance than FC features. The VN–DAN and VN–DMN SCs make the strongest contribution to sertraline response prediction.

Afterward, we examined the multimodal neuroimaging biomarkers for sertraline response as dictated by the prediction model weights. Overall, the predictive pattern exhibited a robust multivariate foundation encompassing SC and FC involving every functional brain network (Fig. 2d). Specifically, the connection between left fusiform gyrus and left middle frontal gyrus showed the most substantial contribution among SCs, while the connection between right precuneus and left amygdala demonstrated the strongest contribution among FCs. Further investigation of pivotal brain regions in the structural and functional profiles of sertraline response biomarkers (Fig. 2e) revealed that the right orbital part of the middle frontal gyrus and the left fusiform gyrus played key roles in the SC-based biomarker, and the bilateral striata and the right precuneus were prominent in the FC-based biomarker. Notably, within the left striatum, the region functionally associated with the ventral attention network (VAN) held particular importance, while the region with rich connections to the limbic network played a crucial role in the right striatum. Lastly but importantly, we examined the network connectivity importance by averaging the absolute non-zero weights of ROI-level connectivity features. As a result, we found that SC exhibited overall greater importance than FC, while the SC between visual network (VN) and dorsal attention network (DAN) and between VN and default-mode network (DMN) showed the strongest contribution to sertraline response prediction (Fig. 2f).

Multimodal neuroimaging biomarkers for placebo response

Subsequently, we applied the TOMMF framework to derive a model for predicting placebo response in 120 placebo-medicated patients with MDD. As a result, the framework exhibited high cross-validation performance for placebo response prediction (R² = 0.2190, r = 0.4682, P = 6.96 × 10⁻⁸; Fig. 3a), whose significance was further confirmed by permutation test (P_perm = 0.001; Supplementary Fig. 1b). Importantly, the placebo response prediction model was unable to predict sertraline response (R² < 0, r = −0.0597, P = 0.5492; Fig. 3b), suggesting its specificity to placebo. Furthermore, our TOMMF framework showed superiority of placebo response prediction over baseline methods (one-tailed Wilcoxon signed-rank test, all P_FDR ≤ 9.77 × 10⁻⁴; Fig. 3c) and clinical regressor-based model (Wilcoxon rank-sum test, P = 9.13 × 10⁻⁵; Supplementary Fig. 3b). Notably, although prediction performance was significantly higher in female patients (Wilcoxon signed-rank test, P = 0.0020; Supplementary Fig. 5f), the model achieved significant predictive accuracy in both sexes (female, R² = 0.2777, r = 0.5317, P = 6.51 × 10⁻⁷, Supplementary Fig. 5d; male, R² = 0.1183, r = 0.3721, P = 0.0140, Supplementary Fig. 5e). This difference in prediction accuracy was not attributable to differences in placebo response magnitude between sexes (P = 0.6820; Supplementary Fig. 4b). These results demonstrated the potential of structure–function covariation in providing enhanced predictive power for placebo response and showcased its generalizability to different clinical prediction targets.

Fig. 3 |. Multimodal neuroimaging biomarkers for placebo-induced antidepressant response.

a, Multimodal-based placebo response prediction model. The model demonstrated robust prediction performance for placebo response evaluated by 10× 10-fold cross-validation. b, Specificity of placebo response prediction model to treatment. The model for placebo response showed no prediction capability for sertraline response. Error bars indicated 95% CIs. c, Comparison of TOMMF framework with baseline methods. The TOMMF framework showed significantly higher prediction performance than baseline methods, including Concat_PLS, LASSO regression with only SC, only FC, sCCA-based fused features or Concat. The statistical significance is evaluated by one-tailed Wilcoxon signed-rank test (P = 9.74 × 10⁻⁶) with 100× 10-fold cross-validation. The error bars indicate 95% CIs. d, Important FC and SC features for placebo response. The predictive pattern exhibited a multivariate foundation encompassing every functional module of the brain. Specifically, the connection between right precentral gyrus (R-PreCG) and left anterior hippocampus showed the strongest contribution among SCs, while the connection between left middle frontal gyrus (L-MFG) showed the strongest contribution among FCs. The top 20 connectivity features are shown for each data modality for clarity. e, Important brain regions in the multimodal predictive pattern for placebo response. The right precuneus and the R-PreCG showed the most substantial contribution in abnormal SC. The left inferior frontal gyrus (L-IFG) and the L-MFG played crucial roles in abnormal FC. The top 10 brain regions are shown for each data modality for clarity. f, Multimodal network connectivity importance for placebo response prediction. The network connectivity importance is calculated as the average absolute value of non-zero ROI-level connectivity weights of the prediction model. Overall, the FC features show greater importance than SC features. The within-FPCN, within-SMN FCs and SMN–SCN SC make the strongest contribution to placebo response prediction.

We then explored the multimodal neuroimaging biomarkers for placebo response as guided by the prediction model weights. Similar to the biomarkers for sertraline response, the predictive pattern for placebo response exhibited a robust multivariate foundation encompassing every functional brain network (Fig. 3d). Specifically, the connection between right precentral gyrus and left anterior hippocampus showed the most substantial contribution among SCs, while the connection between left middle frontal gyrus and right angular gyrus demonstrated the strongest contribution among FCs. Further investigation of important brain regions in the structural and functional profiles of placebo response biomarkers (Fig. 3e) revealed that the right precuneus and the right precentral gyrus played key roles in the SC-based biomarker, while the left inferior and middle frontal gyri were prominent in the FC-based biomarker. Network connectivity importance revealed that FC provided overall greater contribution for placebo response than SC, while the FCs within somatomotor network (SMN) and within fronto-parietal control network (FPCN) and the SC between SMN and subcortical network (SCN) showed particular importance (Fig. 3f).

Generalizability of treatment response biomarkers on independent cohort

We then examined the generalizability of treatment response biomarkers on an independent cohort. Remarkably, despite the differences in medication and MDD symptom assessment, and the potential heterogeneity in populations (see Methods for details), the symptom relief predicted by sertraline response biomarker exhibited a significant correlation with the escitalopram-induced one in the Canadian Biomarker Integration Network in Depression (CAN-BIND)-1 cohort (r = 0.2739, P = 0.0104; Fig. 4a). This result demonstrated the generalizability of the multimodal sertraline response biomarker and suggested a potential overlap of psychopharmacological signatures across selective serotonin reuptake inhibitors (SSRIs). By contrast, no such correlation was observed between the escitalopram-induced symptom relief and the one predicted by placebo response biomarker (r = −0.0499, P = 0.6603; Fig. 4b), consistent with our finding that sertraline and placebo alleviate the symptoms via distinct mechanisms. Notably, cross-cohort harmonization of SC and FC features did not improve generalization performance (sertraline model, r = 0.2253, P = 0.0295, z-test, P = 0.7642; placebo model, r = 0.0503, P = 0.3384, z-test, P = 0.5619; Supplementary Fig. 6). This suggests that the multicenter design of the EMBARC cohort allowed our models to inherently accommodate potential site effects. Consequently, data harmonization may be unnecessary when applying our models to independent test samples, highlighting their practical utility in real-world clinical settings, where such harmonization may not be feasible as patients are enrolled individually.

Fig. 4 |. Generalization analysis of multimodal biomarkers for sertraline and placebo response in the CAN-BIND-1 cohort.

a,b, Multimodal-based prediction models for sertraline (a) and placebo (b) response were applied to an independent cohort of escitalopram-medicated patients with MDD (n = 71), with symptom relief assessed by the MADRS. Predicted HAMD₁₇ changes were transformed to MADRS changes using an equipercentile-based conversion (MADRS ≈ HAMD₁₇ × 1.3). Despite differences in medication, symptom relief predicted by the sertraline response model significantly correlated with actual escitalopram response, validating the generalizability of the sertraline response biomarker and suggesting an overlap of pharmacological signatures across SSRIs. By contrast, the placebo response model did not predict escitalopram response, consistent with our findings that sertraline and placebo alleviate the symptoms via distinct mechanisms. P values are one-tailed as we seek only positive correlations. Error bars indicated 95% CIs.

Distinct data modality importance for sertraline and placebo

After validating the identified biomarkers, we investigated how SC and FC contribute to the sertraline and placebo response, respectively. To this end, we examined the effect of the modality importance parameter ρ_Λ on prediction performance (see Methods for details). Interestingly, our findings indicated that the sertraline response biomarker exhibited a preference for SC, while the placebo response biomarker showcased a more substantial influence of FC (Supplementary Fig. 7). These results suggested the distinct biomarkers for sertraline and placebo responses, implying the potential difference in their psychopharmacological mechanisms.

Prediction capability of multimodal models with missing data modalities

To simulate real-world clinical scenarios where patients may lack either fMRI or diffusion tensor imaging (DTI) data, we conducted a missing-modality test to evaluate the prediction capability of our multimodal models under partial input conditions. Within the cross-validation framework, models were trained using TOMMF on multimodal data, but during validation, either SC or FC weights were set to zero to emulate the absence of that modality.

For sertraline response prediction, the multimodal model maintained strong performance when tested with SC alone (R² = 0.3028, r = 0.5508, P = 1.65 × 10⁻⁹; Supplementary Fig. 8a), but not with FC alone (R² = 0.0199, r = 0.2424, P = 0.0136; Supplementary Fig. 8b), aligning with our earlier observation of SC’s dominant contribution (Supplementary Fig. 7). However, predictions with only SC were significantly less accurate than those with full multimodal input (Wilcoxon signed-rank test, P = 0.0020), highlighting the added value of FC. Notably, even when only SC data were available in test samples, the multimodal model showed a trend toward outperforming the model trained with only SC, although the significance of this comparison was not confirmed (P = 0.1162; Supplementary Fig. 8c).

For placebo response prediction, the multimodal model preserved significant predictability when tested with FC alone (R² = 0.1648, r = 0.4470, P = 3.09 × 10⁻⁷; Supplementary Fig. 8d), but not with SC alone (R² < 0, r = 0.1750, P = 0.0559; Supplementary Fig. 8e), consistent with our finding of FC’s stronger association with placebo response (Supplementary Fig. 7). Although prediction accuracy with FC alone was reduced compared with full multimodal input (P = 9.77 × 10⁻⁴), the multimodal model still significantly outperformed the model trained with only FC (P = 9.77 × 10⁻⁴; Supplementary Fig. 8f), demonstrating the benefit of multimodal training even under application scenarios with missing modality.

Finally, on the independent cohort receiving escitalopram, the sertraline-trained multimodal model achieved significant prediction using SC alone (r = 0.2754, P = 0.0101; Supplementary Fig. 9a), but not with FC alone (r = −0.0247, P = 0.5811; Supplementary Fig. 9b), further supporting the primary role of SC in SSRI response prediction. This finding underscores the robustness and clinical utility of our multimodal models for real-world applications with incomplete neuroimaging data.

Generalizable network constellations with distinct structure–function covariation

To evaluate the generalizability of structure–function covariation and parse the intricate psychopharmacology of MDD, we constructed network constellations by aggregating network components with the highest cross-modality correlations based on the sertraline response biomarker (see Methods for details). Essentially, we aimed to identify distinct components of the overall antidepressant response-predictive patterns that each exhibit generalizable structure–function covariation. As a result, we identified three network constellations spanning the intrinsic–extrinsic functioning axis. We named these network constellations based on the pattern of findings, including the default-mode regulatory constellation, the affective constellation and the sensory processing constellation (Fig. 5a). The default-mode regulatory constellation consisted of the intra-connectivity within DMN, SCN and their connectivity with DAN and FPCN. The affective constellation was primarily composed of the intra-connectivity of cortical limbic network (LIM) and its connectivity with VN, SMN and DAN. The sensory processing constellation encompassed the VN’s connectivity to DMN and SCN, as well as connectivity involving attention networks and cerebellum (CB). For the sertraline response biomarker, the sertraline-medicated patients indeed showed significant correlations between the SC and FC constellation scores (default-mode regulatory, r = 0.4385, P = 1.96 × 10⁻¹²; affective, 0.3027, P = 1.62 × 10⁻⁶; sensory processing, r = 0.1375, r = 0.0190), which were successfully generalized to the placebo-medicated patients (default-mode regulatory, r = 0.1394, P = 0.0154; affective, r = 0.1579, P = 0.0072; sensory processing, 0.1445, P = 0.0126; top row of Fig. 5b).

Fig. 5 |. Network constellations in multimodal biomarkers for sertraline and placebo response.

a, Composition of network constellations. First, network components are quantified as the sum of region-level connectivity scores involving a particular network pair, weighted by antidepressant response model coefficients. Subsequently, network constellations are constructed by aggregating network components with the highest cross-modality correlations. Three generalizable network constellations spanning the intrinsic–extrinsic functioning axis are identified, including the default-mode regulatory constellation, the affective constellation and the sensory processing constellation. Notably, SC and FC contributions are not equivalent. b, Generalizable structure–function covariation of patients with MDD in network constellations. Top row: structure–function covariation in sertraline response biomarker. The SC and FC contributions of constellation scores exhibit significant positive correlation on the sertraline arm, demonstrating that the framework has successfully identified the link between SC and FC. As structure–function covariation is an intrinsic property of pre-treatment neuroimaging data, we expect to observe the same relationship in the other treatment arm. Encouragingly, the patients receiving placebo indeed show significant positive correlation between SC and FC contributions of all three constellation scores, suggesting the generalizability of structure–function covariation of sertraline response. Bottom row: structure–function covariation in placebo response biomarker. The network constellations constructed on sertraline response biomarker also show robust structure–function covariation with placebo response biomarker for all three network constellations. This structure–function covariation is generalizable across treatment arms. c, Treatment-specific traits of network constellations. Top row: association of treatment-specific network constellations with NEO personality traits. For default-mode regulatory constellation, the sertraline-specific biomarker exhibits a robust association with neuroticism and conscientiousness, while the placebo-specific biomarker significantly associates with openness. The sertraline-specific biomarker in affective constellation also exhibits significant association with neuroticism, while no biomarkers in sensory processing constellation showed significant association with NEO personality traits. Bottom row: association of treatment-specific network constellations with behavioral/cognitive task performance. The placebo-specific biomarker in default-mode regulatory constellation shows a significant association with CRT, and the affective constellation score shows significant association with Flk_acc. No sertraline-specific constellations exhibit significant correlations with the task performance that we have investigated. *P < 0.05, **P < 0.01 and ***P < 0.001 indicate the FDR-corrected statistical significance of correlations between constellation scores and personality/task profiles.

Afterward, we examined whether these network constellations captured structure–function covariation in placebo response biomarker. Encouragingly, these network constellations also exhibited generalizable structure–function covariation for the placebo response biomarker (placebo-medicated patients, default-mode regulatory, r = 0.4349, P = 8.48 × 10⁻¹³; affective, r = 0.2909, P = 2.29 × 10⁻⁶; sensory processing, r = 0.1769, P = 0.0032; sertraline-medicated patients, default-mode regulatory, r = 0.1242, P = 0.0305; affective, r = 0.1518, P = 0.0109; sensory processing, r = 0.1801, P = 0.0030; bottom row of Fig. 5b), demonstrating the generalizability of the structure–function covariation within these identified network constellations for patients with MDD.

Treatment-specific traits of network constellations

To further explore the unique characteristics of each network constellation, we then examined how SC–FC constellation scores relate to personality traits and cognitive task performance. Notably, the SC and FC constellation scores were summed together to get a combined score in this analysis. First, as a recent study demonstrated significant association between personality traits and neuropsychopathology²⁹, we examined the correlations between network constellation scores and NEO personality traits of patients with MDD (top row of Fig. 5c). Interestingly, in the default-mode regulatory constellation, the sertraline-specific biomarker exhibited a robust association with neuroticism (r = 0.1897, P_FDR = 2.12 × 10⁻⁴) and conscientiousness (r = −0.1343, P_FDR = 0.0098), while the placebo-specific biomarker was significantly associated with openness (r = 0.1524, P_FDR = 0.0052). The sertraline-specific biomarker in affective constellation also exhibited significant association with neuroticism (r = 0.1216, P_FDR = 0.0451), while no biomarkers in sensory processing constellation showed significant association with NEO personality traits.

Subsequently, we explored the association between network constellation scores and behavioral/cognitive task performance at baseline, including the A-not-B task, choice reaction time (CRT), the flanker test, the probabilistic reward task (PRT) and the word fluency (WF) test (bottom row of Fig. 5c). Results revealed that the placebo-specific biomarker in default-mode regulatory constellation was significantly associated with CRT (r = −0.1909, P_FDR = 0.0264), while the affective constellation score showed a significant association with flanker inference on accuracy (r = 0.2097, P_FDR = 0.0106). No sertraline-specific constellation scores exhibited significant correlations with the task performance that we investigated. Together, these results demonstrate the distinct characteristics of the network constellations, supporting their validity as a conceptual framework for delineating the psychopharmacology of MDD.

Enhanced robustness of biomarkers endowed by TOMMF

Lastly, we examined the association between TOMMF-based multimodal biomarkers and SC/FC biomarkers from unimodal baseline methods for both sertraline and placebo responses. Predictive SC exhibited a strong correlation between unimodal- and multimodal-based patterns (r_SER = 0.8186, r_PLA = 0.8158; Supplementary Fig. 10), while predictive FC showed a relatively modest correlation (r_SER = 0.3401, r_PLA = 0.4352; Supplementary Fig. 10). This highlighted that the TOMMF framework effectively utilized inputs from each data modality for improved prediction. Notably, the pattern correlation for FC aligned with its relative importance for placebo compared with sertraline, and the overall stronger correlation for SC echoed its relative persistence compared with FC.

Subsequently, the robustness of predictive patterns across cross-validation folds was investigated. We respectively examined the intraclass correlation coefficients (ICCs) of predictive SC and FC patterns from 10 × 10 cross-validation folds, which demonstrated robustness for both sertraline and placebo responses in the TOMMF framework (ICC_SC,SER = 0.5326, 95% confidence interval (CI) (0.5253, 0.5400); ICC_FC,SER = 0.3916, 95% CI (0.3846, 0.3988); ICC_SC,PLA = 0.4801, 95% CI (0.4727, 0.4875); ICC_FC,PLA = 0.4926, 95% CI (0.4853, 0.5001)). Importantly, these were higher than the robustness of predictive patterns obtained from unimodal methods (ICC_SC,SER = 0.4395, 95% CI (0.4322,0.4468); ICC_FC,SER = 0.3018, 95% CI (0.2955,0.3081); ICC_SC,PLA = 0.3972, 95% CI (0.3901, 0.4044); ICC_FC,PLA = 0.3169, 95% CI (0.3105, 0.3234)), with statistical significance confirmed by one-tailed two-sample z-tests (P_SC,SER = 1.11 × 10⁻¹⁶, P_FC,SER = 3.09 × 10⁻¹², P_SC,PLA = 2.41 × 10⁻¹², P_FC,PLA < 10⁻³²³). Collectively, these results suggested that the TOMMF framework enhanced the predictability of antidepressant response by integrating information across data modalities and improved the robustness of identified patterns for each data modality.

Discussion

In this study, we examined the structure–function covariation in patients with MDD with the TOMMF framework to establish generalizable neuroimaging biomarkers for sertraline and placebo responses. We also explored the roles and interplay of SC and FC and their treatment-specific patterns that distinguish placebo and drug effects. The integration of SC and FC facilitated the development of predictive models for sertraline- and placebo-induced antidepressant responses that outperformed unimodal methods. These models revealed critical brain connections/regions linked to antidepressant response and the associations between SC and FC abnormalities. Remarkably, the sertraline response biomarker exhibited significant generalizability on an independent escitalopram-medicated cohort, holding promise for enabling applicable neuroimaging-guided antidepressant treatment and suggesting a shared signature across SSRIs. Furthermore, we parsed the overall predictive patterns into distinct components, revealing constitutive network constellations with generalizable structure–function covariation and treatment-specific traits. Structure–function covariation, with its generalizability and high interpretability, enhances the predictive power for antidepressant responses and provides a new tool for future neuroimaging biomarker studies, paving the way for a deep understanding of the psychopharmacology of MDD and its precision medicine.

Generalizability is crucial for translating biomarker discoveries into advances in clinical practice and understanding of mechanisms. In this work, we evaluated our treatment response prediction models using an independent cohort of escitalopram-medicated patients. Our results suggest that the sertraline biomarker may generalize to predict escitalopram response, supporting the previous finding of a potential shared biomarker for serotonin reuptake inhibitors³⁰ and the identified brain signatures truly reflect MDD’s psychopharmacology. However, it is important to acknowledge that sertraline and escitalopram may still differ in their specific mechanisms of action. Our generalizability analysis shows that the sertraline response biomarker explains a smaller proportion of variance in escitalopram response, echoing with previous studies that neuroimaging biomarkers can differentiate responses to these drugs^9,31. This is consistent with sertraline’s additional effects on dopamine receptors³² and escitalopram’s binding to the allosteric site of serotonin transporters³³. While drug-specific biomarkers may offer more accurate treatment response predictions and thus improving drug recommendations, a general biomarker could be valuable for identifying non-responders to first-line antidepressants. This would allow clinicians to expedite alternative treatment strategies without relying on trial and error, facilitating early intervention and personalized care.

Multimodal fusion is a technique emergingly used by neuroimaging-based psychiatric studies, aiming to deepen our understanding of psychopathological neural alterations^34–37 and improve predictability in psychiatric diagnosis^38–40 and treatment effect⁴¹. Our innovative approach incorporates prediction target guidance into the multimodal fusion process, yielding substantial benefits over unsupervised fusion for downstream prediction tasks. As a recent study showed that the general neuroimaging variability across participants mostly represents subclinical heterogeneity⁴², the target guidance ensured that the fused features are relevant to antidepressant responses. Notably, the learning objective minimized the distance between canonical dimensions of SC and FC. Therefore, the fused features essentially reflect the information in agreement, or the structure–function covariation, thus enhancing the reliability of features. Importantly, TOMMF’s linear basis simplifies the complexity, enhancing generalizability and facilitating neurophysiological interpretation of identified biomarkers. Furthermore, the L0-regularization achieved a high degree of orthogonality in latent features without explicit constraints, achieving data-driven dimensionality and subdimension identification in the overall predictive patterns, which elucidates the psychopharmacological intricacies of MDD.

Structure–function coupling has attracted interest among neuroimaging researchers with its strong associations with both subclinical and psychiatric variability across individuals¹⁶. While it offers valuable insights into patient heterogeneity, the factors contributing to disruptions in structure–function coupling are complex and multifaceted, highlighting the need for additional metrics to provide more specific insights. Unlike structure–function coupling, which examines the correlation between SC and FC of the same ROI pairs¹⁶, the structure–function covariation features extracted by our framework expand this analysis to interactions between different brain region pairs. For instance, we observe that changes in SC between LIM and VN, SMN, DAN and LIM itself are significantly correlated with changes in FC between LIM and DAN, DMN and SCN (the affective constellation). These structural and functional changes, while involving different brain regions, coupled with each other and collectively link to antidepressant response. This structure–function covariation provides a more detailed explanation for the observed reductions in structure–function coupling in patients with MDD⁴³ and offers novel insights into how anatomical alterations and functional changes may induce each other.

Interestingly, our findings suggest that SC is more informative for predicting sertraline response, while FC is more indicative of placebo response. Encouragingly, this result echoes with a recent finding based on an independent cohort that SC is more predictive of antidepressant responses induced by a variety of serotonin reuptake inhibitors than FC⁴⁴. Conceptually, SC reflects the stable and enduring anatomical brain structure, while FC signifies the more adaptable active interactions between brain regions¹⁷. As previous studies have noted a significantly higher relapse rate with placebo compared with antidepressant medications after treatment discontinuation following remission^45–47, we speculate that the greater variation in SC may contribute to the relatively persistent sertraline response, whereas the intermittent placebo response is associated with greater variation in FC. Owing to the design of the clinical trial, we were unable to examine the heterogeneity in relapse using the EMBARC dataset. Additional research is warranted to provide further evidence for this theory, including investigating whether patients experiencing relapse earlier exhibit greater FC alteration compared with those experiencing relapse later or without relapse, irrespective of the treatment received. Its generalizability to other antidepressant medications should also be further investigated.

With the structure–function covariation features extracted by our innovative machine learning framework, we identified neuroimaging biomarkers for antidepressant response on a multimodal basis. Notably, while sertraline and placebo responses exhibited distinct biomarkers, the right precuneus emerged as a consistently significant predictor for both treatments. This underscores the crucial role of the precuneus in modulating antidepressant effects, aligning with previous findings indicating its abnormalities in patients with MDD⁴⁸ and predictive value for treatment outcome^11,49. The SCs involving the right middle frontal gyrus and left fusiform gyrus exhibited particular importance to sertraline response, echoing with previous findings that sertraline induces increase in metabolic rate of middle frontal gyrus⁵⁰ and the perfusion level change in both regions⁵¹. The left inferior and medial frontal gyri showed strongest contribution to placebo response, aligning with previous positron emission tomography-based⁵², electroencephalogram-based^49,53 and unimodal FC-based studies¹¹. Our study also provided insights into the contribution of subcortical regions to sertraline response, a facet not extensively explored in previous individual-level antidepressant response prediction studies^11,49. Specifically, we found that FC involving striatum is particularly important for sertraline response, echoing with previous findings that FC involving striatum substantially associates with MDD symptoms⁵⁴ and responses to repetitive transcranial magnetic stimulation⁵⁵. Our models did not attribute as much importance to hippocampal structural abnormalities for antidepressant response, despite the hippocampus being consistently identified as a biomarker for MDD diagnosis⁵⁶. This highlights the unique information provided by SC compared with anatomical volume of brain regions and suggests a possible discrepancy between the psychopathology of psychopharmacology of MDD. Moreover, the subcortical regions exhibited less substantial contribution in placebo response, in line with a recent finding highlighting differences in brain gradient between sertraline and placebo responses⁴⁹. Importantly, our biomarker findings discerned between the contributions of SC and FC, providing novel insights into the respective roles of persistent and adaptable neural connections in treatment efficacy. Inspired by these findings, we advocate for future research into how persistent and adaptable neural connectivity influences antidepressant response and how therapeutics can facilitate the transition from transient treatment-induced neurophysiological changes to persistent ones.

In addition, the construction of network constellations effectively disentangled the overarching predictive patterns for antidepressant response into distinct components. Our findings unveiled three distinct network constellations (that is, clusters of networks) that synergistically informed the prediction of antidepressant response. These constellations, including the default-mode regulatory, affective and sensory processing constellations, encompassed a spectrum from intrinsic to extrinsic functions and demonstrated a high degree of independence from each other. Remarkably, we found that these three network constellations may correspond to three distinct roles in MDD’s psychopharmacology: (1) core pathophysiological substrates, (2) pharmacologically modulated circuits and (3) behavior and symptom mediators. The default-mode regulatory constellation comprises SC and FC centered on the DMN and SCN, which may be primarily mediated by the cingulum bundle and are commonly disrupted in patients with MDD^57,58. Notably, these disruptions are not always reversed following symptom remission and show no significant difference between remitters and non-remitters⁵⁹, suggesting that they reflect underlying pathophysiology rather than direct therapeutic targets. By contrast, the LIM-anchored affective constellation shows early modulation by antidepressants independent of symptom improvement. For example, functional changes in limbic–paralimbic and subcortical regions have been observed within a week of SSRI administration, before changes in symptom severity become detectable⁶⁰. In addition, the orbitofrontal cortex, a LIM hub densely innervated by serotonergic fibers from the brain stem⁶¹, exhibits increased FC with the amygdala following serotonin receptor blockade, disrupting emotional processing⁶². Meanwhile, serotonergic stimulation reduces LIM activation in response to social exclusion and alleviates social pain⁶³. This heavy reliance of LIM circuits on serotonergic pathways and their early changes following SSRI treatment support the affective constellation as a key target of pharmacological action. Finally, the sensory processing constellation, encompassing visual, somatomotor, cerebellar and attention networks, may mediate the expression of internal state changes as observable symptoms. Although these networks are less frequently highlighted in traditional pathophysiological and pharmacological models of MDD, they are increasingly recognized for their roles in mediating behavioral symptoms^64,65. For instance, DMN–CB circuits link to gastrointestinal symptoms in MDD⁶⁶, and the visual cortex abnormalities are associated with psychomotor retardation and overall MDD symptomatology^67,68. Collectively, these patterns suggest that this constellation translates pharmacologically induced brain-state changes into behavioral outcomes. We emphasize that this tripartite framework is a preliminary conceptualization; brain circuits may serve overlapping roles, and their specific contributions to treatment mechanisms warrant further neurobiological validation.

Finally, although our framework successfully identifies generalizable biomarkers for antidepressant response and disentangles brain signatures into interpretable network dimensions, several limitations and open questions remain. First, the current approach prioritizes SC–FC consensus information while de-emphasizing modality-specific contributions. Future extensions that explicitly incorporate modality-specific information may enhance predictive performance and enable effective handling of missing modalities during model training. In addition, integrating task-based fMRI data may further enhance the prediction accuracy and offer novel insights into the neural mechanisms underlying antidepressant response. Furthermore, while the current placebo response model achieves significant predictive performance for both sexes, it shows a notable disparity in accuracy between female and male patients. Future studies with sufficient male samples are critical for developing sex-specific models that may further improve predictive accuracy. As studies have revealed a consistent divergence between the pathophysiological and psychopharmacological biomarkers of MDD⁶⁹, future research should also investigate how these response-predictive SC–FC covariations contribute to the reversal of brain abnormalities in MDD, thereby advancing our understanding of the neural pathways underlying antidepressant action. Lastly, future work should also investigate how these SC–FC covariation signatures evolve over the course of treatment. Such longitudinal analyses will be essential for understanding mechanisms of sustained remission and relapse.

In summary, we present generalizable structure–function covariations in patients with MDD and its association with the heterogeneity in antidepressant response. The structure–function covariation offers a new perspective to inspect the interplay of SC and FC in mental health conditions with an emphasis on their target-relevant consensus information, which enhances the generalizability of its biomarker findings. The identified sertraline response biomarker is replicable on an independent escitalopram-medicated cohort, demonstrating its high trustability and suggesting an overlap of psychopharmacological signatures across SSRIs. Beyond the biomarker identification, our framework also discerns distinct components within the overall predictive patterns, which establishes it as a promising tool for investigating psychiatric disorders with multimodal neuroimaging data. Our key observation emphasizes that sertraline and placebo responses correspond to greater variation in SC and FC, underscoring the importance of persistent neurophysiological changes for optimal treatment outcomes. Furthermore, we have delineated three network constellations constituting the overarching antidepressant response biomarkers, which unveil potential distinct aspects within the complex psychopharmacology of MDD. These novel insights serve as inspiration for future research, advocating the development of therapeutics that induce transient changes to persistent alterations and address each constitutive neural circuitry within the comprehensive psychopharmacological pathways of MDD. With demonstrated generalizability and high explainability, our findings hold promise for enabling neuroimaging-guided precision medicine and provide new insights for MDD that may foster the development of more efficacious antidepressant treatments.

Methods

Participants

Dataset 1: the EMBARC cohort.

The EMBARC recruited 296 patients with MDD and randomly assigned these participants into the sertraline and placebo treatment arms²⁵. After excluding disqualified/withdrawn participants and the individuals with missing or low-quality neuroimaging data, we included 103 sertraline-medicated and 120 placebo-medicated patients with MDD aged between 18 and 65 years in our analyses (Supplementary Fig. 11). The EMBARC study was conducted according to the Food and Drug Administration guidelines and the Declaration of Helsinki and approved by the Institutional Review Board of each clinical site. All participants have given signed informed consent and agreement to all procedures before entry. The inclusion criteria of the EMBARC study included (1) MDD as the primary diagnosis by the Structured Clinical Interview for DSM-IV Axis I Disorders⁷⁰, (2) Quick Inventory of Depressive Symptomatology score ≥14, (3) an MDD episode onset before age 30, either a chronic recurrent episode (duration ≥2 years) or recurrent MDD (≥2 lifetime episodes), and (4) no antidepressant failure during the current episode. The exclusion criteria of the EMBARC study included (1) ongoing pregnancy or breastfeeding, (2) being sexually active without contraception, (3) lifetime history of psychosis or bipolar disorder, (4) substance dependence in the last 6 months or substance abuse in the last 2 months, (5) unstable psychiatric or general medical conditions that require hospitalization, (6) study medication contraindication, (7) clinically relevant laboratory abnormalities, (8) history of epilepsy or conditions requiring an anticonvulsant, (9) electroconvulsive therapy, vagal nerve stimulation, transcranial magnetic stimulation or other somatic treatments in the current episode, (10) ongoing medication intake (including but not limited to antipsychotics and mood stabilizers), (11) ongoing psychotherapy, (12) significant suicide risk and (13) unresponsiveness to any antidepressant at adequate dose and duration in the current episode.

For patients in either treatment arm, an 8-week course of sertraline or placebo was administered. Stratification was enforced by site, depression severity and chronicity for the randomization of treatment arm. The dosing of medications started at 50 mg and increased to a maximum of 200 mg given patient’s tolerance and unresponsiveness to lower dosing. The treatment response was evaluated using HAMD₁₇. Unavailable endpoint HAMD₁₇ scores were imputed from participants’ intermediate treatment response where possible to maximally retain the sample size. In addition, neuroimaging data (including diffusion and functional MRI) were scanned for patients with MDD before the treatment. Clinical trial registration identifier: NCT01407094. URL: http://clinicaltrials.gov/show/NCT01407094.

Dataset 2: the CAN-BIND-1 cohort.

The initial samples of the CAN-BIND-1 study included 196 patients with MDD⁷¹. The inclusion criteria⁷² were (1) outpatients aged 18–60 years, (2) experiencing a major depressive episode diagnosed by DSM-IV Text Revision and confirmed by the Mini International Neuropsychiatric Interview, (3) depressive episode duration ≥3 months, (4) Montgomery–Åsberg Depression Rating Scale (MADRS) score ≥24 and (5) sufficient English proficiency to complete interviews and self-report questionnaires. The exclusion criteria included (1) bipolar I or II disorder, (2) primary diagnosis of psychiatric disorders other than MDD, (3) personality disorder that might interfere with study participation (for example, borderline and antisocial), (4) high suicidal risk, (5) substance dependence or abuse in the past 6 months, (6) neurological disorders, head trauma or other unstable medical conditions, (7) ongoing pregnancy or breastfeeding, (8) psychosis in the current episode, (9) high risk for hypomanic switch, (10) resistance to at least 4 antidepressant medications at therapeutic dosages, (11) previous unresponsiveness or contraindication to escitalopram or aripiprazole, (12) psychological treatment within the past 3 months with intent to continue and (13) contraindications to MRI. The CAN-BIND-1 study was approved by the ethics committees of each participating clinical site. All eligible participants have provided written informed consent for all study procedures at the screening visit.

For patients who had taken psychoactive medications, a wash-out period of at least five half-lives of medication was required. Subsequently, all patients with MDD were treated with 10–20 mg per day escitalopram for 8 weeks. Treatment response was measured as the change in MADRS score from pre- to post-treatment. We calculated the HAMD₁₇ score estimated by equipercentile for a more direct comparison between results obtained with the EMBARC and CAN-BIND-1 cohorts (MADRS ≈ 1.3 × HAMD₁₇)⁷³. Baseline fMRI and DTI scans were also collected.

MRI acquisition

Functional MRI acquisition.

In the EMBARC study, resting-state fMRI data were collected using a single-shot gradient echo-planar pulse sequence with a duration around 5 min. The patients were instructed to keep their eyes open. While different scanners were used at four clinical sites (Columbia University, General Electric 3T; Massachusetts General Hospital, Siemens 3T; University of Texas Southwestern Medical Center, Philips 3T; University of Michigan, Philips 3T), acquisition parameters were standardized. Key parameters included a 2,000 msec repetition time, a 28 msec echo time, a 90° flip angle, a 64 × 64 matrix size, a 3.2 × 3.2 × 3.1 mm³ voxel size, 39 axial slices and 180 image volumes.

In the CAN-BIND-1 study, resting-state fMRI data were acquired over a 10 min scan using a whole-brain T2*-sensitive blood-oxygen-level-dependent (BOLD) echo-planar imaging sequence. Participants were instructed to remain still with their eyes open, focusing on a fixation cross⁷⁴ throughout the scan. Key acquisition parameters, consistent across study sites, included a repetition time of 2,000 msec, an echo time of 30 msec and voxel dimensions of 4 mm × 4 mm × 4 mm (ref. 75). Scanners varied by site (GE 3T for Toronto Western/Toronto General Hospital, Centre for Addiction and Mental Health, McMaster University and University of Calgary; Philips 3T for University of British Columbia; Siemens 3T for Queen’s University). Notably, the University of British Columbia and Queen’s University used a significantly lower pixel bandwidth for fMRI scans⁷⁵, potentially resulting in a low signal-to-noise ratio of their data.

Diffusion MRI acquisition.

The same scanners used for fMRI acquisition were used for diffusion MRI data collection in both cohorts. For the EMBARC cohort, diffusion MRI parameters varied by clinical site: Columbia University used a repetition time of 6 msec, an echo time of 2.4 msec, a 9° flip angle and a 5 min scan duration; Massachusetts General Hospital used a 2.3 msec repetition time, a 2.54 msec echo time, a 9° flip angle and a 4.3 min duration; University of Texas Southwestern Medical Center used an 8 msec repetition time, a 3.7 msec echo time, a 12° flip angle and a 4.24 min duration; University of Michigan used an 8.1 msec repetition time, a 3.7 msec echo time, a 12° flip angle and a 5.29 min duration.

In the CAN-BIND-1 study, DTI data were collected using a single-shot spin-echo echo-planar imaging sequence⁷⁵. Diffusion sensitizing gradients were applied in 31 non-collinear directions (b = 1,000 s mm⁻²) and 6 volumes with b = 0 s mm⁻². The University of British Columbia and Queen’s University also used a significantly lower pixel bandwidth for DTI scans⁷⁵, yielding more distorted images. We therefore excluded all individuals from these two study sites to improve the data quality for replication analysis.

MRI preprocessing

Diffusion MRI preprocessing.

Images with a b-value less than 100 s mm⁻² were first designated as b = 0 images. A denoising procedure was then implemented using the MRtrix3’s method⁷⁶ with a 5-voxel window. Subsequently, correction for B1 field inhomogeneity was conducted using the dwibiascorrect function from MRtrix3 with the N4 algorithm incorporated⁷⁷. Afterward, the mean intensity of the diffusion-weighted imaging (DWI) series was adjusted to calibrate the mean intensity of the b = 0 images across each individual DWI scanning sequence. FSL’s eddy was used for head motion and eddy correction⁷⁸, with a q-space smoothing factor of 10, 5 iterations and 1,000 voxel-based hyperparameter estimation. Linear first- and second-level models were used to characterize eddy current-related spatial distortion and q-space coordinates were assigned to the respective shells. Field offset was separated from subject movement. Post-eddy, eddy’s outlier replacement⁷⁹ was conducted. Data were further organized by slices, ensuring the inclusion of values from slices with a minimum of 250 intracerebral voxels. Outlier groups that deviated by more than 4 standard deviations from the prediction were replaced with imputed values.

Functional MRI preprocessing.

The acquired resting-state fMRI data were preprocessed using the fMRIPrep pipeline⁸⁰. Specifically, the T1-weighted images were corrected for intensity nonuniformity and stripped skull. Spatial normalization was conducted via nonlinear registration with the T1-weighted ref.81. Brain tissue was segmented from the reference brain-extracted T1-weighted image⁸². The fieldmap information was used to correct distortion in low- and high-frequency components caused by field inhomogeneity. Subsequently, a corrected echo-planar imaging reference was obtained from a more accurate co-registration with the anatomical reference. The BOLD reference was then transformed to the T1-weighted image with a boundary-based registration method⁸³. Automatic removal of motion artifacts using independent component analysis⁸⁴ was performed on the preprocessed BOLD time series after removal of non-steady-state volumes and spatial smoothing with an isotropic Gaussian kernel of 6 mm full-width at half-maximum. Lastly, quality control was enforced to remove data with high head motion (>0.5 mm motion framewise displacement or BOLD signal displacements >0.5%)^85,86.

Connectivity feature calculation

SC calculation.

We calculated the SC of all participants using DSI Studio program (https://dsi-studio.labsolver.org). Specifically, we first created a white matter mask with thresholding. Afterward, the spin distribution function of each voxel of masked image was estimated using generalized q-sampling imaging reconstruction⁸⁷ in the T1-weighted space with a diffusion sampling length ratio of 1.25. The tensor metrics were calculated using DWI with b-value lower than 1,750 s mm⁻². The whole-brain fibers were reconstructed using a modified streamline deterministic tracking algorithm⁸⁸ with augmented tracking strategies⁸⁹ and a whole-brain seeding strategy. The parameters were set as angular threshold 45°, step size 1 mm, fiber length extent 20–300 mm and maximum subvoxel search seeds 1,000,000. In each tracking iteration, a random voxel coordinate within the whole-brain seed was selected and fiber tracking algorithm started from the coordinate and tracks in two directions until the fiber reaches the brain border. Tracks with lengths shorter than 30 or longer than 200 mm were discarded. Finally, we extracted SC by calculating the counts of connected fibers linked pair-wise ROIs defined from the Schaefer atlas of 100 parcels²⁶ and subcortical parcellations^27,28. The 35 subcortical ROIs include striatal and cerebellar regions, amygdala regions, anterior and posterior hippocampal regions, and the thalamus. We excluded the SC that do not exist in real brain or only exist in a proportion of participants (<67%) to improve the stability of predictive patterns across cross-validation folds. Lastly, the fiber numbers were log-transformed to enhance normality.

FC calculation.

FC was extracted based on the same cortical and subcortical ROIs as used for SC extraction. The preprocessed fMRI data were averaged into 135 ROI-level time series. For each subject, FC was calculated as the Pearson’s correlation coefficient between the time series of each pair of ROIs. Fisher’s r-to-z transformation was implemented to ensure the normality of the FC features. For participants with two fMRI scans, FC features were respectively calculated for each scan, and SC features were duplicated to form two multimodal samples. Samples corresponding to the same subject were never separated in the cross-validation procedure to prevent information leakage. To achieve and demonstrate maximal generalizability, no data harmonization of SC or FC features was performed across sites within the EMBARC cohort. This approach compelled the models to learn connectivity–response relationships that generalize across subpopulations with potential demographic or clinical variability.

The TOMMF framework (multimodal-based regression model with L0-regularization)

To integrate information from multiple data modalities for improvement in prediction performance, we developed a framework that combines the multimodal fusion and regression task (Fig. 1a). In addition, because the neural connectivity features outnumbered the sample size of participants, sparsity constraint is introduced to control the model complexity, thus alleviating the potential overfitting issue. Notably, here we utilized L0-regularization as the sparsity constraint for its non-shrinking variance and natural interpretability (see Appendix B for details). In brief, compared with the commonly used L1-regularization (also known as LASSO), L0-regularization enables data-driven dimensionality and promotes minimally overlapping components in the latent space (Extended Data Fig. 1), naturally dissecting the overall brain signatures into interpretable subcomponents. We used a technique named continuous sparsification²⁴ to realize the optimization of the non-convex and non-differentiable L0-regularization (see Appendix A for details).

The overall loss function of the TOMMF framework can be formularized as

$$\begin{array}{c}L=\sum _{\mathrm{\Lambda }}^{\{S,F\}}({\rho }_{\mathrm{\Lambda }}{\Vert \mathcal{G}-X_{\mathrm{\Lambda }}{W}_{\mathrm{\Lambda }}\Vert }_F^2+{\lambda }_{\text{fusion}}\sum _{j=1}^P{\Vert W_{\mathrm{\Lambda },j}\Vert }_0)\\ & +K_{\text{pred}}\Vert Y-\mathcal{G}\beta {\Vert }_2^2+{\lambda }_{\text{pred}}\Vert \beta {\Vert }_0.\end{array}$$

The notation $\Lambda$ denotes the set of data modalities; in our case, it contains the structural ($S$) and functional ($F$) modalities. The multimodal fusion term $\sum _{\mathrm{\Lambda }}^{\{S,F\}}{\rho }_{\mathrm{\Lambda }}{\Vert \mathcal{G}-X_{\mathrm{\Lambda }}{W}_{\mathrm{\Lambda }}\Vert }_F^2$ aims to construct a latent space $\mathcal{G}$ that represents the information in agreement across data modalities, where the modality importance ratio ${\rho }_{\mathrm{\Lambda }}$ is a hyperparameter that controls the relative contribution of each data modality. Incorporating ${\rho }_{\mathrm{\Lambda }}$ into the multimodal fusion term enables the framework to lean the latent space toward the data modality that is more informative for a particular prediction target. Remarkably, as we minimize this multimodal fusion term, the distance between $X_S{W}_S\text{and}{X}_F{W}_F$ is also minimized. Therefore, the multimodal fusion step also identifies canonical association between SC and FC features.

The prediction task term $K_{\text{pred}}\Vert Y-\mathcal{G}\beta {\Vert }_2^2$ is a standard loss for prediction residue, where $K_{\text{pred}}$ modulates the relative importance of multimodal fusion and prediction loss. Both multimodal fusion and prediction task are regularized by the L0-norm of their feature weights (W_S, W_F and β). This L0-regularization endows the framework with two important characteristics—data-driven dimensionality and pseudo-exclusive compositions of the latent features (Fig. 1b; see Appendix B for details). These unique properties enable the framework to determine the number of features in the latent space based on their informativeness instead of an empirical percentage, as well as significantly enhance the interpretability of the latent features.

In addition, by incorporating both multimodal fusion and prediction loss, our approach directs the fusion process with guidance from target—the optimal latent space $\mathcal{G}$ is dependent on the target variable $Y$. This crucial relationship ensures that the latent space is tailored to the prediction target, resulting in distinct features for sertraline and placebo responses. In contrast to unsupervised multimodal fusion techniques (Figs. 2c and 3c), our proposed TOMMF framework conducts multimodal fusion in a supervised manner, enhancing the relevance of latent features and thereby improving the prediction performance. The sparsity constraints further eliminate target-irrelevant components in the SC–FC covariation, ensuring its specificity to MDD’s psychopharmacology. With both multimodal fusion and prediction model relying on linear transformations, the overall predictive pattern from original SC and FC features to the antidepressant response is also linear. As a result, the TOMMF framework offers excellent end-to-end interpretability. Specifically, the equation for generating prediction $\left(X_S\text{and}{X}_F\right)$ based on original SC and FC features (X_S and X_F) can be formularized as (see Appendix A for details)

$$\tilde{Y}=\frac{{\rho }_S{X}_S{W}_S+{\rho }_F{X}_F{W}_F}{{\rho }_S+{\rho }_F}\beta .$$

We can observe from the above formula that $\frac{{\rho }_S{W}_S\beta }{{\rho }_S+{\rho }_F}\text{and}\frac{{\rho }_F{W}_F\beta }{{\rho }_S+{\rho }_F}$ are the linear transformation matrices for the original SC and FC features, therefore granting the linear interpretability of the predictive biomarkers for antidepressant response.

Generalization analysis of multimodal treatment response biomarkers

To assess the generalizability of identified treatment response biomarkers, we directly applied the sertraline and placebo response prediction models on the CAN-BIND-1 cohort. Two study sites with substantially different MRI acquisition protocols that might lead to low data quality were excluded from the analysis (see ‘MRI acquisition’ for details). The final model for each treatment was calculated as the average of models in 100 cross-validation folds. No model re-training or fine-tuning was conducted. To simulate real-world application scenarios, where patients are enrolled individually at clinical sites that may lack sufficient data for harmonization, connectivity features from the CAN-BIND-1 cohort were normalized using the mean and variance from the EMBARC cohort. To leverage the longer fMRI scans to improve the data quality, fMRI data of the CAN-BIND-1 cohort were split into two sessions of equal length for FC calculation. An equipercentile-based conversion was used to transform predicted HAMD₁₇ change to symptom relief measured by MADRS⁷³.

Network constellations

The network constellations were constructed to represent the distinct dimensions in the common space (Fig. 1c). To enhance their interpretability, we leveraged the dissection of functional networks, which partition the brain into functional modules⁹⁰. Specifically, we quantified the network components within biomarkers by summing ROI-level connectivity features involving a particular pair of networks, weighted by their coefficients from the antidepressant response prediction model. We calculated these components for SC and FC, respectively. Afterward, we constructed network constellations by parsimoniously aggregating network components with the highest cross-modality correlations. Notably, SC and FC components for the same network were not considered equivalent. To derive psychopharmacological components with high generalizability and reliability, the network constellations were constructed based on sertraline response biomarker and then validated by the placebo response biomarker.

NEO personality traits and cognitive tasks

We aimed to explore the relationship between personality traits and the network constellation components identified within antidepressant response biomarkers, building on recent findings of extensive associations between personality traits and psychopathology²⁹. Personality traits were assessed using the NEO five-factor inventory, including agreeableness, neuroticism, extraversion, openness and conscientiousness in patients with MDD, as part of the EMBARC study²⁵.

In addition, behavioral/cognitive task performance has shown substantial associations with symptom profiles and treatment responsiveness in MDD²⁵. The EMBARC study conducted five cognitive tasks, which we analyzed in this study. These tasks include the A-not-B task, choice reaction, the flanker test, the PRT and the WF test. Specifically, we examined patients’ accuracy in the A-not-B task, CRT, flanker inference on accuracy and reaction time, response bias in the PRT and the total number of valid words in the WF test. To identify potential pre-treatment biomarkers, we included only the cognitive task performance data collected at baseline.

Statistical tests

Comparing ICCs.

To compare the ICCs of predictive patterns derived from unimodal and multimodal frameworks, we performed a two-sample z-test on the z-scores of ICCs. Specifically, Fisher z-transformation was first conducted on ICCs $(Z_i=\frac{1}{2}\mathrm{l}\mathrm{n}(\frac{1+{\mathrm{I}\mathrm{C}\mathrm{C}}_i}{1-{\mathrm{I}\mathrm{C}\mathrm{C}}_i}),i=\mathrm{1,2})$ to ensure the normality of variables⁹¹. The z-statistic for the two-sample z-test can be formularized as $Z_{\text{test}}=\frac{Z_1-Z_2}{{\mathrm{S}\mathrm{E}}_{\mathrm{\Delta }Z}},$ where ${\mathrm{S}\mathrm{E}}_{\mathrm{\Delta }Z}=\sqrt{{\mathrm{S}\mathrm{E}}_1^2+{\mathrm{S}\mathrm{E}}_2^2}$ is the standard error of the difference between means and ${\mathrm{S}\mathrm{E}}_i\left(i=\mathrm{1,2}\right)$ is the standard error of each mean. Given the approximated standard error of Fisher’s z (ref. 92), ${\mathrm{S}\mathrm{E}}_i=\frac{1}{\sqrt{N_i-3}}(i=\mathrm{1,2}),$ we can further formularize the z-statistic as

$$Z_{\text{test}}=\frac{Z_1-Z_2}{\sqrt{\frac{1}{N_1-3}+\frac{1}{N_2-3}}}$$

N₁ and N₂ are the sample sizes for the two ICCs, which are the numbers of total connectivity features. Essentially, we examined the importance of each connectivity feature with the models in each of the cross-validation folds as judges. The P value corresponding to the z-statistic can be found based on standard normal distribution.

Extended Data

Extended Data Fig. 1 |. Data driven dimensionality and dissimilarity of dimension compositions of the latent space.

As the degree of sparsity increases, the latent space dimensions derived from both the L0-regularized (a) and L1-regularized (b) framework first exhibit a shrinkage of degree of freedom (decrease in number of dimensions), demonstrating the data driven dimensionality property that the framework can judge on its own whether a feature is adequately informative to be included in the sparse latent space. However, while the latent dimensions derived with L0-regularization show decent dissimilarity with each other (as quantified by their pairwise cosine similarity), the dimensions derived with L1-regularization gradually converge. After the dimensionality reaches a relative stable number, the L0-regularized framework further enhance the degree of orthogonality across dimensions, while the L1-regularized counterpart further aggregates information from dimensions and emphasizes the dominant dimension. Together, the differences in the properties of latent space demonstrate the unique advantage of L0-regularized framework over its L1-regularized counterpart — the L0-regularization imposes penalty on the occurrence of features in dimensions, especially the features with contribution to multiple dimensions, therefore yielding distinct dimensions with decent dissimilarity with each other. The sparsity parameters (λ_fusion and λ_pred) are harmoniously adjusted and kept same for L0- and L1-regularization. The colormap of latent space shows the weights of informative connectivity features with respect to each of the latent dimensions. The blankness in the rightmost part of latent space is the eliminated dimensions that are automatically set to zero vectors by the framework due to their inadequate informativeness. The latent space of one cross-validation fold is shown for each regularization paradigm as the representative example.

Supplementary Material

Supplementary information The online version contains supplementary material available at https://doi.org/10.1038/s44220-025-00541-0.

Data availability

The EMBARC cohort is publicly available through the National Institute of Mental Health Data Archive (NDA) (https://nda.nih.gov/edit_collection.html?id=2199). The CAN-BIND-1 cohort is available under a data use agreement with Brain-CODE, based at the Ontario Brain Institute (https://www.braincode.ca/content/canadian-biomarker-integration-network-depression-can-bind-0).

Code availability

All analyses were conducted in MATLAB (version R2022b), and the code is available via our GitHub repository at https://github.com/Xiaoyu-Tong/TargetOrientedMultimodalFusion--AntidepressantResponse.