Skip to main content
PMC home page
NeuroImage : Clinical logoLink to NeuroImage : Clinical
. 2019 Nov 6;26:102080. doi: 10.1016/j.nicl.2019.102080

Preliminary prediction of individual response to electroconvulsive therapy using whole-brain functional magnetic resonance imaging data

Hailun Sun a,b, Rongtao Jiang a,b, Shile Qi c, Katherine L Narr d, Benjamin SC Wade d, Joel Upston e, Randall Espinoza d, Tom Jones e, Vince D Calhoun c, Christopher C Abbott e,⁎, Jing Sui a,b,c,f,⁎
PMCID: PMC7229344  PMID: 31735637

Highlights

  • •

    The negative FC networks achieve predictive accuracy of 76.23% for ECT response.

  • •

    The consensus FCs represent predominately frontal, temporal and subcortical regions.

  • •

    FCs that changed significantly were concentrated in frontal and limbic networks.

  • •

    Longitudinal change overlapped with two FCs compared with FC with predictive power.

Keywords: Individualized prediction, Electroconvulsive therapy (ECT), Functional connectivity (FC), Major depressive disorder (DEP), Resting-state fMRI, HDRS, Treatment response

Abstract

Electroconvulsive therapy (ECT) works rapidly and has been widely used to treat depressive disorders (DEP). However, identifying biomarkers predictive of response to ECT remains a priority to individually tailor treatment and understand treatment mechanisms. This study used a connectome-based predictive modeling (CPM) approach in 122 patients with DEP to determine if pre-ECT whole-brain functional connectivity (FC) predicts depressive rating changes and remission status after ECT (47 of 122 total subjects or 38.5% of sample), and whether pre-ECT and longitudinal changes (pre/post-ECT) in regional brain network biomarkers are associated with treatment-related changes in depression ratings. Results show the networks with the best predictive performance of ECT response were negative (anti-correlated) FC networks, which predict the post-ECT depression severity (continuous measure) with a 76.23% accuracy for remission prediction. FC networks with the greatest predictive power were concentrated in the prefrontal and temporal cortices and subcortical nuclei, and include the inferior frontal (IFG), superior frontal (SFG), superior temporal (STG), inferior temporal gyri (ITG), basal ganglia (BG), and thalamus (Tha). Several of these brain regions were also identified as nodes in the FC networks that show significant change pre-/post-ECT, but these networks were not related to treatment response. This study design has limitations regarding the longitudinal design and the absence of a control group that limit the causal inference regarding mechanism of post-treatment status. Though predictive biomarkers remained below the threshold of those recommended for potential translation, the analysis methods and results demonstrate the promise and generalizability of biomarkers for advancing personalized treatment strategies.

1. Introduction

Major depression affects over 350 million lives and contributes to approximately 1 million suicides each year. Though treatable, current interventions are only moderately successful. Two-thirds of patients require two or more antidepressant drug trials and one-third remain unresponsive to multiple trials (McGrath et al., 2006; Trivedi et al., 2006). At least 50% of patients who respond to initial treatment will experience additional episodes, as recurrence is seen in ~80% of patients with a history of two or more episodes (Fava et al., 2006; Greden, 2001; Nemeroff, 2007). The length and frequency of depressive episodes tend to increase and symptoms become more refractory over time (Greden et al., 2011). Chronic depression leads to profound personal suffering, low quality of life, and higher morbidity and mortality (Greden, 2001; Nemeroff, 2007). Personalized treatment strategies offer a potential solution for minimizing these societal and economic crises of depression.

Biomarkers that can predict an individual's response to treatment would provide the opportunity for more successful, personally-tailored treatment strategies (Meng et al., 2019). Biomarkers that change in association with clinical response would inform mechanisms that could also lead to more successful interventions (Osuch et al., 2018). However, a major impedance to identifying biomarkers for antidepressant response stems from the modest overall response rates of standard first-line pharmacotherapies and the typical weeks-to-months delay for these treatments to work. By contrast, ECT is an established treatment that induces an acute and robust clinical response in ~70% of severely depressed patients (Husain et al., 2004;Kho et al., 2003), albeit the ECT response remains variable at the individual level. Consequently, though biological indicators of treatment response are expected to overlap with other forms of treatment, there is also strong rationale for developing evidence-based methods to yield predictors of ECT treatment itself. For example, in the pre-procedural evaluation and consent process, the ECT clinician must balance the anticipated benefits and risks for an individual patient. Ideally, the ECT clinician would be able to accurately predict a patient's likelihood of remission with particular ECT dosage parameters (e.g., unilateral or bilateral electrode placement), socio-demographic information (e.g., family psychiatric history), and symptom severity and phenomenology. However, a recent meta-analysis of ECT predictors has shown that these clinical predictors lack reliability (Haq et al., 2015). Longer symptom duration and antidepressant treatment resistance appear to be the only predictors that impact potential ECT non-response, although alone these factors have been of little practical value for informing clinical practice (Dombrovski et al., 2005; Haq et al., 2015).

Prior neuroimaging studies have utilized data-driven methods to identify predictive biomarkers associated with ECT response (Cao et al., 2018; Jiang et al., 2018a; Leaver et al., 2018; Moreno-Ortega et al., 2019; Redlich et al., 2016; van Waarde et al., 2014; Wade et al., 2016) and relapse (Wade et al., 2017). Both functional and structural pre-ECT MRI data have predicted eventual response with 68 to 90% accuracy. However, most included small sample sizes (between 18 and 53 subjects with a depressive episode), thereby increasing the risk of over-fitting (dependency on the validation data sets) and limiting the generalizability and clinical utility of the identified predictive networks. Furthermore, several of these investigations focused on an a priori defined region of interest such as the striatum (Wade et al., 2016) or hippocampus (Cao et al., 2018). Whole-brain analyses have been performed using structural data in two investigations (Jiang et al., 2018a; Redlich et al., 2016). Both of these investigations have identified predictive (baseline network patterns associated with eventual response) and non-overlapping treatment responsive (network changes during treatment course associated with response) biomarkers. Despite these results, the identified predictive biomarkers from these two investigations have not been replicated with larger datasets and other imaging modalities. Overlap of both predictive- and treatment-responsive biomarkers with different imaging modalities could identify biomarkers for targeted engagement and more focal stimulation for neuromodulation therapies.

In the current investigation, we aim to extend prior findings in smaller samples to identify both functional connectivity predictive and treatment responsive biomarkers from a large, multi-site database (n = 122 subjects with a depressive episode). Functional connectivity has been widely used to describe and predict neuropsychiatric disease severity and cognitive impairment (Jiang et al., 2018b). In this study, we used functional connectivity (FC) to identify predictors of ECT treatment response. Resting state fMRI data permit the assessment of both positive (correlated) and negative (anticorrelated) FCs. In addition, FC are more dynamic and compliment the longer time scale of sMRI neuroplasticity (Batista-Garcia-Ramo and Fernandez-Verdecia, 2018). For the predictive biomarkers, we assessed outcomes with both depression rating scales (continuous outcome) and remitter/non-remitter classification (dichotomous outcomes). For treatment responsive biomarkers, we assessed functional connectivity changes during treatment associated with change in depression ratings. Based on our previous investigation with predictive and treatment responsive biomarkers with structural MRI (Jiang et al., 2018a), we hypothesized that the predictive and treatment responsive biomarkers would be related, but also consist of non-overlapping functional networks.

2. Materials and methods

2.1. Participants and clinical outcomes

Patients with a depressed episode and diagnosis of major depressive disorder or bipolar disorder (DEP) (n = 122) were included from University of New Mexico (UNM) and University of California Los Angeles (UCLA). Exclusion criteria for patients included the following: (1) defined neurodegenerative or neurological disorder (e.g., Alzheimer's disease, epilepsy or head injury); (2) other psychiatric conditions (e.g., schizoaffective disorder, schizophrenia); (3) current alcohol or drug dependence; (4) pregnancy; and (5) contraindication to magnetic resonance imaging (MRI) (e.g., pacemaker). The clinical assessment was the Hamilton Depression Rating Scale-17 items (HDRS) before and after the ECT sessions. ECT response was defined as >50% improvement from baseline in HDRS, and ECT remission defined as final HDRS-17 of <7 (Heijnen et al., 2010). All subjects received their initial assessment prior to the initiation of the ECT series and the final assessment within one week of completing the ECT series. Earlier investigations at UNM included subjects with concurrent psychotropic medications, but the latter investigations at UNM and all UCLA subjects discontinued psychotropic medications prior to commencement of the ECT series. The demographical, clinical and medical characteristics of the sample are summarized in Table 1. All participants provided written informed consent prior to participation in the study. This study was approved by the institutional review boards of UNM and UCLA.

Table 1.

Demographic and clinical information.

UCLA DEP mean (SD) UNM DEP mean (SD) All DEP mean (SD)
Sample size 45 77 122
Age: years 41.2 (13.5) 65.1 (9.0) 56.3(15.9)
Gender: M/F 20/25 23/54 43/79
Education degree 5.8 (1.7) 4.6 (2.3) 5.0 (2.2)
Handiness (R/L) 41/4 77/0 118/4
Pre-ECT HDRS 25.2 (6.2) 25.9 (6.2) 25.6(6.1)
Post-ECT HDRS 16.2 (9.3) 8.9 (8.0) 11.6(9.2)
ΔHDRS (pre-post) 9.0 (9.1) 17.0 (10.0) 14.1 (10.4)
Rate of ΔHDRS ((pre-post)/pre) 0.34 (0.33) 0.64 (0.35) 0.53 (0.37)
Responder (%) 27/45 (60.0) 44/77 (57.1) 71/122(58.2)
Remitter (%) 6/45 (13.3) 41/77 (53.2) 47/122(38.5)
Framewise displacement (pre-ECT) 0.22/0.09 0.26/0.16 0.25/0.14
Framewise displacement (post-ECT) 0.23/0.11 0.27/0.17 0.27/0.16
Antidepressants (%) 0/77 (0.0) 40/77 (51.9) 40/122 (32.8)
Antipsychotic (%) 0/77 (0.0) 22/77 (28.6) 22/122 (18.0)
Open in a new tab

2.2. ECT procedure

ECT procedures, which were similar at both sites, are previously described (Jiang et al., 2018a). Subjects started with right unilateral electrode placement, ultra-brief pulse width (0.3 ms) ECT, with the initial session devoted to seizure threshold (ST) titration and subsequent sessions administered at 6 × ST. Non-responding subjects were transitioned to bitemporal electrode placement for the remainder of the ECT series as clinically determined by the treating psychiatrist. Further adjustments to pulse train duration and frequency occurred as needed to maintain adequate seizure duration (>25 s electroencephalogram seizure activity) and/or acceptable EEG progression and morphology. Treatments occurred three times weekly until maximal sustained clinical response or clinical decision to stop treatment in the context of non-response. Patients were oxygenated throughout the procedure with a disposable bag and mask and received adequate induction (methohexital or etomidate) and relaxation (succinylcholine). Blood pressure, pulse and oxygen saturation were monitored throughout the procedure.

2.3. Data acquisition and preprocessing

The sMRI imaging parameters have been detailed previously (Abbott et al., 2013; Leaver et al., 2016). At UNM, magnetic resonance imaging data were collected on a 3-T Siemens Trio scanner (Siemens Healthcare, Malvern, PA, USA). A whole-brain gradient-echo echoplanar imaging sequence consisted of the following parameters: Repetition time (TR) = 2 s (s), echo time (TE) = 29 milliseconds (ms), flip angle (FA) 75°, voxel size = 3.75 × 3.75 × 4.55 mm, and 154 volumes. At UCLA, magnetic resonance imaging data were collected on a 3-T MAGNETOM Allegra MRI scanner (Siemens, Erlangen, Germany). Functional images consisted of the following parameters: TR = 5 s, TE = 30 ms, FA = 70°, 3.4 × 3.4 × 5 mm3 resolution, and 180 volumes. Standard preprocessing of fMRI in SPM12 (https://www.fil.ion.ucl.ac.uk/spm/) included the following: 1) realignment; 2) slice timing correction; 3) normalization to an EPI template (Calhoun et al., 2017); 4) spatial smoothing using a 6-mm full width half-maximum Gaussian kernel; 5) band-pass temporal filtering (0.01–0.15 Hz); 6) regression of nuisance variables (six parameters obtained by rigid body head motion correction, cerebrospinal fluid, white matter signals and global signal); and 7) regression of the site-covariate from the fMRI data. To further assess motion artifacts, we calculated mean framewise displacement (Table 1). The pre-/post-ECT and UNM/UCLA contrasts were not significant (p < 0.05). We were unable to further censor the data based on framewise displacement secondary to the short acquisition time (154 volumes) (Power et al., 2014).

2.4. Resting-state functional connectivity analysis

The method of calculating resting-state functional connectivity analysis is the same as our previous study (Jiang et al., 2018b). For all subjects, the registered functional MRI volumes in Montreal Neurological Institute standard space were divided into 246 regions of interests (ROI) from the Brainnetome Atlas (Fan et al., 2016) (https://scalablebrainatlas.incf.org/human/BNA), serving as 210 cortical and 36 subcortical nodes for calculating FC. Mean regional time series were obtained for each individual by averaging fMRI time series over all voxels in each of the 246 ROIs. Pearson correlations of time courses between each node pair were calculated, and were normalized to Z scores using Fisher transformation, resulting in a 246 × 246 symmetric FC matrix for each subject. After removing 246 diagonal elements, we extracted the upper triangle elements of the FC matrix as features for prediction, namely, each subject has a feature vector in the dimension of (246 × 245)/2 = 30,135.

2.5. Individualized prediction method

Connectome-based predictive modeling (CPM) (Shen et al., 2017) is a data-driven model integrating several machine-learning methods. This method, which has demonstrated utility for predicting brain-behavior relationships from connectivity data using cross-validation, was used for prediction in the current study. To predict individual ECT response for DEP, we selected pre-ECT (baseline) resting-state functional imaging features to predict eventual change in depression rating scores with proportional change in depression ratings:%HDRS = (pre-ECT HDRS − post-ECT HDRS)/pre-ECT HDRS. Although more conservative than absolute change or post-ECT depression outcomes (Vickers, 2001), the rationale for the use of the proportional change score was to control for the variability of the pre-ECT HDRS.%HDRS and pre-ECT HDRS were correlated (r = 0.2, p = 0.02). Fig. 1 illustrates the individualized prediction framework used in this investigation. We regressed out site prior to the predication analysis and used leave-one-out-cross-validation (LOOCV) to train multiple models.

Fig. 1.

Fig. 1

Open in a new tab

Flowchart of the prediction framework.

In the leave-one-out loop, one subject was left out for the testing dataset, the remaining N-1 subjects were used as the training dataset, which was repeated N times (equal to the subject number) to test all subjects. First, the training stage was divided into three main parts: feature extraction, feature subsets selection and training multiple linear regression (MLR) model. For details of feature extraction, please refer to Section 2.4, Resting-state Functional Connectivity Analysis. Second, feature subsets selection identified representative feature subsets for building the regression model. We computed the spearman correlation (Abubacker et al., 2014; Tripoliti et al., 2010) between%HDRS and the i th feature (i = 1,2,…,30,135) in FC vector, and the r and p values between each group of ith feature and%HDRS were obtained. Then, the K+ features satisfying r>0 and p<positive threshold (PT) were recorded as positive features (PF). The K− features satisfying r<0 and p<negative threshold (NT) were recorded as negative features (NF). The optimal positive and negative threshold were obtained by the grid search method. Finally, all the PF and NF were summed separately to get the final positive features sum (PFS) and negative features sum (NFS). Feature subsets selection thus reduced the degree of redundancy between features. Preferred subsets of features are highly correlated with the predicted measure and have low inter-correlation with other features (Kohavi and John, 1997). Third, combinations of%HDRS and PFS,%HDRS and NFS and%HDRS with summed PFS and NFS trained the MLR. In the testing stage, K+ PF and K− NF were extracted from the testing dataset and by summing PF and NF separately. Finally, PFS and NFS of the testing dataset were combined in the three trained MLR models to predict the continuous value of%HDRS (%HDRSpredict). Based on%HDRSpredict and pre-ECT HDRS, the post-ECT HDRSpredict for each patient could be calculated (i.e., post-ECT HDRSpredict = pre-ECT HDRS -%HDRSpredict × pre-ECT HDRS). Each patient could be further classified as remitter or non-remitter according to post-ECT HDRSpredict (Heijnen et al., 2010). The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and the prediction accuracy for remitter/non-remitter classification were calculated with comparison to true ECT response. Furthermore, in the feature subsets selection, we applied a covariate-control method to retrain the predictive model (Hsu et al., 2018). Specifically, we excluded any FC that was significantly correlated with age, sex, and medication use (two-sample t-test, p < 0.01). This conservative method filters out FCs that covary with%HDRS, potentially decreasing predictive power but minimizing the impact of medication use. We also assessed differences in longitudinal changes in predictive FCs between remitters and not-remitters.

2.6. Identified predictive FCs networks

In the jth leave-one-out loop, Kj+ PF and Kj− NF (j = 1,2,…,122) were extracted from the training dataset to train MLR model. All these functional connectivity networks had the potential to predict%HDRS. The relative weights for all selected FCs (PF and NF) were determined by recording the frequency of each FC in 122 loops. For better interpretation and visualization (Rosenberg et al., 2016), the 246 FC nodes were grouped into 24 relatively larger brain regions anatomically defined by the Brainnetome atlas (Fan et al., 2016; Rosenberg et al., 2016). The contributing weights of these pairs of macroscale regions were estimated by summing the relative weights for the FC (PF and NF respectively) connecting among each pair of macroscale regions. Because different macroscale regions had different numbers of nodes (range from 4 to 16), the contributing weights were divided by the number of nodes in each pair of macroscale regions to get the final mean contribution weights to circumvent the influence of disproportionate nodes. Two groups of predictive FC networks were obtained by performing above process for PF and NF separately. Furthermore, in order to clearly observe the macroscale region most relevant to the prediction of ECT response, we also filtered out PF and NF that were repeatedly extracted in 122 loops (i.e. with a 100% occurrence rate) as consensus FC (Dosenbach et al., 2010; Zeng et al., 2012). We performed the process for two groups of consensus FCs separately to get more concise predictive FC networks.

2.7. Longitudinal analyses and group comparison

For contrasting prediction and treatment response (longitudinal change associated with ECT response), longitudinal analyses and group comparisons were performed. First, we identified FC associated with pre-/post-ECT longitudinal change with paired t-tests. We then examined the degree of overlap between brain regions associated with longitudinal change and predictive brain regions (positive features and negative features) by visual inspection. Second, we identified treatment responsive networks by preforming linear regression (%HRDRS ~ ∆FC + age + sex + medication use + treatment number + site). Third, we assessed longitudinal FC group differences by averaging positive and negative FC between remitters and non-remitters (two-sample t-tests and paired t-tests).

3. Results

3.1. Individualized prediction of ECT outcome

To optimize the hyper-parameters of the prediction model, we performed grid search method using training data. The hyper-parameters for positive (PT) and negative thresholds (NT) were chosen from p = [0.001, 0.002,…, 0.02]. We found the optimal parameters for PT (r > 0, p < 0.012) and NT (r < 0, p < 0.003). These thresholds corresponded to extracted positive (PF) and negative features (NF) of 179 and 101, respectively. The three FC-based prediction regression models achieved significant correlations between%HDRSpredict and the%HDRStrue for 122 DEP (Fig. 2). Specifically, Spearman`s correlations of r = 0.27 (p = 2.30 × 10−3, NRMSE = 0.22), r = 0.51 (p = 6.20 × 10−10, NRMSE = 0.19) and r = 0.46 (p = 7.84 × 10−8, NRMSE = 0.20) were achieved for three combinations of features (PF, NF and PF&NF). To evaluate statistical significance of the r values between true%HDRS and predicted%HDRS generated from LOOCV, we also conducted permutation testing (Hsu et al., 2018). P- values determined with permutation testing were 0.015, 9.99 × 10−4 and 9.99 × 10−4 for PF, NF and PF & NF, respectively. (Supplementary Material Section 2).

Fig. 2.

Fig. 2

Open in a new tab

Scatter plot of the predicted%HDRS with respect to the true%HDRS using PF, NF and PF&NF. Spearman`s correlation of r = 0.27 (p = 2.30 × 10−3), r = 0.51 (p = 6.20 × 10−10) and r = 0.46 (p = 7.84 × 10−8) were achieved for three combinations of features, respectively.

Based on the%HDRSpredict and the pre-ECT HDRS, the post-ECT HDRSpredict could be estimated, and DEP could be further classified as remitter or non-remitter (Heijnen et al., 2010). The classification results using negative features were the most accurate (accuracy=76.23%, sensitivity=51.06%, specificity=92.00%, PPV=80.00%, NPV=75.00%) (Table 2). Please refer to Supplementary Material Section 3 for 10-fold cross-validation, Section 4 for receiver operating curves, and Section 5 for the more conservative covariate-control predictions analysis. Only 47 out of 122 subjects (39%) were ECT remitters. We assessed longitudinal differences in predictive FC by averaging positive and negative predictive FCs and comparing remitters and non-remitters in Supplementary Material Section 6. We performed an analysis with responder criteria (77 out of 122 subjects or 63% were ECT responders) in Supplementary Material Section 7.

Table 2.

The predicted results of%HDRS and ECT remitters using three combinations of features with LOOCV.

Positive features (PF) Negative features (NF) Positive & Negative (PNF)
correlation r/p 0.27/2.30 × 10−3 0.51/6.20 × 10−10 0.46/7.84 × 10−8
RMSE 0.38 0.33 0.34
NMRSE 0.22 0.19 0.20
Predicted Predicted Predicted Predicted Predictedremitter Predictednon-remitter
remitter non-remitter Remitter non-remitter
Ture remitter 14 33 24 23 19 28
True non-remitter 11 64 6 69 6 69
Sensitivitya 29.79% 51.06% 40.42%
Specificitya 85.33% 92.00% 92.00%
PPVa 56.00% 80.00% 76.00%
NPVa 65.98% 75.00% 71.13%
Accuracya 63.93% 76.23% 72.13%
Balanced accuracy 57.56% 71.53% 66.21%
Open in a new tab
a

Abbreviations: ECT, electroconvulsive therapy; HDRS, Hamilton Depression Rating Scale; DEP, major depressive disorder; RMSE, root mean squared prediction error; NRMSE, normalized root mean squared prediction error; NPV, negative predictive value; PPV, positive predictive value.

3.2. Predictive networks for positive and negative FCs

The identified consensus FC (repeatedly extracted in all 122 loops) and the mean contribution weights of consensus FC between each pair of the 24 anatomically defined macroscale areas represented predominately frontal, temporal and subcortical regions (see Fig. 3, details of the relative weights for all selected FC between each pair of macroscale areas in Supplementary Material Section 1). For the positive feature networks, FC demonstrated the most predictive power were between the thalamus (Tha) / superior temporal gyrus (STG), medioventral occipital cortex (MVOcC) / precuneus (Pcun), basal ganglia (BG) / superior frontal gyrus (SFG) and BG / orbital gyrus (OrG). For the negative feature networks, FC demonstrated the most predictive power were between the inferior frontal gyrus (IFG) / the inferior temporal gyrus (ITG), IFG / parahippocampal gyrus (PhG) and IFG / fusiform gyrus (FuG). Other important negative feature networks include the precuneus (Pcun) / middle frontal gyrus (MFG) and BG / insular (INS).

Fig. 3.

Fig. 3

Open in a new tab

The identified consensus FCs (extracted in all 122 loops) and the mean contribution weights of consensus FCs between each pair of the 24 anatomically defined macroscale areas. The left half and right half represented the results of the positive and negative feature networks respectively. Circle plots: the 246 FC nodes were grouped into 24 macroscale brain regions that were anatomically defined by the Brainnetome Atlas. Each line indicated that the FC between the two nodes had been repeatedly extracted 122 times in all loops. Matrix plots: rows and columns represented predefined macroscale regions in Brainnetome Atlas, and a bigger and darker circle represented a higher contribution weight. Names of 24 macroscale regions were colored according to their lobe locations.

In addition, for better visualization, we display the most relevant circuits of positive features and negative features association with treatment response (Fig. 4) with BrainnetViewer (https://www.nitrc.org/projects/bnv/), which showed FCs that demonstrated the most predictive power in Fig. 3.

Fig. 4.

Fig. 4

Open in a new tab

The most relevant circuits of positive features and negative features association with treatment response. Each FC node was colored according to their lobe locations.

3.3. Longitudinal analyses and group comparison

First, to examine longitudinal changes, paired t-tests (pre-/post-ECT) were performed for 30,135 FC. Longitudinal changes were significant (p < 0.01, FDR corrected) in 41 FCs (Fig. 5). FC networks that changed significantly were mainly concentrated in frontal and limbic networks: IFG / SFG, cingulate gyrus (CG) / MFG, inferior parietal lobule (IPL) / OrG, and LOcC / MFG. Compared with FC with predictive power (positive features and negative features, Fig. 3), longitudinal change overlapped with two predictive FCs: right lateral area of OrG (42,31,−9) / left dorsal area of CG (−4, −39, 31) and right lateral area of OrG / right dorsal area of CG (4, −37, 32). To prove that the overlap was not accidental, we performed permutation tests. The dice coefficient (dice=2|x∩y|/(|x|+|y|)) for two edges overlap between the 41 longitudinal edges and the predictive edges (179 positive FC and 101 negative FC) was 0.0125. Then, we randomly selected 41 edges in 1000 times to test the number of overlaps with the predictive edges. There were only 45 times in which the number of overlapping edges was equal to or larger than two, so p value determined with permutation testing were 0.046, which determined that this was more likely than would occur by chance. Second, treatment-responsive networks (remitter/non-remitter contrasts and regression analysis with%HDRS) were not significant (FDR-corrected p > 0.05).

Fig. 5.

Fig. 5

Open in a new tab

Functional connectivity networks changed significantly before and after ECT and the brain regions corresponding to those FCs in response to ECT therapy. Circle plots: the 246 FC nodes were grouped into 24 macroscale brain regions that were anatomically defined by the Brainnetome Atlas. Each line indicated that functional connectivity changed significantly before and after ECT (paired t-test, p < 0.01, FDR corrected). Matrix plots: rows and columns represented predefined macroscale regions in Brainnetome Atlas, and a bigger and darker circle represented a bigger number of FCs. Names of 24 macroscale regions were colored according to their lobe locations.

4. Discussion

One of the goals of identifying biomarkers and the underlying systems-level neural mechanisms of successful response is to achieve individualized prediction of clinical outcomes, which can play a guiding role in clinical practice (Sui et al., 2018). The present study had modest predictive accuracy for individual ECT response based on the combination of whole-brain FCs when implementing a generalized prediction framework. The positive feature network and the negative feature network were extracted as predictors of the change in HDRS-17, achieving 63.93% and 76.23% accuracy of remission and non-remission, respectively. The accuracy for combined positive and negative features was 72.13%. The prediction algorithm also predicted the post-ECT depression rating (maximum r = 0.51, while the corresponding NRMSE is only 0.19). We also assessed longitudinal changes in FC in predominately frontal and limbic networks. Several of the longitudinal change FCs were also associated with prediction, but treatment responsive networks were not identified with the FC data. In the following section, we discuss our findings in the context of ECT predictive and treatment responsive biomarkers and translational implications.

To date, most ECT prediction investigations have focused on sMRI (Cao et al., 2018; Jiang et al., 2018a; Redlich et al., 2016; Wade et al., 2016, 2017). The few resting state fMRI prediction investigations have either utilized data reduction with independent component analysis (van Waarde et al., 2014) or combined rs-fMRI features with other imaging modalities to improve accuracy (Leaver et al., 2018). Our investigation utilized novel methods to assess both positive (increased FC strength) and negative features (decreased FC strength or degree of anti-correlation) associated with ECT response from whole brain rs-fMRI. The predictive performance of the extracted negative features was significantly better than that of the positive features for both predicting post-ECT depression severity (HDRS) and remission status, which may be due to the general decrease of brain functional network connectivity in depressive episodes, and the extent of the decrease may be related to recovery after ECT. Similar to previous investigations (Jiang et al., 2018a; Leaver et al., 2018), our prediction results had strong representation from frontal and temporal networks. Specifically, the positive features were represented subcortical /frontal and subcortical / temporal FC. The negative features had a narrower representation with the inferior frontal gyrus / temporal FC. Our overall accuracy rates were below the 80% accuracy, which is suggested as the standard for clinical utilization (American Psychiatric Association, 2012). The lower accuracy of this investigation may be a function of the larger, multi-site dataset (relative to other ECT prediction studies). A larger dataset may result in less over-fitting or dependency on the test dataset with improved generalizability at the expense of lower accuracy. Furthermore, our rs-fMRI results are less robust than a previous sMRI prediction analysis with a smaller but overlapping sample (Jiang et al., 2018a). Aside from the limitations of the current investigation, comparing these two investigations suggests that pre-ECT sMRI may have more predictive potential than rs-fMRI. Nonetheless, our results are consistent with previous ECT-imaging prediction studies that utilized smaller sample sizes (Redlich et al., 2016; van Waarde et al., 2014).

Previous whole brain sMRI prediction investigations have not detected an intersect between predictive and treatment responsive networks (Jiang et al., 2018a; Redlich et al., 2016). These previous investigations identified smaller predictive and treatment responsive networks based on sMRI features. In the current investigation, the 41 FC networks demonstrating longitudinal change were not related to antidepressant response and are subsequently referred to as “longitudinal-change networks” as opposed to “treatment-responsive networks” for clarity. The predictive FC overlapped with FC networks demonstrating longitudinal change with ECT with the right orbital and cingulate gyri. The right hemisphere changes in longitudinal change networks are consistent with previous ECT-imaging investigations demonstrating neuroplasticity ipsilateral to right unilateral electrode placement (Abbott et al., 2014; Bouckaert et al., 2015). The right orbital frontal cortex was also identified as a top-ranked feature in a recent ECT prediction investigation (Leaver et al., 2018). The overlap of predictive and longitudinal-change biomarkers may have implications in the context of the development of more focal stimulation patterns with an identified target region as well as the elucidating the relationship between predictive and treatment responsive biomarkers.

5. Limitations

Some limitations of this investigation should be acknowledged. First, although the sample size of this paper was larger than previous studies, it was still relatively small considering that the feature dimension of whole-brain FCs was high. Thus, further work in larger samples with validation on multiple independent datasets is preferred, as did in Jiang et al., 2019a, Jiang et al., 2019b. Second, before baseline assessment (pre-ECT), UCLA subjects had gradually reduced and stopped the use of antidepressants. However, a subset of the UNM subjects received psychotropic medications before baseline scanning and throughout the ECT series. Simultaneous use of some antidepressant and antipsychotic treatments may work in conjunction with ECT for some patients and share a similar but less effective mechanism of action (Austin et al., 2001). Specific clinical, illness and treatment features (number of depressed episodes, illness course and onset, total number of ECT treatments, etc.) were not assessed in this investigation and will be the focus of a following study. However, we did perform a more conservative covariate control analysis that demonstrated similar results (Supplemental Material Section 5). Finally, our study design did not include an active control group treated with alternative antidepressant treatments (pharmacotherapy, transcranial magnetic stimulation). Thus, we are unable to compare the probability of response to ECT relative to alternatives (pharmacotherapy), which would be highly relevant to clinical decision making during the ECT consultation. We are unable to exclude other factors (regression to the mean, placebo response, other psychosocial factors) that may be responsible for longitudinal changes in depression severity or the specificity of the identified prediction network to ECT.

6. Conclusion

To the best of our knowledge, this is the largest sample size for an ECT prediction investigation to date. Despite the use of multi-site data and the increased sample size, our data-driven methods were able to produce only comparable accuracy results relative to previous, smaller, single site investigations though benefit from increased generalizability. Accuracy and generalizability may be improved with different atlases, multi-modal imaging (fusion of sMRI and rs-fMRI) and incorporating clinical and demographic features into the predictive analysis. Future work could leverage group independent component analysis (ICA) denoising and/or the use of ICA components instead of atlas-based regions, which provide additional protection against artifacts (Du et al., 2016). In addition, longitudinal study designs where only treatment resistant patients advance to the next phase (for example, pharmacotherapy to transcranial magnetic stimulation, to ECT) could elucidate treatment-specific biomarkers. ECT prediction imaging investigations will not replace but may augment clinical decision making where the probability of benefit (response) are less robust or clear (Dombrovski et al., 2005; Haq et al., 2015). Future research may utilize similar methodology with ECT-mediated neurocognitive impairment to further improve the individualization of the risk and benefit procedural discussion. Finally, prediction studies may improve accuracy with methods to recognize diagnostic and phenomenological heterogeneity such that sub-groups may have unique patterns of predictive biomarkers as opposed to a singular diagnostic or procedural biomarker of eventual clinical outcome (Drysdale et al., 2017).

Declaration of Competing Interest

None.

Acknowledgments

This work is supported in part by the Strategic Priority Research Program of the Chinese Academy of Sciences (grant No. XDB32040100), China Natural Science Foundation (No. 61773380), Beijing Municipal Science and Technology Commission (Z181100001518005), the National Institute of Health (1R01MH117107, R01EB020407, 1R01EB005846, 1R01MH094524, P20GM103472, P30GM122734, U01 MH111826) and (MH092301, MH110008 and MH102743 to UCLA investigators) and the National Science Foundation (1539067).

Footnotes

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.nicl.2019.102080.

Contributor Information

Christopher C Abbott, Email: CAbbott@salud.unm.edu.

Jing Sui, Email: kittysj@gmail.com.

Appendix. Supplementary materials

mmc1.docx (1.4MB, docx)

References

  1. Abbott C.C., Jones T., Lemke N.T., Gallegos P., McClintock S.M., Mayer A.R., Bustillo J., Calhoun V.D. Hippocampal structural and functional changes associated with electroconvulsive therapy response. Transl. Psychiatry. 2014;4:e483. doi: 10.1038/tp.2014.124. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Abbott C.C., Lemke N.T., Gopal S., Thoma R.J., Bustillo J., Calhoun V.D., Turner J.A. Electroconvulsive therapy response in major depressive disorder: a pilot functional network connectivity resting state FMRI investigation. Front. Psychiatry. 2013;4:10. doi: 10.3389/fpsyt.2013.00010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Abubacker N.F., Azman A., Doraisamy S., Murad M.A.A., Elmanna M.E.M., Saravanan R. Correlation-based feature selection for association rule mining in semantic annotation of mammographic medical images. In: Jaafar A., Ali N.M., Noah S.A.M., Smeaton A.F., Bruza P., AbuBakar Z., Jamil N., Sembok T.M.T., editors. Information Retrieval Technology, Airs 2014. 2014. pp. 482–493. [Google Scholar]
  4. American Psychiatric Association, 2012. Consensus Report of the APA Work Group on Neuroimaging Markers of Psychiatric Disorders. In: Association, A.P. (Ed.), Arlington, VA, USA.
  5. Austin M.P., Mitchell P., Goodwin G.M. Cognitive deficits in depression - possible implications for functional neuropathology. Br. J. Psychiatry. 2001;178:200–206. doi: 10.1192/bjp.178.3.200. [DOI] [PubMed] [Google Scholar]
  6. Batista-Garcia-Ramo K., Fernandez-Verdecia C.I. What we know about the brain structure-function relationship. Behav. Sci. 2018;8 doi: 10.3390/bs8040039. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bouckaert F., De Winter F.L., Emsell L., Dols A., Rhebergen D., Wampers M., Sunaert S., Stek M., Sienaert P., Vandenbulcke M. Grey matter volume increase following electroconvulsive therapy in patients with late life depression: a longitudinal MRI study. J. Psychiatry Neurosci. 2015;40 doi: 10.1503/jpn.140322. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Calhoun V.D., Wager T.D., Krishnan A., Rosch K.S., Seymour K.E., Nebel M.B., Mostofsky S.H., Nyalakanai P., Kiehl K. The impact of T1 versus epi spatial normalization templates for fMRI data analyses. Hum. Brain Mapp. 2017;38:5331–5342. doi: 10.1002/hbm.23737. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Cao B., Luo Q., Fu Y., Du L., Qiu T., Yang X., Chen X., Chen Q., Soares J.C., Cho R.Y., Zhang X.Y., Qiu H. Predicting individual responses to the electroconvulsive therapy with hippocampal subfield volumes in major depression disorder. Sci. Rep. 2018;8:5434. doi: 10.1038/s41598-018-23685-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Dombrovski A.Y., Mulsant B.H., Haskett R.F., Prudic J., Begley A.E., Sackeim H.A. Predictors of remission after electroconvulsive therapy in unipolar major depression. J. Clin. Psychiatry. 2005;66:1043–1049. doi: 10.4088/jcp.v66n0813. [DOI] [PubMed] [Google Scholar]
  11. Dosenbach N.U.F., Binyam N., Cohen A.L., Fair D.A., Power J.D., Church J.A., Nelson S.M., Wig G.S., Vogel A.C., Lessov-Schlaggar C.N. Prediction of individual brain maturity using fMRI. Science. 2010;329:1358–1361. doi: 10.1126/science.1194144. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Drysdale A.T., Grosenick L., Downar J., Dunlop K., Mansouri F., Meng Y., Fetcho R.N., Zebley B., Oathes D.J., Etkin A., Schatzberg A.F., Sudheimer K., Keller J., Mayberg H.S., Gunning F.M., Alexopoulos G.S., Fox M.D., Pascual-Leone A., Voss H.U., Casey B.J., Dubin M.J., Liston C. Resting-state connectivity biomarkers define neurophysiological subtypes of depression. Nat. Med. 2017;23:28–38. doi: 10.1038/nm.4246. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Du Y., Allen E.A., He H., Sui J., Wu L., Calhoun V.D. Artifact removal in the context of group ICA: a comparison of single-subject and group approaches. Hum. Brain Mapp. 2016;37:1005–1025. doi: 10.1002/hbm.23086. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Fan L., Li H., Zhuo J., Zhang Y., Wang J., Chen L., Yang Z., Chu C., Xie S., Laird A.R., Fox P.T., Eickhoff S.B., Yu C., Jiang T. The human brainnetome atlas: a new brain atlas based on connectional architecture. Cerebral Cortex. 2016;26:3508–3526. doi: 10.1093/cercor/bhw157. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Fava M., Rush A.J., Wisniewski S.R., Nierenberg A.A., Alpert J.E., McGrath P.J., Thase M.E., Warden D., Biggs M., Luther J.F., Niederehe G., Ritz L., Trivedi M.H. A comparison of mirtazapine and nortriptyline following two consecutive failed medication treatments for depressed outpatients: a STAR*D report. Am. J. Psychiatry. 2006;163:1161–1172. doi: 10.1176/ajp.2006.163.7.1161. [DOI] [PubMed] [Google Scholar]
  16. Greden J.F. The burden of disease for treatment-resistant depression. J. Clin. Psychiatry. 2001;62(Suppl 16):26–31. [PubMed] [Google Scholar]
  17. Greden J.F., Riba M.B., McInnis M.G. 1st ed. American Psychiatric Pub.; Washington, DC: 2011. Treatment Resistant depression: a Roadmap For Effective Care. [Google Scholar]
  18. Haq A.U., Sitzmann A.F., Goldman M.L., Maixner D.F., Mickey B.J. Response of depression to electroconvulsive therapy: a meta-analysis of clinical predictors. J. Clin. Psychiatry. 2015;76:1374–1384. doi: 10.4088/JCP.14r09528. [DOI] [PubMed] [Google Scholar]
  19. Heijnen W.T., Birkenhager T.K., Wierdsma A.I., van den Broek W.W. Antidepressant pharmacotherapy failure and response to subsequent electroconvulsive therapy a meta-analysis. J. Clin. Psychopharmacol. 2010;30:616–619. doi: 10.1097/JCP.0b013e3181ee0f5f. [DOI] [PubMed] [Google Scholar]
  20. Hsu W.T., Rosenberg M.D., Scheinost D., Constable R.T., Chun M.M. Resting-state functional connectivity predicts neuroticism and extraversion in novel individuals. Soc. Cogn. Affect. Neurosci. 2018;13:224–232. doi: 10.1093/scan/nsy002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Husain M.M., Rush A.J., Fink M., Knapp R., Petrides G., Rummans T., Biggs M.M., O'Connor K., Rasmussen K., Litle M., Zhao W., Bernstein H.J., Smith G., Mueller M., McClintock S.M., Bailine S.H., Kellner C.H. Speed of response and remission in major depressive disorder with acute electroconvulsive therapy (ECT): a consortium for research in ect (CORE) report. J. Clin. Psychiatry. 2004;65:485–491. doi: 10.4088/jcp.v65n0406. [DOI] [PubMed] [Google Scholar]
  22. Jiang R., Abbott C.C., Jiang T., Du Y., Espinoza R., Narr K.L., Wade B., Yu Q., Song M., Lin D., Chen J., Jones T., Argyelan M., Petrides G., Sui J., Calhoun V.D. SMRI biomarkers predict electroconvulsive treatment outcomes: accuracy with independent data sets. Neuropsychopharmacology. 2018;43:1078–1087. doi: 10.1038/npp.2017.165. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Jiang R., Calhoun V.D., Cui Y. Multimodal data revealed different neurobiological correlates of intelligence between males and females. Brain Imaging Behav. 2019 doi: 10.1007/s11682-019-00146-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Jiang R., Calhoun V., Fan L., Zuo N. Gender Differences in Connectome-based Predictions of Individualized Intelligence Quotient and Sub-domain Scores. Cereb. Cortex. 2019 doi: 10.1093/cercor/bhz134. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Jiang R., Calhoun V.D., Zuo N., Lin D., Li J., Fan L., Qi S., Sun H., Fu Z., Song M., Jiang T., Sui J. Connectome-based individualized prediction of temperament trait scores. Neuroimage. 2018;183:366–374. doi: 10.1016/j.neuroimage.2018.08.038. [DOI] [PubMed] [Google Scholar]
  26. Kho K.H., van Vreeswijk M.F., Simpson S., Zwinderman A.H. A meta-analysis of electroconvulsive therapy efficacy in depression. J. ECT. 2003;19:139–147. doi: 10.1097/00124509-200309000-00005. [DOI] [PubMed] [Google Scholar]
  27. Kohavi R., John G.H. Wrappers for feature subset selection. Artif. Intell. 1997;97:273–324. [Google Scholar]
  28. Leaver A.M., Espinoza R., Pirnia T., Joshi S.H., Woods R.P., Narr K.L. Modulation of intrinsic brain activity by electroconvulsive therapy in major depression. Biol. Psychiatry. Cogn. Neurosci. Neuroimag. 2016;1:77–86. doi: 10.1016/j.bpsc.2015.09.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Leaver A.M., Wade B., Vasavada M., Hellemann G., Joshi S.H., Espinoza R., Narr K.L. Fronto-Temporal connectivity predicts ect outcome in major depression. Front. Psychiatry. 2018;9:92. doi: 10.3389/fpsyt.2018.00092. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. McGrath P.J., Stewart J.W., Fava M., Trivedi M.H., Wisniewski S.R., Nierenberg A.A., Thase M.E., Davis L., Biggs M.M., Shores-Wilson K., Luther J.F., Niederehe G., Warden D., Rush A.J. Tranylcypromine versus venlafaxine plus mirtazapine following three failed antidepressant medication trials for depression: a STAR*D report. Am. J. Psychiatry. 2006;163:1531–1541. doi: 10.1176/ajp.2006.163.9.1531. quiz 1666. [DOI] [PubMed] [Google Scholar]
  31. Meng X., Jiang R., Lin D., Bustillo J., Jones T., Chen J. Predicting individualized clinical measures by a generalized prediction framework and multimodal fusion of MRI data. Neuroimage. 2019;145:218–229. doi: 10.1016/j.neuroimage.2016.05.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Moreno-Ortega M., Prudic J., Rowny S., Patel G.H., Kangarlu A., Lee S., Grinband J., Palomo T., Perera T., Glasser M.F., Javitt D.C. Resting state functional connectivity predictors of treatment response to electroconvulsive therapy in depression. Sci. Rep. 2019;9:5071. doi: 10.1038/s41598-019-41175-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Nemeroff C.B. Prevalence and management of treatment-resistant depression. J. Clin. Psychiatry. 2007;68(Suppl 8):17–25. [PubMed] [Google Scholar]
  34. Osuch E., Gao S., Wammes M. Complexity in mood disorder diagnosis: fMRI connectivity networks predicted medication-class of response in complex patients. Acta Psychiatrica Scandinavica. 2018;138(5):472–482. doi: 10.1111/acps.12945. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Power J.D., Mitra A., Laumann T.O., Snyder A.Z., Schlaggar B.L., Petersen S.E. Methods to detect, characterize, and remove motion artifact in resting state fMRI. Neuroimage. 2014;84:320–341. doi: 10.1016/j.neuroimage.2013.08.048. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Redlich R., Opel N., Grotegerd D., Dohm K., Zaremba D., Burger C., Munker S., Muhlmann L., Wahl P., Heindel W., Arolt V., Alferink J., Zwanzger P., Zavorotnyy M., Kugel H., Dannlowski U. Prediction of individual response to electroconvulsive therapy via machine learning on structural magnetic resonance imaging data. JAMA Psychiatry. 2016;73:557–564. doi: 10.1001/jamapsychiatry.2016.0316. [DOI] [PubMed] [Google Scholar]
  37. Rosenberg M.D., Finn E.S., Scheinost D., Papademetris X., Shen X., Constable R.T., Chun M.M. A neuromarker of sustained attention from whole-brain functional connectivity. Nat. Neurosci. 2016;19:165–171. doi: 10.1038/nn.4179. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Shen X.L., Finn E.S., Scheinost D., Rosenberg M.D., Chun M.M., Papademetris X., Constable R.T. Using connectome-based predictive modeling to predict individual behavior from brain connectivity. Nat. Protoc. 2017;12:506–518. doi: 10.1038/nprot.2016.178. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Sui J., Qi S., van Erp T.G.M., Bustillo J., Jiang R., Lin D., Turner J.A., Damaraju E., Mayer A.R., Cui Y., Fu Z., Du Y., Chen J., Potkin S.G., Preda A., Mathalon D.H., Ford J.M., Voyvodic J., Mueller B.A., Belger A., McEwen S.C., O'Leary D.S., McMahon A., Jiang T., Calhoun V.D. Multimodal neuromarkers in schizophrenia via cognition-guided MRI fusion. Nat. Commun. 2018;9 doi: 10.1038/s41467-018-05432-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Tripoliti E.E., Fotiadis D.I., Argyropoulou M., Manis G. A six stage approach for the diagnosis of the alzheimer's disease based on fMRI data. J. Biomed. Inform. 2010;43:307–320. doi: 10.1016/j.jbi.2009.10.004. [DOI] [PubMed] [Google Scholar]
  41. Trivedi M.H., Rush A.J., Wisniewski S.R., Nierenberg A.A., Warden D., Ritz L., Norquist G., Howland R.H., Lebowitz B., McGrath P.J., Shores-Wilson K., Biggs M.M., Balasubramani G.K., Fava M., Team S.D.S. Evaluation of outcomes with citalopram for depression using measurement-based care in STAR*D: implications for clinical practice. Am. J. Psychiatry. 2006;163:28–40. doi: 10.1176/appi.ajp.163.1.28. [DOI] [PubMed] [Google Scholar]
  42. van Waarde J.A., Scholte H.S., van Oudheusden L.J., Verwey B., Denys D., van Wingen G.A. A functional MRI marker may predict the outcome of electroconvulsive therapy in severe and treatment-resistant depression. Mol. Psychiatry. 2014 doi: 10.1038/mp.2014.78. [DOI] [PubMed] [Google Scholar]
  43. Vickers A.J. The use of percentage change from baseline as an outcome in a controlled trial is statistically inefficient: a simulation study. BMC Med. Res. Methodol. 2001;1:6. doi: 10.1186/1471-2288-1-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Wade B.S., Joshi S.H., Njau S., Leaver A.M., Vasavada M., Woods R.P., Gutman B.A., Thompson P.M., Espinoza R., Narr K.L. Effect of electroconvulsive therapy on striatal morphometry in major depressive disorder. Neuropsychopharmacology. 2016;41:2481–2491. doi: 10.1038/npp.2016.48. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Wade B.S.C., Sui J., Hellemann G., Leaver A.M., Espinoza R.T., Woods R.P., Abbott C.C., Joshi S.H., Narr K.L. Inter and intra-hemispheric structural imaging markers predict depression relapse after electroconvulsive therapy: a multisite study. Transl. Psychiatry. 2017;7:1270. doi: 10.1038/s41398-017-0020-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Zeng L.-.L., Shen H., Liu L., Wang L., Li B., Fang P., Zhou Z., Li Y., Hu D. Identifying major depression using whole-brain functional connectivity: a multivariate pattern analysis. Brain. 2012;135:1498–1507. doi: 10.1093/brain/aws059. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

mmc1.docx (1.4MB, docx)

Articles from NeuroImage : Clinical are provided here courtesy of Elsevier

RESOURCES