Classifying Schizophrenia Using Multimodal Multivariate Pattern Recognition Analysis: Evaluating the Impact of Individual Clinical Profiles on the Neurodiagnostic Performance

Abstract

Previous studies have shown that structural brain changes are among the best-studied candidate markers for schizophrenia (SZ) along with functional connectivity (FC) alterations of resting-state (RS) patterns. This study aimed to investigate effects of clinical and sociodemographic variables on the classification by applying multivariate pattern analysis (MVPA) to both gray matter (GM) volume and FC measures in patients with SZ and healthy controls (HC). RS and structural magnetic resonance imaging data (sMRI) from 74 HC and 71 SZ patients were obtained from a Mind Research Network COBRE dataset available via COINS (http://coins.mrn.org/dx). We used a MVPA framework using support-vector machines embedded in a repeated, nested cross-validation to generate a multi-modal diagnostic system and evaluate its generalizability. The dependence of neurodiagnostic performance on clinical and sociodemographic variables was evaluated. The RS classifier showed a slightly higher accuracy (70.5%) compared to the structural classifier (69.7%). The combination of sMRI and RS outperformed single MRI modalities classification by reaching 75% accuracy. The RS based moderator analysis revealed that the neurodiagnostic performance was driven by older SZ patients with an earlier illness onset and more pronounced negative symptoms. In contrast, there was no linear relationship between the clinical variables and neuroanatomically derived group membership measures. This study achieved higher accuracy distinguishing HC from SZ patients by fusing 2 imaging modalities. In addition the results of RS based moderator analysis showed that age of patients, as well as their age at the illness onset were the most important clinical features.

Introduction

Multivariate pattern recognition methods, such as support vector machines (SVM)^1,2 have recently emerged as promising tools for the diagnosis of schizophrenia (SZ) due to their capacity to extract the inter-regional dependencies of distributed brain patterns from high-dimensional training data and generalize the learned discriminative rules to unseen patient cohorts.^3–5 Classification results from the recent studies suggest that grey matter (GM) and functional connectivity (FC) alterations provide useful discriminative power to identify patients with SZ, revealing complex patterns of cortical changes underlying the symptoms of the disorder.⁶

Extant magnetic resonance imaging (MRI) findings heavily support the “brain surrogate” of SZ that is not confined to single regions but rather involves distributed pattern of brain alterations in both structural^4,7 and functional MRI (fMRI) modality.^8,9 However, although GM changes in prefrontal, striatal, temporal and parietal regions, as well as FC deficits in a long- and short-range connections between these brain regions are well replicated they are susceptible to the effects of sociodemographic variables and age in particular.^10–12

In the field of neuroanatomical disease classification it has also been shown that clinical factors such as patients’ age, age of illness onset and symptoms may affect diagnostic accuracy. A recent meta-analysis⁶ has shown that although illness duration itself did not have a significant impact on sensitivity and specificity of the unimodal classification, older patients and chronic stage of illness were significantly associated with higher sensitivity in RS fMRI data. These findings might result from more pronounced FC alterations in older subjects with SZ as compared with first-episode patients. The more pronounced FC alterations have been previously shown in functional studies that reported a relationship between altered connectivity (topological measurements) and illness duration in SZ.¹³

However, the majority of these multivariate pattern analysis (MVPA) studies used only 1 MRI modality to determine the diagnostic membership of study participants.¹⁴ Some of the recent multimodal imaging studies investigating neuronal abnormalities in different MRI modalities came to a conclusion that multimodal fusion is an effective approach to elucidate disease factors that are shared across different modalities.^15–17

However, more work needs to be done to explore factors determining the diagnostic performance of MRI-based biomarkers across the life span. Within this context, the first aim of this study was to thoroughly evaluate the moderating impact of sociodemographic and clinical variables on the model generated by MRI-based classifiers.

Previous studies have shown that integration of different MRI modalities into multi-modal disease models may provide superior classification performance compared to each of these single data channels alone.^16,18–21 The potential benefit of combining different MRI modalities lies in the increase of the multivariate pattern classification performance, as well as a broader insight into the neurobiological processes underlying SZ. However, defining the optimal way for combining information from different modalities is challenging. Overfitting needs to be avoided and the fusion process has to lead to disease classifiers that remain robust and operational even if 1 data channel is lost.¹²

We hypothesized that the sociodemographic and clinical moderators heavily determine the applicability of multimodal predictive models. Furthermore, we explored the hypothesis that longer duration of illness and higher level of psychotic symptoms are paralleled by more pronounced GM and FC abnormalities in patients.

Methods

Participants

The study included 74 HC and 71 SZ patients. Imaging and phenotypic information was drawn from a Mind Research Network Center of Biomedical Research Excellence (COBRE), funded by the National Institutes of Health. This data is publicly available via the COINS data exchange (http://coins.mrn.org/dx).²² Detailed study procedures are described at http://cobre.mrn.org/. In summary, patients received the diagnosis of SZ by clinical psychiatrists using the Structured Clinical Interview for DSM-IV.

Psychopathological symptoms were rated using the Positive and Negative Syndrome Scale (PANSS).²³ The antipsychotic medication at MRI scan was converted to chlorpromazine and olanzapine equivalents (table 1).

Table 1.

Sociodemographic and Clinical Characterization of the Sample

Sociodemographic Variables	Patient Group (SZ)	Healthy Group (HC)	Statistics
Complete sample
N	71 M/SD	74 M/SD	—
Mean age at baseline [y] (SD)	38.1 (13.9)	35.8 (11.5)	t = 1.09, P = .278
Handedness (mixed or left)	10 (13.5%)	1 (1.4%)	χ2 = 7.57, P = .001
Sex (male)	57 (80.3%)	51 (68.9%)	χ2 = 2.46, P = .12
Educational level (years of education)	13 (1.8)	14.4 (3.3)	t = 3.08, P = .003
Mean verbal IQ [MWT-B]	98.6 (1.8)	105.1 (14.3)	t = 2.59, P = .011
Matched sample (age and sex)
N	66	66	—
Mean age at baseline [y] (SD)	36.4 (12.9)	37 (10.7)	t = 0.301, P = .764
Sex (male)	52 (78.8%)	47 (71.2%)	χ2 = 1.01, P = .315
Independent validation set
N	5	8	—
Mean age at baseline [y] (SD)	61.0 (2.65)	25.9 (14.7)	—
Sex (male)	5 (100%)	4 (50%)	—
Clinical battery (N = 58)
Age at first psychotic symptoms	20.6 (6.3)	—	—
Age at first psychiatric hospitalization	22.6 (6.7)
Age at first psychiatric illness	21.1 (6.74)		—
Mean PANSS positive symptoms score (SD)	15.0 (4.9)	—
Mean PANSS negative symptoms score (SD)	14.8 (4.9)	—	—
Mean PANSS general score (SD)	29.6 (8.6)	—	—
Current medication dose CPZ equivalent	352.6 (309.7)	—	—
Current medication dose OLZ equivalent	14.9 (10.73)	—	—

Note: SZ, schizophrenia; HC, healthy controls; PANSS, Positive and Negative Syndrome Scale; MWT-B, Mehrfachwahl-Wortschatz-Intelligenztest Version B; CPZ, Chloropromazine; OLZ, Olanzapine.

All subjects were screened and excluded if they had any history of neurological disorder, history of mental retardation, history of severe head trauma with more than 5 minutes loss of consciousness, history of substance abuse or dependence within the last 12 months. Informed consent was obtained from all subjects according to institutional guidelines required by the Institutional Review Board at the University New Mexico (UNM).

Data Acquisition and Preprocessing

All participants were scanned on a 3 Tesla SIEMENS TIM scanner with a 12-channel radio-frequency coil at the Mind Research Network. All the prepocessing steps were performed in the NeuroDiagnostic Applications group of the Psychiatric Hospital of the LMU.

Structural Image Acquisition and Preprocessing

Structural images were obtained using a multi-echo MPRAGE sequence with the following parameters: time of repetition (TR) = 2530, echo time (TE) = (1.64, 3.5, 5.36, 7.22, 9.08), inversion time (TI) = 900ms, flip angle = 7°, field of view (FOV) 256×256mm, slice thickness = 176mm, matrix size 256×256×176, voxel size 1mm × 1mm × 1mm, Pixel band width 650 Hz. Structural MRI data were preprocessed using the VBM 8 toolbox (http://dbm.neuro.uni-jena.de/vbm8/), which was used to segment the brain into white matter (WM), GM, and cerebral spinal fluid (CSF). The VBM8 toolbox extends the unified segmentation model of SPM8²⁴ by the (1) application of the Optimized Blockwise Nonlocal-Means Filter to increase the signal-to-noise ratio of the data,²⁵ (2) segmentation into GM, WM, and CSF using an adaptive maximum a posteriori approach²⁶ extended by a partial volume estimation model,^27,28 and (3) post-processing using a hidden Markov Random Field model.²⁹ This preprocessing produced modulated and GM segments were then normalized to MNI structural template using the DARTEL algorithm³⁰ and re-sampled to 3mm, isotropic resolution.

Functional Image Acquisition and Preprocessing

Blood oxygenation level dependent (BOLD) images of the whole brain using an echo planar imaging (EPI) sequence were acquired in 32 axial slices (TR = 2000ms, TE = 29ms, flip angle =75°, FOV = 24cm, 3.5mm thickness and 1.05mm gap, matrix size= 64×64, voxel size 3.75mm × 3.75mm × 4.55mm) using the intercommissural line (AC-PC) as a reference. RS scans resulted in 304 seconds duration (152 volumes) and subjects were instructed to keep their eyes open during the scan and stare passively at a foveally presented fixation cross, as this is suggested to facilitate network delineation compared to eyes-closed conditions and helps to ensure that subjects are awake.

Functional RS data were processed as follows: First, after discarding the first 10 images (magnetization equilibrium not reached) and 2 dummy scans, the remaining 140 images were unwarped and realigned to the first volume for head-motion correction. Secondly, the time course of head motions was obtained by estimating the translations in each direction and the rotations in angular motion about each axis for each volume using SPM8, (http://www.fil.ion.ucl.ac.uk/spm). The unwarped and realigned images underwent background filtering and exclusive masking of the cerebellum to account for the low signal to noise ratio in this brain region. Subsequently, temporal band-pass filtering (0.01–0.08 Hz) was performed to reduce the effects of low-frequency drift and high-frequency noise using the REST toolbox³¹ (http://restfmri.net/forum/REST). We regressed out the global brain signal, mean white matter signal (0.9 threshold), mean CSF signal (0.7 threshold) and the 6 motion parameters.

Then, the Montreal Neurological Institute structural template was high-dimensionally registered to each subject’s structural scan using the DRAMMS algorithm³² (https://www.nitrc.org/projects/dramms/), which encoded linear and nonlinear deformations in the template space relative to each individual’s subject space. Following a rigid body transformation (6degrees of freedom) from the subject’s structural image to the mean functional image, the 2 transformations were respectively applied to the template-space AAL atlas³³ thus parcellating each subject’s series of functional images using the warped AAL atlas. Then, the mean time series in each of the 90 AAL regions was computed by averaging voxel-level time series. The FC was then estimated by calculating the mutual Information³⁴ between the mean time series of each pair of brain regions for each subject (mutual information matrices; MIM).

We decided to work in the original functional space in order to avoid the reslicing of the functional images to a higher dimensional space.

Multivariate Pattern Classification Analysis

We used our pattern recognition tool NeuroMiner (http://www.pronia.eu/the-project/work-plan/wp2-surrogate-marker/) to implement a fully automated machine learning pipeline, which (1) constructed sets of predictive neuroanatomical and neurofunctional features from the high-dimensional GM maps and MIM respectively, and (2) learned decision rules from these features to classify patients with HC versus SZ at the single-subject level.

Firstly, to account for age, sex and size differences between the samples, the HC and SZ cohort were matched according to age and sex, resulting in 2 smaller samples containing 66 subjects each. The remaining 13 subjects were used as independent validation set, meaning that these subjects were kept out of the model estimation process and used as previously unseen data (table 1).

To strictly separate the training process from the evaluation of the predictor’s generalization capacity, the pipeline was completely embedded into a repeated double cross-validation framework (rdCV).³⁵ As previously described, rdCV computes an unbiased estimate of the method’s expected diagnostic accuracy on new cases, rather than merely fitting the current study population. Furthermore, rdCV produces classifier ensembles that separate single individuals from different groups, while avoiding overfitting to the peculiarities of the training data.

More specifically, the following analysis steps were wrapped into a 20-fold cross-validation cycle at the outer (CV2) and a 20-fold cross-validation at the inner (CV1) cycles of rdCV. Both MIM and GM maps were scaled feature wise and for the GM maps, due to its inherent high dimensionality, principal component analysis (PCA)³⁶ was used to reduce feature dimensionality. The principal components with the highest eigenvalues that cumulatively explained 80% of the variance were selected and the single subject GM maps were projected into the reduced principal components space. The resulting PC features (GM maps) and the scaled MIM (FC) were, independently, fed to a linear L2-regularized Logistic Regression (L2-LR) implemented in the LIBLINEAR toolbox (http://www.csie.ntu.edu.tw/~cjlin/liblinear/). The slack variable, C, was estimated in the inner cycle of the cross validation, for both modalities independently.

Finally, unseen CV1 and CV2 test subjects, as well as individuals in the independent validation set were processed by successively applying all training parameters to the test data. The classifier determined a test subject’s geometric position relative to the learned decision boundary, resulting in a group membership score and predicted membership (determined by sign of the group membership score). This analysis sequence was repeated for each CV1 training partition in a given CV2 training fold, thus generating an ensemble classifier which computed a CV2 test subject’s group membership by averaging the decision scores of its L2-LR base learners. Finally, for each subject, ensemble group membership scores were averaged across those training partitions, in which the subject had not been involved in the training and optimization process, and thus determining its final out-of-training (OOT) group membership score.

The previously described classification procedure was applied independently to both GM maps and MIM resulting, for each subject, in 2 OOT decision score, one based on the structural and one the fMRI data. The fusion of the data domains was then performed by taking the mean of the 2 OOT decision scores for each subject, resulting in a multi-modal class membership score from which the multimodal class membership was derived.

The models described above were then applied to independent test data to further validate the method.

For the clinical moderator analyses, a ν support vector regression (ν-SVR) was applied to predict the single modality OOT group membership scores based on clinical variables (table 1; for more details supplementary materials table 3). The clinical analysis was performed on 62 SZ subjects with complete data sets. Similarly to the classification process an rdCV framework was employed to derive an unbiased estimation of the method’s performance. For each CV2 fold the features were scaled, feature wise and a filter method, the Pearson correlation between the feature and the label, was applied to filter uninformative features.³⁷ The selected scaled clinical features were used to predict the group membership scores with a ν-SVR as implemented in LIBSVM with ν and C estimated in the inner cycle of the CV framework.

Results

Both functional and structural classifiers were able to distinguish between HC and SZ patients with similar accuracies. The GM classifier showed a slightly lower accuracy of 69.7 % (P < .001; specificity 63.4% / sensitivity 82.4%) comparing to 70.5% accuracy of the RS classifier (specificity 69.7% / sensitivity 71.2%).

The accuracy for the OOCV dataset in the RS fMRI was 76.9 % (P = .09; specificity 62.5 % / specificity 100 %) and 92.3 % (P = .003; specificity 87.5 % / sensitivity 100 %) for the structural magnetic resonance imaging (sMRI).

Ensemble-based data fusion outperformed pattern classification based on single MRI modalities by reaching 75 % accuracy classifier (P < .001; specificity 71.2% / sensitivity 78.8%).

For the OOCV data the fusion improved the classification accuracy achieving for this dataset 100% accuracy (P < .001).

Structural Analysis

The structural pattern identified by the classifier (figure 1) consisted of (1) GM volume reductions in bilateral frontal, temporal, angular, supramarginal, insular, anterior cingulate, parahippocampal, and postcentral cortices and bilateral subcortical structures (striatum and thalamus), and (2) GM volume increments in the cerebellum, as well as parietal and occipital brain regions (supplementary materials table 1).

Fig. 1.

Percentage difference in grey matter (GM) between healthy controls (HC) and schizophrenia (SZ) for the 95th percentile of reliable voxels. Reliability is defined as the mean value of the L2-LR weight divided by its standard error across all the generated models.

RS-fMRI Analysis

Decreased FC highly predictive in the RS-fMRI classification of SZ vs HC was detected between fronto-occipital, fronto-parietal, fronto-temporal and cortico-thalamic regions. Increased FC in SZ vs HC was observed between the left inferior temporal gyrus and parahippocampal gyrus (figure 2). For more information please see supplementary materials table 2.

Fig. 2.

Percentage difference in connectivity between healthy controls (HC) and schizophrenia (SZ) for the 99th percentile of reliable connections. Reliability is defined as the mean value of the L2-LR weight divided by its standard error across all the generated models.

Effect of Clinical and Demographical Measures on Neuroimaging Results

The SVR model significantly predicted RS-based OOT group membership probabilities based on clinical and sociodemographic measures (r = .48, P = 1.2×10⁻⁴, T = 4.15, coefficient of determination= 23.5, mean average error = 0.24 df = 57). This model was then applied to the SZ subjects with complete data in the independent test data obtaining a mean average error of 0.24, supplementary materials figure 1.

The identical approach failed when predicting sMRI-based OOT probabilities using the same clinical and sociodemographic battery (r = −.18, P = .17, T = −1.39, Coefficient of Determination= 3.3, df = 57).

Using a filter to perform feature selection in conjunction with the SVR algorithm allowed us to measure the relevance of the clinical and demographic variables based not only on the weight of each variable but also on its probability of being selected by the filter (supplementary materials table 3).

The weight profile of features derived from the SVR approach suggests that older SZ patients and those with a “later” onset of illness (age > 25) seem to be more robustly recognized as patients. Notably, frequently selected demographic and clinical features were: current age, age of the first psychotic symptoms, age at first psychiatric illness, age at first psychiatric hospitalization, difficulty in abstract thinking, emotional withdrawal and PANSS negative summary score.

Discussion

This study aimed to investigate potential effects of clinical and sociodemographic variables on the performance of neurodiagnostic classification. We initially confirmed that combination of GM and RS predictions outperformed pattern classification based on single MRI modalities by reaching 75% accuracy for this analysis compared to ~70% accuracy for the single modalities. The application of the proposed method to the independent test data resulted in accuracies near 100% this might be a consequence of the small size and demographic profile of this subsample, notably the age differences between the groups, young HC versus old SZ (table 1). Furthermore, the filter method employed as feature selection approach in the regression helped us to identify the subset of the clinical and demographic features that were most predictive for the RS decision scores. The particular features as patients’ current age and age of illness onset had the highest probability to be selected. Specifically, older SZ patients with a later onset of illness and more pronounced negative symptoms were more reliably assigned to the SZ group by the RS classifier. Our findings about more pronounced changes in FC in older subjects with SZ could be a result of accelerated „brain aging” in SZ,¹² as previous studies on normal aging already pointed out at reduced FC in healthy elderly.³⁸ Additionally, the age of onset is an important predictor of SZ group membership as patients with a “later” age of illness onset, after adolescence, seem to be characterized by greater FC deficits. Previous MRI studies on biomarkers have shown that brain-specific deviations are dynamically related to age and illness duration trajectories.^13,39,40 For example, several studies showed that structural classifiers are very sensitive to age-related effects across the lifespan,¹² reporting accelerated „brain aging” in SZ putatively due to dysmaturational processes in the early course of the illness. The latest publications in the field are indicating that the acceleration is especially large in the first years after the disease onset.⁴¹ Though we were able to replicate findings from diverse studies that suggest GM changes in SZ in a pattern involving both cortical and subcortical structures the structural classifier based on GM alterations operated independently of sociodemographic and clinical measures in our study. The reason could lie in the heterogeneous cohort that consist of many chronic SZ Patients with different onset and duration of illness in which the difference between chronological and the neuroanatomical age (brain age gap estimation [BrainAGE]) might be exhibiting ceiling effect.

In the RS data domain it seems that the “later” illness onset, after adolescence, may be responsible for the more severely disrupted FC in patients with SZ. This may not be reflecting the innate relationship between the FC and age trajectories, but could be due to the fact that the later onsets of illness are associated with older patients in our sample.

Majority or aberrant connections in the SZ group were subcortical short range connections with a particular emphasis on interhemispheric connections. This is not surprising considering that previous studies on inter-hemispheric connectivity in SZ have shown evidence for weakening of FC between the 2 brain hemispheres involving a number of cortical and limbic regions.⁴² Aberrant long range connections of the SZ group in fronto-occipital, fronto-parietal,⁴³ fronto-temporal^9,44 and cortico-thalamic regions⁴⁵ were also embedded in the most predictive set of functional features.

Importantly, the aberrant fronto-temporal connectivity could explain the impaired recruitment of cognitive verbal processing⁴⁶ which contributes to negative symptoms in the speech domain.^47–49 Aberrant cortico-striatal connections have also been previously shown to be related to negative symptoms of the SZ group.⁵⁰

The limitation of the study is that the chronic effects of antipsychotics could contribute to the altered FC with longer duration of illness. Unfortunately, although the sample contains information on the medication, we had no information on “life time” medication. Importantly, such chronic exposure to antipsychotic medication may also alter FC.⁵¹

Our results from the recent meta-analysis underline the utility of multivariate pattern recognition approaches despite the clinical heterogeneity of the SZ phenotype. We demonstrated that combination of sMRI and RS fMRI provides even higher accuracy in discriminating SZ patients from HC by optimizing deficiencies of single modality prediction. Furthermore, our finding on aberrant connectivity in SZ that starts in early adulthood supports the hypothesis that such psychopathological abnormalities are paralleled by more pronounced brain abnormalities which could be more easily detectable with RS classifiers compared to structural classifiers.

Future studies need to proceed further to identify clinical variables that are relevant for the real-world application, as well as for the clinical reliability of multivariate methods and novel therapeutical approaches.

Supplementary Material

Supplementary material is available at http://schizophreniabulletin.oxfordjournals.org.

Funding

This work was funded by the EU project IN-SENS FP7-PEOPLE-2013 ITN (607616).