Application of a Machine Learning Algorithm for Structural Brain Images in Chronic Schizophrenia to Earlier Clinical Stages of Psychosis and Autism Spectrum Disorder: A Multiprotocol Imaging Dataset Study

Abstract

Background and Hypothesis

Machine learning approaches using structural magnetic resonance imaging (MRI) can be informative for disease classification; however, their applicability to earlier clinical stages of psychosis and other disease spectra is unknown. We evaluated whether a model differentiating patients with chronic schizophrenia (ChSZ) from healthy controls (HCs) could be applied to earlier clinical stages such as first-episode psychosis (FEP), ultra-high risk for psychosis (UHR), and autism spectrum disorders (ASDs).

Study Design

Total 359 T1-weighted MRI scans, including 154 individuals with schizophrenia spectrum (UHR, n = 37; FEP, n = 24; and ChSZ, n = 93), 64 with ASD, and 141 HCs, were obtained using three acquisition protocols. Of these, data regarding ChSZ (n = 75) and HC (n = 101) from two protocols were used to build a classifier (training dataset). The remainder was used to evaluate the classifier (test, independent confirmatory, and independent group datasets). Scanner and protocol effects were diminished using ComBat.

Study Results

The accuracy of the classifier for the test and independent confirmatory datasets were 75% and 76%, respectively. The bilateral pallidum and inferior frontal gyrus pars triangularis strongly contributed to classifying ChSZ. Schizophrenia spectrum individuals were more likely to be classified as ChSZ compared to ASD (classification rate to ChSZ: UHR, 41%; FEP, 54%; ChSZ, 70%; ASD, 19%; HC, 21%).

Conclusion

We built a classifier from multiple protocol structural brain images applicable to independent samples from different clinical stages and spectra. The predictive information of the classifier could be useful for applying neuroimaging techniques to clinical differential diagnosis and predicting disease onset earlier.

Introduction

Several case–control studies have reported on the characteristics of brain anatomical differences in patients with chronic schizophrenia (ChSZ) compared to healthy controls (HCs), such as a reduction in gray matter volume in the frontal and temporal cortices, hippocampus, thalamus, and nucleus accumbens.^1–4 Mega studies have also identified brain characteristics among patients with schizophrenia including an increase in the volume of the putamen and pallidum from the Enhancing Neuro Imaging Genetics through Meta-Analysis (ENIGMA, n = 4568)⁵ and the Cognitive Genetics Collaborative Research Organization in Japan (COCORO, n = 2,564).^4,6 Gray matter loss in the frontotemporal regions is observed in different stages of schizophrenia, such as first-episode psychosis (FEP) and the ultra-high risk for psychosis (UHR).^7–19 Previous studies reported a progressive decrease in gray matter volume in the superior temporal lobe during the transition period among UHR individuals^9,15,16,18 and several years after disease onset.^9,18,19 Although approximately two-thirds of UHR cases did not develop a psychotic disorder (UHR-NP), some neuroanatomical alterations in the frontotemporal regions were seen in UHR overall and UHR-NP.^7,20 On the other hand, the effect size of the volume reduction in the triangular part of the inferior frontal gyrus (IFG) showed similarity among individuals with UHR, FEP, and ChSZ.^21–23 These results suggest that schizophrenia spectrum disorders have specific brain characteristics before onset, which are also present in individuals with subthreshold psychotic symptoms, and these characteristics become more disease-specific with progression from the first psychotic episode to the chronic stage.

Previous studies reported that neuroanatomical alterations may be partially shared among individuals with schizophrenia and autism spectrum disorders (ASDs) in the frontal lobes, anterior cingulate cortex, insula, basal ganglia, and cerebellum.^24–27 The overlap of structural alterations in the insular cortex was observed in individuals with UHR and FEP.^24,28 Although no large sample studies have directly compared the structural brain characteristics of ASDs and schizophrenia, the findings of ENIGMA suggested that ASD individuals showed less volume loss compared to patients with schizophrenia²⁷ and greater cortical thickness in the superior frontal gyrus and frontal pole compared to individuals with typical development.²⁹ We also reported that the volume reduction in the pars triangularis of the IFG is specific to the schizophrenia spectrum, while that in the pars opercularis of the IFG is specific to ASD.^21,23,30 Therefore, investigating the common and spectrum-specific neuroanatomical alterations in schizophrenia and ASD may provide new biological insights beyond case–control studies and render them applicable to possible biological markers in clinical settings.

Recently, machine learning approaches have been applied to structural brain imaging to determine the classification pattern of patients with psychiatric disorders.^3,12,31–34 We previously built a three-class machine learning classifier differentiating ChSZ, ASD, and HC, and showed that UHR and FEP individuals were classified into the ChSZ and HC groups but not into the ASD group.³⁵ Therefore, the machine learning approach can be informative for disease classification applicable to different clinical stages of psychosis. However, several limitations should be addressed in its interpretation and clinical application. First, only a few studies have evaluated a model that achieved good performance in terms of overall accuracy with independent data.^2,31,36,37 Lack of generalizability to unseen data is prone to information leaks between the training and test datasets.³⁸ To minimize data leakage when building models, some strategies can be used, such as repeating the preprocess of brain images for each cross-validation fold or holding out a validation dataset. The former strategy is meant to preprocess the images in every fold while building the classifiers, which requires a high calculation cost for neuroimaging data. Holding out a validation dataset is applied in the classifier evaluation step using the data that were not used in the model building process by holding out parts of the data in advance. Since the data collected from the same site and procedure still includes potential information leakage, independent confirmatory data outside of the site and procedure for building a classifier or newly measured data after building the classifier will be needed. Second, previous studies have used large samples from multiple sites and datasets; however, the differences in measurement protocols and magnetic resonance imaging (MRI) equipment have been neglected.^31,39,40 A recent multisite resting-state functional connectivity study showed that disease-derived information from functional images was smaller compared to machine- and protocol-derived information.⁴¹ The machine learning approach requires a large sample size; however, one machine and protocol from a single site is limited in sampling from various psychiatric disorders. Diminishing the machine- and protocol-derived differences in MRI data should be considered before machine learning classifiers are applied to a clinical setting.

Here, we intend to develop a support vector machine (SVM) classifier to differentiate between individuals with ChSZ and HCs, and test whether the classifier applies to individuals with earlier schizophrenia spectrum, such as UHR and FEP, and other disorders such as ASD. Although SVM and logistic regression similarly perform using brain MRI in clinical psychiatry,³⁵ SVM was the most popular algorithm.^31,32,36,37 Furthermore, SVM can generate coordinates for each data point as predictive performance for unknown data. To overcome the limitations associated with previous multisite/multiprotocol datasets, we proposed the use of ComBat,⁴² a batch-effect correction tool for harmonizing voxel-wise data collected with multiple protocols. ComBat is a suitable method for application to multidataset gray matter volumes or that of cortical thickness. Furthermore, we investigated the neuroanatomical alterations between the ChSZ and HC groups. To prevent information leakage from the data used in building a classifier, we applied a two-step approach using an independent confirmatory dataset and other clinical stages and spectrum data to evaluate the performance of the classifier.

First, we hypothesized that the performance of the classifiers would be retained for independent confirmatory datasets. Second, the classifier would discriminate the various clinical stages of psychosis (UHR and FEP) as schizophrenia, and ASD as HCs. Third, the characteristic of schizophrenia would correlate positively with the clinical stages, as greater anatomical alterations would be observed for patients with longer duration of illness. An SVM classifier was employed with the following purposes: (1) to ascertain how distinguishable ChSZ patients and HCs are from each other using T1-weighted MRI data, (2) to describe patterns of morphological features/neuroanatomical alterations contributing to the classification of psychosis, and (3) to evaluate the performance of the classifier in predicting the decision scores of unseen data such as those of UHR, FEP, and ASD individuals.

Methods

Participants

A total of 359 participants, including 154 individuals with schizophrenia spectrum disorder (UHR, n = 37; FEP, n = 24; and ChSZ, n = 93), 64 individuals with ASD, and 141 HCs were enrolled in this study (table 1). The HC group was matched for sex, age, and premorbid intelligence quotient (IQ) to the ChSZ group (P > .05).

Table 1.

Demographic Characteristics of Study Participants

	HC	UHR	FEP	ChSZ	ASD	Statistical value	P-value
Participants
Total, n	141	37	24	93	64
Protocol1, n	58	27	20	34	37
Protocol2, n	55	2	0	49	5
Protocol3, n	28	8	4	10	22
Sex, male/female	83/58	20/17	18/6	55/38	61/3	χ2 = 32.89	<.001
Age, mean (SD)	29.66 (7.88)	20.59 (3.35)	24.46 (5.88)	31.44 (10.03)	29.44 (6.67)	F = 39.20 HC>FEP>UHR ASD>FEP>UHR	<.001
Handedness (right), %	85.71	86.49	95.83	77.42	82.81
IQ
JART IQ, mean (SD)	104.82 (7.42)	105.02 (9.79)	106.27 (10.46)	101.68 (10.33)	NA	F = 3.61 HC>ChSZ	.018
FIQ, mean (SD)	NA	NA	NA	NA	105.79 (12.26)	NA	NA
VIQ, mean (SD)	NA	NA	NA	NA	112.55 (13.56)	NA	NA
PIQ, mean (SD)	NA	NA	NA	NA	94.98 (16.18)	NA	NA
DOI, mean (SD)	NA	NA	0.21 (0.17)	7.28 (6.91)	NA	t = 9.96	<.001
CPeq, mean (SD)	NA	154.0 (306.0)	452.4 (420.2)	698.3 (582.1)	NA	F = 23.14 ChSZ>FEP>UHR	<.001
PANSS
Positive symptom, mean (SD)	NA	12.5 (3.88)	13.48 (4.57)	16.47 (5.28)	NA	F = 8.77 ChSZ>UHR	<.001
Negative symptom, mean (SD)	NA	16.17 (6.37)	18.1 (4.45)	20.38 (6.69)	NA	F = 6.57 ChSZ>UHR	.002
General psychopathology, mean (SD)	NA	31.2 (8.81)	33.76 (7.95)	36.41 (9.78)	NA	F = 4.42 ChSZ>UHR	.014
ADI-R
Social, mean (SD)	NA	NA	NA	NA	15.63 (5.16)	NA	NA
Com, mean (SD)	NA	NA	NA	NA	9.45 (4.58)	NA	NA
RRB, mean (SD)	NA	NA	NA	NA	3.89 (1.88)	NA	NA
Total, mean (SD)	NA	NA	NA	NA	27.79 (9.54)	NA	NA

Note: HC, healthy control; UHR, ultra-high risk for psychosis; FEP, first-episode psychosis; ChSZ, chronic schizophrenia; ASD, autism spectrum disorder; SD, standard deviation; DOI, duration of illness; CPeq, chlorpromazine equivalent doses; PANSS, Positive and Negative Syndrome Scale; ADI-R, Autism Diagnostic Interview-revised; Social, Social interaction issues; Com, Communication and language skills; RRB, Restricted and repetitive behavior.

We applied a two-step approach to evaluate the performance of the models by dividing the data into four datasets: training, test, independent confirmatory, and independent group datasets (figure 1). First, the training and test datasets comprised the data of individuals with ChSZ and HC collected using protocols 1 and 2. Ninety percent of the data were randomly sorted as the training dataset, and the remaining 10% were sorted as the test dataset. The independent confirmatory dataset comprised the data of the same groups collected using protocol 3, which was completely excluded from the training partition, to perform an independent first-step evaluation without site information leakage. To evaluate the classifier for earlier clinical stages of psychosis as the second step, we defined the independent group dataset as the data of individuals with UHR, FEP, and ASD collected using any protocol.

Fig. 1.

Pipeline employed for the processing and analysis of T1-weighted images. HC, healthy control; UHR, ultra-high risk for psychosis; FEP, first-episode psychosis; ChSZ, chronic schizophrenia; ASD, autism spectrum disorder.

The participants were recruited from the outpatient and inpatient units of the University of Tokyo Hospital, University of Tokyo Health Service Center, psychiatry clinics, and internet referrals. Individuals with ChSZ were diagnosed with schizophrenia by designated psychiatrists. Individuals with ASD were diagnosed according to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR)⁴³ with more than 2 months of follow-up examinations by an experienced psychiatrist (H.Y.). The diagnoses were further confirmed by a certified psychiatrist (H.K.) using the Japanese version of the Autism Diagnostic Interview – Revised (ADI-R). The inclusion criteria were ages 15–40 years for FEP and 15–30 years for UHR, non-receipt of antipsychotic medications for more than 16 cumulative weeks, and continuous psychotic symptoms within the past 6 months. All eligible participants in the UHR and FEP groups were assessed using the Structured Interview for Prodromal Symptoms (SIPS)^44,45 and evaluated using the UHR or psychosis criteria (supplementary materials). The SIPS criteria for psychosis were the same as those of psychotic disorders in the DSM-IV-TR. HCs were not diagnosed with ASD, schizophrenia, or any other psychiatric disorder, and were screened for neuropsychiatric disorders using the Structured Clinical Interview for DSM-IV, Nonpatient Edition.⁴⁶

The exclusion criteria were as follows: (1) previous and/or present severe brain injury and/or neurological illness; (2) a previous history of electroconvulsive therapy; (3) a premorbid IQ of ≤70 as assessed using the 25-item version of the Japanese Adult Reading Test (JART25)^47,48 for the schizophrenia spectrum groups and full scale of the Wechsler Adult Intelligence Scale Revised Japanese version (WAIS-R) for the ASD group; (4) previous and/or present alcohol addiction; and (5) previous and/or present continuous substance use. For the schizophrenia spectrum groups, we also excluded participants with clearly comorbid ASD according to the DSM-IV criteria.⁴⁹ The detailed inclusion and exclusion criteria for the UHR and FEP groups are described in the protocol paper.⁴⁹

Symptom severity for schizophrenia spectrum groups was assessed using the Positive and Negative Syndrome Scale (PANSS)⁵⁰ and designated using the positive, negative, and general psychopathology subscales. For the ASD group, the ADI-R subtypes (social, communication, and restricted and repetitive behavior [RRB]) were assessed. The chlorpromazine equivalent dose was calculated for medications received at the time of scanning.⁵¹

The study protocol was approved by the ethics committee of the Faculty of Medicine, University of Tokyo (approval nos. 397, 629, 630, and 2226). All participants provided written informed consent to participate in the measurements after receiving a complete explanation of the experiment.

MRI Data Acquisition and Preprocessing

T1-weighted images were obtained from the three datasets using different scanners and protocols. All structural MRI images were acquired using 3 Tesla General Electric scanners. The scanning for protocol 1 was performed using an 8-channel head coil on SIGNA HDx (3D-FSPGR sequence, 176 axial slices, slice thickness: 1.0 mm; supplementary table S1), for protocol 2 using a 24-channel head coil on DISCOVERY MR750w (Sag_IR-FSPGR sequence, 196 sagittal slices, slice thickness: 1.2 mm), and for protocol 3 using a 32-channel head coil on DISCOVERY MR750w (3D-FSPGR sequence, 172 axial slices, slice thickness: 1.0 mm). All T1-weighted images were first corrected for intensity nonuniformity with N4BiasFieldCorrection,⁵² distributed using ANTs 2.2.0 (RRID: SCR_004757).⁵³ The intensity corrected images were then segmented to produce images of different tissue types (gray matter, white matter, and cerebrospinal fluid) using CAT12 software (http://www.neuro.uni-jena.de/cat/). We used the Diffeomorphic Anatomical Registration Through Exponentiated Lie Algebra (DARTEL)⁵⁴ option in the CAT12 toolbox to normalize the segmented scans into a standard Montreal Neurological Institute (MNI) space. Default parameters were used for this preprocessing. Smoothing was applied later in the harmonization process. To assess gray matter segment homogeneity and identify possible outliers, the “Check Data Quality” module of CAT12 was used. No participants were excluded from this step. To retain maximum image information, voxels from gray matter images were smoothed using a Gaussian smoothing kernel of 2-mm full width at half-maximum as features in further classification. All features from the smoothed, modulated, and normalized gray matter images were transformed to a two-dimension matrix (participants × features) using Niftimasker, a component of Nilearn (https://github.com/nilearn/nilearn).

ComBat Harmonization

ComBat⁴² is a harmonization method used to remove scanner and protocol effects based on the adjusted general linear model harmonization method. ComBat uses Bayesian criteria to improve the estimation for small sample size data (supplementary materials). Further analyses were conducted using Python version 3.7.4. We applied the transformed two-dimensional data with participants’ age and sex as covariables, along with protocol effects (supplementary figure S1).

Support vector Machine

The main idea behind SVMs is to separate two groups using a contrasted hyperplane (supplementary materials).⁵⁵ In building a classifier, we applied standardization and dimension reduction using principal component analysis (PCA) to the dataset, which included 554 992 features. The usage of value standardization, optimization of the number of PCA components, and hyperparameters of the classifier (penalty parameter C and kernel parameter gamma) were tuned using GridSearchCV implemented in the “scikit-learn” module (version 0.21.3) in Python (https://scikit-learn.org/stable/whats_new/v0.21.html). We plotted the weights of the classifier to determine the importance of the features for generalization (supplementary materials).^56,57 The classifier was optimized using a 10-fold cross-validated grid search over a defined parameter grid. Data from the HC group were randomly downsampled to the same ratio as the ChSZ group in each fold. To reduce downsampling bias, downsampling and grid search were repeated 1000 times and stratified 10-fold for the training data. Then, we applied 10-fold cross-validation and 1000 permutations to evaluate the significance of the cross-validation scores of the model with the best hyperparameters for the training dataset. The best cross-validation accuracy score was averaged across 1000 repeats. Permutation tests were conducted by shuffling the labels in the training data, and the permutation-based P-value was calculated.⁵⁸ The final model with the best hyperparameters was trained using the entire training dataset. One hundred and fifty PCA components were used as features. Finally, the trained classifier was applied to the test set and the independent confirmatory dataset with the best parameters tuned by grid search. The predictive performance of the classifier was evaluated using an independent group dataset (UHR, FEP, and ASD data collected using any protocol).

Statistical Analysis

Evaluation metrics

First, the classifier was evaluated using the test, independent confirmatory, and independent group datasets separately by the given scores of the tuned classifier using the training dataset. We calculated the confusion matrix, macro, and weighted average accuracies to evaluate the classifier because the data used were imbalanced (supplementary materials). We also reported the area under the curve (AUC) of the receiver operator characteristic.

Predictive performance of the classifier

The predictive performance of the classifier and the chi-squared test were applied to the classified labels of the test, independent confirmatory, and independent group datasets. Since we compared 10 pairs of groups, Bonferroni’s correction was applied to post-hoc comparisons (P < .05/10 = .005). Decision scores generated by the SVM were tested using an analysis of variance separately for all samples corresponding to the hyperplane. Bonferroni’s correction was applied to post-hoc comparisons (P < .05/10 = .005).

Correlations between decision scores and clinical severity

To determine the relationship between the decision score and symptom severity, Pearson’s correlation analyses were performed using the PANSS subscores for the schizophrenia spectrum groups and the ADI-R subscale scores for the ASD group. Bonferroni’s correction was applied to the subscores (P < .05/3 = .016). To determine the potential effect of medication on the classification, we also tested the correlation between the decision score and medication dose for the schizophrenia spectrum groups using Spearman’s rank correlation (uncorrected P < .05). For the UHR group, we also tested the difference in decision scores between those with and without medication using a t-test.

Results

Model Evaluation

The best cross-validation accuracy within the training dataset was 74% (±0.68). The permutation test showed that it was significantly higher than that attributable to chance (50%, P < .001). The accuracy with the best estimator for the test and independent confirmatory datasets were 75% (AUC = 0.88) (table 2, supplementary figure S2A) and 76% (AUC = 0.82) (supplementary figure S2B), respectively.

Table 2.

Classification Report of the Test and Independent Datasets

	Precision	Recall	F1-score	N for prediction
Test dataset
HC	0.89	0.67	0.76	12
ChSZ	0.64	0.88	0.74	8
macro avg	0.76	0.77	0.75	20
weighted avg	0.79	0.75	0.75	20
Independent confirmatory dataset
HC	0.88	0.79	0.83	28
ChSZ	0.54	0.70	0.61	10
macro avg	0.71	0.74	0.72	38
weighted avg	0.79	0.76	0.77	38

Since the samples from protocol 1 and 2 were used for Test dataset (HC = 12, ChSZ = 8) and those from protocol 3 for Independent confirmatory dataset (HC = 28, ChSZ = 10), the metrics except for weighted average accuracy (weighted avg) may be affected by sampling imbalance (Supplementary Materials).

The voxel space feature weights of the SVM showed that the clusters including the IFG pars triangularis, superior frontal gyrus, cuneus, superior occipital gyrus, putamen, and pallidum contributed to identifying ChSZ (figure 2A, table 3). Clusters including the inferior parietal gyrus, inferior occipital gyrus, superior parietal gyrus, and middle frontal gyrus contributed to the identification of HCs.

Fig. 2.

Voxel feature contributions and predictive performance comparisons of support vector machine (SVM). (A) Weighted features of SVM classification in the voxel space. Note that positive scores indicate the regions of feature contribution for identifying patients with chronic schizophrenia, while negative score for healthy controls. (B) Predictive performance of HC and ChSZ groups was evaluated using the independent confirmatory dataset, and UHR, FEP, and ASD groups using the independent group dataset. (C) Box and scatter plot of decision scores of support vector machines. P-values of post hoc comparisons were corrected using the Bonferroni method (***P < .001, **P < .01, *P < .05).

Table 3.

Top 10 Weighted Features of Regions Contributed to SVM Classification Included in AAL Atlas.

Cluster ID	Peak			Contribution values			Brain regions
	x	y	z	Cluster mean weight (×10−5)	Peak weight (×10−5)	Volume (mm3)	Cluster	Peak
Top 10 by cluster mean weight for healthy controls
361	−51	−55.5	54	−3.09	−4.42	104.625	90.32% Parietal_Inf_L; 9.68% no_label	Parietal_Inf_L
48	−36	−73.5	−7.5	−3.08	−5.53	1090.12	75.85% Occipital_Inf_L; 13.00% Occipital_ Mid_L; 5.88% Fusiform_L; 5.26% no_label	Occipital_Inf_L
504	25.5	−25.5	37.5	−3.04	−3.84	74.25	100.00% no_label	no_label
63	40.5	13.5	61.5	−3.03	−6.24	772.875	96.51% Frontal_Mid_2_R	Frontal_Mid_2_R
392	12	−60	60	−3.03	−5.27	101.25	80.00% Precuneus_R; 20.00% Parietal_Sup_R	Precuneus_R
148	−4.5	−91.5	36	−3.02	−4.57	273.375	69.14% Cuneus_L; 19.75% no_label; 11.11% Occipital_Sup_L	no_label
105	28.5	−37.5	72	−3.01	−4.47	381.375	100.00% Postcentral_R	Postcentral_R
333	19.5	−61.5	−64.5	−2.98	−5.06	114.75	100.00% no_label	no_label
586	−16.5	−43.5	−52.5	−2.98	−4.47	57.375	88.24% Cerebelum_9_L; 11.76% no_label	Cerebelum_9_L
74	−12	−63	72	−2.96	−5.41	631.125	90.37% Precuneus_L; 5.35% Parietal_Sup_L	Precuneus_L
Top 10 by peak weight for healthy controls
1	7.5	46.5	−24	−2.89	−6.98	118770	23.13% no_label	Rectus_R
26	−19.5	−51	76.5	−2.87	−6.45	1859.62	54.99% Parietal_Sup_L; 22.87% Precuneus_L; 14.34% Postcentral_L; 5.44% Paracentral_ Lobule_L	Parietal_Sup_L
63	40.5	13.5	61.5	−3.03	−6.24	772.875	96.51% Frontal_Mid_2_R	Frontal_Mid_2_R
12	28.5	−7.5	−42	−2.81	−5.83	2504.25	47.04% no_label; 18.06% Fusiform_R; 15.09% Temporal_Pole_Mid_R; 12.67% Temporal_Inf_R; 6.74% ParaHippocampal_R	Fusiform_R
5	−27	−10.5	46.5	−2.87	−5.69	5055.75	31.38% Precentral_L; 27.10% Frontal_Mid_2_L; 26.44% Frontal_Sup_2_L; 14.35% no_label	no_label
21	−45	7.5	28.5	-2.73	−5.66	2052	43.75% Precentral_L; 39.47% Frontal_Inf_ Oper_L; 11.84% Frontal_Inf_Tri_L	Frontal_Inf_ Oper_L
69	57	−31.5	−21	−2.94	−5.57	678.375	90.55% Temporal_Inf_R; 7.46% no_label	Temporal_Inf_R
161	−15	−73.5	63	−2.91	−5.54	239.625	73.24% Precuneus_L; 18.31% no_label; 8.45% Parietal_Sup_L	Precuneus_L
48	−36	−73.5	−7.5	−3.08	−5.53	1090.12	75.85% Occipital_Inf_L; 13.00% Occipital_ Mid_L; 5.88% Fusiform_L; 5.26% no_label	Occipital_Inf_L
11	−37.5	−49.5	−52.5	−2.76	−5.49	2575.12	65.92% Cerebelum_8_L; 21.10% no_label; 10.62% Cerebelum_7b_L	Cerebelum_8_L
Top 10 by cluster mean weight for chronic schizophrenia
432	−49.5	46.5	4.5	3.73	6.02	91.125	96.30% Frontal_Inf_Tri_L	Frontal_Inf_Tri_L
253	−12	51	46.5	3.55	5.73	155.25	58.70% Frontal_Sup_2_L; 32.61% Frontal_Sup_ Medial_L; 8.70% no_label	Frontal_Sup_2_L
72	−9	−84	27	3.26	6.75	648	83.85% Cuneus_L; 16.15% Occipital_Sup_L	Cuneus_L
516	−10.5	−33	−28.5	3.25	5.53	70.875	100.00% no_label	no_label
138	18	−52.5	72	3.18	5.63	293.625	100.00% Parietal_Sup_R	Parietal_Sup_R
224	54	40.5	7.5	3.13	5.01	178.875	100.00% Frontal_Inf_Tri_R	Frontal_Inf_ Tri_R
297	58.5	24	25.5	3.09	4.97	131.625	87.18% Frontal_Inf_Tri_R; 12.82% Frontal_Inf_ Oper_R	Frontal_Inf_ Tri_R
79	−9	0	64.5	3.08	6.39	563.625	79.64% Supp_Motor_Area_L; 16.17% no_label	Supp_Motor_ Area_L
396	−49.5	−45	−45	3.06	4.02	97.875	100.00% Cerebelum_Crus2_L	Cerebelum_ Crus2_L
1308	12	−48	22.5	3.04	3.37	16.875	60.00% Cingulate_Post_R; 40.00% Precuneus_R	Cingulate_Post_R
Top 10 by peak weight for chronic schizophrenia
2	22.5	−4.5	3	2.97	9.32	17465.6	48.10% no_label; 14.40% Putamen_R; 9.80% Pallidum_R; 5.84% Caudate_R; 5.10% Temporal_Sup_R	Pallidum_R
3	−19.5	−1.5	4.5	3.01	8.66	11292.8	35.51% no_label; 19.75% Putamen_L; 16.44% Pallidum_L; 8.70% Parietal_Inf_L	Pallidum_L
72	−9	−84	27	3.26	6.75	648	83.85% Cuneus_L; 16.15% Occipital_Sup_L	Cuneus_L
4	6	70.5	4.5	2.98	6.72	9966.38	17.78% Frontal_Sup_Medial_L; 15.92% Frontal_ Sup_2_R; 14.83% no_label; 12.87% Frontal_ Sup_Medial_R; 11.38% Frontal_Sup_2_L; 9.75% Frontal_Mid_2_R	Frontal_Sup_ Medial_R
79	−9	0	64.5	3.08	6.39	563.625	79.64% Supp_Motor_Area_L; 16.17% no_label	Supp_Motor_ Area_L
80	15	−76.5	42	2.87	6.18	560.25	65.06% Cuneus_R; 19.88% Precuneus_R; 7.83% Parietal_Sup_R	Cuneus_R
432	−49.5	46.5	4.5	3.73	6.02	91.125	96.30% Frontal_Inf_Tri_L	Frontal_Inf_Tri_L
64	42	−6	21	2.83	5.74	752.625	47.09% no_label; 42.60% Rolandic_Oper_R; 7.62% Insula_R	Rolandic_Oper_R
45	64.5	1.5	27	2.77	5.74	1194.75	64.41% Precentral_R; 14.97% Postcentral_R; 11.02% Frontal_Inf_Oper_R; 9.32% Rolandic_ Oper_R	Postcentral_R
253	−12	51	46.5	3.55	5.73	155.25	58.70% Frontal_Sup_2_L; 32.61% Frontal_Sup_ Medial_L; 8.70% no_label	Frontal_Sup_2_L

The 10 largest clusters that contributed to the classification by cluster mean and peak of the brain regions are shown. All features are listed in supplementary tables S2-3. By transforming the weights of each PCA component back into the original feature (voxel) output, a weighted image was generated in a standard MNI space, which including the coordinates of the weights. AAL atlas was used to generate a coordinates table and region labels based on the information given by the weighted image.

Predictive Performance of the Classifier for the Test, Independent Confirmatory, and Independent Group Datasets

A chi-squared test showed a significant difference within the classified labels for the test, independent confirmatory, and independent group datasets, respectively (X²(1, n = 20) = 5.69, P < .05; X²(1, n = 38) = 7.72, P < .01; and X²(2, n = 118) = 11.25, P < .01). Further residual analysis showed that the HC group was significantly more likely to be classified as HCs than the ChSZ group in the independent confirmatory dataset (79% vs. 21%, corrected P < .01). For the independent group dataset, the ASD and UHR groups were significantly more likely to be classified as HCs (classification rate to HC: 81% and 59%, respectively, corrected P < .01), while the FEP group as ChSZ (46%, corrected P < .01; figure 2B). Compared to the ASD group, UHR was more likely to be classified as ChSZ (corrected P < .01). A chi-squared test showed no difference between HCs from the independent confirmatory dataset and ASD participants from the independent group dataset.

Using the decision score generated by the SVM for all groups, we found a significant main effect of the decision score for the five groups (F = 161.99, P < .001). Multiple comparisons showed that UHR was close to FEP but different from HC, ASD, and ChSZ (HC < ASD < UHR and FEP < ChSZ; corrected P < .001) (figure 2C).

Correlations Between Decision Scores and Clinical Severity

No significant correlations were found between PANSS subscores and decision scores. In the schizophrenia spectrum groups, significant correlations between chlorpromazine equivalent dose and decision scores were observed in the UHR group (rho = 0.44, P < .05). However, no significant differences in decision scores between medicated and nonmedicated UHR participants were found (t = −1.78, not significant). No significant correlation was found between the ADI-R subscale scores and decision scores in the ASD group. No significant correlations were found between JART IQ and decision scores in each group.

Discussion

To the best of our knowledge, the current study is the first to apply machine learning to the classifier for HC and ChSZ groups with multi-protocol structural MRI and multidisease spectrum and clinical stages. To evaluate the classifier, we confirmed a two-step approach using an independent confirmatory dataset obtained via a different protocol from the ones used in building the classifier and the earlier clinical stage and ASD dataset. We successfully achieved a 74% accuracy in the 2-class classification within training dataset. The decision scores from the classifier indicated that the characteristics of ChSZ became pronounced in the UHR and FEP groups, which was different from those in the HC and ASD groups.

In this study, we discriminated between the HC and ChSZ groups with 74% and 75% accuracy in the training and test sets, respectively. The performance of the classifier on the independent confirmatory dataset achieved 76% accuracy. To avoid downsampling bias, we performed random execution and 1000 repeats, which may cause a slightly lower training accuracy than the test accuracy (≤1%). Our results are compatible with the accuracy of 72%–77% in a previous study.³⁷ As Rozycki et al.³⁷ treated MRI scanners as a variable in the multivariate analysis, we used ComBat to harmonize the effects of MRI protocols or scanners. ComBat is superior at preserving within-site biological variability and improving the consistency and replicability of the voxels associated with age.^59–61 By applying the two-step approach using multi-protocol samples, we obtained less biased results than evaluating the classifier using data from the same dataset. We obtained a model with a solid predictive performance for new data, unlike previous studies that did not test the performance of the trained classifier.^2,37 As expected, a majority of ASD patients were classified as HCs. The decision scores also indicated that ASD and UHR patients share some characteristics that differ from those of the HC or ChSZ group. Moreover, no significant associations were found between IQ or chlorpromazine equivalent doses and decision scores, indicating that the brain characteristics of schizophrenia found in the present study may serve as biomarkers for improving methods for differential diagnosis.

In line with previous studies of volumetric alterations in the schizophrenia spectrum,^1–6,13 we found that a pattern of morphological features, including the IFG pars triangularis, putamen, and pallidum contributed to the identification of ChSZ patients. Using a manual tracing method within the IFG, we previously found that the volume reduction in the IFG pars triangularis is a disease-specific feature in the schizophrenia spectrum, such as UHR,²¹ FEP,²¹ and ChSZ,²³ and the extent of structural alteration was associated with brain activity during a verbal fluency test in FEP.²² These findings suggest that a decrease in gray matter volume in the region is a trait characteristic of the schizophrenia spectrum. On the other hand, the IFG pars opercularis is related to a disease-specific feature in ASD.³⁰ In this study, the SVM classifier used the structural features of the IFG pars triangularis as one of the contributory regions and is a reasonable selection according to previous studies.

Previous large-scale studies from ENIGMA⁵ and COCORO⁶ found increased volumes of the putamen and pallidum in patients with schizophrenia. Recently, our multi-site study showed that individuals with UHR exhibited increased volumes of the left caudate and pallidum⁶²; however, the ENIGMA clinical high risk for psychosis study failed to replicate these findings.²⁰ This suggests that an altered striatum and pallidum volume is another disease-specific feature of the schizophrenia spectrum. These volumes are inconsistent in terms of FEP.^63–65 Nevertheless, the findings of the present study suggest a major contribution of the putamen and pallidum for classifying ChSZ, and these regions may contribute to some disease-specific pathologies in schizophrenia.

The decision score given by the classifier increased according to the clinical stages of psychosis, suggesting that the gradual transitions of neuroanatomical characteristics were distinguishable among the HC, UHR, FEP, and ChSZ groups. This further implies that prominent symptoms (anatomical alterations) appear after FEP and progressively change after the initial period of psychosis onset (HC < ASD < UHR and FEP < ChSZ) (figure 2C). Although a partial overlap of anatomical alterations has been reported between the ASD and ChSZ groups,^66,67 the classifier in this study weighted otherwise. As ASD is mostly classified as HC compared to other schizophrenia spectrum groups, the neuroanatomical changes used in the classifier during early clinical stages are specific to schizophrenia and weighed both differences in trait and progressions in brain pathology. Recent studies including whole brain (gray matter, white matter, and ventricular cerebrospinal fluid volumes)³³ or diffusion MRI as features reported a high performance rate^32,34; with more features, it is possible to achieve a higher accuracy with relatively smaller samples than this study.

Our study had several limitations. First, we did not build a classification system differentiating ASD from other groups because of the smaller sample size. More samples from multiple sites will yield a multiclass classifier that can be used to learn more about disease-specific structural brain characteristics, which may apply to earlier clinical stages of the schizophrenia spectrum. Moreover, although ComBat was applied protocol-wise, there is little risk of information leak since the data used in the current study were acquired from a single site. However, ComBat was unable to distinguish the sampling bias from measurement bias^.41,60,68 While applying combat harmonization to multiple site datasets, the sampling effects could be a potential information leak. Second, due to the limited samples of UHR individuals who later converted to psychosis, we were unable to evaluate whether the classifier could differentiate a later onset of psychosis. Third, although we showed that the classification was slightly influenced by the medication dose, the potential effect of medication on the structural characteristics should be considered. Fourth, although previous machine learning classification studies did not control features for intracranial volume,^31,35,36 we need to see whether the features adjusted for intracranial volume would increase the sensitivity in further studies.

The present study compared a 2-class SVM classifier (HC vs. ChSZ) using the multiprotocol voxel-based morphometry datasets, which showed a good predictive performance for the unknown data of the UHR, FEP, and ASD groups. The classifier indicated that the characteristic was gradually modified in the UHR, FEP, and ChSZ groups. This method could be the next step to apply brain MRI machine learning algorithms to clinical settings. Further elaboration of the method applied herein may contribute to the early discovery and clinical diagnosis of schizophrenia and ASDs in the future.