Differentiation of schizophrenia using structural MRI with consideration of scanner differences: A real‐world multisite study

Abstract

Aim

Neuroimaging studies have revealed that patients with schizophrenia exhibit reduced gray matter volume in various regions. With these findings, various studies have indicated that structural MRI can be useful for the diagnosis of schizophrenia. However, multisite studies are limited. Here, we evaluated a simple model that could be used to differentiate schizophrenia from control subjects considering MRI scanner differences employing voxel‐based morphometry.

Methods

Subjects were 541 patients with schizophrenia and 1252 healthy volunteers. Among them, 95 patients and 95 controls (Dataset A) were used for the generation of regions of interest (ROI), and the rest (Dataset B) were used to evaluate our method. The two datasets were comprised of different subjects. Three‐dimensional T1‐weighted MRI scans were taken for all subjects and gray‐matter images were extracted. To differentiate schizophrenia, we generated ROI for schizophrenia from Dataset A. Then, we determined volume within the ROI for each subject from Dataset B. Using the extracted volume data, we calculated a differentiation feature considering age, sex, and intracranial volume for each MRI scanner. Receiver–operator curve analyses were performed to evaluate the differentiation feature.

Results

The area under the curve ranged from 0.74 to 0.84, with accuracy from 69% to 76%. Receiver–operator curve analysis with all samples revealed an area under the curve of 0.76 and an accuracy of 73%.

Conclusion

We moderately successfully differentiated schizophrenia from control using structural MRI from differing scanners from multiple sites. This could be useful for applying neuroimaging techniques to clinical settings for the accurate diagnosis of schizophrenia.

Schizophrenia is a mental disorder that affects around 1% of the general population.1 Patients with schizophrenia suffer from various symptoms, including positive symptoms, negative symptoms, and cognitive decline. Indeed, according to the Global Burden of Disease Study, schizophrenia causes a high degree of disability, which accounts for 1.1% of total disability‐adjusted life years and 2.8% of years lived with disability.2

Various neuroimaging studies have investigated structural brain changes caused by schizophrenia using MRI. A region‐of‐interest (ROI) approach revealed that patients with schizophrenia exhibit lateral ventricular enlargement as well as atrophy in medial temporal lobes, including the amygdala, hippocampus, parahippocampal gyrus, and superior temporal gyrus.3 In addition to the ROI approach, several studies have employed a voxel‐based‐morphometry approach, which allows investigation of focal differences in the whole brain. Meta‐analysis of voxel‐based‐morphometry studies has shown that patients with schizophrenia exhibit reduced gray matter volume in the medial temporal lobes, superior temporal lobes, anterior cingulate gyrus, medial portion of prefrontal regions, or insulae.4, 5, 6, 7, 8 A recent mega‐analysis of subcortical structure by ENIGMA–Schizophrenia reconfirmed that schizophrenia patients exhibit smaller hippocampus, amygdala, thalamus, and accumbens volumes and larger pallidum and lateral ventricle volumes.9 Related to this finding, schizophrenia‐specific leftward asymmetry in pallidum volume has also been reported.10

That patients with schizophrenia exhibit gray‐matter volume reduction in certain regions has led to the idea of applying these findings to the diagnosis of schizophrenia. Back in 1999, Leonard and colleagues showed that they could classify patients with schizophrenia from control with 77% accuracy using several variables derived from brain MRI, including hemisphere and third ventricle volume, and normalized location of three associated cortex sulcal landmarks.11 Since then, several studies have investigated discrimination of schizophrenia using thalamic and hippocampal shape12 or ventricle volume13 with multivariate analyses. Subsequent to these reports, automatic preprocessing of MRI as well as various methods of differentiation, such as machine learning or multivariate pattern classification, have been introduced.14, 15, 16, 17, 18, 19 Kambeitz and colleagues performed a meta‐analysis of reports to examine the differentiation of schizophrenia from control. With structural MRI, patients were differentiated from controls with a sensitivity of 76.4% and a specificity of 79.0%.20

Although these reports indicate that structural MRI can be useful for the diagnosis of schizophrenia, it is not widely used in clinical settings. While there may be several reasons for this, one potential reason is that these approaches are not fully tested in the real world. It is well known that a fitting model for a cohort for machine learning cannot be applied easily to another cohort. In addition, differences in MRI scanners have a huge impact on image quality and compartment volume,21 which could affect differentiation models.

Therefore, in this study, we tried to generate a simple model that could be used to differentiate schizophrenia from control subjects with consideration of MRI scanner differences employing voxel‐based morphometry. Then, we evaluated how measurements could allow the differentiation of schizophrenia from control with real‐world samples from multiple sites.

Methods

Subjects

Subjects to generate ROI (Dataset A)

MRI data of 95 patients with schizophrenia (57 men, 38 women; mean age ± SD, 29.8 ± 5.2 years) and the same number of age‐ and sex‐matched control subjects (57 men, 38 women; 29.9 ± 5.1 years) were used to generate ROI. Both patients and control subjects were recruited from the University of Osaka and the University of Tokyo. We chose these two institutes because demographics of patients in these two institutes were similar and both had a sufficient number of healthy control subjects for balanced datasets. As the number of subjects scanned at the University of Osaka was greater than that at the University of Tokyo, we first selected a balanced dataset from the University of Tokyo, and then chose sex‐ and age‐matched subjects from the University of Osaka. The remaining data from the University of Osaka were used for Dataset B, which is described in the following section. Each patient with schizophrenia was assessed and diagnosed according to the DSM‐IV by at least two trained psychiatrists. Controls were recruited through local advertisements and evaluated via the DSM‐IV Structured Clinical Interview, Non‐Patient Version.22 Subjects were excluded if they had neurological or medical conditions that could potentially affect the central nervous system, such as atypical headache, head trauma with loss of consciousness, chronic lung disease, kidney disease, chronic hepatic disease, thyroid disease, active stage cancer, cerebrovascular disease, and epilepsy or seizures. Subject demographics are summarized in Table 1. The total prescribed antipsychotics being taken by patients was calculated using chlorpromazine equivalent (mg/day) based on Inada and Inagaki.23

Table 1.

Demographics of Dataset A subjects

MRI scanner	Diagnosis	n	Female (n)	Age (years) (mean ± SD)	Education (years) (mean ± SD)	Disease duration (years) (mean ± SD)	Chlorpromazine equivalent (mg/day) (mean ± SD)
Osaka A	Schizophrenia	51	19	30.2 ± 5.3	14.5 ± 2.4	9.0 ± 6.1	629.9 ± 528.7
	Control	51	19	30.4 ± 5.3	15.1 ± 2.1	—	—
Tokyo A	Schizophrenia	44	19	29.3 ± 4.9	13.3 ± 2.1	7.6 ± 5.5	759.3 ± 492.7
	Control	44	19	29.2 ± 4.8	16.6 ± 1.6	—	—

Subjects to evaluate the model (Dataset B)

For the evaluation of the model we propose in this study, we used a total sample of 1603 subjects (schizophrenia: 446; control: 1157) from Osaka University, University of Tokyo, University of Toyama, and Kyushu University. The subjects in this dataset were independent of the subjects in Dataset A. As we wanted to estimate how useful our model would be in clinical settings, we did not match age and sex between patients and control subjects. Subject demographics are summarized in Table 2. This study was approved by the ethics committees of each institute and performed in accordance with the guidelines and regulations of these research institutions. All participants gave written informed consent prior to participation.

Table 2.

Demographics of Dataset B subjects

MRI scanner	Diagnosis	n	Female (n)	Age (years) (mean ± SD)	Education (years) (mean ± SD)	Disease duration (years) (mean ± SD)	Chlorpromazine equivalent (mg/day) (mean ± SD)
Osaka A	Schizophrenia	129	50	36.5 ± 12.6	14.2 ± 2.5	11.8 ± 9.5	547.8 ± 501.4
	Control	404	217	35.5 ± 12.6	15.0 ± 2.2	—	—
Osaka B	Schizophrenia	74	40	34.4 ± 12.0	13.3 ± 2.4	11.1 ± 9.0	868.8 ± 844.9
	Control	239	107	31.2 ± 13.2	15.0 ± 1.8	—	—
Osaka C	Schizophrenia	73	37	36.4 ± 14.0	13.6 ± 2.4	11.6 ± 9.2	583.0 ± 567.7
	Control	244	101	29.0 ± 12.1	14.8 ± 1.8	—	—
Tokyo B	Schizophrenia	17	6	39.4 ± 8.3	14.4 ± 2.2	13.5 ± 8.9	814.1 ± 516.3
	Control	54	31	32.9 ± 9.2	15.6 ± 2.2	—	—
Toyama	Schizophrenia	121	58	26.7 ± 6.2	13.7 ± 1.9	4.3 ± 5.2	483.1 ± 424.0
	Control	130	60	26.1 ± 6.3	16.1 ± 2.4	—	—
Kyushu	Schizophrenia	32	22	39.6 ± 9.5	14.0 ± 1.9	15.1 ± 9.4	571.4 ± 409.8
	Control	86	46	33.1 ± 11.5	15.7 ± 2.4	—	—

MRI data acquisition

MRI data were obtained using seven different scanners. A 3‐D volumetric acquisition of a T1‐weighted sequence produced a gapless series of sagittal sections. Table 3 provides a summary of the MRI scanners and the pulse sequences for each scanner.

Table 3.

Summary of MRI scanners and pulse sequences

	Osaka A	Osaka B	Osaka C	Tokyo A	Tokyo B	Toyama	Kyushu
Manufacturer	GE	GE	GE	GE	GE	Siemens	Philips
Scanner name	Signa EXCITE	Signa Hdxt	Discovery MR750	Signa Horizon	Discovery MR750w	Magnetom Vision	Achieva
Magnet strength	1.5 T	3 T	3 T	1.5 T	3 T	1.5 T	3 T
Head coil	Head QD	8HRBRAIN	HNS Head	Circularly polarized head	Head 24	CP head	8ch head
Pulse sequence	Fast SPGR	Fast SPGR	Fast SPGR	SPGR	SPGR	FLASH	3D T1‐TFE
Number of slices	124	172	156	124	200	160	190
Echo time (ms)	4.2	2.9	3.2	7	3.1	10	3.8
Repetition time (ms)	12.6	7.2	8.2	35	7.7	24	8.2
Flip angle (degree)	15	11	11	30	11	40	8
Acquisition matrix	256 × 256	256 × 256	256 × 256	256 × 256	256 × 256	256 × 256	240 × 240
Number of excitations (NEX)	1	1	1	1	1	1	1
Field of view (cm)	24 × 24	24 × 24	26 × 26	24 × 24	26 × 26	25.6 × 25.6	24 × 24
Voxel dimension (mm)	0.9 × 0.9 × 1.4	0.9 × 0.9 × 1.0	1.0 × 1.0 × 1.2	0.9 × 0.9 × 1.5	1.0 × 1.0 × 1.2	1.0 × 1.0 × 1.0	1.0 × 1.0 × 1.0
Slice thickness (mm)	1.4	1.0	1.2	1.5	1.2	1.0	1.0

Preprocessing of imaging

All of the MR images were processed using spm12 (Wellcome Department of Imaging Neuroscience, University College London, UK, http://www.fil.ion.ucl.ac.uk/spm) running on matlab R2015b (MathWorks, Natick, MA, USA) on Ubuntu 16.04 based Lin4Neuro.24 Prior to preprocessing, all data were co‐registered to ‘icbm152’ standard image implemented in spm12 so that the origin of images would be close to the anterior commissure–posterior commissure (AC‐PC) and so that images would be aligned with the AC‐PC line. Each image was segmented into gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) using the segment function of spm12. Subsequently, the segmented GM images were spatially normalized using diffeomorphic anatomical registration through an exponentiated lie algebra (DARTEL) algorithm.25 Custom DARTEL templates were generated from all the GM and WM images of the participants. After spatial normalization, the GM images were modulated to preserve the volume, followed by smoothing with an 8‐mm full width at half maximum Gaussian kernel. For this preprocessing, default parameters were used. In addition to that, total intracranial volume (TIV) was calculated by summing GM image, WM image, and CSF image using the ‘Tissue Volumes’ function of spm12.

ROI for schizophrenia

We used the ‘two‐sample t‐test’ model implemented in spm12 to detect regions where patients with schizophrenia exhibited decreased volume compared with control subjects. Age, sex, and TIV were input as covariates of no interest. In addition, to consider the use of two different MRI scanners, we added another column to the design matrix to account for scanner difference and designated ‘one’ for one scanner and ‘zero’ for the second scanner as done by Mechelli et al. for multi‐scanner data analysis.26 As we wanted to obtain robust ROI, we employed conservative statistical thresholds for both peak‐level and cluster‐level. As for peak‐level, statistical threshold was set to a family‐wise error (FWE)‐corrected P‐value of <0.05. With this threshold as a cluster‐defining threshold, an extent threshold of 250 voxels, which corresponded to a false discovery rate (FDR)‐corrected P‐value of <0.05, was set as a cluster‐level threshold. We should note that we employed FDR instead of FWE for cluster‐level correction because the FWE‐corrected P‐value of <0.05 corresponded to only one voxel. As described in the Results section, the ROI comprised four clusters. Since voxel values within these clusters were highly correlated with each other, handling the values of each cluster separately would introduce a multicollinearity problem. Therefore, we united the clusters to form one ROI, binarized it, and obtained the averaged modulated volume within the united ROI as a single value for each subject.

Feature definition and statistical analysis

In clinical settings, MRI parameters vary from site to site. In this situation, it is almost impossible to accommodate a wide variety of scanner differences with a single model. Therefore, we employed a strategy to determine the coefficients of a model for each MRI scanner. Voxel values of MRI data are subject to MRI scanner differences as well as various subject factors, such as age, sex, TIV, or disease. In order to estimate the effect of MRI scanners accurately, we also considered age, sex, and TIV in the model and estimated the effect of MRI scanners using healthy control MRI data. This process is similar to preparing a standard for a control experiment. We chose a general‐linear‐model‐based approach to extract a feature for differentiation. Though recent studies have tended to employ a machine‐learning approach or multivariate analysis to extract features for classification, it is difficult to evaluate the effect of how MRI scanner differences are considered with these approaches. Therefore, we used a simple linear model to see if the model could minimize scanner differences. Using modulated volume within ROI as an objective variable (Y), we defined the model as follows:

$$\mathrm{Y}={\beta }_1\times \mathrm{Age}+{\beta }_2\times \mathrm{Sex}+{\beta }_3\times \mathrm{TIV}+C+\epsilon ,$$

where the betas are coefficients for each independent variable, C is a constant term in which we expect to include the scanner factor, and ε is the residual, which we use as a classification feature. Given a design matrix Y consisting of modulated volumes within the ROI (n × 1 matrix where n is the number of subjects), X consisting of age, sex, TIV and ‘one’ for each subject (n × 4 matrix), B consisting of β ₁, β ₂, β ₃, and C (4 × 1 matrix), and E consisting of residuals (n × 1 matrix), the formula can also be described as follows:

$$\mathrm{Y}=\mathrm{XB}+\mathrm{E}$$

According to the ordinary least square method, B is estimated by multiplying the inversion matrix of X by Y. However, X is not a square matrix here, so we cannot directly compute the inversion matrix of X. In this situation, considering X^T, which is the transposed matrix of X, X^TX will always be a square matrix, so using the inversion matrix of X^TX, (X^TX)⁻¹, B can be estimated.

$$\begin{array}{l}\mathrm{Y}=\mathrm{XB}\\ {\mathrm{X}}^{\mathrm{T}}\mathrm{Y}={\mathrm{X}}^{\mathrm{T}}\mathrm{XB}\\ {\left({\mathrm{X}}^{\mathrm{T}}\mathrm{X}\right)}^{-1}{\mathrm{X}}^{\mathrm{T}}\mathrm{Y}={\left({\mathrm{X}}^{\mathrm{T}}\mathrm{X}\right)}^{-1}{\mathrm{X}}^{\mathrm{T}}\mathrm{XB}\\ {\left({\mathrm{X}}^{\mathrm{T}}\mathrm{X}\right)}^{-1}{\mathrm{X}}^{\mathrm{T}}\mathrm{Y}=\mathrm{B}\end{array}$$

This computation can be easily done with matlab using the pinv function (B = pinv(X)*Y), so we used matlab R2015b to estimate the Matrix B. After obtaining B, we applied Formula 1 to all subjects for each scanner to obtain Matrix E, which comprises ε for each subject. Matrix E represents the ‘schizophrenia likeness’ feature by minimizing the MRI scanner effect and adjusting for age, sex, and TIV at the same time. Figure 1 is the processing flowchart. We then performed receiver–operator curve (ROC) analysis with the ROCR package in R 3.4.3 (R Foundation for Statistical Computing, Vienna, Austria) to evaluate the accuracy of the differentiation.27

Figure 1.

Scheme of feature extraction. Segmentation of 3‐D T1‐weighted image was performed to extract gray matter image and intracranial volume (ICV), followed by anatomical normalization with diffeomorphic anatomical registration through an exponentiated lie algebra (DARTEL) and smoothing. Then the within‐region‐of‐interest (ROI) volume was calculated. Using this volume as a dependent variable Y, a general linear model was fitted considering age, sex, ICV, and a constant that could reflect scanner differences for each MRI scanner. Then the residual ε for each subject was calculated and this value was treated as a differentiation feature.

Validation of our method

Our method for deciding the coefficient of the model is entirely dependent on control subjects. We explored how coefficients from different datasets would affect the results of ROC analysis. As Osaka A was the largest dataset in Dataset B, we used it for validation. We randomly divided the Osaka A dataset into two subsets. We prepared Matrices Y₁ and X₁ from Subset 1 and Matrices Y₂ and X₂ from Subset 2. From the control subjects of Subset 1, we calculated coefficient Matrix B₁. We also calculated coefficient Matrix B₂ from the control subjects of Subset 2. Then we calculated the residual Matrix E₁ from Subset 1 with E₁ = Y₁ – X₁B₂ and the residual Matrix E₂ from Subset 2 with E₂ = Y₂ – X₂B₁. ROC analyses were performed on these residuals.

Results

ROI for schizophrenia

Figure 2 and Table 4 show the ROI we used for further analysis. Patients with schizophrenia showed a significant gray matter volume reduction in the bilateral insulae, superior temporal gyri, the middle frontal gyri, the medial portion of the superior frontal gyri, and the hippocampi. These results are consistent with a previous meta‐analysis,5, 6, 7 so we used these areas as our ROI. We did not find any volume increase in patients with schizophrenia with the statistical threshold we employed.

Figure 2.

Regions of interest (ROI) for this study. ROI are defined from group comparisons between 95 patients with schizophrenia and age‐ and sex‐matched controls. Statistical threshold was set to a family‐wise‐error‐corrected P‐value of <0.01 with an extent threshold of 100 voxels. These regions include the bilateral superior temporal gyri, the middle frontal gyri, the medial portion of the superior frontal gyri, and hippocampi.

Table 4.

Volume reduction in patients with schizophrenia using Dataset A

Cluster	Region	Cluster size	Cluster‐level FDR corrected P	Peak‐level FWE corrected P	T	MNI coordinates
						x	y	z
Cluster 1	Lt. anterior insula	4787	<0.001	<0.001	8.34	−36	23	−5
	Lt. hippocampus			<0.001	6.58	−14	−6	−15
	Lt. opercular part of inferior frontal gyrus			<0.001	6.12	−50	12	4
Cluster 2	Rt. anterior insula	2792	<0.001	<0.001	7.48	39	21	−3
	Rt. planum polare			<0.001	7.03	57	−1	1
	Rt. posterior orbital gyrus			<0.001	6.9	36	18	−18
Cluster 3	Rt. medial frontal gyrus	5489	<0.001	<0.001	7.46	2	41	−20
	Lt. superior frontal lobe medial segment			<0.001	7.06	−3	51	3
	Lt. medial frontal gyrus			<0.001	6.93	0	54	−11
Cluster 4	Lt. middle cingulate gyrus	810	<0.001	<0.001	6.38	−8	−13	39
	Rt. middle cingulate gyrus			0.003	5.38	6	−25	36

FDR, False discovery rate; FWE, Family‐wise error; MNI, Montreal Neurological Institute.

ROC analysis

Table 5 shows the statistics for Dataset B. Figure 3 and Table 6 show the results of ROC analysis. The purpose of this study was to evaluate how a single model with a certain feature can be applied to different sites, so we adopted a single cut‐off among analyses. ROC analysis with the Osaka A scanner (GE 1.5T) with the largest sample in this study revealed that the cut‐off score of the closest point to (0, 1) in the ROC curve was −1.3. Therefore, we applied this value (−1.3) as a cut‐off score to evaluate sensitivity, specificity, and accuracy for each ROC analysis. As a result, sensitivity ranged from 47% to 67% as well as specificity from 75% to 84%. The area under the curve (AUC) ranged from 0.74 to 0.84 and accuracy ranged from 69% to 76%. ROC analysis with all samples revealed an AUC of 0.76 and an accuracy of 73%. Table S1 shows the ROC results for the two subsets of Osaka A. The results were similar, which indicates the validity of our method to some extent.

Table 5.

Statistics of Dataset B

	Y: volume within ROI (mL) (mean ± SD)		TIV (liter) (mean ± SD)		ε: residuals (mean ± SD)
	Schizophrenia	Control	Schizophrenia	Control	Schizophrenia	Control
Osaka A	24.70 ± 2.97	26.38 ± 3.20	1.41 ± 0.14	1.40 ± 0.13	−1.69 ± 1.71	0.00 ± 1.54
Osaka B	23.88 ± 3.34	26.79 ± 3.48	1.46 ± 0.16	1.49 ± 0.14	−2.14 ± 1.84	0.00 ± 1.60
Osaka C	24.75 ± 4.00	28.54 ± 3.79	1.48 ± 0.15	1.50 ± 0.13	−2.39 ± 2.29	0.00 ± 1.81
Tokyo B	26.29 ± 2.97	25.05 ± 2.66	1.33 ± 0.49	1.56 ± 0.50	−1.08 ± 1.95	0.00 ± 1.57
Toyama	26.58 ± 3.09	28.21 ± 2.83	1.44 ± 0.16	1.43 ± 0.13	−1.66 ± 1.75	0.00 ± 1.59
Kyushu	23.93 ± 2.85	26.65 ± 3.04	1.39 ± 0.14	1.41 ± 0.13	−1.60 ± 2.07	0.00 ± 1.61

ROI, region of interest; TIV, total intracranial volume.

Figure 3.

Receiver–operator curve (ROC) analysis. The area under the curve (AUC) ranged from 0.74 to 0.84, and accuracy from 69% to 76%. ROC analysis with all samples revealed an AUC of 0.76 and an accuracy of 73%.

Table 6.

Results of receiver–operator curve analysis

	Osaka A	Osaka B	Osaka C	Tokyo B	Toyama	Kyushu	All data
Schizophrenia	129	74	73	17	121	32	446
Control	404	239	244	54	130	86	1157
Area under curve	0.78	0.84	0.83	0.68	0.76	0.74	0.76
Sensitivity	0.60	0.67	0.70	0.44	0.57	0.56	0.63
Specificity	0.81	0.80	0.77	0.84	0.79	0.80	0.75
Positive predictive value	0.49	0.50	0.47	0.47	0.71	0.51	0.34
Negative predictive value	0.86	0.88	0.89	0.82	0.66	0.83	0.91
Accuracy	0.76	0.76	0.74	0.74	0.69	0.74	0.73

Discussion

In this study, we tried to minimize the effect of scanner differences for differentiation of schizophrenia patients from control subjects. Even with a single cut‐off score, overall accuracy remained around 70% among different sites/scanners, which implies the reproducibility of our model for different datasets. Our results also have the potentiality for clinical application. Conventionally, in order to find the schizophrenia likeness from brain MRI data, one needs to prepare datasets for large numbers of patients and controls. In this situation, if a facility's MRI scanner is replaced, researchers and physicians must create entirely new datasets with data from the new machine. However, the model we propose in this study requires only control subjects for a specific MRI scanner. Once the coefficients of the model are determined, we can apply the model to as few as one patient, eliminating the need for multiple datasets from multiple individuals, to see if that single patient has a schizophrenia likeness.

Though there are various reports that have discussed the classification of schizophrenia patients from a control group using structural MRI, multisite studies are limited. Rozycki and colleagues used multivariate analysis tools and found a neuroanatomical signature of patients with schizophrenia, with which they achieved a prediction accuracy of 72%–77%.28 They took into account the effect of MRI scanner as a variable in the multivariate analysis. Our results are similar to theirs, which substantiates that structural MRI could be a useful tool for schizophrenia at the individual level and indicates that our approach of considering the MRI scanner as a factor in the model could be as effective as their approach. From a diagnostic point of view, the accuracy of our method did not exceed the previous reports. Meta‐analysis of brain volumes in schizophrenia showed that effect sizes for gray matter structures ranged from −0.22 to −0.58.29 This effect size means that there is substantial overlap between patients with schizophrenia and controls. In this context, it might be difficult to improve the accuracy of classification with structural data alone. Another point of our result is that specificity is generally higher than sensitivity in each site of Dataset B. Florkowski indicates that high sensitivity is the ideal property of a ‘rule‐out’ test while high specificity is the ideal property of a ‘rule‐in’ test.30 Considering this, our results suggest that our method might be suitable to rule‐in the possibilities of schizophrenia rather than rule‐out.

Not only structural MRI, but also other modalities, such as functional MRI (fMRI) or diffusion tensor imaging, could contribute to the differentiation of schizophrenia at the individual level. Recent studies using the Alzheimer's Disease Neuroimaging Initiative dataset to investigate the combined biomarkers from different modalities, such as structural MRI, fMRI, fluorodeoxyglucose–positron emission tomography, or CSF, to discriminate between Alzheimer's disease, mild cognitive impairment, and control reported that classification accuracy with a combination of modalities was better than that with a single modality.31, 32, 33 Similar results have been reported for the classification of schizophrenia. Yang et al. reported that a classifier of schizophrenia using combined features of structural and functional MRI data achieved higher accuracy than did a single‐modal‐features method.34 However, no multisite studies on the differentiation of schizophrenia using multimodal imaging have been reported to date. Scanner differences need to be considered for diffusion tensor imaging or fMRI as well as for structural MRI. Our approach might provide a means to identify appropriate features of different modalities while also considering scanner differences.

There are several limitations in our study. Our method for deciding the coefficient of the model was entirely dependent on control subjects. Though we showed that coefficients from different control subjects did not change the result much, we have not explored the minimum number of subjects required in order to obtain robust results. In Dataset B, Tokyo B and Kyushu resulted in low sensitivities. The sample sizes of the control group for the Tokyo B and Kyushu datasets were relatively small (less than 100) compared with those for other facilities, which might have affected the results. We must also consider the variation in control subjects. In Dataset B, the subjects for Toyama were young and the standard deviation of age was small. In this situation, two things need to be considered: (i) brain volume changes might be subtle due to the shorter duration of disease; and (ii) overfitting might occur when calculating coefficients, as the range of variables is limited. We used as many control subjects as possible for this study, but in order for this approach to become feasible in a clinical setting, further study is necessary to determine the minimal sample size for control subjects. At the same time, the Alzheimer's Disease Neuroimaging Initiative uses phantom and common pulse sequence to obtain images of similar quality across MRI scanners. This kind of approach might also be useful in our model. Another limitation is that the ROI we employed was determined using only 1.5‐T MRI scanners. Though we set a statistically conservative threshold in order to generate a robust ROI and the results were consistent with a previous meta‐analysis,5, 6, 7 larger balanced datasets using different MRI scanners might be necessary to generate a more robust ROI. One other limitation is that most of the subjects with schizophrenia were on medication. It is known that antipsychotics can affect brain volume, especially subcortical volume.35 As our model depends upon control subjects, we did not take the medication dose into account. However, we defined the ROI with a conservative threshold (family wise error P < 0.05 with extent threshold of 250 voxels) in order to ensure the use of robust regions for the present analysis.

In conclusion, we demonstrated that in considering scanner differences we could differentiate schizophrenia from control using structural MRI across multiple sites. This could be a useful method for applying neuroimaging techniques to a clinical setting in order to achieve an accurate diagnosis of schizophrenia.

Disclosure statement

The authors declare no conflicts of interest.

Author contributions

K.N. was critically involved in analysis and wrote the first draft of the manuscript. T.S., M.F., F.Y., and M.T. were involved in the data analysis and contributed to the interpretation of the data. H.Y., Y.Y., H.A., N.K., Y.W., M.K., T.T., S.K., N.O., Y.H., T.O., H.Y., M.S., and K.K. were involved in data collection and contributed to the interpretation of the data. T.A. was involved in the interpretation of the data and critical review of the manuscript. R.H. supervised the entire project, collected the data, and was critically involved in the design and interpretation of the data. All authors contributed to and approved the final manuscript.

Supporting information

Table S1. Results of ROC analysis on Osaka A subset data.