Multivariate classification of schizophrenia and its familial risk based on load-dependent attentional control brain functional connectivity

Abstract

Patients with schizophrenia (SCZ), as well as their unaffected siblings (SIB), show functional connectivity (FC) alterations during performance of tasks involving attention. As compared with SCZ, these alterations are present in SIB to a lesser extent and are more pronounced during high cognitive demand, thus possibly representing one of the pathways in which familial risk is translated into the SCZ phenotype. Our aim is to measure the separability of SCZ and SIB from healthy controls (HC) using attentional control-dependent FC patterns, and to test to which extent these patterns span a continuum of neurofunctional alterations between HC and SCZ. 65 SCZ with 65 age and gender-matched HC and 39 SIB with 39 matched HC underwent the Variable Attentional Control (VAC) task. Load-dependent connectivity matrices were generated according to correct responses in each VAC load. Classification performances of high, intermediate and low VAC load FC on HC-SCZ and HC-SIB cohorts were tested through machine learning techniques within a repeated nested cross-validation framework. HC-SCZ classification models were applied to the HC-SIB cohort, and vice-versa. A high load-related decreased FC pattern discriminated between HC and SCZ with 66.9% accuracy and with 57.7% accuracy between HC and SIB. A high load-related increased FC network separated SIB from HC (69.6% accuracy), but not SCZ from HC (48.5% accuracy). Our findings revealed signatures of attentional FC abnormalities shared by SCZ and SIB individuals. We also found evidence for potential, SIB-specific FC signature, which may point to compensatory neurofunctional mechanisms in persons at familial risk for schizophrenia.

Introduction

Schizophrenia is a brain disease whose risk pathways are heterogeneous and multi-factorial [1, 2]. Such complex architecture is mainly due, on one hand, to the multiple gene-by-gene additive and epistatic interactions [1, 3, 4] and, on the other hand, to the interaction between genetic risk and environmental risk factors [2]. Within this framework, a wide variety of studies [1, 5–8] have proved that the investigation of people with familial risk for schizophrenia (e.g., unaffected siblings of patients-SIB) may uncover biological traits with shared heritability with schizophrenia, which may thus have a role in the risk pathways of the disorder, as SIB share about half of the genetic variation with probands [9].

Within this context, studies have indicated brain functional anomalies in patients with schizophrenia (SCZ) and SIB during cognitive processing [7, 8, 10], which are more prominent in both groups when a high cognitive demand is required. Furthermore, whole-brain functional connectivity (FC) alterations in SCZ are a largely replicated finding [8, 11–14]. Previous evidence [15–19] indicated that such FC anomalies are also detectable in SIB, as well as in other first-degree relatives of SCZ [15–17]. In particular, a previous investigation [5] has revealed that SCZ exhibit thalamic hypoconnectivity within an attentional-control related network during task-related functional Magnetic Resonance Imaging (fMRI). Interestingly, this disease-associated phenotype was present also in SIB, even if to a lesser extent. Indeed, SIB’s FC pattern was intermediate between those displayed by SCZ and healthy controls (HC).

Overall, this body of evidence suggests that abnormalities in attentional control-related FC is core, state-independent schizophrenia phenotype [7] and is potentially representative of an endophenotypic biomarker. However, schizophrenia-associated FC phenotypes have been mostly described at the group level through univariate statistics, and studies have so far had limited success in identifying clinically meaningful markers of disease [20, 21]. Indeed, several fMRI studies did not investigate complex brain network properties but have focused on single, a-priori defined regions of interest (ROIs), assessing significance through group-level tests [22, 23]. However, to understand whether a disease-associated intermediate phenotype could also be qualified as a disease biomarker (i.e., as a measurable feature associated with a certain condition or biological process [24]), one should investigate its sensitivity and specificity in identifying the respective patient population [21, 25, 26]. To take into account the complexity of FC at the network level, as well as to investigate such performance metrics, Support Vector Machine (SVM) techniques can be applied to brain imaging data. SVM can “learn” whole-brain FC patterns which characterize brain pathologies from training data and generalize the learned discriminative rules to unseen patients [27]. SVM techniques are therefore considered a promising tool to identify individualized biomarkers for numerous psychiatric illnesses [28, 29].

Previous studies [30–32] suggested that FC patterns have good discriminative power in classifying SCZ from HC and allow to identify disease-associated patterns of alterations. Nevertheless, only three studies have investigated whether FC patterns may have the potential to identify a brain signature able to classify at the multivariate level both people with the disease and people at familial risk for the disease. Findings from two studies have revealed the existence of network-based FC signatures that allow the discrimination of patients with SCZ, SIB, and HC [33, 34], but only using resting state data. On the other hand, another resting state study [35] reported through multivariate techniques the existence of both FC commonalities and differences between unaffected siblings and their probands. These shared and unique FC patterns could, therefore, reflect the contemporary familial risk-related and compensatory-related mechanisms occurring in SIB at the brain functional level.

However, no study has yet been conducted to assess the ability of task-related FC to identify a pattern of connectivity alterations which characterizes both SCZ and SIB at the multivariate level. Notably, cognitive deficits associated with specific neural substrates predate the onset of the disorder [36]. Furthermore, studies have demonstrated that a large portion of resting-state fMRI BOLD signal change is associated with vascular effects [37], as well as with individual emotional and mood variation that is difficult to control [38, 39], thus limiting the utility of “pure” FC patterns at rest. Therefore, we believe that the use of task-related fMRI data for multivariate classification purposes would (i) offer complementary information to resting state on schizophrenia pathophysiology, and (ii) overcome some of the shortcomings of resting state MRI. Therefore, the aim of this study is to identify both common and unique patterns of functional alterations between SCZ and SIB through SVM techniques. We employed a “reversal discovery-validation strategy” (Fig. 1), in which FC signatures for SCZ and SIB were built independently and subsequently cross-validated between the two cohorts. Furthermore, we investigated the potential clinical associations of these findings by testing the association between such discriminative signatures and well-established clinical phenotypes of schizophrenia (i.e., symptomatology levels), and of risk for the disease (i.e., schizotypy), given findings revealing that familial risk for psychosis disorder predicts schizotypy in SIB [40].

Fig. 1.

The reversal discovery-validation machine learning strategy employed to identify. a disease and risk-related functional connectivity signature. HC-SCZ and HC-SIB Support Vector Machine-based classification models were built within a repeated nested cross-validation scheme (10 permutations × 10 folds both for HC-SCZ and HC-SIB classifications, and related validations)

Materials and methods

Sample determination

65 SCZ on stable antipsychotic treatment for at least one month and 39 SIB were enrolled in this study. In order to avoid demographic and sample size confounds within the machine learning pipeline, two independent age and gender propensity-matched [41, 42] HC cohorts (N = 65 and N = 39; Supplementary Information, section 1) were selected from a larger sample (N = 324) of HC who previously took part in an fMRI experiment. Diagnosis of schizophrenia for SCZ, as well as exclusion of any psychiatric diagnoses for HC, were assessed with the Structured Clinical Interview for DSM-IV [43]. Symptoms severity was assessed through the Positive And Negative Symptoms Scale [44] (PANSS). Schizotypal personality characteristics were assessed in SIB through the Schizotypal Personality Questionnaire [45] (SPQ), a self-assessment questionnaire consisting of 74 items with dichotomous response format. Inclusion and exclusion criteria are reported in Supplementary Information, section 1. All procedures performed in this study were in accordance with the ethical standards of the institutional and/or national research committee and with the 2013 Helsinki Declaration or comparable ethical standards. Informed consent was obtained from all participants included in the study.

The neuropsychological task

Participants underwent fMRI while performing the Variable Attentional Control (VAC) task [6, 46, 47], which requires three increasing levels of attentional control. It is composed of stimuli made of arrows with 3 different sizes either pointing to the right or to the left. In the task, 42 small arrows are embedded in 6 medium arrows, which are in turn embedded in a single large arrow (Fig. S1). Stimuli are presented with a cue word above each stimulus (big, medium, or small). Individuals are instructed to focus on the cue word and press either the left or right button on an optic fiber button box corresponding to the direction of the big, medium, or small arrows indicated by the cue word. To increase the level of attentional control required (VAC load), the direction of the arrows may be congruent or incongruent across all three sizes. This resulted in three different conditions and in three different VAC loads, according to the level of attentional control required (high, intermediate, low; Supplementary Information, section 2).

Analysis on demographics and behavioral data

Two sample t-tests and χ² tests were used to compare demographic characteristics across groups (Table 1). Furthermore, repeated-measures ANOVA (between factor: group, repeated-measures factor: VAC load) was employed to investigate any significant group by VAC load interaction on behavioral performance within the HC-SCZ and HC-SIB cohorts. Specifically, behavioral accuracy (expressed as the percentage of correct responses) and reaction time (RT, in milliseconds) were calculated per VAC level of attentional control. Details on BOLD fMRI data acquisition and analysis are reported in Supplementary Information, section 3.

Table 1.

Demographic and neuropsychological characteristics (mean(SD)) of the (A) SCZ and HC-matched cohort, and (B) SIB and HC-matched cohort

A	N	Age	Gender ratio (m:f)	VAC low load behavioral accuracy	VAC low load RT	VAC int load behavioral accuracy	VAC int load RT	VAC high load behavioral accuracy	VAC high load RT	PANSS total score	Chlorpr. Eq.
HC	65	29 (5)	53:12	99.4 (2.05)	876.23 (178.77)	96.29 (3.97)	1008 (170.89)	91 (11.11)	1084.75 (193.4)	n.a.	n.a.
SCZ	65	29 (6)	50:15	94.07 (8.01)	838.28 (172.97)	82.78 (13.18)	955.13 (220.51)	80 (14.52)	984.26 (224.9)	98.44 (29.3)	678.5 (351.31)
p value	1	0.5	0.7	<0.0001	0.23	<0.0001	0.13	<0.0001	0.008	n.a.	n.a.

B	N	Age	Gender ratio (m:f)	VAC low load behavioral accuracy	VAC low load RT	VAC int load behavioral accuracy	VAC int load RT	VAC high load behavioral accuracy	VAC high load RT	SPQ total score
HC	39	35 (10)	20:18	99.18 (2.18)	799.64 (139.85)	91.46 (11.65)	951.63 (175.93)	85.88 (15.60)	1021.21 (184.74)
SIB	39	35 (9)	15:24	98.31 (3.61)	864.82 (150.54)	91.98 (9.03)	1043.48 (154.61)	88.65 (12.84)	1076.22 (160.84)	12.42 (10.79)
p value	1	0.9	0.2	0.24	0.07	0.83	0.02	0.42	0.18	n.a.

C	Age	Gender ratio (m:f)	VAC low load behavioral accuracy	VAC low load RT	VAC int load behavioral accuracy	VAC int load RT	VAC high load behavioral accuracy	VAC high load RT
HC(SCZ) vs. HC(SIB)	<0.001	<0.001	0.62	<0.001	<0.001	0.14	0.07	0.12
SCZ vs. SIB	<0.001	<0.001	<0.001	0.44	<0.001	<0.001	<0.001	<0.001

Table 1/c reports significant and non-significant differences between the SCZ-matched and the SIB-matched HC cohorts, and between SCZ and SIB groups. All p < 0.05

VAC variable attentional control, int intermediate, RT reaction time, PANSS positive and negative symptoms scale, SPQ schizotypal personality questionnaire

Load-dependent connectivity matrices generation

Vector of onsets reflecting the timing of correct responses during each VAC load, as well as slice-time corrected, unwarped, spatially normalized (MNI template) and smoothed (gaussian kernel = 8 mm) VAC-related fMRI volumes (Supplementary Information, section 3) entered the BetA Series COrrelation (BASCO) toolbox [48], which was specifically designed to investigate inter-regional FC in event-related fMRI data. Individual ROIxROI VAC load-related connectivity matrices (Pearson’s r, Fisher-z transformed [48]; Supplementary Information, section 4) based on the Dosenbach atlas [49], composed of 160 ROIs, were generated, resulting in 12720 fMRI features for every participant within each load-dependent matrix. Following this process, high, intermediate and low attentional control-related connectivity matrices were obtained.

Machine learning strategy

Following our aim, we employed a reversal discovery-validation strategy (Fig. 1) across the two samples through the following steps:

1. a HC-SCZ load-dependent FC classifier was built to identify a disease-related FC signature. To investigate whether this disease-related signature could also be associated with familial risk for schizophrenia, models generated for HC-SCZ classification were applied to the HC-SIB cohort. Here, we expected to identify a pattern of FC anomalies that not only classified HC and SCZ with high accuracy, but that could also discriminate between HC and SIB with good validation performance.

2. a HC-SIB load-dependent FC classifier was built and then applied to the HC-SCZ cohort. We expected to generate a classifier able to discriminate between HC-SIB, but with poor or null validation performance on the HC-SCZ cohort. Indeed, some previous literature suggests that functional activity and connectivity abnormalities in SIB are qualitatively similar to those exhibited by SCZ, but quantitatively present only to a lesser extent [5, 7]; on the other hand, other studies show the presence of potential compensatory mechanisms occurring in SIB, and not in SCZ, at the functional level [35].

Machine learning pipeline

The machine learning platform NeuroMiner (www.pronia.eu/neurominer), version 1.0 [50], was used to set up a machine learning analysis pipeline in which (i) the individual classification ability (HC vs. SCZ and HC vs. SIB) of the three VAC load-dependent FC matrices (high, intermediate, low, hereby called “uni-modal classifiers”), and (ii) the classification performance of the three uni-modal classifiers together (HC vs. SCZ and HC vs. SIB), was tested. For this purpose, uni-modal classifiers were combined within the machine learning environment through a stacking procedure [51]. To avoid overfitting, test the estimation of the model’s generalizability and prevent information leakage between training and test participants, repeated nested cross-validation (CV) was employed (Supplementary Information, section 4). This CV structure implies the presence of an inner CV cycle, where models are generated, embedded in another, super-ordinate, outer CV cycle which is ultimately used to test for models’ generalizability [20]. In the inner CV cycle, matrices were pruned of zero-variance features. Then, a dimensionality reduction procedure was applied using Principal Component Analysis (PCA). Consistently with recent work [26, 50, 52], PCA models were trained with a limited number of Principal Components (15, 20, and 25 eigenvectors) in the CV1 training data in order to reduce the risk of overfitting and increase the generalizability of classification models [53] . PC scores were 0–1 scaled and underwent a greedy sequential forward feature selection procedure [21, 54] that employed linear SVM [55] to detect a set of PCs that optimally predicted the training and test cases’ labels in given CV1 partition (Supplementary Information, section 4). To avoid overfitting, we stopped the forward feature selection procedure when 20% of the PCs had been discarded from the feature pool.

This analysis pipeline was applied to the outer CV cycle determining the participant’s classification (HC vs. SCZ and HC vs. SIB) through majority voting within every uni-modal classifier. This way, we obtained SVM decision models for high, intermediate and low attentional control-related FC features. Decision models from each of the three VAC load-related uni-modal classifiers were then used to build a stacking-based classifier [51, 56]. Stacked generalization [51] trains a new higher-level machine-learning algorithm which learns from the decisions of lower-level models [21]. Specifically, we used the decisions models from our three VAC load uni-modal classifiers altogether (high, intermediate, low) to build a new, final learning algorithm which did not learn from raw features (i.e., pairwise functional connections), but from the VAC load-related decisions scores of the uni-modal classifiers. This was done in order to investigate whether using all the information coming from the VAC task, i.e., combining decisions from high, intermediate and low VAC load classifiers, would have led to an increase in HC-SCZ and HC-SIB classification accuracy, compared with the classification ability of VAC load-related uni-modal classifiers. Statistical significance of individual and ensemble classifiers was assessed through permutation testing [57, 58], with α = 0.05 and 1000 permutations (Supplementary Information, section 5). To better understand which variables might inform SCZ and SIB classes at the single-subject level, we extracted the percentage difference in connectivity between HC and SCZ and between HC and SIB for the 99th percentile of reliable connections, following published procedures [30]. Reliability for each connection is defined in terms of a Cross-Validation Ratio (CVR = mean(w)/standard error(w)) [50]. In this formula, w represents the normalized individual weights from SVM models generated in the repeated nested CV scheme. Normalization is performed using the Euclidean norm of w, defined as s = w/||w||₂ [50]. A positive CVR for each pairwise connection indicates greater connectivity for SCZ or SIB compared to HC, while a negative CVR for each pairwise connection indicates a decrease in connectivity for SCZ or compared to HC. CVR for the 99th percentile of most reliable connections [30] of the VAC high load uni-modal classifiers discriminating between SCZ and HC and between SIB and HC (see sections “HC-SCZ classification validated in the HC-SIB cohort” and “HC-SIB classification validated in the HC-SCZ cohort”) are reported in Table S2/A and S2/B, respectively.

Validation analyses of individual and stacking-based classifiers

Following the strategy described in section “Load-dependent connectivity matrices generation”, after building classification models, we respectively validated: (i) the HC-SCZ model in the HC-SIB cohort, and (ii) the HC-SIB model in the HC-SCZ cohort. To do so, all SVM decision models obtained from our individual and stacked analyses were applied without any in-between re-training step to the respective external validation cohort.

Additional classification analyses

To exclude that demographic and neuropsychological differences between the two HC cohorts (Table 1/c) would have affected the discrimination performance of HC-SCZ and HC-SIB models, we performed additional classification analyses between the two HC cohorts. Related methods and results are reported in Supplementary Information, section 6, and Table S3.

Correlations between models’ decision scores and disease and risk-related phenotypes

To further explore the relationship between the performance of the VAC load-high uni-modal classifier generated in HC-SCZ and validated in HC-SIB (Section “Results”) and disease and risk-related prototypical phenotypes, we performed Pearson’s r correlation analysis between (i) subject-specific linear SVM decisions scores (generated in SCZ and validated in SIB) and, respectively (ii) PANSS total score, for SCZ (data were available for 37 out of 65 participants), and (iii) SPQ total score for SIB (data were available for 27 out of 39 participants).

Correlations between models’ decision scores and task performance

To further investigate classifiers performance, correlation analyses between the VAC load-high uni-modal classifiers generated, respectively, in HC and SCZ and in HC and SIB, and neuropsychological performance during the high VAC load were performed. We used both behavioral accuracy and RT computed over high attentional control stimuli. Correlations were run in the whole HC-SCZ cohort (N = 130) and in the whole HC-SIB cohort (N = 78). All p values are <0.05, FDR-corrected for multiple comparisons [59]. Further correlation analyses have been conducted in order to exclude any association between HC-SCZ and HC-SIB high VAC classifiers’ decision scores and potential demographic and pharmacological confounds (Supplementary Information, section 7, and Table S4).

Results

Neuropsychological differences between samples

ANOVA on load-dependent behavioral accuracy in the HC-SCZ cohort revealed a main effect of group (F = 55.25, p = 0.0001), with SCZ having poorer behavioral accuracy than HC. In addition, a main effect of VAC load (F = 65.50, p = 0.00001) was found, i.e., participants showed a linear dose effect between behavioral accuracy and VAC load. Analyses revealed also a group by VAC load interaction (F = 8.874, p = 0.0001), such that the behavioral accuracy percentage significantly decreased as the VAC load increased, with SCZ having poorer behavioral accuracy compared with HC in every VAC load (Table 1/a). Furthermore, ANOVA on load-dependent RT revealed a main effect of VAC load (F = 178.5, p = 0.0001), characterized by a linear RT increase as long as the VAC load request increased, but no main effect of group (F = 3.8, p = 0.054). A significant group by VAC load interaction on RT (F = 5.749, p = 0.004) was found, such that VAC RT significantly increased as the VAC load increased, with SCZ having lower RT compared with HC just in the high VAC load (Table 1/a).

HC-SIB neuropsychological comparisons revealed a main effect of VAC load on behavioral accuracy, with all participants showing a linear behavioral accuracy percentage drop as long as the VAC load increased (F = 33.223, p = 0.0001). On the other hand, no main effect of group (F = 0.19, p = 0.655) and no significant group by VAC load interaction (F = 0.84, p = 0.434) on behavioral accuracy were found. For what concerns RT, no main effect of group (F = 3.7, p = 0.059) was found, but a main effect of VAC load, with all participants showing a linear RT increase as long as the VAC load request increased, was present (F = 233.9, p = 0.0001). No significant group by VAC load interaction on RT was found (F = 1.587, p = 0.208). Further sample comparisons are reported in Table 1/c.

HC-SCZ classification validated in the HC-SIB cohort

The high attentional control classifier correctly discriminated SCZ from HC with a cross-validated accuracy of 66.9% and was significant at p = 0.002. The intermediate attentional control classifier produced a significant classification accuracy of 62.3% (p = 0.013). The low attentional control classifier separated study groups with 64.6% accuracy (p = 0.002). The stacking model, fusing the decisions generated by all classifiers, discriminated HC from SCZ with a 66.9% accuracy and was significant at p = 0.005. Detailed statistics of all individual classification models are reported in Table 2/a. The HC-SCZ classification based on VAC high load uni-modal classifier was mainly characterized by decreased FC between ROIs located in prefrontal cortex, basal ganglia, inferior parietal lobule, insula, cerebellum and thalamus, with SCZ having decreased FC compared to HC (Fig. 2/a, Table S2/A). Applying the models generated within the HC-SCZ cohort to the HC-SIB cohort revealed that individual and stacked classifiers all predicted HC-SIB group membership at chance level (for detailed statistics, see Table 2/b) except for the high attentional control classifier, which was able to discriminate between HC and SIB above chance level (accuracy = 57.7%, Table 2/b).

Table 2.

(A) Validated classification performance of uni-modal and stacked classifiers in the HC-SCZ cohort; (B) classification performance of the models generated in HC-SCZ and applied to the HC-SIB cohort without any in-between re-training; (C) validated classification performance of uni-modal and stacked classifiers in the HC-SIB cohort; (D) classification performance of the models generated in HC-SIB and applied to the HC-SCZ cohort without any in-between re-training

A. Classification: HC vs. SCZ	True negatives	True positives	False negatives	False positives	Sensitivity	Specificity	Accuracy	Positive predictive value	Negative predictive value	Number needed to diagnose	Positive likelihood ratio	Diagnostic odds ratio	Permutation test, p value
High VAC load classifier	56	31	34	9	47.7	86.2	66.9	77.5	62.2	3	3.4	11.9	0.002
Int VAC load classifier	40	41	24	25	63.1	61.5	62.3	62.1	62.5	4.1	1.6	2.7	0.013
Low VAC load classifier	29	55	10	36	84.6	44.6	64.6	60.4	74.4	3.4	1.5	2.3	0.002
Stacking-based classifier (High, Int, Low)	38	49	16	27	75.4	58.5	66.9	64.5	70.4	2.9	1.8	0.4	0.005

B. Out-of-sample validation: HC vs. SIB	True negatives	True positives	False negatives	False positives	Sensitivity	Specificity	Accuracy	Positive predictive value	Negative predictive value	Number needed to diagnose	Positive likelihood ratio	Diagnostic odds ratio
High VAC load classifier	38	7	32	1	17.9	97.4	57.7	87.5	54.3	6.5	7	49
Int VAC load classifier	22	16	23	17	41.0	56.4	48.7	48.5	48.9	−39	0.9	0.9
Low VAC load classifier	12	16	23	27	41.0	33.3	35.9	37.2	34.3	−3.5	0.6	0.4
Stacking-based classifier (High, Int, Low)	22	12	27	17	30.8	56.4	43.6	41.4	44.9	−7.8	0.7	0.5

C. Classification: HC vs. SIB	True negatives	True positives	False negatives	False positives	Sensitivity	Specificity	Accuracy	Positive predictive value	Negative predictive value	Number needed to diagnose	Positive likelihood ratio	Diagnostic odds ratio	Permutation test, p value
High VAC load classifier	37	17	22	2	43.6	94.9	69.2	89.5	62.7	2.6	8.5	72.2	0.003
Int VAC load classifier	25	16	23	14	41.0	64.1	52.6	53.3	52.1	19.5	1.1	1.3	0.317
Low VAC load classifier	28	14	25	11	35.9	71.8	53.8	56.0	52.8	13	1.3	1.6	0.22
Stacking-based classifier (High, Int, Low)	32	18	21	7	46.2	82.1	64.1	72.0	60.4	3.5	2.6	6.6	0.031

D. Out-of-sample validation: HC vs SCZ	True negatives	True positives	False negatives	False positives	Sensitivity	Specificity	Accuracy	Positive predictive value	Negative predictive value	Number needed to diagnose	Positive likelihood ratio	Diagnostic odds ratio
High VAC load classifier	58	5	60	7	7.7	89.2	48.5	41.7	49.2	−32.5	0.7	0.5
Int VAC load classifier	36	24	41	29	36.9	55.4	46.2	45.3	46.8	−13	0.8	0.7
Low VAC load classifier	46	8	57	19	12.3	70.8	41.5	29.6	44.7	−5.9	0.4	0.2
Stacking-based classifier (High, Int, Low)	48	8	57	17	12.3	73.8	43.1	32.0	45.7	−7.2	0.5	0.2

Int intermediate

Fig. 2.

Depicting of the cross-validation ratio-based most reliable connections driving the classification between (a) HC and SCZ (mainly characterized by FC decreases in SCZ), and (b) HC and SIB (mainly characterized by FC increases in SIB). Reliability is defined as the mean value of the SVM weight divided by its standard error across all the generated models in the cross-validation scheme

HC-SIB classification validated in the HC-SCZ cohort

Only the high attentional control classifier correctly discriminated SIB from HC significantly (p = 0.003) with a cross-validated accuracy of 69.2%. The intermediate attentional control classifier produced a 52.6 classification accuracy (p = 0.313) and the low attentional control classifier separated study groups with a 53.8% accuracy (p = 0.22). The stacking-based model discriminated HC from SIB with a 64.1% accuracy (p = 0.031). Detailed statistics of all individual classification models are reported in Table 2/c. Increased FC highly predictive in the SIB vs HC classification based on high load was detected between temporal, dorsolateral prefrontal, cingulate, occipital, parietal, and insular ROIs (Fig. 2/b, Table S2/B), with SIB having increased FC compared to HC. The application of the models generated within the HC-SIB cohort to the HC vs SCZ classification revealed that all individual and stacked classifiers predicted HC-SIB group membership at chance level (for detailed statistics, see Table 2/d).

Decision scores and disease and risk-related phenotypes correlations results

Results in SCZ showed a positive association between high classifier decision scores and PANSS total score (r = 0.328, p = 0.048). Coherently, in SIB, validated high classifier decision scores and SPQ total score were positively associated (r = 0.395, p = 0.05).

Decision scores and task performance correlations results

Correlation analyses performed between VAC load-high uni-modal classifier decision and VAC high load-related behavioral performance revealed no significant association (p = 0.685 for behavioral accuracy, p = 0.098 for RT). In the HC-SIB cohort, there was a significant association between decision scores of the VAC load-high uni-modal classifier (accuracy = 69.2%) and high VAC behavioral accuracy (r = −0.2776, uncorrected p = 0.014, FDR-corrected p = 0.028), but not between decision scores and high VAC RT (r = −0.024, p = 0.833). Moreover, the association of decision scores generated in HC-SCZ and validated in HC-SIB with VAC high load behavioral performance was not significant in SIB (p = 0.414 for behavioral accuracy, p = 0.556 for RT).

Discussion

To the best of our knowledge, this is the first multivariate study assessing the classification ability of attentional control-related FC with respect to both schizophrenia and its familial risk. Furthermore, this is the first machine learning study in which VAC load-related FC patterns, rather than overall task performance-related [60] or resting state FC [33, 34] are used for multivariate classification purposes. Therefore, we believe that our results provide novel insights about (1) the multivariate performance of task-related fMRI connectivity measures in classifying both psychiatric and risk conditions, and (2) gain understanding of the continuum of neurofunctional processes and their clinical correlates linked to attentional control across the risk spectrum.

Our findings reveal that high attentional control-related FC significantly delineates SCZ from HC individuals with an accuracy of 66.9%. Such results are consistent with previous investigations in which functional MRI features were used to discriminate between HC and SCZ with good classification performance [30–32, 61]. Specifically, high VAC load had the highest accuracy out of the three attentional control levels tested, suggesting a dose effect of the stimulus on neurofunctional separability. Furthermore, the stacking-based classifier involving all VAC loads and the high VAC-based classifier performed equally (both accuracies = 66.9%), thus suggesting that intermediate and low-related FC patterns did not yield additional information for discrimination purposes. This is in line with a previous investigation employing the VAC task [6] revealing that the brain activation anomalies in SCZ are load-dependent and that such anomalies are detectable in nodes of the attentional control network [62] when a high cognitive demand is requested.

Furthermore, our results reveal that the majority of aberrant connections in the SCZ group involved prefrontal cortex, basal ganglia, inferior parietal lobule, cerebellum, and thalamus, thus identifying a widespread pattern of decreased FC in SCZ. This observation is consistent with the literature showing that these regions have a crucial role in both general cognitive abilities and in attentional control [3, 5, 6, 8, 62–64]. Furthermore, existing evidence highlights how decreases in FC in such attentional control nodes is associated with schizophrenia-related phenotypes [5, 8, 65–67]. The disease-derived FC pattern also discriminated between HC and SIB above chance level (57.7% accuracy) and is independent of the HC samples included in the model and of potential demographic differences between the two HC cohorts (accuracy = 52.8%, Table S3). Our finding is coherent with previous structural and functional investigations [33, 34, 68] which identified multivariate structural and FC signatures reflecting familial risk for schizophrenia with good multivariate classification performance. However, this is one of the first studies in which task-related fMRI data are used within a machine learning environment for classification purposes in both SCZ and SIB cohorts. While results are consistent with the extant literature, we are convinced that the potential added value of these findings is to qualify attentional control-related FC anomalies as a schizophrenia biomarker [20, 24, 25]. Biomarkers have intrinsic potential for translation into clinical practice. For example, they may be employed to monitor disease progression, or treatment outcome, especially if combined with other multivariate signatures (i.e., clinical and cognitive). The accuracy reported in the present work is below what is needed for clinical translation, but our study shows that more research into attention control-related FC anomalies is warranted to develop novel biomarkers.

Nonetheless, our analysis shows a difference in accuracy between the model separating in HC-SCZ (accuracy = 66.9%) and separating HC-SIB (accuracy = 57.7%). We believe that the decrease in the classification performance of the model when applied to the HC-SIB cohort might be due to the fact that functional (and structural) anomalies in SIB are qualitatively similar to those exhibited by SCZ but quantitatively present at a lesser extent [69]. Therefore, our analyses suggest the existence of a common multivariate FC pattern associated with the psychosis spectrum, but that this FC pattern is expressed to a different extent in SCZ patients and individuals with a familial risk for the disease. The potential clinical relevance of the identified disease and risk-related hypoconnectivity signature is further confirmed by the results of our correlation analyses, which revealed that such signature performs better in classifying SCZ with higher PANSS total score and SIB with higher overall schizotypal score.

Results from the HC-SIB multivariate classification analysis, on one hand, confirm that high attentional control–related connectivity features, out of all individual and stacking-based classifiers, reach the highest classification performance (accuracy = 69.2%, p = 0.003). However, the HC-SIB classification pattern was different from the previously identified HC-SCZ network, mainly characterized by FC decreases in SCZ. Indeed, the majority of aberrant connections in SIB involve temporal, cingulate, dorsolateral prefrontal and insular connections nodes. Of note, differently from the HC-SCZ decreased FC signature, (i) HC-SIB most discriminative functional connections identified an increased FC pattern, and (ii) this increased FC pattern was negatively associated with VAC behavioral accuracy during the high load, such that the more the likelihood of classifying individuals as SIBs, the less the VAC behavioral accuracy at the high VAC load. Taken together, these findings allow to hypothesize a compensatory pattern of increased FC in temporal, cingulate, prefrontal and insular nodes in SIB compared to HC. This interpretation is in line with previous univariate fMRI studies [6, 19, 70–74] testifying region-specific hyperactivity and hyperconnectivity anomalies in SIB. Further, they are consistent with models of the relationship between brain activity and task performance [75] positing that an increase in brain recruitment parallelized by either intact or decreased behavioral performance is an indicator of a compensatory, or even inefficient, recruitment of brain resources. This compensatory interpretation is also consistent with the fact that, when the HC-SIB discriminative hyperconnectivity pattern is applied to the HC-SCZ cohort, it performs at the chance level. To the best of our knowledge, only this work and a previous resting state study [35] reported the possible existence of SIB-specific FC pattern which they do not share with their probands. Together, we believe that these findings shed light on how functional compensatory mechanisms in SIB may be concurrent to FC anomalies associated with familial risk for schizophrenia.

This study has some limitations. First, there were significant age and gender differences between SCZ and SIB and the related HC cohorts. However, (i) no significant associations were found between decision scores and demographic variables, and (ii) models classifying between the two HC cohorts performed at chance levels. These findings, therefore, make it less likely that our classification models were affected by demographic confounds. Another limitation is that SCZ were chronic patients under stable antipsychotic treatment. Even if there was no association between SCZ decision scores and pharmacological treatment levels (Supplementary Information, section 7), this may limit the generalizability of our findings. Replication studies involving individuals at the first episode of the disease, which have shown to have poorer treatment courses [76], may, therefore, give further information about the reliability of our results. A further limitation is the sample size, as we are aware that small sample sizes could be more easily overfitted [77]. In this line of reasoning, it should also be pointed out that another important limitation is the lack of an external validation analysis in independent SCZ and SIB samples, which are currently unavailable in sufficient sample size to the best of our knowledge. Therefore, the generalizability of both the unique and shared FC patterns between SCZ and SIB identified here remains to be tested. Even though a stringent separation of training and test set was used, and a robust, repeated-nested CV scheme was employed, replication in independent and larger samples is needed to ensure that our findings are replicable, as external validation is considered the gold standard in the field to assess model generalizability.

In conclusion, on one hand our results provide evidence of a shared, common schizophrenia and familial risk-related decreased FC signature during high attentional control demand. On the other hand, the increased FC signature able to discriminate between HC and SIB may reflect specific compensatory mechanisms associated with the risk for the disease. Overall, our findings suggest that employing cognitive load-related FC information for classification purposes in schizophrenia may lead to reliable insights on specific cognitive stimuli subtending the existence of both commonalities and differences of both disease and risk-related biomarkers. Future studies are needed to validate both these shared and compensatory functional signatures in first episode patients and in at-risk individuals that subsequently convert psychosis risk symptoms in full-blown disease.

Supplementary information

Supplementary information is available for this paper at (10.1038/s41386-019-0532-3).