On the integrity of functional brain networks in schizophrenia, Parkinson's disease, and advanced age: Evidence from connectivity‐based single‐subject classification

Abstract

Previous whole‐brain functional connectivity studies achieved successful classifications of patients and healthy controls but only offered limited specificity as to affected brain systems. Here, we examined whether the connectivity patterns of functional systems affected in schizophrenia (SCZ), Parkinson's disease (PD), or normal aging equally translate into high classification accuracies for these conditions. We compared classification performance between pre‐defined networks for each group and, for any given network, between groups. Separate support vector machine classifications of 86 SCZ patients, 80 PD patients, and 95 older adults relative to their matched healthy/young controls, respectively, were performed on functional connectivity in 12 task‐based, meta‐analytically defined networks using 25 replications of a nested 10‐fold cross‐validation scheme. Classification performance of the various networks clearly differed between conditions, as those networks that best classified one disease were usually non‐informative for the other. For SCZ, but not PD, emotion‐processing, empathy, and cognitive action control networks distinguished patients most accurately from controls. For PD, but not SCZ, networks subserving autobiographical or semantic memory, motor execution, and theory‐of‐mind cognition yielded the best classifications. In contrast, young–old classification was excellent based on all networks and outperformed both clinical classifications. Our pattern‐classification approach captured associations between clinical and developmental conditions and functional network integrity with a higher level of specificity than did previous whole‐brain analyses. Taken together, our results support resting‐state connectivity as a marker of functional dysregulation in specific networks known to be affected by SCZ and PD, while suggesting that aging affects network integrity in a more global way. Hum Brain Mapp 38:5845–5858, 2017. © 2017 Wiley Periodicals, Inc.

INTRODUCTION

Schizophrenia (SCZ) and Parkinson's disease (PD) are two of the most prevalent and socio‐economically relevant brain diseases [Andlin‐Sobocki et al., 2005]. Although SCZ onset typically emerges during adolescence and early adulthood [Häfner et al., 2013], PD is characterized by an onset during late adulthood [Hughes et al., 1992; Poewe et al., 2017] and has been associated with premature aging, that is, earlier and more rapid neurodegeneration as compared with the course of normal aging (NA) [Rodriguez et al., 2015]. Both SCZ and PD are characterized by disease‐specific pathophysiological changes of the dopaminergic system [Jankovic, 2008; Toda and Abi‐Dargham, 2007], contrasting with a more global dopamine decline in NA [Bäckman et al., 2006]. However, it has been proposed that dopaminergic dysfunction in SCZ arises as a secondary effect due to alterations of the glutaminergic system [Laruelle et al., 2003]. In contrast, in PD dopaminergic deficiency represents the primary cause leading to pathophysiological upstream dysregulations of different neural systems [Obeso et al., 2008]. These neurobiological features of SCZ, PD and NA [Bäckman et al., 2006; Jankovic, 2008; Laruelle et al., 2003; Obeso et al., 2008; Rodriguez et al., 2015; Toda and Abi‐Dargham, 2007] may manifest themselves in functional connectivity alterations at the level of large‐scale brain networks [Cole et al., 2013; Kelly et al., 2009; Narr and Leaver, 2015; Prodoehl et al., 2014; Sala‐Llonch et al., 2015]. However, some putative commonalities (neurodegeneration, dopaminergic dysregulations, and altered connectivity) need to be juxtaposed with the prominent phenotypical differences between SCZ, PD, and NA [Bäckman et al., 2006; Jankovic, 2008; Narr and Leaver, 2015; Prodoehl et al., 2014; Sala‐Llonch et al., 2015; Toda and Abi‐Dargham, 2007] and the fact that the clinical presentations of SCZ and PD are very different [Eaton et al., 1995; Jankovic, 2008; Kalia and Lang, 2015; van Os and Kapur, 2009], raising the question whether various functional systems are differentially affected in the three conditions. Rather than assessing altered activations in different functional systems by conducting task‐based functional magnetic resonance imaging (fMRI) studies, we examined altered functional connectivity within various functional networks robustly defined by meta‐analyses of task‐based neuroimaging studies in a comparative fashion [cf. New et al., 2015; Schilbach et al., 2016]. This has the practicable advantage of using easily accessible, short and standardized resting‐state (RS) data while at the same time incorporating the consolidated knowledge based on task‐based imaging into the analysis. We argue that such an approach is particularly relevant given that in contrast to RS imaging, task‐based assessments will rarely be feasible in a routine clinical setting.

Alterations in functional network integrity patterns in SCZ, PD or older adults (compared with respective healthy/young controls) can be captured by using machine learning‐based classification. For extracting a diagnostically relevant marker that allows the classification of individual subjects based on the connectivity in functional brain networks, multivariate decoding algorithms like support vector machine (SVM) should provide the most appropriate approach for this endeavor. Rather than testing each connection independently for group differences, SVMs are trained on part of the data by weighting all connections in order to separate the known clinical status from healthy controls (HCs). Classification accuracy can then be determined by assessing the ability to predict group membership of previously unseen subjects. Applied to (whole‐brain) connectivity data, this approach has previously been found to distinguish SCZ patients [cf. Arbabshirani et al., 2016; Kambeitz et al., 2015; Wolfers et al., 2015] or PD patients [cf. Chen et al., 2015; Long et al., 2012] from HCs, as well as aged from young subjects (NA) [cf. Meier et al., 2012; Vergun et al., 2013].

Previous pattern‐classification studies aimed at providing the best possible classification performance on whole‐brain connectivity. In contrast, the aim of this work was to assess whether specific functionally defined networks are altered in SCZ, PD, and NA. Although previous studies mainly used Independent Component Analysis (ICA) based data‐driven methods to extract major RS networks [Damoiseaux et al., 2006; Smith et al., 2009], our work is based on a priori meta‐analytically defined networks associated with specific sets of behavioral functions such as working memory [Rottschy et al., 2012] or emotional processing [Sabatinelli et al., 2011]. In contrast to well‐established RS networks, these networks represent the consolidated information from hundreds of task‐based fMRI studies and hence those locations in the brain that are reliably activated when subjects perform tasks pertaining to a particular mental function. We thus argue that these nodes define robust functional networks in the brain related to specific mental domains. In turn, the functions associated with RS networks are usually derived from a reverse inference approach, as these lack any direct relationship to mental functions [Poldrack, 2011]. We suggest that this more direct relationship between the network‐nodes and actual task‐demands is an important advantage of our approach. Moreover, the employed strategy results in an a priori, unbiased definition of the respective networks, whereas ICA‐based networks are usually defined from the current data [Cole et al., 2010]. Our meta‐analytically derived network model approach thus offers the potential to investigate functional connectivity within robust a priori brain networks that are implicated in processing a specific mental process.

Therefore, this study aimed to examine whether the known impairment of different functions in SCZ, PD, or aging, respectively, would equally translate into a high classification accuracy for a given network in the respective group, based on the connectivity pattern within this network. As a “proof‐of‐principle” approach we therefore intended to investigate whether various a priori networks based on task‐activation findings carry differential disease‐related information assessable by RS imaging. To this end, we examined two diseases which are clinically very disparate but well studied in the previous neuroimaging literature. The findings were then juxtaposed to findings on age‐related effects in the same networks. Thereby, we could evaluate whether the respective networks carry differential information related to the different conditions or, conversely, whether the different networks carry differential information related to a particular condition. Given some putative commonalities and especially phenotypical differences, the aim was to examine the possibility for differential classification of SCZ, PD, und age, rather than to primarily study the specific diseases and their clinical separation from each other or aging per se. In our investigation, these three groups thereby serve as examples to evaluate this approach. For example, we assume that connectivity in the reward (Rew) network will be potent in differentiating SCZ patiens from matched HCs, as several studies have shown impairments related to reward learning in SCZ, and the neurobiology of this network has been linked to psychosis [Deserno et al., 2013; Heinz and Schlagenhauf, 2010; Radua et al., 2015]. Likewise, we would expect a good classification accuracy for PD patients based on FC in the motor network, given that motor impairments represent the core feature of this disease [Jankovic, 2008], and motor circuits in the brains of PD patients are altered during motor tasks and at rest [Herz et al., 2014; Prodoehl et al., 2014; Tessitore et al., 2014]. Finally, NA is accompanied by cognitive decline in various domains [Glisky, 2007], such as deterioration in working memory function [Braver and West, 2008]. For the latter, age‐related neural changes have repeatedly been shown at task [Dennis and Cabeza, 2008; Rajah and D'Esposito, 2005] and rest [Keller et al., 2015]. Accordingly, we assume that the working memory (WM) network allows a clear distinction between old and young adults.

In an explorative manner, we furthermore assessed a broad set of networks associated with different behavioural domains (cognitive, social‐affective, motivational, and motor‐related) since all three conditions (PD, SCZ, and NA) show alterations in various functional domains on the behavioral and neural level [Barch, 2005; Duncan et al., 2013; Seidler et al., 2010]. Importantly, in our approach, we reasoned that classification performance may be interpreted as an indication for the amount of information contained in a given network regarding a particular disease or age status, and thus of the degree of change observed in the integrity of particular networks under these conditions.

We assume that classification performance will be best for connectivity in those networks that subserve mental functions known to be affected in SCZ and PD. SCZ is characterized by prominent social‐affective/motivational alterations [Brunet‐Gouet and Decety, 2006; Deserno et al., 2013; Heinz and Schlagenhauf, 2010; Kring and Elis, 2013; Radua et al., 2015], whereas in PD motor impairments are most affected [Herz et al., 2014; Rowe and Siebner, 2012; Tessitore et al., 2014]. We, therefore, hypothesized that social‐affective/motivational and motor‐related networks provide a superior classification of SCZ and PD patients, respectively. As both diseases are accompanied by cognitive impairments as well, we assumed that cognitive networks may also be predictive to some degree [Barch, 2005; Duncan et al., 2013; Elgh et al., 2009; Nieoullon, 2002]. As NA is associated with a broad spectrum of decline affecting various functional systems (albeit to a varying degree) [Hedden, 2007; Mather, 2016; Seidler et al., 2010], we expected that most networks allowed for an accurate discrimination of old from young adults.

MATERIALS AND METHODS

Samples

Schizophrenia

RS fMRI data and phenotypical information of 86 SCZ patients and 84 HCs obtained from the COBRE sample (http://fcon_1000.projects.nitrc.org/indi/retro/cobre.html) and the University Hospital of Göttingen, Germany, were included in the analysis. SCZ diagnosis was assigned as assessed by the DSM‐IV‐TR based on the structured clinical interview (SCID‐P) and the International Classification of Diseases (ICD‐10), respectively. SCZ symptom severity was assed using the Positive and Negative Symptom Scale (PANSS) [Kay et al., 1987] evaluating the severity of positive and negative symptoms as well as the general psychopathology. Patients received their regular medication therapy with considerable variability in the exact compounds used and a high prevalence of combination drug therapy (medicated patients but exact medication and dose unknown for Olanzapine equivalent dose [Gardner et al., 2010]: COBRE: 50.9%; Göttingen: 25.8%; medication status unknown: COBRE: 1 SCZ patient; Göttingen: 2 SCZ patients).

Parkinson's disease

RS fMRI data of 80 PD patients and 84 HCs obtained from the RWTH Aachen University Hospital and the University Hospital Düsseldorf, Germany, were included in the analysis. Diagnosis of PD was assigned by consultant neurologists with longstanding expertise in movement disorders based on clinical examination and review of the medical history. Included PD patients fulfilled the standard UK Brain Bank criteria for PD and had on average a mild cognitive impairment as confirmed by the Montreal Cognitive Assessment (MoCA) but no major depression symptoms [Hoops et al., 2009; Hughes et al., 1992; Nasreddine et al., 2005].

To assess PD symptom severity and evaluate motor impairments the Unified Parkinson's Disease Rating Scale Part III [Movement Disorder Society Task Force on Rating Scales for Parkinson's Disease, 2003] (UPDRS) and Hoehn and Yahr Scale (H & Y Scale) [Hoehn and Yahr, 1967] were applied. All patients were medicated with their regular individual PD‐related treatment (medication and dose unknown for Levodopa equivalent daily dose [Tomlinson et al., 2010]: Aachen: 28.1%; Düsseldorf: 12.5%).

Healthy controls

RS fMRI data of HC (HC_SCZ and HC_PD) were obtained from the four different sites as respective clinical subjects (SCZ and PD), and were without any record of neurological or psychiatric disorders as confirmed via structured clinical screening.

Normal aging

RS fMRI data of 95 old (age range: 55–70 years) and 93 young (age range: 20–35 years) participants with an age range of 15 years in each group were obtained from the population‐based 1000BRAINS study [Caspers et al., 2014] and another separate study at the Research Centre Jülich, Germany. This relative small age‐range aims to enhance the subsample homogeneity. “NA” in old participants refers to the absence of neurodegenerative diseases. Older adults showed cognitive performance adequate for their age (DemTect > 13) as assessed by the Mild Cognitive Impairment and Early Dementia Detection (DemTect) assessment [Kalbe et al., 2004] and all participants did not exhibit clinically relevant symptoms for depression (BDI‐II < 13) as evaluated via the Beck Depression Inventory‐II [Beck et al., 1996].

Importantly, target and control groups (i.e., patients vs. HCs, old vs. young adults) of all three samples (PD, SCZ, NA) represent subsamples from larger samples that were post‐hoc matched for gender, within‐scanner movement and (only for the clinical samples) age (cf. Table 1 for sample and group matching characteristics). Written informed consent from all subjects and approval by the local ethics committees was obtained from all sites. Joint reanalysis of the anonymized data was approved by the ethics committee of the Heinrich Heine University Düsseldorf.

Table 1.

Sample and group matching characteristics

Sample	n (males)	Age (years)	Head movement (DVARS)	Age at onset (years)	Illness duration (years)	Antipsychotic/dopaminergic medication	Neuropsychology and psychopathology
SCZ sample						OZP‐equivalent	PANSS: Total/PS/NS/GEN
COBRE
SCZ patients	55 (46)	38 ± 14	1.66 ± 0.55*	20 ± 8	17 ± 14	13 ± 8	58 ± 14/14 ± 5/14 ± 5/29 ± 8
HCscz	55 (42)	38 ± 12	1.44 ± 0.41
Göttingen
SCZ patients	31 (25)	32 ± 10	1.47 ± 0.30*	25 ± 8	7 ± 8	14 ± 9	52 ± 11/12 ± 3/13 ± 4/28 ± 6
HCscz	29 (22)	32 ± 9	1.31 ± 0.23
Total
SCZ patients	86 (71)	36 ± 13	1.59 ± 0.48*
HCscz	84 (64)	36 ± 11	1.39 ± 0.36
PD sample						LEDD	H & Y Scale	UPDRS‐III	MoCA
Aachen
PD patients	32 (21)	64 ± 9	0.51 ± 0.16	59 ± 8	6 ± 5	449 ± 238	2 ± 1	23 ± 12	27 ± 2
HCPD	33 (20)	63 ± 6	0.62 ± 0.29
Düsseldorf
PD patients	48 (30)	59 ± 9	0.69 ± 0.26	51 ± 9	8 ± 6	1029 ± 416	2.5 ± 1	16 ± 8	24 ± 4
HCPD	51 (30)	57 ± 9	0.68 ± 0.22
Total
PD patients	80 (51)	61 ± 9	0.62 ± 0.24
HCPD	84 (50)	59 ± 8	0.66 ± 0.25
NA sample							DemTect		BDI‐II
Jülich
Old	48 (26)	61 ± 5	1.58 ± 0.41*				16 ± 2		5 ± 5
Young	52 (26)	26 ± 3	1.24 ± 0.24						5 ± 4
1000BRAINS Jülich
Old	47 (25)	64 ± 4	1.79 ± 0.43*				15 ± 2		6 ± 5
Young	41 (23)	28 ± 4	1.28 ± 0.26						4 ± 4
Total
Old	95 (51)	63 ± 5	1.68 ± 0.43*
Young	93 (49)	27 ± 4	1.26 ± 0.25

SCZ, schizophrenia; HC_SCZ, matched healthy controls (HCs) of SCZ sample; PD, Parkinson's disease; HC_PD, matched HCs of PD sample; NA, normal aging; characteristic values in mean ± standard deviation; DVARS, derivative of root mean squared variance over voxels (head movement parameter) [Power et al., 2012]; significant difference in age (clinical samples), gender and movement are marked with * for P < 0.05; SCZ: OZP‐equivalent [Gardner et al., 2010], Olanzapine equivalent dose; PANSS, Positive and Negative Symptom Scale, (PS, Positive Symptoms Scale/NS, Negative Symptoms Scale/GEN, General Psychopathology Scale); PD: LEDD [Tomlinson et al., 2010], Levodopa equivalent daily dose; H & Y Scale, Hoehn and Yahr Scale; UPDRS‐III, Unified Parkinson's Disease Rating Scale Part III; MoCA, Montreal Cognitive Assessment; NA: DemTect, Mild Cognitive Impairment and Early Dementia Detection, BDI‐II, Beck Depression Inventory II.

RS fMRI Data Acquisition, Preprocessing, and Analysis

During image acquisition (see Supporting Information Table SI for fMRI parameters), participants were instructed to lie still, let their mind wander and not fall asleep (confirmed at debriefing). SPM8 (http://www.fil.ion.ucl.ac.uk/spm) was used for image realignment, spatial normalization to the MNI‐152 template using the unified segmentation approach [Ashburner and Friston, 2005], and smoothing “5‐mm full‐width at half‐maximum Gaussian kernel”.

We investigated 12 functional networks, robustly defined by previous quantitative meta‐analyses, to reflect neural correlates of a broad set of cognitive, social‐affective/motivational and motor functions (see Table 2 for an overview and Supporting Information Table SII for detailed network coordinates and corresponding brain regions). Only meta‐analytic networks with a minimum of 10 nodes were included, since a lower number of features are uninformative for robust classification. RS functional connectivity (RSFC) within each network was computed per subject by first extracting the time‐series for each node within 6 mm of the meta‐analytic peaks. To reduce spurious correlations, variance explained by the six movement parameters and their derivatives (modeled as first and second order effects) as well as the mean white‐matter and cerebrospinal fluid signal time‐courses was removed from the time series [Satterthwaite et al., 2013; Varikuti et al., 2016]. Subsequently, time series were high‐pass filtered retaining frequencies above 0.01 Hz. Connectivity was computed as the Fisher's Z‐transformed Pearson correlation between the time series of each network's nodes; connectivity values were adjusted for effects of acquisition site, gender, movement, total brain volume, and (only for the clinical samples) age [cf. Schilbach et al., 2016, 2014] to avoid classification based on spurious between‐subject effects.

Table 2.

Network overview

Network (Abbr.)	Network	Contrast	Nodes	Publications
EmoSF	emotional scene and face processing	emotional scene > neutral scene and emotional face>neutral face	24	Sabatinelli et al. [2011]
ER	cognitive emotion regulation	reappraise > naturalistic emotional responses	14	Buhle et al. [2014]
ToM	theory‐of‐mind cognition	ToM > non‐social baseline	15	Bzdok et al. [2012]
Empathy	empathic processing	“feel into” affect‐laden social situations > watched or listened passively	19	Bzdok et al. [2012]
Rew	reward‐related decision making	ME: reward valence and decision stages	25	Liu et al. [2011]
AM	autobiographical memory	autobiographical memory > non‐autobiographical baseline	22	Spreng et al. [2009]
SM	semantic memory	access to word meaning > processing word structure	23	Binder et al. [2009]
WM	working memory	ME: n‐back, sternberg, delayed matching to sample and delayed simple matching tasks	23	Rottschy et al. [2012]
CogAC	cognitive action control	ME: stroop‐task, spatial interference task, stop‐signal task and go/no‐go task	19	Cieslik et al. [2015]
VigAtt	vigilant attention	ME: detection task, discrimination task	16	Langner and Eickhoff [2013]
MNS	mirror neuron system	action observation ∩ action imitation	11	Caspers et al. [2010]
Motor	motor execution	finger tapping > baseline; excl. regions associated with visually paced finger‐tapping tasks	10	Witt et al. [2008]

ME, main effect.

SVM Features and Classification

To examine whether the RSFC pattern of a network contains predictive information on the respective groups (SCZ vs. HC_SCZ, PD vs. HC_PD, old vs. young) non‐sparse linear two‐class SVMs were computed using LibSVM [Chang and Lin, 2011] (https://www.csie.ntu.edu.tw/~cjlin/libsvm). SVMs' were trained separately for each of all three analyses (PD, SCZ, NA) and each of the functional networks. Of note, we did not attempt between‐patient classification (i.e., PD vs. SCZ), as the different groups were closely matched to their respective controls but substantially different from each other with respect to age, gender, and movement. The input variables (features) to the SVM consisted of edge‐wise RSFC between all nodes of a given network. Each SVM was trained and tested by a nested 10‐fold cross‐validation scheme for each individual group (see e.g., Fig. 1 [Xia et al., 2013]) [cf. Lemm et al., 2011]. The inner loop used a 10‐fold cross‐validation within the training group to optimize the soft‐margin slack parameter. For each fold of the outer loop, the left‐out (unseen) 10% were then classified using the SVM trained on the (entire) training‐set using the optimized parameter. This nested scheme ensured that classifier optimization and evaluation was performed independent of each other [Kriegeskorte et al., 2009]. Classification performance was evaluated based on accuracy (Acc.) balanced accuracy (bAcc.), sensitivity (Sens.), and specificity (Spec.) as well as two measures derived from signal‐detection theory: the area under the receiver operating characteristics (ROC) curve (AUC) [Fawcett, 2004] and d'. Acc. denotes the overall proportion of subjects correctly classified as patients (PD, SCZ) or advanced age versus healthy or younger age, respectively. The bAcc. is calculated as the average proportion of subjects correctly classified as patients (PD, SCZ) or advanced age versus healthy or younger age, respectively. Sens. indicates the percentage of patients (SCZ or PD) correctly classified as ill or subjects correctly classified as old in the aging sample (true positives). Spec. in turn represents the fraction of HCs correctly classified as healthy or subjects correctly identified as young in the aging sample (true negatives). AUC refers to the area under the ROC curve. An ROC curve depicts the relationship between true positive rate and false positive rate, and its AUC value indicates the sensitivity of the diagnostic process independent of any specific decision criterion. Finally, we assessed d', an alternative index of diagnostic sensitivity independent of the decision criterion, calculated as z(true positive rate) – z(false positive rate). To increase robustness, the entire procedure was repeated 25 times, and each performance measures was averaged across repetitions. To examine significant differences in classification performance between networks within each group, pairwise t‐tests were performed for each of the 12 networks based on the accuracies obtained from the 25 cross‐validation outer loop replications of the separate SVMs (significance threshold of P < 0.05, Bonferroni‐corrected for the number of pairwise network comparisons).

Figure 1.

Linear two‐class SVM nested 10‐fold cross‐validation scheme. Illustration of a SVM example for classification of the SCZ sample based on the EmoSF network. As input variables (DATA) (= features) served the subjects' RSFCs of all edges of every network. The inner loop was performed in a 10‐fold manner with 10 repetitions conducted as parameter setting optimization on a training sample. The outer loop was performed in a 10‐fold manner with 25 repetitions conducted as classification accuracy testing on an unseen test set. Classification performance measures are computed based on the confusion matrix. Acc., accuracy; Sens., sensitivity; Spec., specificity; AUC, area under the ROC curve and d' (see “Materials and Methods” section for explanation). [Color figure can be viewed at http://wileyonlinelibrary.com]

To compare the separately conducted classifications for SCZ versus HC_SCZ and PD versus HC_PD subgroups, accuracies obtained for each individual analysis for every network were converted to standardized z‐scores by reference to the binomial distribution reflecting chance level and corrected for multiple comparisons by the amount of networks‐based classifications. Log‐likelihood ratios were estimated to identify networks showing better classification performance for one patient group than the other. To investigate significant differences in classification performance between the groups, t‐tests were calculated based on the 25 accuracies obtained from the cross‐validation outer loop replications of the separate SVMs performed in each group (SCZ, PD, NA) for each of the 12 networks (significance threshold of P < 0.05, Bonferroni‐corrected for the number of groups and networks).

RESULTS

As expected, SCZ patients could be distinguished above chance from matched HCs based on RSFC in the Rew network (Acc. = 68%; AUC =0.73). In turn, PD patients were distinguished above chance from their matched HCs based on RSFC in the motor network (Motor; Acc. = 70%; AUC = 0.77). Finally, old and young subjects were differentiated very well from each other based on RSFC in the WM network (Acc. = 79%; AUC = 0.84). Results are summarized as follows: Figure 2A for polar plot of group classification accuracies, Table 3 for Acc., Sens., Spec. and AUC, Supporting Information Table SIII for bAcc., Supporting Information Table SIV for d', Supporting Information Figure S1 for z‐standardized accuracies of all groups and Supporting Information Figure S2 for variance of accuracies.

Figure 2.

Group classification results of the SVM. (A) Polar plot of group classification accuracies based on all 12 networks for SCZ (in green), PD (in blue) and NA (in yellow). Accuracy refers to the proportion of subjects correctly classified as patients (PD, SCZ) or older age and subjects correctly classified as being HCs or younger age. (B) Polar plot of z‐standardized accuracies (corrected for multiple comparisons) of patients classification for SCZ (in green) and PD (in blue). (C) Log‐likelihood ratios of classification performance for networks showing higher classification for one patient group vs. the other. [Color figure can be viewed at http://wileyonlinelibrary.com]

Table 3.

Classification results of the SVM of all groups based on specific networks

Network (Abbr.)	SCZ vs. HCSCZ Acc. (Sens./Spec.) AUC	PD vs. HCPD Acc. (Sens./Spec.) AUC	Old vs. Young Acc. (Sens./Spec.) AUC
EmoSF	72% (77%/68%) 0.79	63% (64%/63%) 0.68	88% (89%/86%) 0.93
ER	71% (77%/65%) 0.76	69% (74%/64%) 0.74	78% (79%/76%) 0.86
ToM	61% (74%/46%) 0.62a	67% (70%/64%) 0.71	78% (77%/80%) 0.84
Empathy	71% (73%/69%) 0.78	63% (61%/65%) 0.69	78% (80%/75%) 0.83
Rew	68% (73%/62%) 0.73	66% (70%/63%) 0.71	87% (85%/88%) 0.93
AM	62% (67%/57%) 0.71	75% (78%/73%) 0.76	80% (80%/80%) 0.89
SM	61% (67%/54%) 0.68a	69% (65%/73%) 0.75	84% (85%/83%) 0.90
WM	62% (65%/60%) 0.66	65% (68%/63%) 0.71	79% (80%/77%) 0.84
CogAC	68% (73%/63%) 0.69	62% (66%/57%) 0.67	73% (73%/74%) 0.83
VigAtt	68% (72%/63%) 0.72	65% (68%/63%) 0.67	80% (78%/83%) 0.89
MNS	64% (65%/63%) 0.73	57% (64%/51%) 0.53a	84% (83%/84%) 0.91
Motor	61% (72%/50%) 0.61	70% (68%/73%) 0.77	80% (79%/81%) 0.90

Abbreviations: Acc., Accuracy (in %)/Sens., sensitivity (in %)/Spec., specificity (in %)/AUC, area under the ROC curve.

^aNetwork with no significant classification result.

Acc. refers to the proportion of subjects correctly classified as patients (PD, SCZ) or older age and subjects correctly classified as being healthy or younger age (mean of sensitivity and specificity). Sensitivity relates to the percentage of patients (SCZ or PD) correctly classified as being ill or else subjects correctly identified as old in the aging sample (true positives). Specificity relates to the percentage of healthy subjects correctly classified as being healthy or else subjects correctly identified as young in the aging sample (true negatives). AUC refers to the area under the ROCs curve. The ROC curve depicts the relationship between true positive rate and false positive rate.

Considering the performance of all functional networks in distinguishing SCZ and PD patients from their respective HCs, a clear differentiation between networks becomes evident, even though only 2 (SCZ) and 1 (PD) out of 12 networks, respectively, did not significantly exceed chance accuracy (Fig. 2B). The following results and discussion are focused on networks with superior classification performance for the respective disorders. In this context, we would like to re‐iterate that we did not attempt to train any classifier to distinguish SCZ from PD patients, since the two samples differed substantially from each other in various confounding factors such as age, gender distribution, and within‐scanner movement.

For SCZ, the emotional scene and face processing (EmoSF) network (Acc. = 72%; AUC = 0.79) as well as the networks for empathic processing (Empathy; Acc. = 71%; AUC = 0.78) and cognitive action control (CogAC; Acc. = 68%; AUC = 0.69) distinguished patients most accurately from their HCs. Hence these networks' connectivity patterns may be considered to contain the highest level of information with respect to SCZ. The EmoSF network was significantly better in the SCZ classification compared with all other networks (P < 0.001). For PD, the networks subserving autobiographical memory (AM; Acc. = 75%; AUC = 0.76), motor execution (Motor; Acc. = 70%; AUC = 0.77), semantic memory (SM; Acc. = 69%; AUC = 0.75), and theory‐of‐mind cognition (ToM; Acc. = 67%; AUC = 0.71) yielded the highest classification accuracies, that is, contained the most informative PD‐related differences in RSFC. The AM network was significantly better in the PD classification compared with all other networks (P < 0.001). All network comparison results within the patient groups are summarized in Supporting Information Tables SV and SVI.

The between‐network comparison of classification performance with respect to SCZ and PD revealed that the networks discriminating either disorder from their respective controls were highly specific (Fig. 2B,C), indicating that these networks carry differential amounts of information regarding SCZ and PD, respectively. In particular, both EmoSF and Empathy networks showed the best performance at distinguishing SCZ patients from HCs (EmoSF: z = 5.9; Empathy: z = 5.5) but were notably worse at discriminating PD patients from their HCs (EmoSF: z = 3.2; Empathy: z = 3.2). Similarly, the CogAC network exhibited high accuracy at classifying SCZ patients and their respective HCs (z = 4.7) but inferior performance at distinguishing PD patients from their HCs (z = 2.7).

In turn, the motor network very well classified PD patients and their HCs (z = 5) but was remarkably ineffective at classifying SCZ patients and their HCs (z = 2.9). Likewise, the AM and SM networks achieved high accuracies in classifying PD patients and controls (AM: z = 6.3; SM: z = 4.5) but performed much less well when classifying SCZ patients and controls (AM: z = 3.2; SM: z = 2.5). Networks which were most accurate in distinguishing SCZ from HCs (EmoSF, Empathy, and CogAC) exhibited significant better classification performance in the SCZ group compared to the PD group (EmoSF: P < 0.001; Empathy: P < 0.001; CogAC: P < 0.001; Supporting Information Table SVII). Likewise, networks which performed best at discriminating PD patients from HCs (AM, Motor, SM, and ToM) showed significant better classification performance in the PD group compared with the SCZ group (AM: P < 0.001; Motor: P < 0.001; SM: P < 0.001; ToM: P < 0.001; Supporting Information Table SVII).

This differential picture markedly contrasted with the results obtained for the classification between old and young subjects. In the aging sample, each network yielded accuracies ≥ 73% (see Supporting Information Table SVIII for network comparison results within NA), significantly outperforming every classification obtained in the SCZ or PD samples (P < 0.001; see Fig. 2A, Supporting Information Figure S1, Table 3, Supporting Information Tables SIX and SX).

In particular, for each network the accuracy for classifying a previously unseen participant as young or old was about 10% higher than any clinical classification based on the same network. Additionally, the comparison of all three separate group classifications revealed that the variance of the classification accuracies over the 25 replications of the outer loop was distinctively lower for the classification of age, as compared with classifying the clinical status (Supporting Information Fig. S2).

DISCUSSION

We assessed whether RSFC patterns in a diverse set of functionally defined brain networks allowed for a classification of patients with SCZ or PD or healthy older adults on the one hand, and their respective healthy or young controls on the other. Thereby, we evaluated which functional system was most informative for a given condition (i.e., SCZ, PD, or higher age). Conversely, our analysis also assessed the amount of information on each condition found in a given network. Our results show in a proof‐of‐principle manner that networks pertaining to functions known to be affected by SCZ, PD, or aging indeed exhibited good classification performance for the respective condition. Furthermore, each network's young–old classification outperformed any disease‐related classification. This indicates that specific networks are affected by and associated with the diseases, whereas for healthy older adults RSFC appears to be altered rather globally.

Conceptual Considerations

Our study demonstrates that machine‐learning techniques can be successfully used to assess whether RSFC in functional systems known to be affected in SCZ, PD, or advanced age exhibits high classification capacity for the respective condition. Further, our approach compared the classification capacity of RSFC patterns between different functional networks and between several clinical and physiological states. Of note, for each classification, target and control groups (i.e., SCZ vs. HC_scz, PD vs. HC_PD, old vs. young) were well matched with respect to gender and (for the clinical samples) age. In addition, RSFC variance attributable to these confounding factors or within‐scanner movement was regressed out of the data before the SVM analyses. Therefore, these confounds were evidently heterogeneous across the three groups (SCZ, PD, NA) but should not have influenced classification accuracy within each condition. In spite of proper matching and state‐of‐the‐art removal of variance related to motion [cf. Power et al., 2012; Satterthwaite et al., 2013], residual effects that only manifest in the multivariate pattern cannot be fully ruled out. However, one factor worth noting is that, for example, we observed differential classification performance across networks in the SCZ sample, largely ruling out a dominant general effect of head motion.

Given that both groups were assessed under their regular medication, differences in classification performance may be influenced by pharmacological treatment. In particular, we cannot exclude that classification results of networks modulated via dopaminergic transmission (e.g., reward or motor system) might originate from interactions between disease condition and medication. Unfortunately, however, we could not perform a more detailed assessment of the influence of medication, as the compounds, duration of treatment and doses varied considerably between subjects, with many receiving a combination of drugs.

When comparing classification performance to previous work based on whole‐brain functional connectomes [cf. Chen et al., 2015; Long et al., 2012; Meier et al., 2012; Su et al., 2013; Tang et al., 2012; Vergun et al., 2013; Yu et al., 2013], we note that our approach yielded higher functional specificity, allowing inference on the amount of disease‐specific information in well‐defined functional systems. We acknowledge that even though most of the classifications well exceeded chance level, the achieved network‐based classification accuracies are not strong enough for successful connectivity‐based single‐subject diagnosis. Still, our “sparse” approach achieved classification accuracies comparable to those reported in previous whole‐brain studies, whose feature space obviously was substantially larger than ours. This is particularly noteworthy given that two further aspects besides feature space could be expected to decrease classifier performance in our study [Arbabshirani et al., 2016; Haller et al., 2014; Kambeitz et al., 2015; Schnack and Kahn, 2016; Varoquaux et al., 2016]: First, all of our three groups were based on relatively large samples that were combined from two different measurement sites and hence should be more heterogeneous than usual. Second, we used replicated 10‐fold cross‐validation, rather than the more optimistic leave‐one‐out approach [Varoquaux et al., 2016]. We thus argue that the chosen combination of examining robustly defined functional networks and optimized analysis through replicated and nested 10‐fold cross‐validation may provide valuable new insights into the pathophysiology of brain disorders that is not attainable through global analyses of the entire functional connectome.

Classification of SCZ Patients and Controls

We found that the networks subserving EmoSF, empathic processing as well as CogAC yielded the best performance. Aberrant processing of emotional stimuli [Takahashi et al., 2004] and impaired abilities to relate to others' affective states [Benedetti et al., 2009; Derntl et al., 2012; Harvey et al., 2012] are features of SCZ and mirrored in the degree of SCZ‐related information that is contained in the EmoSF (AUC = 0.79) and Empathy (AUC = 0.78) networks. Further, the good classification performance of the CogAC network resonates well with alterations in cognitive control processes as a core deficit in SCZ [cf. Lesh et al., 2011].

Somewhat surprisingly, the Rew network did not differentiate SCZ from HCs with high accuracy, given the prominent role of the dopaminergic system [Toda and Abi‐Dargham, 2007] and aberrant salience processing in psychosis [Heinz and Schlagenhauf, 2010; Radua et al., 2015] and the association with the reward system in this disorder. We conjecture that this lack of predictive information could arise from the fact that in contrast to task‐activation data, RSFC analyses primarily capture the tonic rather than phasic state of these networks [Schultz et al., 1997].

Classification of PD Patients and Controls

The superior classification performance observed for the motor execution network (AUC = 0.77) is hardly surprising, since motor impairments represent a key clinical feature of PD, and differences in action‐related brain circuitry are well established in this disorder [Herz et al., 2014; Rowe and Siebner, 2012; Tessitore et al., 2014]. The finding that the AM (AUC = 0.76) and SM (AUC = 0.75) networks also achieved a very good differentiation of PD patients from HCs was rather surprising, though. Although PD is a neurodegenerative disorder and dementia is common in PD patients [Aarsland et al., 2001, 2003], several patients showed evidence for mild cognitive impairment, using the MoCA for screening. We can hence only speculate that the RSFC differences in AM and SM networks may pick up these deficits as revealed by standard behavioral screening instruments.

Finally, the good classification performance achieved by the ToM network (AUC = 0.71) was unexpected but matches a growing literature of impaired social cognition in PD patients [Bora et al., 2015; Poletti et al., 2011; Díez‐Cirarda et al., 2015].

Age Group Classification

One of the most striking observations from this study was that every single network achieved a better classification with respect to age group than with respect to SCZ or PD. While we hypothesized that the broad spectrum of age‐related changes in various mental functions [Craik and Salthouse, 2011; Glisky, 2007; Seidler et al., 2010] would be reflected by changes in several networks [Craik and Salthouse, 2011; Hedden, 2007; Mather, 2016; Seidler et al., 2010], the consistency (across both networks and replications) of high classification accuracies is intriguing. It stands to reason that the mechanisms underlying the discriminative changes in functional connectivity patterns may be diverse. In particular, they should include neurodegeneration (cognitive networks [Hedden, 2007]), neurochemical changes (Rew networks [Bäckman et al., 2006]), altered affective processing (social‐affective networks [Mather, 2016]) and use‐dependent plasticity (motor networks [Demirakca et al., 2016]). In addition, it may be argued that in spite of all inter‐individual variability age‐related changes represent a more homogeneous change of the neuro‐functional architecture [Ferreira et al., 2016; Meier et al., 2012] relative to the inevitable heterogeneity among clinical populations.

Given that connectivity patterns of all systems differentiated very well between young and old participants, we acknowledge the possibility that the relevant drivers may be of non‐neural origin. In particular, despite of our optimized confound removal [Power et al., 2012; Satterthwaite et al., 2013; Varikuti et al., 2016], we cannot exclude that residual effects related to motion or brain atrophy as well as physiological effects such as macro‐ and microvascular changes and their cumulative impact on hemodynamic signals [D'Esposito et al., 2003] may have contributed to our findings.

Although the contributions of neural and non‐neural effects outlined in this section certainly warrant further investigation, one of the most critical conclusions that should be taken from the high classification accuracy between younger and older participants is the danger of obtaining spuriously high accuracies in clinical classification studies if patients and HCs are not carefully matched for age.

Conclusions and Outlook

We investigated the potential of RS connectivity patterns in a wide variety of functional networks to distinguish SCZ and PD patients from matched HCs as well as old from young adults. We showed that networks defined by robust activation due to mental operations known to be affected in the respective condition indeed contained information on the respective condition that is captured by our pattern‐classification approach and translates into good classification accuracies. Classification accuracies obtained through replicated, nested 10‐fold cross‐validation were not only generally comparable to those obtained from whole‐brain analyses but also revealed a differentiated picture for both disorders in comparisons. Both SCZ and PD were specifically well predicted by distinct networks that resonate well with known clinical and pathophysiological features. The presented approach thus opens an avenue toward robust and more specific assessments of clinical and developmental differences in functional systems than previous whole‐brain analyses. One of the most striking findings of this work was the fact that integrity in all networks was much better at identifying participants with advanced age than with any of the two disorders. While the most likely heterogeneous mechanisms behind this phenomenon certainly need to be addressed in more detail, the current findings highlight the importance of considering age‐related effects as a potential source of bias in clinical classification studies.

FINANCIAL DISCLOSURES

These authors declare following conflicts of interest: K.R. was partly funded by the German Federal Ministry of Education and Research [BMBF 01GQ1402] and the Alzheimer Forschung Initiative e.V. [AFI 13812] and the position of J.H. was subsidized by the START‐Program of the Faculty of Medicine at the RWTH Aachen University, Germany [23/12 to K.R.]. K.R. has received honoraria for presentations from Lilly and research grants from Pfizer and Merck.

Supporting information

Supporting Information.