Dynamic functional connectome predicts individual working memory performance across diagnostic categories

Highlights

• •We created transdiagnostic predictive working memory models using connectome-based predictive modeling (CPM).
• •Dynamic functional connectivity-based CPM models successfully predicted working memory.
• •Static functional connectivity-based CPM models fell short in prediction.
• •Frontoparietal, somato-motor, default mode and visual networks contributed most to prediction.

Abstract

Working memory impairment is a common feature of psychiatric disorders. Although its neural mechanisms have been extensively examined in healthy subjects or individuals with a certain clinical condition, studies investigating neural predictors of working memory in a transdiagnostic sample are scarce. The objective of this study was to create a transdiagnostic predictive working memory model from whole-brain functional connectivity using connectome-based predictive modeling (CPM), a recently developed machine learning approach. Resting-state functional MRI data from 242 subjects across 4 diagnostic categories (healthy controls and individuals with schizophrenia, bipolar disorder, and attention deficit/hyperactivity) were used to construct dynamic and static functional connectomes. Spatial working memory was assessed by the spatial capacity task. CPM was conducted to predict individual working memory from dynamic and static functional connectivity patterns. Results showed that dynamic connectivity-based CPM models successfully predicted overall working memory capacity and accuracy as well as mean reaction time, yet their static counterparts fell short in the prediction. At the neural level, we found that dynamic connectivity of the frontoparietal and somato-motor networks were negatively correlated with working memory capacity and accuracy, and those of the default mode and visual networks were positively associated with mean reaction time. Moreover, different feature selection thresholds, parcellation strategies and model validation methods as well as diagnostic categories did not significantly influence the prediction results. Our findings not only are coherent with prior reports that dynamic functional connectivity encodes more behavioral information than static connectivity, but also help advance the translation of cognitive “connectome fingerprinting” into real-world application.

1. Introduction

Working memory refers to the ability to maintain and manipulate information over a brief period of time (several seconds to minutes) during the execution of ongoing tasks and activities (Chai et al., 2018, Constantinidis and Klingberg, 2016). In recent years, working memory has motivated research in neuroscientific, cognitive and clinical domains since it has been considered a core component of higher-order cognitive functions (D'Esposito and Postle, 2015, Eriksson et al., 2015). A fundamental property of working memory is its limited capacity, that is, restricted amount of information can be held active simultaneously (Baddeley, 2003, Luck and Vogel, 2013). There are substantial differences between individuals in working memory capacity. These inter-individual differences are highly stable over time and have strong associations with individual variability in a range of cognitive functions, such as learning (Unsworth and Engle, 2005), comprehension (Engle et al., 1992), mathematics (Tuholski et al., 2001), attentional control (Bleckley et al., 2003, Kane et al., 2007), fluid intelligence (Conway et al., 2003, Unsworth et al., 2014), and overall academic performance (Gathercole et al., 2003). Using neuroscience methodologies including electrophysiological, neuroimaging and lesion-deficit approaches, investigators have examined the neural bases that underlie working memory processes in non-human primates and humans (Constantinidis and Klingberg, 2016, Eriksson et al., 2015). It is now generally accepted that working memory involves complex and dynamic interactions among a large number of brain regions mainly including the prefrontal and parietal cortex, sensory cortex (visual and auditory areas), motor system, basal ganglia, and cerebellum (Brissenden and Somers, 2019, Chai et al., 2018, Christophel et al., 2017, Constantinidis and Klingberg, 2016, D'Esposito and Postle, 2015, Lara and Wallis, 2015, Marvel et al., 2019, Rottschy et al., 2012).

Significant overlap in genetic risk factors (Pettersson et al., 2016), clinical symptoms (Russo et al., 2014, Tamminga et al., 2013), cognitive dysfunction (Millan et al., 2012) across psychiatric disorders, and high comorbidity rates (Kessler et al., 2011) raise the possibility that current categorical classifications might not be carving nature by its joints. In response, transdiagnostic initiatives, such as the Hierarchical Taxonomy of Psychopathology (Kotov et al., 2018, Kotov et al., 2017) and the Research Domain Criteria (Cuthbert, 2014, Insel et al., 2010), have worked towards developing new dimensionally oriented approaches by integrating findings from genetics, cognition, and neuroimaging. There are many precedents showing impaired working memory in schizophrenia (SZ) (Gilmour et al., 2019, Grot et al., 2017), bipolar disorder (BD) (Soraggi-Frez et al., 2017), and attention deficit/hyperactivity disorder (ADHD) (Maehler and Schuchardt, 2016), supporting the idea that working memory impairment may be a transdiagnostic signature of psychiatric disorders. In addition, prior studies have revealed substantial overlap in neural circuits with connectivity alterations in different mental disorders, suggesting common neurobiological mechanisms (Baker et al., 2019, Baker et al., 2014, Reinen et al., 2018, Sha et al., 2019, Sha et al., 2018). Accordingly, a central objective of the present study was to examine the association between working memory and brain connectivity in a transdiagnostic sample with a normative background, which may offer a workable route towards translation of findings into real-world application.

Resting-state functional magnetic resonance imaging (fMRI) has emerged as a non-invasive brain imaging technique enabling researchers to measure functional connectivity, i.e., the temporal coherence of the blood-oxygen-level-dependent (BOLD) signal between distinct brain regions (Biswal et al., 1995, Fox and Raichle, 2007). Conventionally, resting-state functional connectivity is thought to be temporally static and evaluate the average functional organization, with the assumption that the interaction between brain regions is fixed throughout a whole resting-state fMRI scan period. However, this assumption may underestimate the dynamic repertoire of brain functions (Calhoun et al., 2014, Chang and Glover, 2010, Hutchison et al., 2013, Preti et al., 2017), whereby the resting brain navigates through a range of putative different functional connectivity states at much faster timescales (Abrol et al., 2017, Liegeois et al., 2017). Recently, an extensive literature has been published to exploit the rich temporal information contained in dynamic functional connectivity (Calhoun et al., 2014, Chang and Glover, 2010, Chen et al., 2016, Hutchison et al., 2013, Liu et al., 2017a, Liu et al., 2017b, Preti et al., 2017, Sakoglu et al., 2010, Sun et al., 2018). Indeed, recent research demonstrates that dynamic connectivity captures task-based phenotypes, whereas self-reported phenotypes are equally well explained by static and dynamic connectivity, indicating that dynamic functional connectivity encodes more behavioral information (Liegeois et al., 2019). Here, we also sought to compare performances of dynamic and static functional connectivity in predicting working memory profile.

To realize these goals, we constructed whole-brain dynamic and static functional connectomes based on resting-state fMRI data from a large transdiagnostic sample of healthy controls and individuals with SZ, BD, and ADHD. Then, we built predictive working memory models by leveraging a connectome-based predictive modeling (CPM) approach, which is a recently developed machine learning method for creating brain-behavior predictive models and identifying functional networks underlying specific behaviors (Shen et al., 2017). Based on prior evidence for more behavioral information contained in dynamic functional connectivity, it was predicted that dynamic connectivity would outperform its static counterpart in predicting individual working memory performance in a transdiagnostic fashion.

2. Materials and methods

2.1. Participants

The participants were from a publicly-available dataset, the Consortium for Neuropsychiatric Phenomics (CNP) (Poldrack et al., 2016). The CNP dataset is available via the OpenfMRI project (http://openfmri.org) (Poldrack et al., 2013) and includes neuroimaging data, neuropsychologic assessments, neurocognitive tasks, demographic and clinical information from a large cohort of right-handed adults aged 21–50 years: healthy controls (n = 130) and individuals with SZ (n = 50), BD (n = 49), and ADHD (n = 43). Diagnoses were based on the Structured Clinical Interview for DSM-IV (SCID) supplemented by the Adult ADHD Interview. Full details on these participants have been described in the data descriptor publication (Poldrack et al., 2016). Out of all subjects, 4 had incomplete or outlier neurocognitive data used in this study, 12 had missing MRI data, 13 had excessive head motion defined a priori as translational or rotational motion parameters > 3 mm or 3° during fMRI scanning, and 1 had errors in the spatial normalization step during resting-state fMRI preprocessing. These subjects were excluded from subsequent analyses. The participant numbers and demographic information of the final sample are presented in Table 1.

Table 1.

Demographic characteristics of the sample.

Characteristics	HC	SZ	BD	ADHD
Number of subjects	115	43	44	40
Gender (female/male)	53/62	11/32	19/25	19/21
Age (years)	31.1 ± 8.6	36.3 ± 9.0	35.0 ± 9.1	32.1 ± 10.4
Education (years)	15.1 ± 1.7	12.6 ± 1.8	14.7 ± 2.0	14.7 ± 1.8
Overall working memory capacity	4.08 ± 0.83	3.38 ± 1.01	3.72 ± 1.07	3.71 ± 1.08
Overall accuracy	0.89 ± 0.06	0.82 ± 0.09	0.86 ± 0.08	0.86 ± 0.09
Mean reaction time at load 1 (ms)	821.0 ± 222.2	1001.2 ± 262.7	915.1 ± 240.7	891.1 ± 248.6
Mean reaction time at load 3 (ms)	978.7 ± 263.3	1251.1 ± 570.1	1144.4 ± 429.1	1014.2 ± 263.7
Mean reaction time at load 5 (ms)	1062.9 ± 271.0	1310.9 ± 439.6	1190.2 ± 348.9	1049.0 ± 238.0
Mean reaction time at load 7 (ms)	1114.5 ± 290.3	1302.2 ± 368.9	1184.8 ± 280.2	1140.7 ± 267.3
FD (mm)	0.17 ± 0.09	0.24 ± 0.13	0.18 ± 0.09	0.18 ± 0.12

The data are presented as the mean ± standard deviation. Abbreviations: HC, healthy controls; SZ, schizophrenia; BD, bipolar disorder; ADHD, attention deficit/hyperactivity; FD, frame-wise displacement.

2.2. Spatial working memory task

During the spatial capacity task (SCAP) (Cannon et al., 2005, Glahn et al., 2003), participants were shown a target array of 1, 3, 5 or 7 yellow circles positioned pseudorandomly around a central fixation. After a fixed delay, subjects were shown a single green circle and were required to indicate whether or not that circle was in the same position as one of the target circles. A relatively long stimulus presentation of two seconds was used to allow subjects to fully encode the target array, minimizing a potential encoding bias on the basis of set size interaction. Likewise, decision requirements were kept constant across set sizes to reduce possible effects of set size on response processes. Trial events included a 1-sec fixation to orient attention, 2-sec target-array presentation, a 3-sec delay period, and a 3-sec fixed response interval during which the subject responded via keyboard presses. A central fixation was visible throughout each of the 48 trials (12 per memory set size). Before starting the scored trials, subjects underwent a supervised instruction and training period (4 trials). Half of the trials were true positive, and half were true negative. The schematic representation of the SCAP design is shown in Fig. 1A.

Fig. 1.

Inter-group differences in primary working memory variables. (A) Schematic representation of the spatial capacity task design. A combination of violin and box plots shows the distribution and group differences of working memory capacity (B), accuracy (C), and mean reaction time (D). * P < 0.05. Abbreviations: HC, healthy controls; SZ, schizophrenia; BD, bipolar disorder; ADHD, attention deficit/hyperactivity.

The primary variables of interest are overall working memory capacity, overall accuracy, and mean reaction time of correct responses at each load (Table 1). Briefly, working memory capacity at each load is calculated according to the formula by Cowan: k = n*(H + CR-1), where k = capacity, n = load, H = hits (out of those responded to), CR = correct rejections (out of those responded to). Overall working memory capacity is the maximum of capacities at 4 loads. Overall accuracy is the average percent correct across 4 loads. Considering the sample heterogeneity, we excluded outliers with values greater than mean + 6 × standard deviation (SD) or smaller than mean − 6 × SD. Group differences in these SCAP variables among the four groups were assessed using a general linear model with age, gender, and education as nuisance covariates.

2.3. Imaging acquisition

MRI data were collected using the same scanning protocol on one of two 3.0-Tesla Siemens Trio scanners, located at the Ahmanson-Lovelace Brain Mapping Center and the Staglin Center for Cognitive Neuroscience at UCLA. High-resolution structural MPRAGE images were acquired with the following parameters: repetition time (TR) = 1900 ms; echo time (TE) = 2.26 ms; field of view (FOV) = 250 mm × 250 mm; matrix = 256 × 256; slice thickness = 1 mm, no gap; and 176 sagittal slices. Resting-state BOLD fMRI data were collected using a T2*-weighted echo planar imaging (EPI) sequence with the following parameters: TR = 2000 ms; TE = 30 ms; flip angle = 90°; FOV = 192 mm × 192 mm; matrix = 64 × 64; slice thickness = 4 mm; 34 axial slices; and 152 time points. Despite the same scanning protocol on two same MR scanners, we included scanner as a covariate of no interest in our analysis to rule out its potential effect.

2.4. fMRI data preprocessing

Resting-state BOLD data were preprocessed using Statistical Parametric Mapping software (SPM12, http://www.fil.ion.ucl.ac.uk/spm) and Data Processing & Analysis for Brain Imaging (DPABI, http://rfmri.org/dpabi) (Yan et al., 2016). The first 5 volumes for each participant were discarded to allow the signal to reach equilibrium and the participants to adapt to the scanning noise. The remaining volumes were corrected for the acquisition time delay between slices. Then, realignment was performed to correct the motion between time points. Head motion parameters were computed by estimating the translation in each direction and the angular rotation on each axis for each volume. All BOLD data of the final sample were within the defined motion thresholds (i.e., translational or rotational motion parameters less than 3 mm or 3°). We also calculated frame-wise displacement (FD), which indexes the volume-to-volume changes in head position (Table 1). Several nuisance covariates (the linear drift, the estimated motion parameters based on the Friston-24 model, the spike volumes with FD > 0.5, the white matter signal, and the cerebrospinal fluid signal) were regressed out from the data. The datasets were then band-pass filtered using a frequency range of 0.01 to 0.1 Hz. In the normalization step, individual structural images were firstly co-registered with the mean functional image; then the transformed structural images were segmented and normalized to the Montreal Neurological Institute (MNI) space using a high-level nonlinear warping algorithm, that is, the diffeomorphic anatomical registration through the exponentiated Lie algebra (DARTEL) technique (Ashburner, 2007). Finally, each filtered functional volume was spatially normalized to MNI space using the deformation parameters estimated during the above step and resampled into a 3-mm cubic voxel.

2.5. Functional connectivity calculation and functional connectome construction

Whole-brain functional connectivity analyses were performed using GRETNA software (http://www.nitrc.org/projects/gretna) (Wang et al., 2015). Network nodes were defined using the Shen brain atlas, which consists of 268 nodes and provides whole-brain coverage of the cortex, subcortex, and cerebellum (Shen et al., 2013). For each of the 268 nodes, the representative mean time course was obtained by averaging BOLD time courses over all voxels within the node. Then, we computed the node-by-node pairwise Pearson’s correlation coefficients and transformed them using Fisher’s z-transformation, resulting in a symmetric 268 × 268 correlation matrix for each subject. In the matrix, each element represents the strength of static functional connectivity between two individual nodes. To characterize functional connectivity temporal dynamics, sliding time-window analysis was conducted. Specifically, Hamming windows (window size = 50 TR = 100 s, which satisfies the 1/f₀ wavelength criterion for a minimum cutoff frequency of 0.01 Hz (Leonardi and Van De Ville, 2015, Pedersen et al., 2018a, Pedersen et al., 2018b, Zalesky and Breakspear, 2015); window step = 1 TR = 2 s) were applied to each participant’s preprocessed fMRI data to obtain a series of BOLD signal windows (98 time windows for the current study). The above-described whole-brain functional connectivity analysis was performed for each window, resulting in a total of 98 correlation matrices for each subject. Here, the standard deviations of the sliding-windowed correlation time series were used as a proxy of dynamic functional connectivity, where higher standard deviation indicates greater signal dispersion from average sliding-windowed correlation time series. For each subject, a 268 × 268 standard deviation matrix was created and each element in this matrix represents the strength of dynamic functional connectivity between two individual nodes (Bosma et al., 2018a, Bosma et al., 2018b, Choe et al., 2017).

2.6. Connectome-based predictive modeling

CPM is a recently developed approach for identifying brain networks associated with a behavioral variable of interest from whole-brain functional connectivity, which can be then used to predict novel participants’ behavior at the single-subject level (Shen et al., 2017). Here, CPM was conducted using previously validated custom MATLAB scripts that are freely available online (https://www.nitrc.org/projects/bioimagesuite/). Overall, CPM took edge weights (i.e., whole-brain dynamic and static functional connectivity matrices) and behavioral data (i.e., the primary variables of SCAP) as inputs. First, the input data were divided into a training set and a testing set. In the training set, the behavioral data were correlated with each edge in the connectivity matrices using partial correlation analyses (adjusting for age, gender, education, scanner, and mean FD) with a statistical significance threshold of P < 0.01 to identify positive and negative predictive networks. For positive networks, edges were significantly positively associated with the behavioral data; for negative networks, edges were significantly negatively associated with the behavioral data. Two networks were considered separately because they may provide different yet complementary information on prediction of behaviors. Next, a single-subject summary value was created by summing the significant edge weights in each network. Then, we built a predictive model that assumes a linear relationship between the single-subject summary value of connectivity data (independent variable) and the behavioral data (dependent variable). In the testing set, the summary value was calculated for each subject and was then input into the predictive model. The resulting value was the predicted behavioral variable for the current test subject. Here, we employed a leave-one-out cross-validation analysis (i.e., internal validation) to test the prediction performance (Liu et al., 2015). Briefly, one subject (i.e., the testing set) was left out and all other subjects (i.e., the training set) were used to build the predictive model; the left-out subject’s predicted behavioral variable was generated by the predictive model; this step was repeated in an iterative manner until all subjects had a predicted behavioral variable. Model performance was evaluated by the magnitude and statistical significance of the Pearson’s correlation between actual and predicted behavioral values. Permutation testing was performed to assess the statistical significance of the correlation. To generate an empirical null distribution of the test statistic (i.e., prediction correlation values), we randomly shuffled the correspondence between connectivity matrices and behavioral variables 1000 times and reran the CPM pipeline using the shuffled data. On the basis of the null distribution, the P value for the leave-one-out prediction was calculated as the proportion of sampled permutations that were greater than or equal to the true prediction correlation, i.e., P value = the number of permutations that generated correlation values greater than or equal to the true correlation values/1000. Bonferroni correction was applied to adjust significance levels for multiple testing, i.e., P < 0.05/12 = 0.0042 corresponding to 2 brain networks (positive and negative networks) and 6 behavioral variables.

2.7. Validation analyses

The following procedures were performed to further test the reproducibility of our findings. First, a significance threshold of P < 0.01 was used to identify edges that were positively and negatively associated with SCAP scores. To determine whether our results depended on the choice of different thresholds, we repeated the CPM analyses using two other thresholds (i.e., P < 0.05 and 0.005). Second, given the strong influence of different parcellation strategies on brain network analysis, we constructed functional connectome using another parcellation scheme based on automated anatomical labeling (AAL) atlas with 116 nodes and then repeated the entire prediction procedure. Third, two other cross-validation methods, i.e., 10-fold and leave-one-group-out, were utilized to further assess the CPM prediction performance. In the 10-fold cross-validation, the sample was randomly divided into 10 equalized divisions; the CPM model was trained on 9 divisions and tested on the excluded 10th division in an iterative manner. In the leave-one-group-out cross-validation, the model was trained on 3 groups (e.g., controls, SZ, and BD) and tested on the excluded fourth group (e.g., ADHD) iteratively. Critically, we also created predictive models based on only healthy controls (i.e., the training set) and then examined their performances in predicting SCAP variables of individuals with psychiatric disorders (i.e., the testing set). Fourth, considering that the transdiagnostic sample heterogeneity may influence the results, we reran the CPM analyses additionally controlling for diagnostic categories. Fifth, for the dynamic functional connectivity-based CPM analyses, we calculated a single-subject mean value instead of the original summary value to build predictive models and then retested their predictive ability. With respect to dynamic functional connectivity measurement, we also calculated the correlation time series based on the Dynamic Conditional Correlation model (Lindquist et al., 2014) and used their standard deviations as dynamic functional connectivity to repeat the CPM analyses. Sixth, considering the fact that none of these SCAP variables was actually independent, we utilized a multi-output regression model with regularization to predict the entire set of six SCAP variables in each iteration. The multioutput. MultiOutputRegressor meta-estimator of scikit-learn 0.24.1 used herein represents a simple strategy (https://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputRegressor) for extending single output estimators to multioutput estimators. Here, a ridge regressor was used and the regularization parameter was set to 1.0. Finally, we performed CPM prediction of mean reaction time by filtering the edges based on overall correlations regardless of load.

3. Results

3.1. Group differences in spatial working memory

Inter-group differences in 6 primary SCAP variables are illustrated in Fig. 1B-D. In comparison to healthy controls, individuals with SZ showed lower overall working memory capacity (Fig. 1B). Individuals with SZ and ADHD both exhibited lower overall accuracy relative to healthy controls (Fig. 1C). With respect to mean reaction time, individuals with SZ demonstrated higher levels than controls at all loads and a higher level than those with ADHD at load 5 (Fig. 1D).

3.2. Prediction of spatial working memory

CPM performances in predicting SCAP variables based on dynamic and static functional connectivity are shown in Table 2. Specifically, the CPM models, using dynamic functional connectivity within the negative networks, successfully predicted overall working memory capacity (r = 0.229, P = 0.001) (Fig. 2A) and overall accuracy (r = 0.221, P = 0.002) (Fig. 2B) (P < 0.05, Bonferroni corrected). The CPM models based on dynamic functional connectivity within the positive networks effectively predicted mean reaction time (load 3: r = 0.185, P = 0.001; load 5: r = 0.235, P = 0.002; load 7: r = 0.174, P = 0.003) (Fig. 3A-C) (P < 0.05, Bonferroni corrected). However, the CPM predictability of SCAP variables using static functional connectivity was low and did not reach statistical significance (Table 2). In addition, Table S1 shows other measures of CPM performances in predicting SCAP variables, including mean absolute error (MAE) and root mean square error (RMSE).

Table 2.

CPM performances in predicting SCAP variables.

SCAP variables	Dynamic functional connectivity				Static functional connectivity
	r_positive	P	r_negative	P	r_positive	P	r_negative	P
Overall working memory capacity	−0.078	0.769	0.229	0.001**	−0.355	0.994	0.076	0.194
Overall accuracy	−0.188	0.949	0.221	0.002**	0.004	0.532	0.092	0.132
Mean reaction time at load 1	0.142	0.030*	−0.344	0.997	0.032	0.385	−0.078	0.809
Mean reaction time at load 3	0.185	0.001**	−0.269	0.993	0.036	0.320	−0.047	0.736
Mean reaction time at load 5	0.235	0.002**	−0.221	0.979	0.107	0.086	−0.021	0.624
Mean reaction time at load 7	0.174	0.003**	−0.356	0.999	0.035	0.353	−0.053	0.733

*P < 0.05, uncorrected; ** P < 0.05, Bonferroni corrected. Abbreviations: CPM, connectome-based predictive modeling; SCAP, spatial capacity task.

Fig. 2.

Connectome-based predictive modeling (CPM) of overall working memory capacity (A) and overall accuracy (B). On the left, scatter plots show the correspondence between actual (x-axis) and predicted (y-axis) values generated using CPM based on the negative networks. On the right, circle plots show high-degree nodes and their connections in the negative networks. Green lines represent interhemispheric connectivity and blue lines intrahemispheric connectivity. Abbreviations: HC, healthy controls; SZ, schizophrenia; BD, bipolar disorder; ADHD, attention deficit/hyperactivity. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 3.

Connectome-based predictive modeling (CPM) of mean reaction time at load 3 (A), 5 (B) and 7 (C). On the left, scatter plots show the correspondence between actual (x-axis) and predicted (y-axis) values generated using CPM based on the positive networks. On the right, circle plots show high-degree nodes and their connections in the positive networks. Yellow lines represent interhemispheric connectivity and red lines intrahemispheric connectivity. Abbreviations: HC, healthy controls; SZ, schizophrenia; BD, bipolar disorder; ADHD, attention deficit/hyperactivity. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

3.3. Network anatomy

Because of the nature of cross-validation, it is likely that a slightly different set of edges will be selected as features in each iteration of the cross-validation. For illustrative purpose, we defined final dynamic functional connectivity networks using data from all 242 subjects. Overall, anatomies for the dynamic functional connectivity networks associated with SCAP variables were complex and included edges between nodes across the brain. For overall working memory capacity, the negative network consisted of 890 edges (Fig. 2A). Highest-degree nodes (i.e., nodes with the most edges) were mainly located within the frontoparietal, somato-motor and visual networks (Fig. 4). For overall accuracy, the negative network was comprised of 640 edges (Fig. 2B). Highest-degree nodes were predominantly located within the frontoparietal, somato-motor and default mode networks (Fig. 4). For mean reaction time, the positive networks were composed of 1532 edges at load 3, 1488 edges at load 5, and 992 edges at load 7, respectively (Fig. 3A-C). The most consistently identified highest-degree nodes were located within the default mode and visual networks (Fig. 4).

Fig. 4.

Polar plots showing the 25 highest-degree nodes in each CPM network summarized by overlap with canonical neural networks. Abbreviation: CPM, connectome-based predictive modeling.

3.4. Validation analysis

First, using edges selected by thresholds of P < 0.05 and 0.005, we found that the prediction performances of SCAP variables were similar to those at the threshold of P < 0.01 (Tables S2 and S3). Second, when constructing functional connectome using parcellation scheme based on AAL atlas with 116 nodes, the patterns of results held although the CPM predictability of overall working memory capacity lost significance (Table S4). Third, when using 10-fold cross-validation analyses, the prediction results of SCAP variables remained unchanged (Table S5); while using leave-one-group-out cross-validation analyses yielded relatively poor prediction, the tendency of results was still present (Table S6); the predictive models built in healthy controls successfully predicted SCAP variables of individuals with psychiatric disorders (Table S7), which was similar to the CPM performances using the leave-one-out cross-validation analyses. Fourth, we found that our main results were reproducible when additionally considering the effect of diagnostic categories (Table S8). Fifth, CPM performances in predicting SCAP variables using the mean value of selected dynamic functional connectivity were nearly identical to those using the summary value (Table S9). However, the CPM models using dynamic connectivity derived from the Dynamic Conditional Correlation model failed to predict any of the SCAP variables (Table S10), suggesting that the predictive ability was largely dependent on dynamic functional connectivity measurement approaches. Sixth, multi-output regression models based on dynamic connectivity yielded results comparable to those from the CPM models; strikingly, we also observed that static connectivity-based multi-output regression models could predict overall accuracy and mean reaction time (Table S11). Finally, we found that the overall predictive power was improved when performing CPM prediction of mean reaction time by filtering the edges based on overall correlations regardless of load (Fig. S1).

4. Discussion

This is the first machine learning study focusing on the comparison of dynamic and static functional connectivity in building transdiagnostic predictive working memory models. Our data demonstrated that the dynamic functional connectivity-based CPM models successfully predicted overall working memory capacity and accuracy as well as mean reaction time, yet their static counterparts showed inability in the prediction. At the neural level, we found that dynamic functional connectivity of the frontoparietal and somato-motor networks were negatively correlated with working memory capacity and accuracy, and those of the default mode and visual networks were positively associated with mean reaction time. Our findings not only are congruent with previous reports of the superiority of dynamic functional connectivity in capturing task-based phenotypes, but also add to the current knowledge by identifying a “connectivity fingerprint” that can predict individual working memory transdiagnostically.

Inter-group comparisons in the primary SCAP variables revealed that individuals with psychiatric disorders manifested shared working memory impairment, but to differing degrees. Individuals with SZ demonstrated the most prominent abnormalities, characterized by lower overall working memory capacity and accuracy along with higher mean reaction time at all loads. Individuals with ADHD only showed lower overall accuracy relative to healthy controls. By contrast, there was no significant alteration in any of the primary SCAP variables in individuals with BD. This may be partially due to limited statistical power from relatively small samples and clinical heterogeneity related to variation in illness profile, as we observed a non-significant trend for lower overall accuracy and higher reaction time in BD individuals relative to controls. The current findings, taken with previous reports of lower working memory capacity in these diseases (Gilmour et al., 2019, Grot et al., 2017, Maehler and Schuchardt, 2016, Soraggi-Frez et al., 2017), provide strong evidence that working memory impairment is a transdiagnostic feature of psychiatric disorders.

There have been several brain imaging studies using machine learning approaches to predict individual working memory in healthy subjects or individuals with a certain clinical condition. In a recent resting-state fMRI study of 1003 healthy young adults, a spatiotemporal multilayer modularity model was applied to the time-resolved fMRI connectivity data to quantify network switching rate; using cross-validated elastic net regression, moreover, this newly developed dynamic functional connectivity measure was found to predict inter-subject variation in working memory performance (i.e., average accuracy) (Pedersen et al., 2018b). Another task-based fMRI study on healthy subjects quantified functional connectivity between occipital and parietal regions during working memory encoding, and subsequent multivariate pattern regression analysis using the Classification and Regression Tree algorithm revealed that individual differences in working memory performance (specifically the working memory precision but not the individual number of items stored in memory) could be reliably predicted on the basis of voxel-wise occipito-parietal functional connectivity patterns (Galeano Weber et al., 2017). Taking advantage of a combination of fMRI and diffusion tensor imaging (DTI) data, McKenna and colleagues employed least angle regression with least absolute shrinkage and selector operator (LASSO) estimation to select the best joint neuroimaging predictors of working memory performance for BD patients and healthy participants, separately (McKenna et al., 2015). For working memory accuracy, BD-specific predictors included gray matter activation in bilateral dorsolateral prefrontal cortex, and white matter integrity in splenium of the corpus callosum, left uncinate fasciculus, and bilateral superior longitudinal fasciculi. Compared to prior studies, our design has the advantage of generating transdiagnostic predictive models using CPM. In comparison with the machine learning approaches that have previously been utilized to study working memory, CPM is optimized for neuroimaging data because it is entirely data driven and requires no a priori selection of networks. The predictive power of CPM has been demonstrated in studies of fluid intelligence (Finn et al., 2015), attention (Rosenberg et al., 2016, Yoo et al., 2018), creativity (Beaty et al., 2018), personality (Cai et al., 2020b), and decision impulsivity (Cai et al., 2020a). More importantly, performing prediction in a sample of healthy and multiple psychiatric populations may permit identification of transdiagnostic neurobiological mechanisms underlying working memory processes, which is of higher clinical and translational importance.

Traditional resting-state functional connectivity is assumed to have static nature and thereby provides a measure of brain function averaged within an fMRI experiment. Yet, the brain is not a static organ, and time-varying profiles of functional connectivity are evident across a broad range of task states and during periods of unconstrained rest (Hutchison et al., 2013, Preti et al., 2017). As a consequence, a rapidly increasing number of studies have emerged to attempt a more thorough characterization of brain activity dynamics (Calhoun et al., 2014, Hutchison et al., 2013, Preti et al., 2017). In this study, we selected the variance (i.e., the amplitude of the sliding-windowed functional connectivity time series) as the dynamic functional measure, as it offers important information on the relationship between brain activity and behavior that can be used to detect between-individual variations (Choe et al., 2017). Remarkably, we found that dynamic functional connectivity outperformed its static counterpart in the prediction of working memory. The findings here not only corroborate previous reports that dynamic connectivity specifically encodes task-based behavioral phenotypes (Liegeois et al., 2019), but also add important context to the growing literature on the more behavioral information contained within the temporal features of spontaneous neural activity. Indeed, the execution of cognitive tasks relies on a coordinated activation of multiple networks at faster timescales, which can be captured by dynamic rather than static functional measures. Moreover, our data showed that lower functional connectivity dynamics variance was associated with better working memory performance, i.e., higher working memory capacity and accuracy as well as shorter reaction time. Converging evidence suggests that the amplitude of dynamic functional connectivity variability decreases during task as compared to resting-state (Elton and Gao, 2015, Hutchison and Morton, 2015). Although speculative, a potential explanation may be that lower neural activity variability during resting-state may facilitate rapidly switching to task states, resulting in greater ability to be recruited according to task demands.

The present observations indicate that dynamic functional connectivity of the frontoparietal and somato-motor networks contribute the most to inter-subject differences in working memory capacity and accuracy. Working memory has long been linked to the prefrontal cortex, since lesion-deficit studies have demonstrated that damage to this brain region can critically impair the ability to maintain and update mnemonic information (Pribram et al., 1952). Electrophysiological evidence from non-human primates has made it possible to understand the basis of working memory at the single-neuron level, and revealed neural correlates of working memory in the prefrontal cortex of monkeys (Fuster and Alexander, 1971). Furthermore, many supporting research studies have pointed to the frontoparietal network as the working memory neural network in the monkey brain (Constantinidis and Procyk, 2004). Although human neuroimaging studies have found that performances of working memory tasks are associated with activity in a wide range of areas across the brain, there are commonalities in a frontoparietal network of regions activated during different working memory tasks with different types of stimulus (Ikkai and Curtis, 2011, Jerde et al., 2012, Rottschy et al., 2012). Moreover, stronger frontoparietal functional connectivity (Palva et al., 2010, Stevens et al., 2012) and structural connectivity (Darki and Klingberg, 2015, Ostby et al., 2011, Vestergaard et al., 2011) have been shown to be associated with higher working memory capacity. Notably, there is also evidence for brain activity changes in the frontal and parietal regions as well as increased frontoparietal connectivity after working memory training (Astle et al., 2015, Constantinidis and Klingberg, 2016, Jolles et al., 2013, Kundu et al., 2013, Thompson et al., 2016). In addition, a convergent body of neuroimaging data show that motor networks are highly integrated into working memory processes and are critical for normal performance (Marvel et al., 2019). On one hand, secondary motor regions, including the premotor cortex, supplementary motor area, basal ganglia, and cerebellum are often simultaneously activated during the maintenance phase of working memory assessments in healthy individuals (Wager and Smith, 2003). On the other hand, damage to secondary motor regions, e.g., the cerebellar motor regions (Cooper et al., 2012, Stoodley et al., 2016) and supplementary motor area (Canas et al., 2018), has been associated with subsequently observed working memory deficits. Our findings, in conjunction with those of previous studies, support an interaction between cognitive effort and motor system engagement in working memory.

With respect to reaction time of working memory, our data suggest a strong contribution of dynamic functional connectivity of the default mode and visual networks to the prediction. The default mode network exhibits correlated activity at rest and has shown deactivation across a range of cognitive tasks (Raichle, 2015, Raichle et al., 2001). Therefore, the default mode network and task-positive networks (e.g., working memory-related network) are usually considered to be anti-correlated. However, recent studies increasingly implicate that working memory is also modulated by the default mode network, and thus it is likely that this network is potentially involved in the neural mechanism underlying the core working memory processes. For example, previous task fMRI studies have demonstrated greater activation of the default mode network during n-back tasks (Konishi et al., 2015, Spreng et al., 2014). From a functional connectivity perspective, the within-network connectivity strength between default mode network hubs was found to correlate with percent correct responses (Hampson et al., 2006) and reaction times to correct responses (Vatansever et al., 2017) of working memory paradigms. In addition, Bluhm et al. reported working memory task-related changes in functional connectivity between the default network nodes and brain areas typically activated during cognitive tasks (i.e., task-positive brain areas) (Bluhm et al., 2011). Moreover, such between-network functional connectivity were shown to predict behavioral performance (Hampson et al., 2010) and to display variable dynamics during the three distinct working memory stages of encoding, maintenance, and retrieval (Piccoli et al., 2015). Using positron emission tomography and DTI techniques, Yakushev et al. demonstrated that metabolic and structural connectivity within the default mode network also related to working memory performance in young healthy adults (Yakushev et al., 2013). Recently, it is quite apparent that the visual network plays a pivotal role in complex cognitive functions, such as visual working memory. For instance, using task-based fMRI, researchers have revealed that neural activity in the visual system is generally recruited to contribute to visual working memory (Albers et al., 2013, Lawrence et al., 2018, Xing et al., 2013). Bergmann et al. reported that individuals with a larger gray matter volume of primary visual cortex tended to have greater visual working memory storage (Bergmann et al., 2016). During visual working memory, functional connectivity patterns between visual processing and frontoparietal control systems were predictive of the behavioral mean precision (Galeano Weber et al., 2017). More directly and interpretably, Rademaker and colleagues found that transcranial magnetic stimulation targeting the visual cortex had an impact on visual working memory precision and guess rate (Rademaker et al., 2017).

Our study has several limitations that need to be considered. First, because of the lack of an independent dataset that could replicate the CNP data, we were not able to perform an additional external validation of the models’ predictive ability. Second, our interpretation may be influenced by the confounding factors such as medication use and/or long illness duration in the patient groups. Future studies with drug-naive first-episode patients are warranted to further determine the reliability of the current findings. Third, during resting-state fMRI scanning, we did not assess subjects’ levels of drowsiness or vigilance that might have influenced the dynamic functional connectivity analysis (Laumann et al., 2017). However, the variance measure of dynamic functional connectivity used in our study has been proved to exhibit good test-retest reliability (Choe et al., 2017), which may alleviate the concern about noise interference to some extent. Finally, the CPM models using dynamic functional connectivity derived from the Dynamic Conditional Correlation model failed to predict any of the SCAP variables, suggesting that the predictive ability is strongly influenced by dynamic functional connectivity measurement approaches. This issue warrants further investigation that is beyond the scope of the present study.

In conclusion, by applying CPM to resting-state fMRI data from a transdiagnostic sample, we demonstrate that dynamic connectivity patterns of functional connectome can effectively and reliably predict working memory performance at the single-subject level, whereas their static counterparts fall short in the prediction. Our data also reveal that individual variations in dynamic functional connectivity of the frontoparietal and somato-motor networks contribute the most to inter-subject differences in working memory capacity and accuracy, and those of the default mode and visual networks contribute to inter-individual variability in reaction time. These findings not only endorse prior observations that dynamic functional connectivity encodes more behavioral information than static connectivity, but also help advance the translation of cognitive “connectome fingerprinting” into real-world clinical settings.

Funding

The work was supported by the National Natural Science Foundation of China (grant numbers: 81801679, 82071905 and 81771817).

CRediT authorship contribution statement

Jiajia Zhu: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Software, Validation, Visualization, Writing - original draft. Yating Li: Data curation, Formal analysis, Investigation, Methodology, Software. Qian Fang: Data curation, Formal analysis, Investigation, Methodology, Software. Yuhao Shen: Data curation, Formal analysis, Investigation, Methodology, Validation, Visualization. Yinfeng Qian: Data curation, Formal analysis, Investigation, Methodology, Validation, Visualization. Huanhuan Cai: Data curation, Formal analysis, Investigation, Methodology, Software. Yongqiang Yu: Conceptualization, Formal analysis, Funding acquisition, Project administration, Resources, Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following are the Supplementary data to this article: