Task-induced brain connectivity promotes the detection of individual differences in brain-behavior relationships

Abstract

Although both resting and task-induced functional connectivity (FC) have been used to characterize the human brain and cognitive abilities, the potential of task-induced FCs in individualized prediction for out-of-scanner cognitive traits remains largely unexplored. A recent study Greene et al. (2018) predicted the fluid intelligence scores using FCs derived from rest and multiple task conditions, suggesting that task-induced brain state manipulation improved prediction of individual traits. Here, using a large dataset incorporating fMRI data from rest and 7 distinct task conditions, we replicated the original study by employing a different machine learning approach, and applying the method to predict two reading comprehension-related cognitive measures. Consistent with their findings, we found that task-based machine learning models often outperformed rest-based models. We also observed that combining multi-task fMRI improved prediction performance, yet, integrating the more fMRI conditions can not necessarily ensure better predictions. Compared with rest, the predictive FCs derived from language and working memory tasks were highlighted with more predictive power in predominantly default mode and frontoparietal networks. Moreover, prediction models demonstrated high stability to be generalizable across distinct cognitive states. Together, this replication study highlights the benefit of using task-based FCs to reveal brain-behavior relationships, which may confer more predictive power and promote the detection of individual differences of connectivity patterns underlying relevant cognitive traits, providing strong evidence for the validity and robustness of the original findings.

1. Introduction

Owing to the well-recognized convenience in data acquisition, robustness to practice effects, flexibility in analysis, and an unconstrained nature in task demands, resting state functional magnetic resonance imaging (rs-fMRI) has become the default condition for probing individual differences (Finn et al., 2017). Using neuroimaging features of functional connectivity (FC) derived from rs-fMRI, together with machine learning approaches, researchers have demonstrated that connectome-based models are strongly predictive of various behavioral aspects (Dubois et al., 2018; Finn et al., 2015; Hsu et al., 2018; Jiang et al., 2019, Jiang et al., 2019; Lake et al., 2019; Liu et al., 2018; Wang et al., 2018; Yamashita et al., 2018; Yip et al., 2019). However, FC can also depend on different task conditions. Importantly, converging evidence from previous studies has highlighted the consistent organization of functional networks at rest and during various tasks (Bzdok et al., 2016; Cole et al., 2014; Tavor et al., 2016). Specifically, Tavor et al. found that the FC measurement derived from rs-fMRI can accurately predict individual differences in task-evoked brain activation maps. These studies suggested that the large-scale brain networks were similar across distinct cognitive states, and tasks only moderately modified FC patterns throughout the brain (Cole et al., 2014). Nonetheless, recent studies substantiated that although subtle, these task-based FC contributed strongly to cognitive performance by reorganizing behaviorally relevant neural networks, which can better reveal individual differences (Finn et al., 2017; Gratton et al., 2016). Consequently, the study and analysis of FCs derived from rs-fMRI may provide just a partial understanding of the brain’s functional architecture (Mennes et al., 2013), signifying that to appreciate the repertoire of the functional dynamics of the human brain, task-based FCs should also be considered.

However, the application of task-induced FCs (connectivity features measured during task) in individualized prediction for cognitive traits remains relatively limited. Most existing studies are commonly limited to predicting in-scanner completion metrics quantified by performing tasks, e.g. (Beaty et al., 2018) accomplished robust prediction of individual creative ability using FCs acquired from 163 participants engaging in a classic divergent thinking task. By contrast, the predictability of out-of-scanner cognitive behaviors using FCs derived from a wide range of task conditions remains largely unexplored. Importantly, only few studies successfully established the relevance of FCs from a variety of cognitive states to cognitive behavior using connectome-based machine learning approaches. An impressive example is a recent study that predicted the fluid intelligence scores using FCs derived from rest and multiple task conditions, suggesting that task-based model improved prediction of individual traits (Greene et al., 2018). This pioneering study provided crucial foundations for future studies to better reveal brain-behavior relationships. Moreover, since individual differences in task-based FCs consist of state-dependent aspects (Geerligs et al., 2015), distinct tasks may exert different but specific influences on the brain topology properties. Putatively, integrating FC features from multiple cognitive conditions may generate better predictions. Accordingly (Elliott et al., 2019), developed a brain measurement termed general functional connectivity (GFC) by combining resting and task scans, and suggested that GFC could perform as well as or better than specific tasks in cognition predictions. Additionally (Gao et al., 2019), proposed a novel prediction framework by integrating multiple task connections into a single predictive model to capture the complementary information, which exhibited superior performance in the prediction of fluid intelligence over single connectome-based model.

Recent advances in fMRI studies have established the uniqueness and stability of FC across cognitive conditions. Specifically, Finn et al. found that although changes in brain states may modulate FC patterns to some degree, an individual’s intrinsic functional architecture was reliable enough regardless of how the brain was engaged during scanning (rest or task), and distinct enough to identify the individual (Finn et al., 2015), i.e., FC fingerprint. More interestingly, the FC-based matching algorithm achieved varied identification accuracies between different pairs of fMRI conditions, implying that brain state may be manipulated to emphasize individual differences in functional connectivity (Finn et al., 2017). Compared to self-identification, predicting cognitive traits with continuous values requires more dedicated design and techniques (Rosenberg et al., 2016). Interestingly, Greene and colleagues also tested the cross-condition generalization but limited the analysis only to the best- and worst-performing models.

Despite such promising progress, there is still a scarcity of more evidence demonstrating the superiority of task-induced FCs in cognition predictions. In the current study, we are inspired to replicate the findings of Greene et al. and Gao et al. to confirm (1) whether models built from task-based FCs outperform those built from resting FCs, (2) does combining multiple connectomes from different conditions improve cognition predictions, (3) and whether the connectome-based models can generalize between different task-task and rest-task conditions. Importantly, we integrated these two studies into a unified framework to demonstrate that the large-scale brain networks were dominated by a stable intrinsic architecture that ensured successfully cross-condition generalizations, and tasks can amplify individual differences in trait-related FCs, which conferred more predictive power.

We followed a similar analytical pipeline but employed a different machine learning approach named partial least square (PLS) regression, and applied the method to predict fluid intelligence as well as three other different cognitive metrics including reading comprehension abilities, cognitive flexibility, and working memory capacity. PLS regression can establish the brain-behavior relationship without a feature selection step, and is especially useful in situations where feature dimension considerably overwhelms the sample size. It has been successfully used for the prediction of several cognitive behaviors like episodic memory and sustained attention (Fong et al., 2018; Meskaldji et al., 2016; Yoo et al., 2017). Notably, the reading comprehension was used as the leading measure being investigated in the present study. As a general cognitive ability, reading comprehension plays a ubiquitous role in modern society, from facilitating language acquisition, communication and information sharing to improving daily functioning and health. Investigating this cognitive metrics could enhance our understanding of the underlying neural correlates. On the other hand, the human connectome project (HCP) dataset provides 2 specific tests (oral reading recognition test [ORRT] and picture vocabulary test [PVT]) for measuring 2 core components of reading skills: reading decoding and linguistic comprehension abilities (Cui et al., 2017; Hoover et al., 1990), which allows the comparison of predictive models built from different cognitive conditions for the same reading measure, and models built from the same cognitive condition for two different reading measures.

2. Materials and methods

2.1. Subjects

We used the released HCP S500 data, which incorporated high-resolution fMRI data in rest and 7 distinct task conditions, and a battery of cognitive tests (Barch et al., 2013). There were 512 subjects’ records in the data we obtained from the HCP. Detailed inclusion/exclusion criteria can be found in (Van Essen et al., 2012) and our previous work (Zuo et al., 2018). After excluding subjects with either missing imaging data or missing cognitive scores, 463 healthy subjects (269 females, mean age 29.1 ± 3.5 years, in range of 22–36 years) were retained, all of whom participated 2 tests for measuring reading decoding and linguistic comprehension, i.e., ORRT (HCP: ReadEng_AgeAdi) and PVT (HCP: Pic-Vocab_AgeAdj) (Gershon et al., 2013). Specifically, the PVT and ORRT were assessed using the NIH Toolbox Cognition Battery (Gershon et al., 2013). In PVT, participants are presented with an audio recording of a word and four photographic images and are asked to select the picture that most closely matches the meaning of the word. In ORRT, participants are asked to read and pronounce letters and words as accurately as possible. According to the NIH Toolbox national norms, the raw scores of the 2 tests were transferred into the age-adjusted scores, with mean of 100 and standard deviation of 15 (Cui et al., 2017). We used fMRI data from 8 separate conditions: one rest session and 7 distinct task sessions (Emotion, Gambling, Language, Motor, Relational, Social, and Working Memory [WM]; Table S1).

2.2. MRI data acquisition and preprocessing

Data acquisition for HCP has been described in detail elsewhere (Ugurbil et al., 2013), as well as in our previous work (Zuo et al., 2018). For the sake of completeness, we repeat these main descriptions for data collection here: the dataset was collected on a 3T Skyra (Siemens, Erlangen, Germany) with a 32-channel head coil. The primary scanning parameters were repetition time (TR), 720 ms; echo time (TE), 33.1 ms; flip angle, 52°; field of view, 208 × 180 mm; slice thickness, 2.0 mm; and voxel size, 2.0 mm isotropic cube (Smith et al., 2013). The HCP data were already preprocessed, well aligned, and registered to the Montreal Neurological Institute (MNI) 2-mm standard space when we received it. The main preprocessing steps included (Glasser et al., 2013): (1) gradient nonlinearity distortion; (2) 6 degrees of freedom (DOF) FSL/FLIRT-based motion correction; (3) FSL/top-up-based distortion correction; (4) registration to a T1 space image; and (5) FSL/FNIRT-based registration to MNI 2-mm space. After receiving the above preprocessed data from HCP, we further band-pass-filtered the data at 0.009–0.08 Hz to reduce low-frequency drift and high-frequency noise (Vatansever et al., 2015). The mean signal of the white matter, cerebrospinal fluid (CSF), and the movement parameters and its derivatives (in the Movement_parameters.txt file in the HCP S500 release) were regressed out as confounding factors.

2.3. Functional connectivity analysis

The registered fMRI volumes in the Montreal Neurological Institute (MNI) template were divided into 246 nodes (regions of interest, ROI) from the Brainnetome Atlas (Fan et al., 2016). For each condition, mean regional time series were obtained by averaging voxel-wise fMRI time series in each of the 246 nodes for each individual. Pearson correlations of time courses between any two nodes were then calculated and Fisher transformed, resulting in a 246 × 246 symmetric FC matrix for each subject (Jiang et al., 2018). After removing 246 diagonal elements, we extracted the upper triangle elements of the FC matrix as features for prediction, namely, each subject has a feature vector in the dimension of (246 × 245)/2 = 30135.

2.4. Individualized prediction of reading comprehension abilities using distinct fMRI states

Following the study of Greene and colleagues, we replicated the prediction procedure to predict cognitive metrics using FC matrices from each of the 8 fMRI conditions. Different from the original study, we employed PLS regression to predict the reading comprehension scores (PVT and ORRT) of new subjects. PLS regression bears some relation to principal component analysis and multiple linear regression, and works by representing variables with a few number of latent components (Yoo et al., 2017). The prediction was conducted within a 10-fold cross-validation. Specifically, one fold subjects (10%) were designated as the testing sample while the remaining 9-fold subjects (90%) were used as the training set. During training, PLS regression (Geladi and Kowalski, 1986) was adopted to model the relationship between observed reading comprehension scores and whole-brain FCs in training subjects, yielding a prediction model. Then, the model built in the training set was applied to the one-fold left out testing subjects’ connectivity data to generate predicted reading comprehension scores. By exchanging the roles of testing and training sets in turn (with each fold of subjects excluded once), we obtained the predicted PVT and ORRT scores for all subjects. The optimal number of latent component was tuned in a nested 10-fold cross-validation loop. The parameter was tested ranging from 1 to 20, and the value that yielded the highest prediction accuracy was finally determined as the optimum number of components. In our experiments, the optimal number of latent components was 3–5 for most conditions. It failed to model the brain-behavior relationship when setting the parameter to a too small value. Setting the parameter to a large value led to high prediction accuracy in training samples but low prediction performance in testing samples (overfitting). Since the full dataset was randomly divided into 10 folds, performance might depend on data division (Feng et al., 2018). Consequently, to ensure the robustness and reliability, we repeated the prediction procedure 100 times with subjects randomly shuffled. The prediction performance was measured by the mean of the 100 correlations between observed and predicted reading comprehension scores. Notably, the above prediction procedure was separately performed for PVT and ORRT. Moreover, we also conducted a non-parametric permutation test to determine the significance of the prediction accuracy. Specifically, we randomly shuffled the observed reading comprehension scores 5000 times and reran the prediction pipeline each time, generating a null distribution for significance testing. The p-values was determined by (1 × the number of permutated r values greater than or equal to the empirical r)/5001 (Beaty et al., 2018).

2.5. Replicability of the prediction under different conditions

As demonstrated in the original study, some potential confounds may influence the predictions. To validate our main results, we performed the following analyses. (1) The reading comprehension scores are negatively correlated with the mean framewise displacement (FD) for all conditions (r = −0.23~−0.16, p < 0.05). To confirm that our predictive models captured FC variations specific to reading comprehension ability independently of this contamination, we calculated the partial correction between predicted and observed reading comprehension scores while factoring out the mean FD (Rosenberg et al., 2016). (2) To further control for the effect of head motion, we reran the prediction procedure only on subjects whose mean FD was less than 0.14 mm or 0.10 mm, which are the two most commonly applied inclusion criteria (Finn et al., 2015; Greene et al., 2018). (3) To ensure the prediction results were robust to cross-validation approaches, we repeated the entire prediction procedure using a leave-one-out cross-validation (LOOCV) strategy. (4) Many subjects in the HCP dataset are genetically related. To account for the family structure, we reran the prediction procedure using a leave-one-family-out cross-validation (Dubois et al., 2018). (5) To investigate the influence of brain parcellation, we repeated the prediction procedure using connectivity matrices in a feature dimension of 34716 calculated from the 264-node Power atlas (Power et al., 2011). (6) Functional scans in the HCP protocol differed considerably in duration (176–1200, Table S1). Previous studies have shown that longer scan durations better preserve individual difference in FCs and have a higher test-rest reliability (Elliott et al., 2019; Finn et al., 2015). To investigate its effect on prediction, we recalculated the FC matrices by truncating the time courses from all conditions to include the same number of frames as the shortest condition (Emotion, 176 vol), and repeated the prediction pipeline. (7) To investigate whether our results that task-based model outperformed rest-based model were specific to the cognitive metrics adopted, we repeated the prediction procedure to predict another three cognitive constructs: the fluid intelligence, cognitive flexibility, and working memory capacity using connectivity data from each of the 8 fMRI conditions. Specially, the fluid intelligence was used as the phenotypic measure of interest in the study of Greene et al.

2.6. Integrating multi-task FC features improves prediction

To replicate the results of Gao et al., we further investigated whether combining FCs from multiple fMRI conditions can generate improved predictions. We concatenated whole-brain FCs from all 8 fMRI conditions horizontally together as input features (in a feature dimension of 30135 × 8), and then repeated the prediction procedure as discussed above. Furthermore, we adopted a backward selection strategy to find the optimal combination of cognitive conditions which can yield the highest prediction performance. Because our primary objective is to explore whether integrating the more cognitive conditions could yield higher prediction accuracy, we will not test the prediction performance across all possible state combinations. The backward selection finds the optimal task combination by excluding each of the fMRI states in a stepwise way (Gao et al., 2019). Specifically, we searched for the cognitive condition, after excluding which can improve prediction the most in each step. Then, this fMRI condition was pruned from the current set of cognitive conditions. This procedure was iteratively repeated until excluding any cognitive condition won’t lead to improvement in prediction. Notably, the multi-task predictions were also run within 10-fold cross-validation with 100 repetitions for PVT and ORRT separately.

2.7. Reading comprehension-predictive functional edges

We investigated the similarity of FC or node weights within- and between-conditions. The similarity was quantified by the correlations of whole-brain FC or node weights between each condition pairs. Node weight was computed by summarizing beta coefficients of all FCs connected with a given node. Notably, since we employed a 10-fold cross validation strategy with 100 repetitions, FC weight was averaged across 1000 loops.

PLS model includes all features in prediction and assigns each a different weight. To investigate whether the beta coefficient represented edge’s real contribution to prediction or was just assigned by chance, we employed permutation test to assess the significance of the beta coefficient for each edge, as described in (Yoo et al., 2017). We randomly shuffled the behavioral scores (i.e., reordering the rows of Y and leaving X unchanged), and then fitted a PLS model using mismatched brain and behavioral data on all participants. PLS beta coefficient can be obtained for each edge from each permutation, and repeating this process 100000 times allowed us to create a null edge weight distribution. Significance was determined by whether its real beta value differed (two-tailed p < 0.05) from the empirical distribution acquired from 100000 permutations (Yoo et al., 2017). We then counted the number of significantly predictive edges between each macroscale brain region pair. In this study, the permutation test was performed separately for PVT and ORRT in cognitive conditions of rest, language and WM.

Finally, to explicitly characterize the contribution of each functional network to prediction, we grouped the 246 nodes into 8 canonical networks, which included 7 networks mapped from the Yeo’s 7-network parcellation scheme (visual, somatomotor, dorsal and ventral attention, limbic, frontoparietal, default mode [DMN]) (Yeo et al., 2011) and a subcortical network. Following the same procedure as in (Greene et al., 2018), we computed the fraction of the most significantly predictive FCs in each pair of canonical networks, normalized by the fraction of total edges belonging to that pair. Therefore, a value > 1 indicated overrepresentation of the network pair to the prediction model.

2.8. Generalizability of predictive models across fMRI conditions

We next demonstrated that prediction models can generalize across distinct cognitive conditions. To this end, we reran the prediction procedure within 10-fold cross-validation, however, this time we used FCs in training and testing subsets from different cognitive conditions, as implemented in the original study. Specifically, the model was trained on 9-fold participants’ whole-brain FCs, but tested on the left-out one-fold participants’ connectivity data from a different fMRI condition. The cross-condition generalization was tested between all possible condition pairs by training the prediction model with each of the 8 fMRI conditions and testing with the other 7 conditions, instead of limiting the analysis only to the best- and worst-performing models as in the original study. Overall, 56 possible condition pairs were tested for each of the two reading comprehension measures (PVT and ORRT).

3. Results

3.1. Individualized prediction of reading comprehension abilities using distinct fMRI conditions

Fig. 1 demonstrates the prediction results of reading comprehension scores using whole-brain FC from each of the 8 fMRI states. Overall, all 8 prediction models achieved significant estimations of PVT and ORRT scores (Fig. 1a and b, Table S2). Specifically, for PVT prediction, the WM task and language task achieved the top two highest prediction performance (r[WM] = 0.430 ± 0.011, Fig 1c; r[Lang] 0.408 ± 0.013; p = 2.0 × 10⁻⁴; Fig. 1d). For ORRT, the language task achieved the highest prediction accuracy (r[Lang] = 0.465 ± 0.012; p = 2.0 × 10⁻⁴; Fig. 1e), and the WM task achieved the second-highest accuracy (r[WM] = 0.425 ± 0.014; p = 2.0 × 10⁻⁴; Fig. 1f). The emotion task yielded the worst-performing model for both PVT and ORRT (r[PVT] = 0.204 ± 0.013, p = 1.0 × 10⁻³; r[ORRT] = 0.223 × 0.017, p = 2.0 × 10⁻⁴). Our results successfully replicated the findings of Greene et al. by demonstrating that rest might be not the optimal condition for predicting individual cognitive traits.

Fig. 1. Prediction results of two reading comprehension abilities (PVT and ORRT).

All 8 fMRI conditions yielded significant correlations between observed and predicted (a) PVT or (b) ORRT scores. Specifically, for PVT, the WM task yielded the best-performing model (c), and the language task yielded the second-best model (d). For ORRT, the language task achieved the highest prediction accuracy (e), and the WM task achieved the second-highest prediction accuracy (f). Combining FC features from all 8 fMRI conditions generated improved prediction performance (red bar in a, b) than using any single condition alone. An optimal combination of 6 cognitive conditions achieved the best predictions for both (g) PVT and (h) ORRT (r[PVT] = 0.503 ± 0.009, r[ORRT] = 0.498 ± 0.012; pink bar in a, b). Values in the x-axis and y-axis were normalized for visualization. (i) Given equal scan durations, all task-based models except emotion again achieved higher prediction accuracies than rest-based models. Abbreviation: Emo, emotion; Gam, gambling; Lang, language; Mot, motor; Rel, relational; Soc, social; WM, working memory; PVT, picture vocabulary test; ORRT, oral reading recognition test.

3.2. Replicability of the prediction under different conditions

All predictions remained significant and largely unchanged when controlling for mean frame-wise displacement (Fig. S1), rerunning the prediction pipeline only on subjects with a small head motion (Fig. S2), using a LOOCV approach (Fig. S3), controlling for the family structure (Fig. S4), or using FC data calculated from a different brain parcellation scheme (Fig. S5). Additionally, we observed that longer time courses improved prediction accuracy (Fig. S6). This is consistent with previous evidence suggesting that longer scan lengths better preserved individual difference in connectivity profiles (Finn et al., 2015). Given equal scan durations, the rest state generated largely attenuated prediction performance, and all task-based models except emotion again achieved higher prediction accuracies than rest-based models (Fig. 1i). A Steiger’s z-test for testing differences between two dependent correlations (Steiger, 1980) revealed that for ORRT, prediction performance based on cognitive conditions of motor, language, relational and WM was significantly higher than that based on rest (p < 0.02). For PVT, models based on social, gambling, relational, language and WM significantly outperformed rest-based model (Steiger’s z-test, p < 0.04). Moreover, in the prediction of fluid intelligence, cognitive flexibility, and working memory capacity, most task-based models again outperformed rest-based models (Fig. 2), suggesting that our conclusion that, rest might be not the optimal cognitive state for investigating individual difference, was not constrained by the specific cognitive metrics adopted.

Fig. 2. Prediction results of fluid intelligence, cognitive flexibility and working memory capacity.

(a) For the prediction of fluid intelligence, the WM task achieved the highest prediction accuracy (r[WM] = 0.378 ± 0.0139, p = 2.0 × 10⁻⁴), and the emotion task achieved the lowest prediction accuracy (r[Emotion] = 0.222 ± 0.0164, p < 1.0 × 10⁻³). (b) Importantly, integrating 8-condition connectivity data achieved improved prediction performance than using any single state alone (r = 0.409 ± 0.0116, p = 2.0 × 10⁻⁴). (c) For cognitive flexibility, the WM task yielded the best-performing model (r[WM] = 0.311 ± 0.0114, p = 2.0 × 10⁻⁴), and the emotion task yielded the worst-performing model (r[Emo] = 0.099 ± 0.0198, p > 0.05). (d) Combining all 8-condition connectivity data also achieved improved prediction performance than using any single state alone (r = 0.330 ± 0.0150, p = 2.0 × 10⁻⁴). (e) For the prediction of working memory capacity, the WM task yielded the best-performing model (r[WM] = 0.302 ± 0.0154, p = 2.0 × 10⁻⁴), and the rest yielded the worst-performing model (r[Rest] = 0.083 ± 0.0191, p > 0.05). (f) Moreover, combining all 8-state connectivity data also achieved improved prediction performance than using any single state alone (r = 0.339 ± 0.0151, p = 2.0 × 10⁻⁴).

3.3. Integrating multi-task FC features improves prediction

The finding that combining multiple connectomes from different cognitive states improved prediction performance was replicated by our results. Specifically, integrating 8-state FC features achieved improved prediction accuracy than using any single condition alone (Fig. 1a and b, red bar: r[PVT] = 0.467 ± 0.089; r[ORRT] = 0.476 ± 0.011). However, integrating the more cognitive conditions could not yield the higher accuracy. As a result, the optimal FC combination includes 6 fMRI conditions for both PVT (r[PVT] = 0.503 ± 0.009; the two excluded states were emotion and gambling; pink bar in Fig. 1a, g) and ORRT (r[ORRT] = 0.498 ± 0.012; the two excluded states were emotion and motor; pink bar in Fig. 1b, h). More results can be found in Fig. S7. For PVT prediction, the optimal multi-task predictions significantly outperformed predictions based on any single task (Steiger’s z-test, p < 0.015). For ORRT, the optimal multi-task predictions significantly outperformed single task predictions of WM, relational, gambling, social, rest, motor, and emotion (Steiger’s z-test, p < 0.01), but not significantly for language task (Steiger’s z-test, p = 0.15).

3.4. Reading comprehension-predictive edges

The predictive models were similar across conditions, with within-condition similarities (r[FC weights] = 0.654–0.757, r[node weights] = 0.622–0.734) higher than between-condition similarities (r[FC weights] = 0.185–0.289, r[node weights] = 0.224–0.338; Fig. 3a and b). Under permutation test, the resting state identified 481 and 565 edges for PVT and ORRT, showing diffuse patterns widely spanning the entire brain (Fig. 4a, Fig. 5). Nodes within the DMN and their connections with frontoparietal regions were overrepresented for PVT model, while regions within visual network and their connections with DMN and subcortical network were overrepresented for ORRT model (Fig. 4a). By contrast, FCs derived from language and WM tasks exhibited considerably denser patterns and greater degrees in predominantly language processing-related regions (Fig. 5). Specifically, the language task detected 477 and 487 edges for PVT and ORRT. Regions in the DMN and frontoparietal network exhibited the greatest involvement in HCP language-based models for both PVT and ORRT (Fig. 4b). The WM task revealed 411 and 262 FCs for PVT and ORRT (Fig. 4c). The DMN, visual and frontoparietal networks were overrepresented for PVT model, while the ventral attention and frontoparietal networks were overrepresented for ORRT model. When generating predictive FCs using a conservative threshold of p < 0.01, we derived similar patterns (Figs. S8 and S9). Moreover, consistent with the original study, models demonstrated a substantial edge overlap between each pair of conditions (shared edges ranged from 11 to 127, Fig. 3b).

Fig. 3.

(a) Similarity of FC or node weights within (main diagonal) and between-conditions (off-diagonal). The similarity was quantified by the correlations of whole-brain FC or node weights between each condition pairs. The weight distributions demonstrated higher within-condition similarities (r [FC weights] = 0.654–0.757, r[node weights] = 0.622–0.734) than between-condition similarities (r [FC weights] = 0.185–0.289, r[node weights] = 0.224–0.338). (b) Edge overlap within (main diagonal) and between (off-diagonal) each pair of fMRI conditions. Cells in the matrix plots are plotted as number of shared edges within and between each pair of cognitive conditions. Values in the row or column names represent the number of edges identified by permutation test under a threshold of p < 0.05. The PVT and ORRT models are represented in the lower and upper triangles respectively.

Fig. 4. The FCs that contributed to reading comprehension prediction and the overrepresented networks.

The most predictive FCs determined by permutation test were demonstrated in the circle plot for cognitive conditions of (a) rest, (b) language and (c) working memory. As shown in the circle plot, functional edges for PVT and ORRT are visualized in orange and blue, respectively. Cells in the matrix plots are plotted as the fraction of the most significantly predictive FCs in each pair of canonical networks, normalized by the fraction of total edges belonging to that pair. A value > 1 indicated overrepresentation of that network pair to the prediction model. Abbreviation: VIS, visual; MOT: somatomotor; DAN: dorsal attention network; VAN: ventral attention network; LIM: limbic; FPN: frontoparietal network; DMN: default mode network; SUB: subcortical.

Fig. 5.

The number of significantly predictive edges between each macroscale brain region pair determined by permutation tests under the threshold of p < 0.05. As shown in the circle and matrix plot, the 246 FC nodes are grouped into 24 macroscale brain regions that are anatomically defined by the Brainnetome atlas. Cells in the matrix plots are plotted as number of edges within and between each pair of brain regions.

3.5. Generalizability of predictive models across fMRI conditions

The cross-condition generalizability was successful across all condition pairs (p < 0.001, FDR corrected), with accuracies ranging from r = 0.188 (relational to rest) to r = 0.315 (rest to language) for PVT (Fig. 6a), and from r= 0.172 (language to motor) to r = 0.336 (rest to language) for ORRT (Fig. 6b).

Fig. 6.

The identified reading comprehension predictive models demonstrated a robust generalizability across fMRI conditions. Namely, the prediction models built on one fMRI state could be applied to FC data from other different conditions to predict (a) PVT and (b) ORRT with appreciable accuracy.

4. Discussion

In this study, we successfully replicated the findings of Greene et al. by demonstrating that task-based models generally outperformed rest-based models in cognition prediction, especially when using equivalent scan durations. Additionally, we found that combining multi-task fMRI improved prediction performance; however, integrating more fMRI conditions cannot ensure the higher accuracy, which replicated the results of Gao et al. Moreover, the reading comprehension-predictive models can be generalized across distinct cognitive conditions. Together, our results demonstrated the superiority of using task-based FCs to predict individual cognitive traits, which may confer more predictive power and promote the detection of individual differences of FC patterns in cognitive-specific and cognitive-general networks. Although there are some difference between the original studies and our current study in terms of the employed prediction method and the investigated cognitive measures, our results are supportive of their findings, providing strong evidence for the validity and robustness of the original findings.

4.1. Task-induced FC may better predict individuals’ cognitive traits

Although both resting and task-based FCs have been used to characterize the human brain and cognitive abilities in health and psychiatric disease, resting state has been playing a dominating role in the field (Finn et al., 2017). In this study, we confirmed the findings of Greene et al. by employing a different prediction method and applying it to predict two reading comprehension measures. Consistent with their results, we found that models built from task conditions (e.g., language, WM) predicted 2 reading-related out-of-scanner cognitive traits better than those built from rest data, implying that the resting state might be not the optimal condition for probing individual differences. There are three explanations for this.

First, the unconstrained nature of rs-fMRI brings many uncertain parameters. Across reported studies, resting-state scans were variably performed with eyes open or closed, with visual fixation points or without, and with various types of subjects’ instructions, all of which can substantially affect the acquisition of scans (Buckner et al., 2013; Zou et al., 2009). Considering that no explicit input and output is specified, it is hard to distinguish between meaningful trait-related variance and less interesting state-related components from the individual resting-state signal (Finn et al., 2017). By contrast, participants are more engaged during task conditions. Consequently, more of the brain is devoted to task-relevant processes, promoting the task effect through more of the connectivity matrix in a way that reduces the task-irrelevant non--stationarities like noise. Consistently, a recent study also concluded that tasks afforded a better ratio of within-to between-subject variability by bringing meaningful idiosyncrasies across participants, which can significantly enhance the individual signal in functional networks of interest beyond what can be measured at rest (Finn et al., 2017).

Second, limitations of resting state also include the susceptibility to head motion artifacts and participant drowsiness or sleep (Vanderwal et al., 2015), both of which can alter FC patterns dramatically (Boly et al., 2012; Power et al., 2015). Tasks can facilitate holding participants’ attention with less risk of falling asleep, and subjects practically are inclined to move less during imaging if they are engaged in a task. Participants may devote more attention to the task-relevant processing, and consequently more individual difference in FC patterns can be preserved (Finn et al., 2017). When comparing the difference in head motion between rest and tasks, we found that rest had higher mean FD values than five of the seven task conditions (only two were statistically significant) (Table S3).

Finally and most importantly, tasks can perturb functional connections in the brain and further amplify individual differences in the neural circuitry underlying related traits (Greene et al., 2018). Numerous studies have pointed that the brain would reveal a stable intrinsic network architecture during resting state, while, this standard state of brain origination can be modified as necessary to meet task demands during task conditions (Cole et al., 2014). Consequently, more context-specific modulations would be made on FCs that are directly related to the given task (Mennes et al., 2013). In our results, two language-relevant tasks (language and WM), which were among those with the highest cognitive demands, achieved the best prediction performance of reading comprehension abilities. However, models built from rest and less reading-relevant tasks (e.g., emotion, motor) achieved relatively lower prediction accuracies. Probably, modifications of functional organization evoked by these less reading relevant tasks contributed nothing or even negatively to the prediction. And this might also be the reason that integrating more fMRI conditions did not necessarily ensure higher prediction accuracy. Since task conditions provide state-dependent aspects, distinct tasks may exert different but specific modifications to the brain network topology properties (Geerligs et al., 2015). Combining multi-task FCs could leverage the complementary information encoded in each condition, yet, redundant or overlapped information brought by some tasks may impair the predictive power.

Interestingly, the predictive models demonstrated higher within-condition similarities and greater edge overlap than between-condition similarities and edge overlap. This is probably because these two reading comprehension measures are similar and closely related (r = 0.69). Additionally, this may suggest that the magnitude of FC changes generated by tasks is greater than that generated from different reading measures.

Moreover, the predictive FCs derived from the language and WM tasks, compared with those derived from rest, exhibited a denser pattern in predominantly default mode, and frontoparietal networks, and brain nodes implicated in these networks were highlighted with more predictive power. These networks may gain its multi-function ability by rapidly updating their patterns of global FC according to task demands and flexibly interacting with various functionally specialized networks throughout the brain (Cole et al., 2010; Cole and Schneider, 2007). In support with our findings, a study investigating the task-evoked network properties found that resting-state networks tended to be more broadly distributed, while networks during semantic decision making were more distinct and interconnected, especially in language-related areas (DeSalvo et al., 2014). They posited that this reorganization in functional networks during the semantic decision task reflected the involved cortical computations including phonetic, lexical, orthographic and semantic processing. Collectively, these results suggest that rs-fMRI discloses only part of the relevant individual differences in brain function, while studying individual differences across a wider range of cognitive conditions will provide a more complete understanding of the brain’s functional architecture (Geerligs et al., 2015; Mennes et al., 2013).

4.2. Generalizability of prediction models across distinct cognitive conditions

Our results demonstrated that connectome-based prediction models have a robust generalizability across distinct cognitive conditions. Importantly, the model generalization was successful across all possible pairs of cognitive conditions, which may be due to the great correspondence between them (Betti et al., 2013; Krienen et al., 2014). A wealth of independent studies converged in demonstrating a grossly high degree of spatial overlap between functional networks estimated at rest and across distinct tasks, with a shared variance in FC reaching 80% (Krienen et al., 2014; Mill et al., 2017). These findings suggested the existence of a stable functional backbone that was present across rest and tasks (Fox and Raichle, 2007; Vincent et al., 2007), and the intrinsic FC pattern did not substantially change under distinct states of consciousness or task engagement (Cole et al., 2014; Greicius et al., 2008; Larson-Prior et al., 2009), which was potentially driven by anatomical connectivity between regions (Dosenbach et al., 2007). We speculated that it is this stable intrinsic architecture that drives the noted high correspondence in FC patterns between distinct cognitive conditions and consequently ensures the successful cross-condition generalization. However, compared with predictions by training and testing with the same type of fMRI condition (Fig. 1), accuracies achieved in the cross-condition predictions were significantly attenuated (p < 10⁻¹⁰), presumably due to the subtle changes brought by different task conditions as mentioned above.

4.3. Biological implications of the predictive functional patterns

Apart from demonstrating that FCs derived from both rest and task conditions are strongly predictive of individual cognitive traits, our results also support the recent findings of reading comprehension abilities as emerging from the coordination of several core language-related regions. Generally, the reading comprehension constitutes multifaceted and complex skills that can break down at several levels including acoustic-phonological analysis, local synaptic structure building, lexical-semantic processes and domain-general processes such as executive functioning (Binder et al., 2009; Fedorenko and Thompson-Schill, 2014; Herrmann et al., 2011; Leaver and Rauschecker, 2010). Over the years, a myriad of works on the neurobiological basis of reading comprehension have accumulated, establishing that the neural circuits supporting language functions primarily locate in the prefrontal and temporal cortices, which are connected via ventral and dorsal pathways (Brauer et al., 2013; Friederici and Gierhan, 2013). Consistently, predictive functional patterns identified in our results converged well on the core regions in the language-processing networks, including STG, IFG, MTG, angular gyrus, fusiform gyrus, and some regions in the executive control and default mode networks (Fedorenko and Thompson-Schill, 2014; Friederici, 2012).

4.4. Limitation and future directions

Some issues need to be mentioned. First, our discovery cohort came from the HCP S500 release, instead of the newly released S1200, which incorporated more subjects. The huge computational burden was the major concern that prevented us from using the larger sample datasets. Apart from the multitask prediction and permutation test, we had to repeat our prediction procedure for more than 10000 times: 100 repetitions × 8 cognitive conditions × (2 reading measures × 6 control analyses + 3 other cognitive measures). Future studies can be performed on larger study samples whenever more computational resources are available. In addition, a recent study showed that predictive models would have a high stability once the sample size reached 300 (Cui and Gong, 2018). Accordingly, our current prediction models built on a sample of 463 participants achieved relatively high prediction accuracies and results remained largely unchanged when controlling for a series of potential confounds. Second, we didn’t perform global signal regression (GSR). As one of the most contentious and debated preprocessing strategy, there was no consensus regarding physiological interpretation of GSR (Murphy et al., 2009). Moreover, previous studies found that GSR discarded globally distributed neural information and introduced negative correlations (Ben Simon et al., 2017; Matsui et al., 2016). Finally, the multi-task predictions were implemented by simply concatenating FCs from different fMRI conditions horizontally into a combined feature space. To maximally capitalize on the strength of each cognitive condition, future studies can develop novel methods to combine these conditions (Elliott et al., 2019). Additionally, multimodal neuroimaging features can also be leveraged to achieve improved predictions (Sui et al., 2018).

5. Conclusions

In summary, this replication study highlights the benefit of using task-based FCs to reveal brain-behavior relationships, which may confer more predictive power and promote the detection of individual differences of FC patterns underlying related cognitive traits, adding strong evidence for the validity of the original studies. Besides, our results show high stability of connectome-based prediction models to be generalizable across distinct cognitive states. Overall, our results suggest that task-based FC patterns could serve as potentially effective predictors of human cognitive traits.