Functional connectome stability and optimality are markers of cognitive performance

Abstract

Patterns of whole-brain fMRI functional connectivity, or connectomes, are unique to individuals. Previous work has identified subsets of functional connections within these patterns whose strength predicts aspects of attention and cognition. However, overall features of these connectomes, such as how stable they are over time and how similar they are to a group-average (typical) or high-performance (optimal) connectivity pattern, may also reflect cognitive and attentional abilities. Here, we test whether individuals who express more stable, typical, optimal, and distinctive patterns of functional connectivity perform better on cognitive tasks using data from three independent samples. We find that individuals with more stable task-based functional connectivity patterns perform better on attention and working memory tasks, even when controlling for behavioral performance stability. Additionally, we find initial evidence that individuals with more typical and optimal patterns of functional connectivity also perform better on these tasks. These results demonstrate that functional connectome stability within individuals and similarity across individuals predicts individual differences in cognition.

Introduction

Although essential for daily life, individual abilities vary for cognitive processes such as sustaining attention across time and holding items in working memory. Identifying brain-based markers of such abilities has been a goal of much recent work in cognitive neuroscience. Brain-based predictive modeling may provide insight both into mechanisms broadly underlying cognitive ability as well as the ways in which individuals meaningfully differ.

Network neuroscience posits that cognitive processes arise from large-scale interactions between distributed sets of brain regions, and work characterizing these interactions has found evidence that they relate to behavior (for a review, see Bassett and Sporns 2017). In particular, network neuroscientific approaches have demonstrated that whole-brain fMRI functional connectivity predicts individual differences in behavioral phenotypes using methods such as connectome-based predictive modeling (CPM; Rosenberg et al. 2016; Shen et al. 2017). In CPM, a set of functional connections between brain regions are identified to serve as model features for a particular predicted phenotype. This method has successfully been implemented to predict cognitive processes and behavioral measures, including sustained attention, fluid intelligence, processing speed, and working memory (Finn et al. 2015; Rosenberg et al. 2016; Avery et al. 2020; Gao et al. 2020; Yoo et al. 2022a). The growing list of CPM’s predictive applications supports functional connectivity’s utility as a predictive tool for a diverse set of outcome variables.

In addition to these supervised network-based predictive models, models based on individual- or group-related summary features of individuals’ whole-brain patterns of functional connectivity, or connectomes, may predict aspects of cognitive and attentional function. One approach to quantifying connectome features is characterizing their network properties using graph theory measures. These network properties, which characterize interactions between brain regions, have been related to variation in cognitive abilities such as general intelligence (van den Heuvel et al. 2009; Hilger et al. 2017) and working memory (Stanley et al. 2015), providing further support that differences in structural and functional brain organization underlie behavioral and cognitive performance. Here, we take a complementary approach, investigating whether similarities—or differences—in task-based network patterns within and across individuals relate to cognitive ability. Specifically, we define features of the whole-brain connectome that may meaningfully vary across individuals: connectome similarity to oneself (stability), connectome similarity to others (typicality and optimality), and their relationship with each other.

Connectome stability reflects the similarity of one’s own functional connectivity pattern over time. Despite evidence that the connectome is stable enough to function as a “fingerprint” (Miranda-Dominguez et al. 2014; Finn et al. 2015; Gratton et al. 2018), there is some variability in stability across individuals which may be useful for prediction. Previously, resting-state stability, measured as the similarity of an individual’s connectome or network connectivity across repeated fMRI runs, was related to cognitive and social function in adolescents and young adults (Vanderwal et al. 2021; Fu et al. 2022), such that higher connectome stability was associated with better cognitive performance and social ability. Furthermore, connectome stability increases over the course of development but is reduced for individuals with psychiatric disorders (Kaufmann et al. 2017; Vanderwal et al. 2021; Fu et al. 2022). As functional connectivity patterns dynamically shift to reflect changes in cognitive state in response to task (Gonzalez-Castillo et al. 2015; Xie et al. 2018), one possibility is that connectome stability reflects the variability of an individual’s cognitive states over time, such that more stable connectivity patterns between repeated resting-state or task runs may reflect a more on-task cognitive state which results in higher task performance. Additionally, stability (i.e., similarity) between resting-state and task connectomes may reflect less task-specific reconfiguration which has previously been related to cognitive function (Schultz and Cole 2016). Here, we examine whether stability across task runs or between task types predicts cognitive performance.

Connectome typicality measures the similarity of an individual’s connectome to the average connectome of others in a group. For the current study, we hypothesized two potential ways in which connectome typicality might relate to cognitive performance. First, previous work has shown that activity in large-scale brain networks (e.g., the frontoparietal network) relates to attention and working memory performance in children and young adults (Satterthwaite et al. 2013; Rosenberg et al. 2020). Therefore, to the extent that a similar task-related pattern of connectivity is captured in the group-average, the degree to which an individual resembles others may be indicative of task-related processing and predictive of performance. Indeed, connectome typicality during movie watching was predictive of social function in development (Vanderwal et al. 2021).

This hypothesis about how connectome typicality relates to performance assumes that the group-average task connectome captures a meaningful “template” pattern of task-specific functional connectivity. However, it is also possible that patterns related to high task performance get lost in the group average. Instead, perhaps, a better proxy of a task-optimal template of connectivity is the average connectome of individuals who perform best on a given task. To investigate this possibility, we calculated each participant’s connectome “optimality,” which reflects the similarity of one’s average task connectome to the average connectome of the highest task performers. A relationship between task connectome optimality and performance would suggest that high performers show patterns of connectivity that are ideal for performing particular cognitive tasks.

Finally, the relationship between connectome stability and typicality may also predict cognitive performance. Individuals whose connectomes are more similar to themselves than they are to others are more easily discriminated from the group, and this uniqueness could hypothetically reflect an individualized strategy by which an individual accomplishes a task. Differences in this discriminability, defined in the current study as the ratio of one’s similarity to oneself vs. one’s similarity to others, may also be meaningful for predicting individual phenotypic differences. Indeed, previous work has found that connectome discriminability increases over development but is delayed in individuals with greater numbers of psychiatric symptoms (Kaufmann et al. 2017), suggesting that connectome distinctiveness may capture variance in behavior. Although complementary work has emphasized the importance of connectome discriminability in prediction such that better identifiability may improve predictive power (Amico and Goñi 2018; Elliott et al. 2019), other research has demonstrated more limited utility of connectome individuation for phenotype prediction (Finn et al. 2017; Noble et al. 2017; Greene et al. 2018) and suggested that the functional connections that contribute to identifiability are distinct from those that best predict behavior (Mantwill et al. 2021). While the relationship between discriminability and prediction is an ongoing topic of discussion, the question of whether greater discriminability relates to cognitive ability has not been explored. Here, we investigate this relationship, asking whether discriminability adds unique predictive power above and beyond its component parts, stability and typicality.

While previous literature has investigated the connectome’s predictive utility, existing studies have typically focused on the strength of sets of functional connections (e.g., CPM, machine learning approaches) or graph theoretic measures rather than connectome similarity within and between individuals. The current study tests whether overall features of the connectome—stability, typicality, optimality, and discriminability—predict cognitive performance in adults. In particular, we investigated the critical cognitive abilities of sustained attention, the ability to maintain attention over time, and working memory, the capacity to hold information in mind. We identified three datasets whose task battery included repeated fMRI runs of a sustained attention and/or working memory task. We then constructed linear models to determine which connectome features—stability, typicality, optimality, or discriminability—best predict cognitive performance across datasets. Finally, we examined whether these whole-brain features of the connectome offer additional predictive power to previously validated network-based models.

Methods

We tested the extent to which overall features of the functional connectome, similarity to oneself (connectome stability), similarity to others (connectome typicality and optimality), and the ratio of stability to typicality (connectome discriminability, or stability ÷ typicality), were related to performance on sustained attention and working memory tasks (Fig. 1). To evaluate the replicability of any effects, we performed analyses on three independent datasets. In each dataset, participants performed a sustained attention task during at least two fMRI runs. In two of the three datasets, participants also performed two runs of a working memory task. All data were collected with IRB approval and secondary analysis was approved by the University of Chicago Institutional Review Board.

Fig. 1.

Connectome features were calculated by correlating vectorized task connectomes across runs (stability) or people (typicality and optimality). Connectome discriminability was calculated as the ratio between fisher-normalized connectome stability and typicality values. The relationship of these features with task performance was determined using partial Spearman’s rank correlation accounting for in-scanner head motion.

Data

Dataset 1. In Dataset 1 (n = 25), described in detail in Rosenberg et al. 2016, sustained attention was measured by performance on a gradual-onset continuous performance task (gradCPT; Esterman et al. 2013). Participants (13 females, ages 18–32 years, mean age = 22.7 years) performed three gradCPT runs in the same scan session (13:44 min/run). Two 6-min resting-state runs (one before and one after the gradCPT runs) were also collected. Task data are missing or were excluded for excessive head motion from one gradCPT run for six participants. For these participants, analyses are performed on their two available runs. Additionally, one resting-state run is missing or excluded for excessive head motion for two participants.

Each gradCPT run consisted of four 3-min blocks interleaved with 32-s rest periods. During the task, grayscale images of city and mountain scenes gradually transitioned from one to the next over a period of 800 ms. Participants were instructed to respond with a button press to city scenes (90% of trials) but withhold a response to mountains (10%). Sustained attention performance was operationalized as sensitivity (d’), calculated as z(hit rate) − z(false alarm rate), averaged over each participant’s two or three available runs. Behavioral inclusion criteria were defined a priori as only those participants whose performance was within 2.5 standard deviations from group mean performance. No participants were excluded from Dataset 1 based on this criterion.

MRI data were collected on a 3T Siemens Trio TIM system equipped with a 32-channel head coil. Preprocessing was performed as described in Rosenberg et al. (2016) using BioImage Suite. Data were motion corrected using SPM8. Linear and quadratic drift, mean signal from cerebrospinal fluid, white matter, and gray matter, and a 24-parameter motion model including 6 motion parameters, 6 temporal derivatives, and their squares were regressed from the data. Additionally, data were temporally smoothed with a zero mean unit variance Gaussian filter.

Dataset 2. In Dataset 2 (n = 94), volunteers (61 females, ages 18–36 years, mean age = 23.1 years) participated in two separate scanning sessions, collected ~2 weeks apart (mean = 17.2 days, SD = 20.0 days). Participants were drawn from the dataset described in Yoo et al. (2022a).

As in Dataset 1, sustained attention was assessed with performance on two runs of the gradCPT task (10 min/run). Unlike in Dataset 1, gradCPT runs were collected during different days rather than during the same scan session. Dataset 2 also included a visual short-term memory task (VSTM; two 10-min runs collected on different days), which measures visual working memory (Luck and Vogel 1997). In the VSTM, an array of 2, 3, 4, 6, or 8 colored discs was presented on the screen for 100 ms. Following the presentation and a 900-ms fixation cross, a second array was presented either with or without a color change, with a color change occurring on 50% of trials. Participants were given 2,000 ms to respond with a button press if they detected a color change. Working memory performance on the VSTM was measured with Pashler’s K (Pashler 1988). In addition to the gradCPT and VSTM, participants performed a multiple object tracking task (MOT; 10 min/run), viewed the naturalistic movie Inscapes (Vanderwal et al. 2015, 2021; 7:16 min/run), and rested (10 min/run).

For sustained attention and working memory analyses separately, we defined inclusion criteria as only those participants whose performance was within 2.5 standard deviations from the group mean performance. No participants were excluded from Dataset 2 based on this criterion. Additionally, participants were excluded from stability and discriminability analyses if they did not complete the first or second sustained attention or working memory run because connectome stability and discriminability could not be calculated for these individuals. This resulted in a sample size n = 65 for sustained attention and n = 72 for working memory analyses, and 56 participants were included in stability and discriminability analyses of both tasks. For analyses of connectome typicality, the full sample size n = 94 was used. Finally, connectome optimality analyses were conducted on a sample of n = 89 which corresponded to the full sample minus the top performers in either the sustained attention or working memory task, whose connectomes were averaged as the “optimal” pattern in analyses. If participants only completed one sustained attention or working memory run, task-specific functional connectivity was calculated using one run. Otherwise, functional connectivity matrices were averaged between sustained attention or working memory runs.

Functional MRI data were collected on a 3T Siemens Prisma system with a 64-channel head coil. Preprocessing was performed with AFNI (Cox 1996) and included the removal of the first three volumes and censoring of volumes with outliers in more than 10% of voxels and those for which the Euclidean norm of the head motion parameter derivatives were >0.2 mm. Data were despiked, slice-time corrected, and motion corrected. Mean signal from the CSF, white matter, and whole brain was regressed from the data, as well as 24 motion parameters. Finally, data were aligned to a high-resolution anatomical image (MPRAGE) and normalized to MNI space.

Dataset 3. Dataset 3 (n = 316) included data provided by the large-scale open-source Human Connectome Project S1200 release (HCP; Van Essen et al. 2013; Glasser et al. 2013). All fMRI data were acquired on a 3T Siemens Skyra scanner. Participants performed seven tasks (emotion, gambling, language, social, motor, N-back, and relational) and completed two resting-state runs over two-day visits. Each condition involved two runs with opposite phase encoding directions (LR and RL) which were completed in the same scanning session. Data were minimally preprocessed using HCP pipelines (Barch et al. 2013; Glasser et al. 2013). Additionally, the first 15 volumes of each run were discarded and nuisance covariates were regressed from each run, including 24 motion-related parameters (6 translational and rotational motions, 6 derivatives, and their squares), three mean tissue signals (global, white matter, and cerebrospinal fluid), and linear and quadratic trends (Yoo et al. 2022b).

Our analyses included data from 316 participants (154 females, ages 22–36+ years) who completed all nine fMRI conditions with low head motion in all runs (<3-mm translation, <3° rotation, and <0.15-mm mean frame-to-frame displacement), had a behavioral fluid intelligence score, and were unrelated to any of the other subjects included in the sample. Fluid intelligence scores were required for inclusion in separate analysis (Yoo et al. 2022b) but were not used in the current study. Relatedness was based on family structure verified with genetic information, and one subject was randomly selected from each family to be included in the sample.

Sustained attention performance in the HCP sample was operationalized as percent accuracy on 0-back blocks of the N-back task. During 0-back blocks, a target cue was presented at the start of the block and participants were instructed to respond with a button press any time the target reappeared. With this design, the 0-back, or target-detection, task functioned as a type of continuous performance task (in which participants make one response to stimuli in a frequent category and another to rare targets), typically used to measure sustained attention. Working memory performance was measured with performance on 2-back runs of the N-back task, during which participants responded with a button press if an image was the same as the image shown two trials previously. The N-back task consisted of two 5-min runs with eight blocks per run. Of these, four blocks were 0-back blocks and four were 2-back blocks. Both 0-back and 2-back blocks contained 10 trials per block, and a target image was displayed on two trials, with two–three nontarget lures displayed on other trials. Images were presented for 2 s with a 500-ms intertrial interval.

As in Datasets 1 and 2, participants were excluded if behavioral performance was greater than 2.5 standard deviations below mean performance for sustained attention or working memory runs. As a result, sustained attention and working memory analyses included 304 and 311 participants, respectively, and 300 participants were included in both analyses.

Functional connectivity calculation

Functional connectivity was calculated in the same way for all three datasets. For each task run, functional data were parcellated using the 268-node Shen functional atlas (Shen et al. 2013) and the time series for all voxels within a parcellation were averaged. In Dataset 3, because 0-back and 2-back blocks were collected in the same fMRI run, time series were constructed by concatenating 0-back and 2-back blocks separately. A lag of 8 TRs (~5.76 s; Lee et al. 1995) was used to adjust for the hemodynamic delay. Then, the Pearson correlation was calculated between all pairs of averaged time series and Fisher z-transformed. Each value in the resulting 268 × 268 matrix, referred to as an “edge,” reflects the strength of the connection between two brain regions within an individual. To test whether results depended on parcellation scheme, we also replicated our main analysis in Dataset 1 using functional connectivity matrices constructed using the 122-node Yeo functional atlas (Yeo et al. 2011). These results are included in Supplementary Table 1.

Measuring features of the connectome

Connectome stability

We assessed the stability of individuals’ functional connectivity patterns across fMRI runs, controlling for head motion inside the scanner. Our primary analyses focused on whether within-task stability—that is, similarity between repeated sustained attention or working memory runs—was related to performance on the respective task. As a secondary analysis, we investigated whether stability within or between unrelated task or resting-state runs reflected sustained attention and working memory performance.

To calculate connectome stability, we first vectorized connectomes by flattening the lower triangle of the symmetric 268 × 268 whole-brain functional connectivity matrix from each participant and fMRI run, resulting in a vector of 35,778 edges. Stability of sustained attention task run connectomes was calculated for each participant as the Pearson correlation between vectorized connectomes (i.e., mean pairwise correlation value for three gradCPT runs for Dataset 1, two gradCPT runs for Dataset 2, 0-back blocks from two N-back runs for Dataset 3). Stability of working memory task run connectomes was calculated for each participant as the Pearson correlation between vectorized working memory connectomes (i.e., two VSTM runs for Dataset 2, 2-back blocks from two N-back runs for Dataset 3). Additionally, overall stability values were obtained for each participant. These were calculated as the mean Pearson correlation value between all pairs of runs for a dataset. Pearson correlation coefficients were Fisher’s z-transformed before averaging. Pearson correlation was used as our measure of stability in keeping with previous work (Kaufmann et al. 2017; Vanderwal et al. 2021) and because we were interested in relative rather than raw edge strength values which could vary across sessions and site due to scanner-related and other noise.

We measured the relationship between connectome stability and task performance by calculating the partial Spearman rank correlation between all pairs of runs, including terms for average frame-to-frame head displacement during both runs being correlated as well as the absolute difference in displacement between runs. As a note, we used Pearson correlation to calculate relationships between functional connectivity patterns, as we expected these values to be linearly related to one another. We used Spearman rank correlation to relate characteristics of functional connectivity patterns (e.g., stability and typicality) and to relate these characteristics with performance, as we expected these variables to be monotonically but not necessarily linearly related.

To test whether connectome stability was related to overall task performance and not consistency in task performance, we conducted an additional control analysis. Again, we calculated the partial correlation between stability and performance but included a term controlling for absolute difference in task performance between repeated runs of either the sustained attention or working memory task. As a second control, we calculated the partial Spearman rank correlation between connectome stability and performance on only one sustained attention or working memory run—i.e., the first, second, or third run, again controlling for mean head motion and change in head motion between runs. These control analyses were conducted on Datasets 1 and 2 because they required run-specific behavioral data which was unavailable for Dataset 3.

Connectome typicality

Do better performers show a more typical task-based connectome—i.e., a connectome that is more similar to the group average? Previous work suggests that cognitive tasks such as the N-back task elicit characteristic patterns of activation across individuals (Satterthwaite et al. 2013; Rosenberg et al. 2020). If these patterns are reflected in the group average connectome, the extent to which an individual resembles the group may reflect task-related processing and therefore performance. Alternatively, connectome typicality may be indicative of more typical task performance such that better and worse performers have less typical patterns of connectivity. To quantify typicality, we first found an individual’s mean functional connectivity pattern for a given run type by averaging connectomes across repeated runs of either sustained attention or working memory tasks. Connectomes were vectorized by extracting the lower triangle of the matrix. Finally, typicality values were calculated for each participant by computing the Pearson correlation between their vectorized connectome and the mean connectome of all other participants within each dataset. This process was repeated until every participant served as the left-out participant. This resulted in two typicality values for each participant: typicality on sustained attention runs and typicality on working memory runs. We tested whether connectome typicality was related to better or more typical task performance by calculating the correlation between connectome typicality values and participants’ overall task performance scores or their absolute difference in performance from the mean.

Connectome optimality

Perhaps taking an average across the group adds noise and does not reflect a theoretical “optimal” task-specific pattern of connectivity. Instead, similarity to the average connectome of the top performers—rather than similarity to the group average—may be a better indicator of task performance. We tested this by calculating the similarity of each participant’s mean task-specific connectome (calculated by averaging functional connectomes over repeated task runs) to the mean connectome of participants who performed best on either the sustained attention or working memory tasks. This optimal functional connectivity pattern was calculated as the average task connectome (from either sustained attention or working memory runs) of the top 5% of performers in each task. The resulting “optimal” connectome was drawn from the top two participants in the Dataset 1 sustained attention task, and the top five performers in the Dataset 2 sustained attention and working memory tasks, respectively. In Dataset 3, 46 participants achieved the top behavioral performance score (100%) on the sustained attention task so the optimal connectome was calculated as the average across these performers. For the working memory task in Dataset 3, 19 participants achieved the top 5% of scores. Connectome optimality values (i.e., similarity of participants’ mean task-specfic connectome to the optimal connectome) were not calculated for top performers included in the optimal connectome. For comparison, optimality analyses were repeated using the average of performers who achieved the single highest score and the top 10% of performers as optimality thresholds. Results from these analyses are included in Supplementary Table 4.

Connectome discriminability

Finally, it might be the case that performance on sustained attention and working memory tasks is best explained not by the stability of an individual’s task-based connectome nor by the similarity of their connectome to others’, but by a relationship between the two. For example, previous work has suggested that connectome distinctiveness, or the extent to which an individual’s connectome can be identified from the group, is lower in adolescence and early adulthood for individuals with increased neurological dysfunction, such as symptoms of attention deficit disorder and depression and prodromal symptoms of schizophrenia (Kaufmann et al. 2017).

Here, we were interested in whether differences in connectome distinctiveness, which we term discriminability, are observed in the nonclinical adult populations as well and, further, whether they relate to sustained attention and working memory performance. To investigate this possibility, we calculated the discriminability of each participant’s sustained attention and working memory connectomes, or the similarity of an individual’s connectome across fMRI runs relative to their similarity to the group within each dataset. Discriminability was defined as the ratio of an individual’s connectome stability (similarity to oneself) to their connectome typicality (similarity to all other participants) for sustained attention and working memory runs separately. Pearson r values reflecting stability and typicality were Fisher-z transformed before their ratio was taken.

Discriminability is related to subject identifiability or distinctiveness, which have previously been quantified as classification accuracy (Kaufmann et al. 2017), or whether an individual’s connectivity most closely resembles oneself across runs (Finn et al. 2015). While these measures are conceptually related, each individual’s connectome identifiability is a binary (because their functional connectivity pattern in one run is either most similar to their own pattern in another run or not), whereas their discriminability is continuous. In the datasets included in the current study, connectome identifiability and discriminability were significantly related for both sustained attention (r_s = 0.437, P = 2.14^*10⁻²⁰) and working memory runs (r_s = 0.533, P = 7.80^*10⁻³⁰). We use discriminability rather than identifiability here to ask whether an individual’s behavior scales with their relative within-subject vs. between-subject variation in functional connectivity patterns.

Modeling behavior across datasets

We evaluated the unique variance in sustained attention and working memory task performance explained by connectome features using mixed-effects models. We constructed eight models to determine which model best predicted performance across datasets. To control for differences between datasets, behavioral performance was z-scored within dataset and stability, typicality, and discriminability values were z-scored across datasets.

We first constructed four separate models that included a predictor of stability, typicality, optimality, or discriminability with a random intercept term for dataset. Additionally, these models included covariates for mean in-scanner motion and difference in motion between runs. Next, to examine whether similarity within and across individuals explain unique variance in performance, we constructed four additional models. To compare the unique contributions of stability and typicality, we built a model with fixed effects of stability and typicality and another model with effects of stability, typicality, and their interaction. Next, we constructed two additional models comparing stability and optimality—one model with fixed effects of stability and optimality and a second model with stability, optimality, and their interaction. Models did not include both typicality and optimality predictors because these features were highly correlated across participants (Table 1). Additionally, because discriminability is the ratio of stability and typicality, we did not include a discriminability term in these models. However, the interaction term of stability and typicality may be theoretically related to discriminability. Models were constructed using R’s lmer function.

Table 1.

Across-subject Spearman’s correlation values between connectome features. Significance values are indicated with asterisks.

Dataset 1
		Stability			Typicality			Optimality
		rho	sig.		rho	sig.		rho	sig.
Typicality	Sustained attention	0.2446	0.2376
Optimality	Sustained attention	0.4941	0.0166	*	0.4990	0.0154	*
Discriminability	Sustained attention	0.8477	<0.001	* * *	−0.2577	0.2128		0.2510	0.2480
Dataset 2
		Stability			Typicality			Optimality
		rho	sig.		rho	sig.		rho	sig.
Typicality	Sustained attention	0.2580	0.0380	*
	Working memory	0.2740	0.0198	*
Optimality	Sustained attention	0.4818	<0.001	* * *	0.8498	<0.001	* * *
	Working memory	0.3304	5.930*10−3	* *	0.9150	<0.001	* * *
Discriminability	Sustained attention	0.8389	<0.001	* * *	−0.2264	0.0697		0.1076	0.4052
	Working memory	0.8057	<0.001	* * *	−0.2569	0.0294	*	−0.1267	0.3031
Dataset 3
		Stability			Typicality			Optimality
		rho	sig.		rho	sig.		rho	sig.
Typicality	Sustained attention	0.7049	<0.001	* * *
	Working memory	0.6743	<0.001	* * *
Optimality	Sustained attention	0.6926	<0.001	* * *	0.9853	<0.001	* * *
	Working memory	0.6635	<0.001	* * *	0.9871	<0.001	* * *
Discriminability	Sustained attention	0.8735	<0.001	* * *	0.3523	<0.001	* * *	0.3321	<0.001	* * *
	Working memory	0.9135	<0.001	* * *	0.3951	<0.001	* * *	0.3945	<0.001	* * *

^* P < 0.05.

^{* *} P < 0.01.

^{* * *} P < 0.001.

Contributions of connectome features above and beyond CPMs

In addition to testing whether connectome stability, typicality, and optimality independently predict performance on cognitive tasks, we asked whether they provide additional utility by improving the predictive power of previously established CPMs. Such models, which identify functional connections whose strength scales with behavior and use the strength of these connections to predict behavior in novel individuals, have been validated for prediction of both sustained attention (Rosenberg et al. 2016) and working memory performance (Avery et al. 2020). To test whether connectome features of stability, typicality, and optimality improved behavioral prediction over and above that achieved by CPM network strength alone, we constructed linear models with terms for CPM network strength, connectome features, and their interaction for sustained attention and working memory runs separately.

CPM network strength was calculated for sustained attention runs using the sustained attention CPM (Rosenberg et al. 2016; https://github.com/monicadrosenberg/Rosenberg_PNAS2020) and for working memory runs using the working memory CPM (Avery et al. 2020). Specifically, sustained attention network strength values were computed by taking the difference between the average functional connection strength in a high-attention network (whose strength predicts better sustained attention performance) and a low-attention network (whose strength predicts worse sustained attention performance), as characterized in Rosenberg et al. (2016). Working memory strength values were calculated analogously using the predefined high- and low-working memory networks (Avery et al. 2020). Network strength and connectome feature values were z-scored within-dataset for sustained attention and working memory separately. Because the sustained attention CPM was originally trained on data from Dataset 1, the sustained attention models were tested only in Datasets 2 and 3 in the current study. Similarly, because the working memory CPM was originally trained on data from Dataset 3, the current working memory models were tested only in Dataset 2. We compared the output of these models to models including a single term of CPM network strength or connectome feature.

Anatomy of connectome stability and typicality

To investigate whether the stability and typicality of functional connectivity differs throughout the brain, we calculated the stability and typicality of functional connections within and between canonical functional networks. Network parcellations were defined in Noble et al. (2017).

We next investigated the degree to which any relationship between connectome stability and task performance was driven by individual functional networks. To do so, we “computationally lesioned” networks from the connectome—that is, removed all nodes within a network from the connectivity matrix—and recalculated stability values for the lesioned connectome. We then recalculated the relationship between lesioned connectome stability and task performance iteratively for each network and evaluated the change in correlation relative to that observed with the full connectome.

We used a similar method to determine how individual functional networks contributed to the relationship between the typicality of functional connectivity patterns and task performance. Typicality was calculated after networks were computationally lesioned, and the relationship with task performance was recalculated using the lesioned connectome. The change in correlation indicates how each network contributed to the relationship observed between typicality of the full connectome and behavior.

Changes in the correlation between computationally lesioned connectome stability and task performance were compared to a distribution of 1,000 null permutations. To create the distributions, network labels were shuffled and the change in correlation between randomly lesioned connectome stability and performance was recalculated 1,000 times. Because calculating lesioned typicality values was more computationally intensive, lesioned typicality values were compared against 250 null permutations.

Results

Individual and group-level connectome features are related

Connectome stability, typicality, optimality, and discriminability are not independent of one another (Table 1). Stability was correlated with typicality and optimality for both sustained attention and working memory runs in Datasets 2 and 3 and correlated with discriminability in all datasets. Typicality and optimality were also strongly correlated across all datasets in both sustained attention and working memory runs. Discriminability was positively correlated with typicality and optimality in Dataset 3 but negatively correlated with typicality in Dataset 2 working memory runs. Because these connectome variables are related but not perfectly correlated, they may explain unique variance in sustained attention and working memory performance.

Within-task connectome stability uniquely predicts behavior

We first investigated whether stability of an individual’s functional connectome during sustained attention and working memory tasks predicts task performance in three independent datasets. To do so, we calculated the partial correlation between connectome stability and task performance controlling for mean head motion on both runs as well as the difference in mean head motion between runs (Fig. 2).

Fig. 2.

A) Connectome stability, b) typicality, c) optimality, d) and discriminability were differentially related sustained attention and working memory performance across datasets. Spearman’s rank correlations were used to mitigate effects of outliers on the relationship between connectome features and performance.

Mean connectome stability and sustained attention performance were positively correlated in all three datasets, and significant in Datasets 2 and 3 (DS1: partial r_s = 0.344, P = 0.109; DS2: partial r_s = 0.411, P = 8.22^*10⁻⁴; DS3: partial r_s = 0.168; P = 3.34^*10⁻³; Fig. 2;Supplementary Table 2). For Datasets 2 and 3, which also included working memory tasks, connectome stability during working memory runs was significantly related to working memory performance (DS2: partial r_s = 0.264, P = 0.027; DS3: partial r_s = 0.207; P = 2.54^*10⁻⁴; Fig. 2;Supplementary Table 2). These results suggest that individuals who express more similar functional connectivity patterns across repeated fMRI runs of a sustained attention or working memory task tend to perform better on these tasks. Additionally, connectome stability during sustained attention tasks was significantly correlated with stability during working memory tasks in both datasets that included these tasks after controlling for mean head motion and difference in head motion between runs (DS2: partial r_s = 0.337, P = 0.013; DS3 partial r_s = 0.693, P = 3.28^*10⁻⁴⁶), suggesting that participants tend to have similar levels of connectome stability between sustained attention and working memory tasks. Note, however, that in Dataset 3, sustained attention and working memory tasks were performed in interleaved blocks, potentially inflating estimates of connectome stability similarity across task types.

Perhaps higher performers have more stable functional connectivity patterns regardless of task. We next tested whether it was stability within sustained attention and working memory runs specifically or across-run stability more generally that predicted performance. To do so, we calculated stability between all pairs of fMRI runs within a dataset and again found the partial correlation between stability and performance controlling for head motion (Fig. 3). To control for the increased likelihood of observing positive but spurious correlations when performing multiple comparisons, we note here which correlations survive Bonferroni correction.

Fig. 3.

Stability between pairs of runs differentially predicted a) sustained attention and b) working memory performance. Colors indicate the partial Spearman correlation between pairwise run stability and performance, controlling for head motion within and between runs. Repeated task runs are outlined in gray. Repeated task runs whose performance was used as the predicted measure are indicated with heavy black lines. Stars within black and gray outlines represent uncorrected significant correlation values, while all stars outside of the black and gray outlines represent Bonferroni-corrected significance values.

Mean stability during resting-state runs was not significantly related to sustained attention performance (DS1: partial r_s = 0.260, P = 0.269; DS2: partial r_s = 0.150, P = 0.168; DS3: partial r_s = 3.80^*10⁻³, P = 0.948) nor working memory performance (DS2: partial r_s = −.088, P = 0.422; DS3: partial r_s = 0.076, P = 0.185; Fig. 3). This suggests that the relationship between connectome stability and performance is relatively specific to task-based functional connectivity.

Stability between resting-state and task runs was not reliably related to either sustained attention or working memory performance across datasets. Average stability in functional connectivity between rest and sustained attention runs (i.e., less change from rest to task runs) predicted sustained attention performance in Dataset 3 only (DS1: r_s = 5.64^*10⁻⁴, P = 0.998; DS2: r_s = 0.137, P = 0.197; DS3: r_s = 0.115, P = 0.046). Similarly, smaller changes between rest and working memory runs predicted working memory performance in Dataset 3 but not Dataset 2 (DS2: r_s = −.029., P = 0.787; DS3: r_s = 0.118, P = 0.037). While previous work using data from the Human Connectome Project found that similarity between resting-state and task FC is related to task performance (Schultz and Cole 2016), we only observed this relationship in Dataset 3, a subsample of the Human Connectome Project dataset. We did not observe this relationship consistently across other datasets.

In some cases, stability during different tasks was predictive of performance on either the sustained attention or working memory tasks. Specifically, in Dataset 2, stability during the MOT task, which taxes attention, predicted both sustained attention and working memory performance (Sustained attention: partial r_s = 0.297, P = 0.015; Working memory: partial r_s = 0.350, P = 3.67^*10⁻³; Fig. 3). However, these relationships do not survive Bonferroni correction for 45 comparisons ((N_tasks² − N_tasks)/2). In Dataset 3, connectome stability during 2-back, motor, language, and social cognition tasks was related to sustained attention performance (2-back: partial r_s = 0.168, P = 3.37^*10⁻³; motor: partial r_s = 0.116, P = 0.044; language: partial r_s = 0.147, P = 0.011; social: partial r_s = 0.141, P = 0.015). Connectome stability during 0-back, motor, language, and social cognition tasks similarly predicted working memory performance in Dataset 3 (0-back: partial r_s = 0.219, P = 1.03^*10⁻⁴; motor: partial r_s = 0.236, P = 2.82^*10⁻⁵; language: partial r_s = 0.271, P = 1.38^*10⁻⁶; social: partial r_s = 0.227, P = 5.75^*10⁻⁵). Only correlations with working memory performance (0-back, language, social, and motor tasks) survive Bonferroni correction. Taken together, these results suggest that the relationship between performance and stability is specific to stability in certain tasks.

Finally, to examine whether performance was related to stability more broadly—that is, across all run types—we correlated performance with a more general measure of stability for each participant. To evaluate participants’ overall connectome stability, we calculated the mean stability between all pairs of runs in each dataset. We then calculated the partial Spearman rank correlation between stability and performance, controlling for individuals’ mean head motion on all runs. Overall stability was not related to sustained attention performance in Dataset 1 or 2 but was marginally related in Dataset 3 (DS1: partial r_s = 0.205, P = 0.336; DS2: partial r_s = 0.167, P = 0.109; DS3: partial r_s = 0.106, P = 0.065). For working memory performance, there was no relationship observed in Dataset 2, but there was a significant correlation with overall stability in Dataset 3 (DS2: partial r_s = 0.045, P = 0.670; DS3: partial r_s = 0.160, P = 4.70^*10⁻³). While inconsistent, the strength of the relationship between performance and overall stability increased with the total number of runs included in the calculation of overall stability. Future work can investigate whether individuals who perform better on sustained attention and working memory tasks express a more stable pattern of functional connectivity across a variety of cognitive and task states.

Connectome stability predicts behavior beyond task performance consistency

An alternative explanation of these results is that instead of reflecting overall performance, connectome stability is related to consistency in performance, which in turn relates to overall performance. We performed two control analyses to test this account. These analyses were not performed in Dataset 3 because this sample lacked run-specific behavioral measures.

First, we recalculated the partial Spearman rank correlation between connectome stability and performance controlling for the absolute difference in performance between runs. Absolute difference in performance was inversely related to sustained attention (DS1: r = −0.301, P = 0.144; DS2: r = −0.051; P = 0.688) and working memory performance (DS2: r = −0.401, P = 4.78^*10⁻⁴) indicating that better performers were indeed more consistent (i.e., had less change in performance) between runs. After controlling for consistency in sustained attention performance, the correlation between connectome stability and performance remains positive in Dataset 1 and significant in Dataset 2 (DS1: partial r_s = 0.325, P = 0.140; DS2: partial r_s = 0.445, P = 2.96^*10⁻⁴). After controlling for consistency in working memory performance, the correlation between connectome stability and performance numerically weakens in Dataset 2 (partial r_s = 0.198, P = 0.103).

Second, we calculated the partial Spearman rank correlation between connectome stability and performance on a single run from each task. The relationship between connectome stability and sustained attention task performance was positive for each of the three runs in Dataset 1 (run 1 partial r_s = 0.431, P = 0.051; run 2 partial r_s = 0.467, P = 0.028; run 3 partial r_s = 0.120, P = 0.604) and for both runs in Dataset 2 (run 1 partial r_s = 0.231, P = 0.068; run 2 partial r_s = 0.522, P = 1.16^*10⁻⁵). For working memory runs in Dataset 2, the partial correlation between connectome stability and individual run performance was also positive, although numerically weaker than the relationship with average working memory performance (run 1 partial r_s = 0.226, P = 0.060; run 2 partial r_s = 0.179, P = 0.139). In combination, these control analyses suggest that relationships between task-based connectome stability and sustained attention and working memory task performance may be partially, but not fully, driven by the fact that better performers tend to be more consistent performers.

Connectome typicality also predicts task performance

It is possible that individuals express patterns of functional connectivity specific to particular cognitive tasks. If this is the case, we may expect a group-averaged task-based connectome to partially reflect a task-engaged pattern and the extent to which a participant’s connectome resembles the group average to reflect the degree to which they achieve this characteristic task pattern. Thus, it may be that behavioral performance is related to connectome typicality, or how similar an individual is to the rest of the group.

To test this possibility, we calculated the typicality of functional connectivity patterns for each dataset and computed the partial correlation between connectome typicality and task performance, again controlling for head motion in the scanner. For sustained attention runs, connectome typicality was significantly related to performance in Datasets 1 and 3. In Dataset 2, however, there was no significant relationship between connectome typicality and sustained attention performance (DS1: partial r_s = 0.446, P = 0.033; DS2: partial r_s = 0.091, P = 0.476; DS3: partial r_s = 0.164, P = 3.34^*10⁻³; Fig. 2; Supplementary Table 2). In working memory runs, connectome typicality was not related to working memory performance in Dataset 2 but was significantly related to performance in Dataset 3 (DS2: partial r_s = 0.168, P = 0.163; DS3: partial r_s = 0.220, P = 2.54^*10⁻⁴). Similar to analyses of connectome stability, the strongest positive relationships between connectome typicality and task performance are largely specific to typicality during sustained attention and working memory tasks (Supplementary Fig. 1), supporting the hypothesis that some task-specific patterns of behavior are captured in the group-average connectome.

The inconsistency of the relationship between connectome typicality and performance, however, suggests that better performers may not simply be expressing a pattern of connectivity resembling the group average. Alternatively, it may be the case that a more typical pattern of connectivity is reflective of more typical performance, in which case better performers would deviate more from the group average. To further investigate these hypotheses of how functional connectivity typicality relates to performance, we performed two additional analyses.

First, we tested the possibility that connectome typicality does not relate to better performance but instead relates to more typical performance. For this analysis, we measured the relationship between participants’ average task-specific connectome and their absolute difference from the mean performance score. We found little evidence for this relationship across datasets for both sustained attention (DS1: partial r_s = −0.153, P = 0.497; DS2: partial r_s = −0.228, P = 0.073; DS3: partial r_s = −0.001, P = 0.984) and working memory tasks (DS2: partial r_s = −0.062, P = 0.610; DS3: partial r_s = 0.071, P = 0.210), suggesting that more typical connectomes are not reliable markers of more typical performance. Furthermore, average connectome typicality values generally increase across task performance quartiles (Supplementary Fig. 2), indicating that better performers are more similar to the group average than low performers.

Similarity to the optimal performer predicts behavior

Another explanation for the inconsistent relationship between connectome typicality and performance may be that the group-average connectome is not the best proxy for task-specific patterns of functional connectivity. Instead, similarity to the functional connectivity patterns of the best performers may better reflect adherence to a task-specific connectivity pattern. To test this possibility, we examined whether similarity to the connectomes of the best performers was predictive of performance. To do so, we calculated the similarity of each participant’s average task functional connectivity pattern to the mean connectivity pattern of the participants who achieved the top 5% of scores. Then, we calculated the Spearman’s rank correlation between this similarity to the “optimal” connectome and task performance. Across all datasets with sustained attention runs, similarity to the optimal connectome predicted performance during sustained attention runs (DS1: partial r_s = 0.809, P = 9.10^*10⁻⁶; DS2: partial r_s = 0.438, P = 4.60^*10⁻⁴; DS3: r_s = 0.240, P = 1.06^*10⁻⁴; Fig. 2; Supplementary Table 2). For working memory as well, similarity to the optimal pattern of connectivity was positively correlated with task performance in both Dataset 2 and 3, and significant in Dataset 3 (DS2: partial r_s = 0.231, P = 0.062; DS3: r_s = 0.246, P = 2.35^*10⁻⁵). This consistency across datasets and tasks suggests that some patterns of connectivity may be “optimal” for specific tasks, such that the extent to which an individual’s connectome resembles this optimal pattern reflects performance on that task.

Do “optimal” patterns of connectivity generalize across task and dataset? We tested whether similarity to the top performer connectome of another dataset similarity predicted task performance within sustained attention and working memory domains. We observed limited generalization such that similarity to the top performers in Dataset 2 significantly predicted gradCPT performance in Dataset 1 (partial r_s = 0.542, P = 7.55^*10⁻³) but not 0-back performance in Dataset 3 (partial r_s = 0.098, P = 0.088). Similarity to the mean optimal connectome in Dataset 1 predicted 0-back performance in Dataset 3 (partial r_s = 0.183, P = 1.37^*10⁻³) and, conversely, similarity to the optimal 0-back connectome in Dataset 3 predicted gradCPT performance in Dataset 1 (partial r_s = 0.455, P = 0.029). However, gradCPT performance in Dataset 2 was not predicted by similarity to the top connectomes of either Dataset 1 (partial r_s = −0.004, P = 0.972) or Dataset 3 (partial r_s = −0.093, P = 0.470). For working memory tasks, similarity to the top performers’ connectome in Dataset 2 did not generalize to predict 2-back performance in Dataset 3 (partial r_s = 0.078, P = 0.171) nor did similarity to the top 2-back connectome predict VSTM performance in Dataset 2 (partial r_s = 0.177, P = 0.143). Full comparisons are included in Supplementary Table 3. Thus, “optimal” functional connectivity patterns may be relatively task- and/or dataset-specific.

Connectome discriminability inconsistently predicts behavior

Previous work demonstrated increases in connectome distinctiveness across development and delays in this trajectory in clinical populations (Kaufmann et al. 2017). These findings raise the possibility that individuals who perform better on cognitive and attentional tasks are achieving a more unique connectivity pattern, such that their connectomes are more similar to themselves than they are to the group. To investigate this, we calculated each individual’s discriminability, or the ratio of their connectome stability to typicality on either sustained attention or working memory runs. While discriminability values could theoretically range from negative infinity to positive infinity, in the current study, values ranged from 0.173 to 3.46. For sustained attention runs, connectome discriminability was not related to performance in Dataset 1 but did significantly predict performance in Datasets 2 and 3 (DS1: partial r_s = 0.109, P = 0.622; DS2: partial r_s = 0.431, P = 4.17^*10⁻⁴; DS3: partial r_s = 0.124; P = 0.031; Fig. 2; Supplementary Table 2). In working memory runs, connectome discriminability did not predict performance in Dataset 2 but was significantly related to working memory performance in Dataset 3 (DS2: partial r_s = 0.154, P = 0.203; DS3: partial r_s = 0.150; P = 8.33^*10⁻³). Although there is some evidence for a relationship between connectome discriminability and performance, inconsistency across datasets suggests that better performers may not simply be expressing more unique—i.e., identifiable—patterns of connectivity.

Models based on connectome stability and optimality best predict behavior

We constructed four linear models, each containing one feature of the connectome (i.e., stability, typicality, optimality, and discriminability) to evaluate how each feature explained variation in sustained attention and working memory performance separately. Model output is summarized in Table 2. To determine which of our candidate models best explained variations in both sustained attention and working memory performance, we compared the Akaike Information Criteria (AIC) values of each model. AIC values quantify to what extent a model minimizes both bias and variance, such that lower AIC values indicate a more optimal minimization (Burnham and Anderson 2004). AIC values are unitless but enable the comparison of models fitting the same data to determine which candidate model least overfits the data. Using these criteria for models explaining sustained attention performance, we observed the lowest AIC value for the model containing only a fixed effect for optimality. A model containing only a stability or typicality term also significantly captured variability in sustained attention performance. When predicting working memory performance, we observed the lowest AIC value for the stability model, suggesting that this is the best-fit model for predicting working memory performance. However, typicality, optimality, and discriminability models explained significant variability in working memory performance.

Table 2.

Output from mixed-effects models including model weights (coefficient) and residual standard error (std error) for connectome feature predictors. Bolded AIC values indicate the best model for predicting sustained attention and working memory performance. Significance levels are indicated with asterisks.

	Sustained attention					Working memory
	coefficient	std error	significance		AIC	coefficient	std error	significance		AIC
Stability	0.4384	0.1055	1.005*10−4	* * *	951.0	0.4054	0.0913	1.590*10−5	* * *	1005.2
Typicality	0.2023	0.0562	0.0112	*	952.8	0.1616	0.0587	0.0111	*	1015.2
Optimality	0.3143	0.0524	5.141*10−9	* * *	931.6	0.1540	0.0528	7.993*10−3	* *	1013.9
Discriminability	0.0781	0.0518	0.1329		963.6	0.1053	0.0509	0.0394	*	1017.7
Stability	0.3264	0.1192	7.642*10−3	* *	952.7	0.3572	0.0975	3.035*10−4	* * *	1008.5
Typicality	0.1463	0.0772	0.0589			0.1010	0.0669	0.1317
Stability	0.3474	0.1217	5.315*10−3	* *	958.0	0.3949	0.1028	1.563*10−4	* * *	1013.3
Typicality	0.1496	0.0775	0.0543			0.0876	0.0683	0.2002
Stability * Typicality	0.0384	0.0632	0.5442			0.0597	0.0579	0.3034
Stability	0.1165	0.0927	0.2899		936.8	0.3529	0.0958	2.887*10−4	* * *	1007.6
Optimality	0.2914	0.0616	4.750*10−6	* * *		0.1099	0.0596	0.0661
Stability	0.2182	0.1131	0.0704		939.3	0.3818	0.1026	2.643*10−4	* * *	1013.0
Optimality	0.2713	0.0630	2.409*10−5	* * *		0.0985	0.0621	0.1134
Stability * Optimality	0.1042	0.0514	0.0434	*		0.0399	0.0582	0.4931

^* P < 0.05.

^{* *} P < 0.01.

^{* * *} P < 0.001.

We were further interested in whether interactions between variables explained additional variance in performance. To determine whether similarity within and between individuals capture unique variability in performance, we constructed four additional models. Two models included terms for stability and typicality and stability, typicality, and their interaction. An additional two models compared stability and optimality, and stability, optimality, plus their interaction. Importantly, discriminability was not included in these models because it was derived from stability and typicality variables and typicality and optimality were not include in the same model to avoid multicollinearity. Results indicated that, for sustained attention runs, a model with fixed effects for stability and optimality best predicted sustained attention performance of these additional models. However, only the fixed effect of optimality was a significant predictor in this model. Similarly, for working memory runs, a model with effects of stability and optimality best predicted working memory performance of these models. However, only stability was a significant predictor in this model. These results suggest that a model considering connectome similarity within (stability) and across (optimality) participants may be the most useful predictor of cognitive performance.

Connectome features are generalizable predictors

Do connectome features capture unique variance in behavioral prediction when used in combination with previously validated network models of sustained attention and working memory? We constructed linear models with terms for CPM network strength, stability, typicality, and their interactions to predict sustained attention and working memory performance separately. Average and mean difference in in-scanner motion between runs were also included as covariates in all models. CPM network strength was calculated using the previously published sustained attention CPM (Rosenberg et al. 2016) for sustained attention runs and previously published working memory CPM (Avery et al. 2020) for working memory runs. Model results are summarized in supplementary materials (Supplementary Table 5).

CPM network strength alone was the best predictor of sustained attention performance, as measured by the gradCPT in Dataset 2. Models with a single term of connectome stability or optimality predicted gradCPT performance in this dataset but were less optimal models based on AIC values. In Dataset 3, connectome feature models predicted 0-back performance better than sustained attention CPM strength. Sustained attention CPM strength alone did not significantly predict 0-back performance in the present study. However, recently published work found that sustained attention CPM strength did significantly predict 0-back performance in a larger HCP sample (Kardan et al. 2022). Differences in how data were preprocessed as well as fewer participants (n = 316 vs. n = 754) and higher and less variable behavioral scores in the current sample may have contributed to differences in model prediction between datasets. Instead, the best model for predicting 0-back performance in Dataset 3 was the model that included terms for sustained attention CPM strength, optimality, and their interactions. However, the only terms that significantly predicted performance in this model were connectome optimality and the interaction between CPM strength and optimality.

In working memory runs in Dataset 2, a model including working memory CPM strength, connectome stability, and their interaction was the best predictor of VSTM working memory performance. However, only the connectome stability and interaction terms had significant beta coefficients in this model. In combination, these results suggest that connectome features of stability and optimality may be reliable predictors of sustained attention and working memory performance in datasets where supervised CPM models do not generalize. Furthermore, in datasets where pretrained models do not generalize on their own, incorporating connectome stability or optimality as model predictors may improve model performance.

Individual network contributions to behavioral prediction vary

We were interested in exploring whether the relationship between connectome stability and cognitive performance relies on whole-brain stability, or whether certain networks contribute to this relationship more than others. First, we tested whether stability in CPM networks, a subset of the connectome edges whose strength predicts sustained attention or working memory performance, also predicts performance. Stability in the sustained attention CPM network was not significantly related to sustained attention performance in any of our three datasets (DS1: r_s = 0.256, P = 0.238; DS2: r_s = 0.083, P = 0.518; DS3: r_s = 0.058, P = 0.315). Working memory CPM network stability predicted 2-back performance in Dataset 3 but not VSTM performance in Dataset 2 (DS2: r_s = 0.218, P = 0.070; DS3: r_s = 0.383, P = 2.94^*10⁻¹²). These results suggest that stability of the whole-brain connectome may be more predictive than stability in CPM networks.

Are certain functional networks more stable or typical than others? We visualized within- and between-network connections in terms of their average stability and typicality (Fig. 4) across individuals within a dataset. Stability and typicality values were normalized based on network size. This visualization revealed high consistency in both network stability and typicality values for both sustained attention and working memory task connectivity patterns. In particular, visual networks were both highly stable and highly typical in both tasks across datasets. Matrices visualizing within- and between-network connections for additional connectome features optimality and discriminability, as well as network connections for individual datasets, are included in the supplement (Supplementary Figs 3 and 4).

Fig. 4.

Stability and typicality of within- and between-network connections for a) sustained attention and b) working memory task connectomes. Matrices on the left represent average Pearson’s correlations across datasets, while matrices on the right are average correlations normalized by network size.

Next, we computationally lesioned all nodes belonging to each of 10 canonical networks and recalculated the correlation between stability in the remaining nodes and either sustained attention or working memory performance. The change between intact and lesioned correlation values is plotted in Fig. 5.

Fig. 5.

Canonical networks differentially contribute to the correlation between connectome stability and cognitive performance. Dashed lines indicate correlation values between intact connectome stability and performance. Heavy red lines indicate the Spearman rank correlation between lesioned connectomes and a) sustained attention and b) working memory performance. Decreased lesioned correlations indicate that the network contributed to the relationship between connectome stability and performance, while increased lesioned correlations suggest that the network added noise. Significance P < 0.05 (uncorrected) is indicated with asterisks.

Networks inconsistently contributed to the relationship between stability and cognitive performance. No network significantly contributed to the correlation between stability and sustained attention performance across all datasets. The visual I network significantly added noise to this correlation in Datasets 1 and 2 which both measured sustained attention performance using the gradCPT task. The visual association network significantly contributed to the correlation between stability and sustained attention performance in Datasets 1 and 2 but displayed a contradictory pattern of results for working memory, contributing to the relationship between stability and working memory performance in Dataset 2 but adding significant noise in Dataset 3. These results suggest that the relationship between connectome stability and sustained attention and working memory performance does not rely on specific contributions from individual networks. Instead, networks appear to contribute in a pattern that is more task-specific rather than cognitive process-specific.

We were similarly interested in whether the relationship between connectome typicality and cognitive performance relies on particular networks. We again computationally lesioned 10 canonical networks and recalculated the relationship between typicality in the intact networks and either sustained attention or working memory performance for each dataset. The change in correlation between intact and lesioned connectome typicality and cognitive performance is shown in Fig. 6.

Fig. 6.

Canonical networks differentially contribute to the correlation between connectome typicality and cognitive performance. Dashed lines indicate correlation values between intact connectome typicality and performance. Heavy blue lines indicate the Spearman rank correlation between lesioned connectomes and a) sustained attention and b) working memory performance. Decreased lesioned correlations indicate that the network contributed to the relationship between connectome typicality and performance, while increased lesioned correlations suggest that the network added noise. Significance P < 0.05 (uncorrected) is indicated with asterisks.

Network contribution to the correlation between typicality and cognitive performance largely differed across datasets. However, we observed significant contribution of the frontoparietal network to the relationship between connectome typicality and performance in working memory runs in both Dataset 2 and Dataset 3. This result is in accordance with previous work that found frontoparietal activity to be a biomarker of working memory performance in preadolescents (Rosenberg et al. 2020). The medial frontal network significantly contributed to the relationship between connectome typicality and sustained attention performance in Datasets 1 and 2 but interestingly added significant noise to this relationship in Dataset 3. Again, it is possible that network contribution to the relationship between connectome typicality and cognitive performance is specific to the task being used to measure a given cognitive ability.

Discussion

The current study investigated whether features of the functional connectome predict individual phenotypes. Connectome stability, or similarity in functional connectivity patterns across runs, may reflect a more consistent on-task state and has been related to phenotypes in development (Kaufmann et al. 2017; Vanderwal et al. 2021; Fu et al. 2022). More typical connectomes may reflect a characteristic task-specific functional connectivity pattern that is captured in the group-average. Alternatively, this optimal pattern of connectivity may be best captured in the mean connectome of the highest task performers. Finally, the ratio of stability to typicality (discriminability) reflects a connectome’s uniqueness and has been related to clinical symptoms in development (Kaufmann et al. 2017). We compared the predictive ability of these features for sustained attention across three independent datasets and working memory across two datasets. Our findings suggest that connectome features, particularly connectome stability and optimality, can serve as generalizable predictors of sustained attention and working memory abilities.

Connectome stability was a consistent predictor of both sustained attention and working memory performance. This relationship was relatively specific to within-task (i.e., sustained attention or working memory) stability, such that the strongest relationships between connectome stability and task were observed when stability was calculated between repeated sustained attention or working memory runs. This relationship was observed across all datasets in the current study, with stability during attention tasks positively correlated with attentional performance and stability during working memory tasks positively correlated with working memory performance. Better performers in both tasks expressed more similar functional connectivity patterns across scans, even when scans were separated by weeks as in Dataset 2. Stability during different tasks (e.g., an MOT task in Dataset 2; language, social, and motor tasks in Dataset 3) was also at times related to task performance. We hypothesize that connectome stability reflects the extent to which an individual’s cognitive state changes over time, with greater stability indicating a more similar cognitive state between fMRI runs. Therefore, connectome stability may indicate a more on-task state which may also be reflected in task performance.

Our feature of connectome stability is related to test–retest reliability of the functional connectome across scan sessions (Noble et al. 2019). However, while test–retest reliability tests the extent to which functional connectivity as a measure remains stable over time, here, we aim to leverage meaningful differences in test–retest reliability across individuals to tell us something about cognitive ability. Future studies of reliability may consider to what extent a lack of test–retest reliability reflects methodological noise versus meaningful variability across tasks and individuals.

Notably, we found a lack of predictive ability from connectome stability during rest and between task and rest runs. The latter observation is in contrast to previous work that found that more similar rest and task connectivity patterns was related to higher cognition, including better working memory performance (Schultz and Cole 2016). The current results also demonstrate a benefit of utilizing task-based functional connectivity data as a predictive tool, rather than resting-state which has been used widely in the field. For the current study, we consider what features of the functional connectomes themselves contribute to tasks’ predictive ability. One possibility is that the variability of connectivity patterns during specific tasks boosts within-individual similarity’s predictive power. However, a visualization of connectome standard deviation across participants suggests that tasks whose stability predicted performance and those that do not are similarly variable (Supplementary Fig. 5). Another possibility is that predictive tasks induce a more variable functional connectivity pattern within-individuals than other tasks, thereby allowing stability to be a more reliable measure. However, a comparison of mean task functional connectivity standard deviation does not support this hypothesis (Supplementary Fig. 6). Finally, scan length may affect the overall stability values, which may in turn affect behavior prediction. However, while stability between task runs was higher for longer tasks across datasets (correlation between number of TRs and task stability: r = 0.675, P = 4.04^*10⁻⁴), overall task stability did not reliably lead to better prediction of sustained attention (relationship between task stability and stability-behavior correlation: r = 0.236, P = 0.278) nor working memory (r = −0.304, P = 0.158) performance (Supplementary Table 6). Therefore, while longer scans may lead to more stable patterns of connectivity, more stable run types are not always most predictive of behavior. Future work may seek to further investigate task-based connectivity’s predictive advantage. However, the current results add to the growing literature emphasizing the benefit of task-specific over resting state functional connectivity as a predictive tool.

We found initial—albeit less consistent—evidence that connectome typicality, or the extent to which an individual resembles the rest of the group, predicts cognitive and attentional performance. Tasks have been shown to induce characteristic changes in the connectome (Shine et al. 2016; Lynch et al. 2018; Greene et al. 2020), and children and adults who show activity in canonical task-positive regions, such as regions of the frontoparietal network, perform better on attention and working memory tasks (Rypma et al. 2002;Satterthwaite et al. 2013 ; Rosenberg et al. 2020). Thus, we hypothesized that if this task-relevant connectivity is reflected in the group-average connectome, the extent to which an individual resembles the group may scale with their task engagement and performance. In the present datasets, connectome typicality was related to sustained attention performance in two of three datasets and working memory performance in one of two datasets. A linear model with a single fixed effect of typicality significantly predicted both sustained attention and working memory across datasets. These results suggests that individuals who express shared, task-specific connectivity patterns may indeed show more on-task cognitive performance as evidenced by higher behavioral scores.

Following these initial inconsistent results, we further explored how connectome typicality might relate to behavior. We did not find evidence that connectome typicality is related to typicality in performance. Instead, we found that, for both sustained attention and working memory tasks, connectome optimality—or, similarity to the connectome of the highest performer on a given task—was positively correlated with task performance across all datasets. In combination with the previous analysis, these results support the notion of optimal task-specific connectivity patterns but suggest that these patterns may become obscured when averaged across participants. We also observed limited generalizability of connectome optimality, such that the similarity to the optimal sustained attention or working memory connectome in one dataset did not reliably predict sustained attention or working memory performance in other datasets, respectively. Future studies may seek to further characterize these “optimal” patterns of connectivity and explore the extent of their generalizability across contexts and tasks.

Finally, we tested the predictive utility of connectome discriminability, or an individual’s ability to be distinguished from others. To do so, we defined connectome discriminability as the extent to which an individual looks more like themselves than others or, intuitively, the ratio of connectome stability and typicality. This measure is related but not identical to previous work that has characterized connectome identifiability using fingerprinting techniques which identify individuals who are more like themelves than any other individual across runs (Finn et al. 2015). Although both measures capture related measurements, connectome fingerprinting binarizes identifiability (i.e., an individual is either most like themselves or not), while connectome discriminability values are continuous. In addition to task-relevant changes, in-scanner tasks amplify individual differences in functional connectivity patterns allowing for improved identification of individuals—i.e., better connectome fingerprinting (Finn et al. 2015; Rosenberg et al. 2016; Greene et al. 2020) and, in some cases, improved behavioral predictions (Finn et al. 2017; Greene et al. 2020, but see Finn and Rosenberg 2021, Mantwill et al. 2021, and Noble et al. 2017). One possibility is that individuals who show stable, unique connectivity profiles during task performance may be reliably engaging individualized networks supporting task performance and thus may show better task performance. Thus, we tested whether connectome discriminability predicts sustained attention and working memory performance. We observed an inconsistent relationship between discriminability and performance across datasets and tasks. Fixed effects models predicting both sustained attention and working memory were significant but less optimal than other models tested. These results suggest that connectome discriminability may not be a reliable predictor of cognitive performance, at least in the case of sustained attention and working memory. Furthermore, discriminability as defined in the current study as the ratio between one’s stability and typicality did not add unique predictive power above and beyond these component measures.

Stability and typicality of individual network connections was consistent across datasets and tasks. While no network reliably related to connectome stability’s prediction of performance across datasets, computational lesioning of the frontoparietal network significantly impacted the relationship between typicality and working memory performance in both datasets with a working memory measure. These results are in line with previous work that identified frontoparietal activity as a biomarker of working memory performance in preadolescents (Rosenberg et al. 2020). Inconsistent contributions from other canonical networks, in combination with limited behavioral prediction from stability within previously validated network-based models, suggest that connections involved in connectome stability and typicality’s prediction of both sustained attention and working memory performance are broadly distributed. One possibility is that the canonical networks used here are too large-scale to identify reliable anatomical regions meaningful for prediction. Additionally, future studies may investigate whether the stability and typicality of functional connections relate to performance in a manner specific to the unique task being performed. Perhaps specific task demands influence which networks’ stability most contribute to performance prediction.

In the current study, we observed utility of including individual- and group-related functional connectivity features in behavioral prediction models. In particular, connectome stability and optimality were able to predict sustained attention and working memory performance in datasets where network-based strength models did not generalize. Functional connectivity features have the additional benefit of being inherent to the connectome, such that models incorporating these features do not require separate training and testing sets. This suggests that unsupervised models based on functional connectivity features may capture unique variance in cognitive performance more broadly and may be valuable to include in future predictive models, particularly in cases when supervised CPMs do not generalize. The specific utility of connectome features for prediction may depend on task and dataset. While we observed significant prediction of both sustained attention and working memory using features of connectome typicality and optimality, these features are dataset-dependent such that they require meaningful patterns of connectivity to be captured in the average connectomes of the group and top performers. Alternatively, connectome stability is relatively agnostic to characteristics of the larger dataset and may therefore be a useful predictor in more idiosyncratic datasets.

Limitations

As is the case in many studies relating task-based functional connectivity to behavior, it is possible that our results are influenced by in-scanner task performance. For example, behavior such as motion (e.g., task-related button presses or head motion) may have systematically varied with our measures of sustained attention and working memory performance. If this were the case, our neural measures may not be solely markers of these cognitive mechanisms but instead reflect some behavioral covariates. While the results of our behavioral stability control analysis suggest that stability in in-scanner performance does not fully explain our findings, we cannot rule out the possibility that our results are influenced by other, systematically varying behaviors.

Although the replication of effects across datasets that measured sustained attention and working memory with different tasks is a strength of the current study, that differences between tasks may have limited our findings. For example, while the 0-back task used as a measure of sustained attention in Dataset 3 resembles a target detection task, commonly used in previous literature, it likely differs in many ways from a continuous performance task such as the gradCPT used in Datasets 1 and 2. Similarly, the working memory tasks used—the VSTM in Dataset 2 and 2-back task in Dataset 3—likely involve different processes despite being used to measure the same cognitive ability. Future work may seek to investigate task-specific influences on the relationship between functional connectivity features and performance to account for these differences.

Conclusion

Here, we demonstrate the utility of functional connectome features as predictors of cognitive ability in adults. Of the features tested, connectome stability and optimality consistently predicted both sustained attention and working memory across three independent datasets. Stability during sustained attention and working memory tasks was related to performance on the respective task, suggesting that better performers display more similar task-specific functional connectivity patterns over time. Additionally, connectome optimality, or similarity to high performers’ task connectome predicted performance across datasets, indicating the potential for task-specific “template” patterns of connectivity. Finally, both connectome stability and optimality explained unique variance in task performance when generalizing network-based connectivity models to novel tasks. Altogether, the current study provides evidence for the utility of connectome features that summarize similarity within and across individuals as brain-based markers of cognition.

Funding

National Science Foundation (BCS-2043740 to M.D.R.); National Science Foundation (BCS-1558497 to M.M.C.); National Institutes of Health (MH 108591 to M.M.C.). Resources provided by the University of Chicago Research Computing Center.

Conflict of interest statement: None declared.

Data availability

Human Connectome Project (HCP) data from Dataset 3 are available at https://db.humanconnectome.org. Family structure and genetic information are restricted (https://www.humanconnectome.org/study/hcp-young-adult/document/quick-reference-open-access-vs-restricted-data) and available only to qualified investigators who agree to HCP Restricted Data Use Terms (https://www.humanconnectome.org/study/hcp-young-adult/document/restricted-data-usage).

Raw data from Dataset 2 is available at https://nda.nih.gov/study.html?id=1050.

For inquiries about data from Dataset 1, contact the authors of the original manuscript https://doi.org/10.1038/nn.4179.

Code availability

Analysis code and average optimal connectivity patterns are available at https://github.com/AnnaCorriveau/FCStabilityOptimality_share.