Using connectome-based predictive models to reveal the systems standardized tests and clinical symptoms are reflecting

Anja Samardzija; Xilin Shen; Wenjing Luo; Abigail S Greene; Saloni Mehta; Fuyuze Tokoglu; Jagriti Arora; Scott W Woods; Rachel B Katz; Gerard Sanacora; Vinod H Srihari; Dustin Scheinost; R Todd Constable

doi:10.1038/s41467-026-73941-0

. 2026 Jun 16;17:7650. doi: 10.1038/s41467-026-73941-0

Using connectome-based predictive models to reveal the systems standardized tests and clinical symptoms are reflecting

Anja Samardzija ^1,^✉, Xilin Shen ², Wenjing Luo ¹, Abigail S Greene ^3,^4,⁵, Saloni Mehta ², Fuyuze Tokoglu ², Jagriti Arora ², Scott W Woods ⁶, Rachel B Katz ⁶, Gerard Sanacora ⁶, Vinod H Srihari ⁶, Dustin Scheinost ^1,^2,³, R Todd Constable ^1,^2,^3,^✉

PMCID: PMC13434263 PMID: 42303975

Abstract

We depart from the feed-forward approach of brain-behavior modeling, which identifies the functional brain connectivity networks associated with performance on external tests, and instead introduce a feedback approach that reveals the brain systems those external tests reflect. In fMRI data from n = 302 demographically and clinically diverse participants, we a priori define connectivity networks for six cognitive constructs and employ kernel ridge regression to quantify each network’s contribution to test performance. This approach provides a ranking of test scores according to the predictive power of each cognitive network, revealing which tests probe which brain networks. It further identifies combinations of measures that optimally probe predefined brain systems and evaluates how specific subtests influence composite scores, revealing when subset inclusion reinforces or weakens specific brain circuit and composite score relationships. This work opens an avenue of research by providing a framework for the development of test instruments guided by quantitative brain metrics.

Subject terms: Cognitive neuroscience, Computational neuroscience

Predictive modeling quantifies how a priori defined human brain networks vary with test performance across a range of out-of-scanner test instruments. This framework supports the development of new tests guided by quantitative brain metrics.

Introduction

In the 1950s and 60 s, as psychology expanded its focus from behavioral studies to a detailed examination of cognitive processes, the field of cognitive psychology emerged as a distinct discipline^1,2. Initially, constructs such as memory, attention, language, problem-solving, and perception were defined, and over time, these definitions have undergone expansion and refinement³. Substantial efforts have been directed toward the development of tests aimed at quantitatively measuring these components of cognition^4–10. Today, standardized tests and normative scores are widely utilized for the quantitative assessment of specific mental processes.

Construct validation has taken various forms¹¹, including theoretical frameworks¹², content validity¹³, convergent and discriminant validity¹⁴, criterion-related validity¹⁵, and lesion studies¹⁶, among others. Lesion studies, in particular, have played a pivotal role in some of the earliest validations of brain-construct associations in terms of the localization of the constructs^17,18. With the advent of neuroimaging, task-based fMRI studies have further contributed to establishing the validity of the tests in terms of the brain systems the tests are reflecting^19–21. These studies consistently demonstrate that the brain regions associated with a given construct exhibit differential activation during tasks involving that specific cognitive domain.

More recently, leveraging extensive data from the Human Connectome Project²², methodologies have been devised to establish relationships between the connectome and cognitive traits^23,24, measured through standardized tests administered outside of MRI scanners. The functional connectome has characteristics that are unique to individuals, to the extent that individuals can be identified via their connectome²⁵, and there are common features that vary across individuals related to brain state and behavioral or clinical traits^26–28. These findings have led to an explosive growth in connectome-based predictive modeling (CPM) that has created new insights about the complexity of brain function and investigations into potential clinical applications^4–10. The key point is that intricate networks of functional connections in the brain form an infrastructure closely linked to traits that can be measured by external tests, leading to generalizable models that can predict scores in previously unseen individuals^25,29.

The circuits revealed by these brain-construct models are those wherein connection strength varies in relation to performance across a group of individuals. This approach can be conceptualized as feed-forward, as it employs standard cognitive tests to reveal the circuits supporting performance on those tests. The models are constructed within a predictive framework to validate the identified brain-construct associations³⁰. This powerful approach has successfully predicted performance on various cognitive tests administered outside the MRI environment in previously unseen individuals, with the most robust models often utilizing functional connectome data. Work has largely focused on what such modeling can reveal about the brain, but the process can be reversed, and the models (and hence the brain) can also be used to inform the external measures³¹. Given the primacy of behavioral research in psychology³² and the long list of important findings made through behavioral studies, it is useful to study the intersection of the brain and behavior in a manner that informs both. The functional connectome, proven to contain rich individual-level information on both brain states and traits^25–28, can play a pivotal role in understanding what the tests are testing. To date, most of the research around CPM has utilized the entire connectome and a feature extraction step is employed to identify connections that vary as a function of the external measure being modeled (test score or symptom measure, for example)^29,33. In many instances, the edges (connections between different regions of the brain) selected are then back-projected to nodes (regions of the brain usually defined by an atlas) to identify the network associated with the score. Graph theory measures^34,35 can then be applied to summarize the properties of the nodes and networks revealed by the model, or the data can be summarized in terms of the relative contributions of specific canonical networks in the brain, such as the Yeo 7 or 17 networks^25,36.

Here, we reformulate connectome-based predictive modeling by predefining networks for each cognitive construct and quantifying the extent to which each brain network varies with task performance for 16 different test scores listed in Table 1. The goal is not to identify the specific circuit supporting an external measure but instead to assume we have networks of interest (the cognitive construct networks in our examples) and to ask the question of how each network contributes to the external measure score. This is then testing the tests in that the results provide an interpretable quantitative brain measure of what brain system(s) the test is reflecting.

Table 1.

Standardized tests modeled, test names used throughout the manuscript text, standard manual names or calculations, and descriptions

Test name throughout text	Test full name	Test naming/ calculation from standard manuals	Test description
Boston Naming Test	Boston Naming Test¹⁰³	Calculated as: standard Z scores = (Total raw score - Age-normed Mean BNT score)/Age-normed standard deviation	Drawings are shown to the participant one at a time, and the participant is asked to name the drawings.
Reading	Reading	Wide Range Achievement Test (WRAT5)⁹⁶: Word Reading Standard score	The participant is asked to slowly and correctly read words.
Verbal Learning	Verbal Learning Immediate Recall	Wide Range Assessment of Memory and Learning (WRAML2)⁹⁵: Verbal Learning Scaled score	The participant is immediately asked to recall a list of common words the evaluator presents.
Verbal Learning with Delay	Verbal Learning Delayed Recall	Wide Range Assessment of Memory and Learning (WRAML2)⁹⁵: Verbal Learning Delayed Recall Scaled score	The participant is asked to recall with delay (approximately 10 minutes after administration) a list of common words the evaluator presents.
Finger Windows	Finger Windows	Wide Range Assessment of Memory and Learning (WRAML2)⁹⁵: Finger Windows Scaled score	The examiner shows a pattern of placing a pencil eraser in a sequence of holes, and the participant is asked to duplicate the sequence in the same order.
Symbol Search	Symbol Search	Wechsler Adult Intelligence Scale (WAIS-IV)¹⁰⁴: Symbol Search Scaled score	The participant is asked to quickly mark whether they identify certain symbols within a row of symbols.
Coding	Coding	Wechsler Adult Intelligence Scale (WAIS-IV)¹⁰⁴: Coding Scaled score	The participant is asked to copy symbols that are paired with numbers.
Letter-Number Sequencing	Letter-Number Sequencing	Wechsler Adult Intelligence Scale (WAIS-IV)¹⁰⁴: Letter-Number Sequencing Scaled score	The participant is read a sequence of letters and numbers and they are asked to recall the numbers in ascending order followed by the letters in alphabetical order.
Cancellation	Cancellation	Wechsler Adult Intelligence Scale (WAIS-IV)¹⁰⁴: Cancellation Total Scaled score	The participant is asked to identify all of the shapes that match the target shape in color and shape.
Trail Making	Trail Making	Delis-Kaplan Executive Function System (DKEFS)⁴¹: Trail Making Test Number-Letter Sequencing Condition 4 Scaled score (Trail Making B)	The participant is asked to alternatively connect in ascending order numbers and letters that are spread across a page.
Verbal Fluency 1	Verbal Fluency I	Delis-Kaplan Executive Function System (DKEFS)⁴¹: Verbal Fluency Test Letter Fluency Condition 1 Total Correct Scaled score	The participant is asked to within a minute produce as many unique words that begin with a letter.
Verbal Fluency 2	Verbal Fluency II	Delis-Kaplan Executive Function System (DKEFS)⁴¹: Verbal Fluency Test Category Fluency Condition 2 Total Correct Scaled score	The participant is asked to within a minute produce as many unique words that belong to a semantic category.
Color-Word Interference	Color-Word Interference	Delis-Kaplan Executive Function System (DKEFS)⁴¹: Color-Word Interference Test Inhibition Condition 3 Scaled score	The participant is asked to name the color of the ink words are written in rather than the words themselves (eg. the word blue is written in the color red).
20 Questions	20 Questions	Delis-Kaplan Executive Function System (DKEFS)⁴¹: Twenty Questions Test Total Weighted Achievement Scaled score	The participant is asked to come up with up to 20 yes/no answer questions to guess which object the examiner is thinking of from a picture of common objects.
Vocabulary	Vocabulary	Wechsler Abbreviated Scale of Intelligence (WASI-II)¹⁰⁵: Vocabulary T-Scores	The participant is asked to name objects presented visually and to define words that are presented visually and orally.
Matrix Reasoning	Matrix Reasoning	Wechsler Abbreviated Scale of Intelligence (WASI-II)¹⁰⁵: Matrix Reasoning T-scores	The participant is shown matrices or visual patterns with something missing and they are asked to select the missing piece from a collection of options.

Open in a new tab

All of the tests included in the table are scaled based on the age-specific norms provided in the test manuals.

Predictive performance for each network, in a feedback sense, reflects the degree to which a particular cognitive test relies on specific brain networks. This work directly assesses contribution of each cognitive construct network to standardized cognitive test performance. The approach allows tests to be objectively ranked according to the systems engaged, as defined by the predictive power for that network. This predictive power reflects the contribution of that network of interest, providing a biologically grounded method for test evaluation and comparison, and an understanding of the brain systems that are relied upon for test performance. This provides an objective approach for assessing subscores and can guide the formation of composite scores that rely on the same brain networks in a manner that provides a more reliable outside-the-magnet test system. Composite test scores formed in this manner provide a robust assessment of specific brain systems, as indicated by enhanced predictive power in CPM. The input imaging data and the network definitions do not change, and thus, improved performance observed comes from more accurate test scores. Subscores can be evaluated based on the systems they reflect for performance, informing the inclusion or exclusion of each score in composite scores to specifically interrogate the network of interest. This also provides a brain-based mechanism for developing improved test strategies that more specifically target the network of interest.

Results

A transdiagnostic fMRI data set was collected from 302 subjects with diverse demographic backgrounds (described in Supplementary Table 1), including healthy controls and individuals with various psychiatric conditions (listed in Supplementary Table 2). The fMRI data collected consists of two resting runs and six task runs (described in Supplementary Table 3). The fMRI data was preprocessed with standard preprocessing methods, and the Shen268³⁷ atlas was applied to the preprocessed data, parcellating it into 268 functionally coherent nodes. To generate connectivity matrices, the mean time-courses of each node pair were correlated, and the correlation coefficients were Fisher z-transformed, generating eight (for the eight runs) 268 × 268 connectivity matrices per subject. Cognitive construct networks are extracted from the functional connectivity data. While any network of interest can be predefined, for this work, we chose to define the six cognitive construct networks, including attention, perception, declarative memory, language, cognitive control, and working memory. The networks are defined as the edges between the nodes that were extracted from NeuroSynth³⁸. In Neurosynth, the construct term (attention, for example) was input, and coordinates from the composite Neurosynth maps were extracted. These coordinates were then associated with nodes in the Shen268 atlas³⁷. In the connectivity matrices, the entire row for each node selected for a given network was included in that network. The coordinates/ nodes of the cognitive construct networks are included in Supplementary Table 4, and the cognitive construct networks for each imaging run vary in size from 7 × 268 nodes to 12 × 268 nodes.

The cognitive measures used in this study were collected through the administration of tests that are frequently used in behavioral studies, listed in Table 1. The complete list of post-scan data collected is displayed in Supplementary Table 5. Predictive modeling of each of the individual cognitive construct networks on each of the test measures was performed using separate kernel ridge regression models. All of the fMRI acquisition data was used such that the connectivity matrices from the two resting runs and the six task runs were concatenated as described previously³¹. Thus, the input feature to the model that predicts behavioral measures based on connectivity data consists of edges of matrices that vary in size from 7 × 268 × 8 to 12 × 268 × 8. More details on the cognitive construct network definitions and the predictive models are included in the Methods section. Here, predictive power is quantified using the coefficient of determination (R²), which captures the proportion of variance in the observed behavioral scores explained by the model’s predictions. In addition, in Supplementary Figs 1 and 2, we report the predictive power as the Pearson correlation (r) between the predicted behavioral measure and the observed behavioral measure. There are limitations to using Pearson correlation (r) as the metric for predictive modeling, as it is not sensitive to the scaling of the data. However, it remains a widely used metric for reporting modeling accuracy in brain-behavior modeling research and is used in recent high-impact studies^39,40, thus, we include it in the supplementary information.

Ranking the tests by cognitive construct network weight

To investigate the role of the construct networks in cognitive test performance, we calculated the predictive power of each construct network for each test. The predictive power of the network edges is taken as a measure of the contribution of that network to test score. Higher predictive power for a given construct network implies a larger role for that network in task performance. Figure 1 shows the predictive power the cognitive construct networks have on 16 behavioral tests, with an average predictive power of ~0.047 (coefficient of determination (R²)). For each network, we can rank the 16 tasks according to the predictive power for that network. The rankings are displayed with bars that indicate the performance of the predictive modeling across the 100 10-fold cross-validations (each cross-validation gives one value of the prediction performance, training and testing sets are randomized for each cross-validation). The attention network, for example, plays a major role in color-word interference performance but a minimal role on the 20 questions task, while the working memory network has the highest contribution on matrix reasoning and the lowest on letter-number sequencing.

Fig. 1 — N = 302 participants. Colored box plots show the coefficient of determination (R²) between observed and predicted scores across 100 10-fold cross-validations (100 iterations per model). Colors for each behavioral test are consistent across panels. Each cross-validation gives one value of the prediction. Training and testing sets are randomized for each cross-validation. Gray box plots represent null-model results obtained by permuting behavioral scores across connectivity data (1000 iterations per model). On each box, the red central line indicates the median, and the left and right edges of the box respectively indicate the 25th and 75th percentiles. Whiskers extend to the most extreme non-outliers. One-sided permutation tests (1000 shuffles) assessed whether observed predictive accuracies of each cognitive construct network on the 16 behavioral test scores exceeded chance. P-values were adjusted using the Benjamini-Hochberg procedure to control the false discovery rate (FDR), accounting for multiple comparisons (n = 16 comparisons) across tests. Asterisks denote significance after FDR correction (*p < 0.05, ** p < 0.01, ***p < 0.001). P-values are reported in Supplementary Table 6. Source data are provided as a Source Data file.

It is important to keep in mind that the goal is to understand the contribution of each a priori-defined network to test score. Once we understand the brain systems that the tests rely upon, it is then possible to make better use of combinations of tests and subtests in terms of interrogating specific brain networks or developing new tests altogether that maximally target the networks of interest.

To confirm that our findings were not driven by random associations, we conducted a null analysis by permuting the behavioral test values across the connectivity data (displayed in gray in Fig. 1). The randomized null analysis failed to reproduce the effects observed in the empirical data. As shown in Fig. 1, predictive performance significantly exceeded null-model results in 16/16 tests for the attention network, 15/16 for the perception network, 12/16 for the declarative network, 14/16 for the language network, 15/16 for the cognitive control network, and 14/16 for the working memory network. We report the p-values in Supplementary Table 6. The same analysis using Pearson correlation (r) between observed and predicted values as the measure of predictive performance is shown in Supplementary Fig. 1, with corresponding p-values reported in Supplementary Table 7 and in the Source Data.

The predictive performance of each cognitive construct network within a test is observed in Fig. 2. Here, we observe the predictive power, reflecting the contribution of each network to performance on a given test. For example, color-word interference is a subtest of the Delis-Kaplan Executive Function System (D-KEFS)⁴¹ that was designed to improve on the Stroop test^42,43 by including an inhibition/switching trial. As shown in Fig. 2 (row 4, column 1), the working memory, attention, and cognitive control networks best predict the color-word interference score (mean predictive power of 0.084, 0.081, and 0.073, respectively). This is consistent with previous studies (examining either the color-word interference test or Stroop test), which have shown that such test scores reflect working memory, attention, and cognitive control (or conflict monitoring)^42–44.

Similarly, the contribution of each cognitive construct network on the remaining 15 behavioral tests can be analyzed (displayed in Fig. 2, p-values are included in Supplementary Table 8 and in the Source Data). These results show that this feedback approach to predictive modeling provides a quantitative assessment of the network loading of task performance for each test. The analysis of the predictive power of the cognitive construct networks across each test in terms of the Pearson correlation (r) is shown in the Supplementary Fig. 2, and we report the corresponding p-values in Supplementary Table 9 and in the Source Data.

Testing the tests can be performed with any network of interest

To demonstrate that this approach can be applied to any network choices, we performed this analysis with alternate brain network definitions. In Supplementary Fig. 3, we include the performance of this model on LanA (a probabilistic functional atlas of the language network)⁴⁵ and the default mode network (DMN), as well as the seven canonical Yeo networks⁴⁶ and similar assessments of the contributions of component networks are made. The approach is general and could be applied to the user’s specific networks of interest.

Testing the tests can be performed with any functional connectivity state

Predictive performance is higher when performing predictive modeling with fMRI data collected while subjects participate in a continuous performance task, compared to models obtained from resting-state data⁴⁷. Of course, there are innumerable different tasks that could be presented (each varying in how they alter brain-state^26,48) and the impact of tasks on models is complicated. An advantage of resting-state data is in the overt simplicity of data collection with the caveat that brain-state is uncontrolled. However, to show that this approach can be applied to only data collected in the resting-state, we compare the predictive modeling results when using resting-state data alone with those obtained using resting- and task-state data. As shown in Supplementary Fig. 4, for five out of the six networks (other than the declarative memory network), the ranking of the behavioral tests for predicting with only the resting-state data was highly aligned with the ranking obtained using resting/task states, with an average correlation of 0.73 across these five networks (p < 0.05). These results show that the analysis approach can be applied to datasets consisting of only resting-state data.

Cognitive construct network-driven test combinations yield models with higher predictive power

We next examined the formation of composite test scores aimed at better interrogating a specific cognitive construct network. Forming composite test scores based on predictive performance for a single network yielded strong predictive performance, indicating that those test combinations are ideal for assessing the functional organization of that network via these external tests. To choose the external tests to combine, we split the dataset in a 2:1 ratio for training and testing. Within the training dataset, across each cognitive construct network, we ranked the individual tests according to predictive power (analogous to the results in Fig. 1). Next, within the testing dataset, starting from most predictive test, we progressively group the tests to form composite scores, adding one test at a time in order of descending predictive power, and perform predictive modeling on the composite test scores (the ranking of the tests to form the composites is based on modeling with the training dataset). As shown in Fig. 3, test combinations tend to outperform the single best-predicted test. Since administering the tests requires time, we aimed to identify the optimal balance between predictive power and testing composite size (i.e., testing duration). We compared the predictive power of the highest individual test (purple) to the predictive power of the test combinations, beginning with the smallest composites and progressively increasing the composite size. If the predictive power of the combinations of tests was not significantly greater than the highest individual test (purple), the boxplot is labeled in yellow. Once a combination of tests is significantly greater than the highest individual test, the boxplot is labeled in green. The following combinations are also labeled in green if each one shows a significant improvement over the previous. When a combination no longer shows a significant gain, it and all the following combinations are labeled in red. The final, green-labeled composite, is the optimal composite, as it has the optimal balance between predictive power and test duration (number of tests). To ensure that the high predictive power of the optimal composite is not simply due to noise reduction from combining multiple tests, we compare it to a control composite of random tests, matched in size and shown in light gray and to a composite of worst tests (defined in the training set), matched in size and shown in dark gray. For five out of six of the cognitive construct networks (only not for the perception network), the optimal composite significantly outperforms the random test composite, and for all six of the cognitive construct networks, the optimal composite significantly outperforms the worst test composite. P-values are reported in Supplementary Table 10 and in the Source Data.

The matrices on the right side of Fig. 3 display the percentage distribution of the 16 tests across ranking positions within the optimal test composite. The rightmost column of each matrix shows how often each behavioral test appears in the optimal composite. While a few tests frequently appear in the optimal test composites of all the cognitive construct networks (eg. matrix reasoning), the overall composition of the optimal test composites varies between the cognitive construct networks. Furthermore, the size of the optimal test composites varies across the different cognitive construct networks: six for the attention, four for the perception, four for the declarative memory, three for the language, five for the cognitive control, and four for the working memory construct network. Here, we showed that this brain-based method of forming composite scores leads to higher predictive power, suggesting that brain-derived composite score formation is a powerful approach to improve assessments made outside of the magnet.

Brain-construct modeling provides a means to evaluate the contributions of subtests

A comprehensive battery for cognitive and clinical testing typically encompasses numerous subtests. Two scores collected as part of this data include a clinical test score, the BRIEF-A⁴⁹ (Brief Rating Inventory of Executive Function – Adult Version) (which is a standardized measure that captures “inter-related higher-order cognitive abilities that are involved in self-regulatory functions which organize, direct, and manage cognitive activities, emotional responses, and overt behaviors”)⁴⁹ and a behavioral score, the DKEFS color-word interference score⁴¹ [which as described earlier in the Results, is a subtest of Delis-Kaplan Executive Function System (D-KEFS)⁴¹ that was designed to improve on the Stroop test⁴² by including an inhibition/switching trial⁴³]. We include these scores here as they are generally reported as composite scores with different subscores combined to yield a composite score.

Connectome-based predictive modeling can be used to measure the association between each network of interest and each subscore. The predictive power of these networks provides insight as to whether the subscores are reflecting the same brain systems or if they rely on the networks in a differential manner. Fundamentally, the subtests are aimed at measuring different aspects of the same function and thus may or may not involve the brain networks in the same manner. If subscores do involve the same brain networks, then the composite score should lead to increased predictive power for the model. As the results shown in Fig. 4 indicate, the predictive performance of composite scores and subscores do not always align in terms of the brain networks engaged. In certain instances, predictive performance improves when composite scores are modified to exclude the subscores with low predictive power. The BRIEF-A composite score (Combo S1-S4) is calculated as the sum of four BRIEF-A subscores: the inhibit raw score (S1), the shift raw score (S2), the emotional control raw score (S3), and the self-monitor raw score (S4). Shown in Fig. 4a, the predictive performance of the composite score on BRIEF-A⁴⁹ (Combo S1-S4) across all six cognitive construct networks is enhanced by removing a poorly predictive subtest (the self-monitor raw score = S4) from the calculation, resulting in Combo S1-S3. Notably, subtest S3 exhibited higher predictive power than either composite (S1–S3 or S1–S4) for most of the cognitive construct networks. These findings illustrate that individual subscores can, in some cases, outperform composite measures. Because subtests are designed to probe distinct aspects of cognition, their subscores likely capture different network contributions. This highlights the utility of brain-based metrics for assessing the component processes that each subscore reflects. Without such models, systems-level interpretations of subtest contributions would not be possible. In cases like DKEFS color-word interfence⁴¹, subscores consistently demonstrate better predictive power than composite scores, suggesting that the subscores are indeed measuring different aspects of cognition that rely on different brain correlates such that when combined they decrease predictive model performance (Fig. 4b). This was shown for a commonly used composite score for DKEFS color-word interfence⁴¹ (Comp1) which is calculated as the scaled sum of color-word interference test color naming condition 1 raw score (RS1) and color-word interference test word reading condition 2 raw score (RS2), and for another commonly used composite score for color-word interference (Comp2) which is calculated as the scaled difference of the color-word interference test inhibition condition 3 scaled score (SS3) and the color-word interference test color naming condition 1 scaled score (SS1). In both cases, at least one of individual subscores has higher predictive power than its corresponding composite scores. The subscores for BRIEF-A and color-word interference are further defined in the Supplementary Information Table 11. P-values are reported in Supplementary Table 12 and 13 and in the Source Data.

Fig. 4 — N = 302 participants. a Prediction performance of cognitive construct networks on subtests, the combination of all four subtests (red), and the combination of three out of four subtests (green) for the BRIEF-A. The standard BRIEF-A composite score (Combo S1-S4) is the sum of four BRIEF-A subscores: the inhibit raw score (S1), the shift raw score (S2), the emotional control raw score (S3), and the self-monitor raw score (S4). The sum of subtests 1-3 (S1-S3) [removal of the subtest with the lowest predictive power (S4) from Combo S1-S4] yields significantly higher predictive power (Combo S1-S3) for all of the cognitive construct networks [evaluated using two-sample one-tailed t-tests at the 5% significance level, with Bonferroni correction applied (n = 6 comparisons); p-values are reported in Supplementary Table 12]. b Prediction performance of cognitive construct networks on individual subtests and composite scores for color-word interference. Composite score 1 (yellow oval) has significantly lower predictive power than at least one of the subtests it is composed of [color-word interference test color naming condition one raw score (RS1) and color-word interference test word reading condition two raw scores (RS2), yellow rectangles]. Similarly, composite score 2 (purple oval, Comp2) has significantly lower predictive power than both of the subtests it is composed of [color-word interference test color naming condition one scaled score (SS1) and color-word interference test inhibition condition three scaled score (SS3), purple rectangles]. These differences in prediction performance are significant at significance testing of 5% [evaluated using two-sample one-tailed t-tests at the 5% significance level, with Bonferroni correction applied (n = 24 comparisons); p-values are reported in Supplementary Table 13]. Predictive performance is evaluated as the coefficient of determination (R²) between the observed and predicted values across 100 iterations. Each cross-validation gives one value of the prediction. Training and testing sets are randomized for each cross-validation. On each box, the red central line indicates the median, and the bottom and top edges of the box respectively indicate the 25th and 75th percentiles. Whiskers extend to the most extreme non-outliers. Source data are provided as a Source Data file.

Together, these findings highlight that this approach to network modeling of behavioral, clinical, and cognitive test scores can be used to assess the relative role of subscores in reflecting specific brain-circuits. Enhanced predictive power of the composite test scores suggest that these combinations are ideal for interrogating these specific networks. Thus, the brain data is used in this context to identify those composite test scores that better reflect specific brain circuits.

Test composite formation based on high prediction and cognitive construct network isolation

Thus far, we have focused on identifying test composites that yield the highest predictive power for each individual cognitive construct network without consideration as to predictive performance for the other cognitive construct networks. Here, we are interested in finding highly predictable test composites for which the predictive performance of each network is distinct from each other. In other words, we wish to identify composite tests that are networks specific – yielding high predictive performance for one network and low predictive performance for the other networks. To do this we sequentially evaluated growing composites scores for combinations of n = 2, 3, and 4 tests. The dataset was divided in 200 participants for training and 102 for validation.

In the first step, we measured the predictive power for all possible combinations of two-tests (120 unique pairs generated from the 16 behavioral tests). A combination was considered desirable if it met the following criteria: (1) The predictive power of the cognitive construct network on the test composite was in the top 5% most predictive composites (6/120 test composites). (2) The predictive power of the cognitive construct network on the two-test composite was significantly greater than its predictive power on either of the individual tests that composed the composite. (3) The isolated difference (defined below) in predictive power for the construct of interest was greater than zero. (4) The isolated difference of the predictive power of the construct of interest on the two-test composite was in the top 5% highest isolated differences (6/120 test composites). We define the isolated difference as the predictive power of the cognitive construct of interest on a given test composite minus the maximum predictive power of the other five cognitive constructs on that composite. Tests were ranked based on predictive power and isolated differences across 20 random data splits (100 iterations each), and the top 5% were determined from the overall ranking results. Only three distinct two-test combinations met the criteria: finger windows & matrix reasoning and vocabulary & matrix reasoning for the language network and finger windows & verbal learning for the cognitive control network. The remaining four cognitive construct networks did not predict any two-test combinations that met the criteria described above. Figure 5a displays the ranking of the 120 two-test combinations by predictive power (left) and by isolated difference (right) for the example language network. Figure 5b shows a zoomed-in view of Fig. 5a, illustrating the top 5% of test pairs for predictive power and isolated difference for the language and cognitive control networks. The two-test composites that pass the criteria for each respective network are highlighted in magenta. Building on this, we evaluated how these cognitive construct networks performed as test composites increased in size. Specifically, we generated three-test-combinations by adding each of the remaining 14 behavioral tests to the two-test pairs. We were interested in three-test combinations for which: (1) The predictive power of the construct of interest was significantly greater than the predictive power of that network on the two-test combination, and (2) The isolated difference of the construct of interest for the composite was greater than zero. Four three-test combinations met these criteria for the language network (finger windows & matrix reasoning & Boston naming test, finger windows & matrix reasoning & reading, finger windows & matrix reasoning & color-word interference, finger windows & matrix reasoning & vocabulary), while none of the combinations met these criteria for the cognitive control network. We extended this approach to four-test composites, where the language network exhibited one composite (finger windows & matrix reasoning & vocabulary & Boston naming test) that met both network isolation criteria (isolated difference>0) and that the predictive power is significantly higher than the higher predictive power of the three-test composites. The predictive performance of the language and cognitive control networks on individual tests and on qualifying composites (up to n = 4) is shown in Fig. 5c. P-values are reported in Supplementary Table 14 and in the Source Data.

Fig. 5 — N = 200 participants. a Left: Predictive performance of the language network across all 120 unique two-test composites formed from 16 behavioral scores, sorted by predictive power. Right: Isolated difference of the language network for the same composites, sorted by isolate difference. The isolated difference is defined as the predictive performance of the target network (e.g., language) minus the maximum predictive power of the remaining five cognitive construct networks for that composite. b Zoomed view of (a) highlighting the top 5% of test combinations (6/120). Two two-test composites for the language network and one two-test composite for the cognitive control network meet both high predictive power and network isolation criteria, and are shown in magenta. c Predictive power of individual tests and of the composites meeting the selection criteria. For these composites, the predictive power of the target construct networks is significantly higher than that of the subsequent (n–1) composites. Two-test composites are highlighted in magenta, three-test composites are highlighted in yellow, and four-test composites are highlighted in purple. Predictive performance is calculated as the coefficient of determination (R²) between the observed and predicted values across 100 iterations. Box plots included indicate the performance of the model across the average of each of the 20 splits across 100 10-fold cross-validations (each cross-validation gives one value of the prediction, training and testing sets are randomized for each cross-validation). For each cognitive construct network, separate kernel ridge regression models predict the scores. On each box, the red central line indicates the median, and the left (in part b, and bottom in part c) and right (in part b, top in part c) edges of the box respectively indicate the 25th and 75th percentiles. Whiskers extend to the most extreme non-outliers. Paired-sample one-tailed t-tests were conducted on the mean performance across the 20 splits over 100 iterations of 10-fold cross-validation: * indicates p < 0.05, ** indicates p < 0.01, and *** indicates p < 0.001. P-values labeled by asterisks are reported in Supplementary Table 14. Source data are provided as a Source Data file.

The same pattern of results was observed in an isolated dataset consisting of 100 of the 302 original subjects. The two-test composites exhibited greater predictive power than their individual components (p < 0.05 for 5/6 of the two-test composites in comparison to the individual tests, p > 0.05 only for the comparison between matrix reasoning and vocabulary & matrix reasoning for the language network). The predictive power of the top two-test composites among the 120 evaluated for each network consistently ranked in the top 11% (language: finger windows & matrix reasoning rank 1/120, vocabulary & matrix reasoning rank 13/120; cognitive control: finger windows & verbal learning rank 5/120). The isolated difference for the top two-test composite was within the top 6% or greater than zero (language: finger windows & matrix reasoning rank 1/120 and isolated difference = 0.0184, vocabulary & matrix reasoning rank 2/120 and isolated difference = 0.0092; cognitive control: finger windows & verbal learning rank 7/120 and isolated difference = −0.0049). Extending the analysis to three-test composites for the language network, the predictive power of the three-test composites exceeded that of the two-test composites for three out of four of the three-test composites (p < 0.05 for 3/4 three-test composites, p > 0.05 only for finger windows & matrix reasoning & vocabulary), and the isolated differences were greater than zero for all four three-test composites. Furthermore, the four-test composite for the language network significantly outperformed the predictive power of the three-test composites (p < 0.05), and the isolated difference was greater than zero.

Discussion

Assessing what systems test performance relies upon

By reversing the common brain-behavior modeling strategy and using pre-defined networks of interest, we show how modeling can be used to better understand brain measures obtained through test instruments performed outside of the magnet. This approach yields a quantitative method for assessing out-of-magnet tests in terms of the brain systems they rely upon for performance. It also provides a platform for the development of new tests that better isolate the networks of interest. While results are shown for a limited number of cognitive and symptom metrics and six cognitive construct networks, the approach is general and can be applied to any test metric and any a priori-defined network. While we used kernel ridge regression for our predictive modeling in this work, the contribution of each construct network can be assessed in terms of its predictive power using a range of machine learning methods^25,29,50. Other approaches such as canonical correlation analysis (CCA) and iterative optimization could also be used to identify the behavioral test composites that reflect the brain networks with high specificity. One of the strengths of this feedback approach for testing-the-tests is that it can be combined with a feedforward approach to iteratively develop better test instruments and with better defined brain networks. Once the imaging data is collected, it doesn’t need to be repeated, and the same subjects could be used to test new measures as they are developed outside of the magnet, and then these new test results can be modeled with the existing functional MRI data.

Brain-based assessment of the systems the tests are testing

In this study, we investigated the predictive power in models linking networks to performance for six predefined cognitive construct networks and 16 standardized test scores. In most imaging applications of brain-behavior modeling, the measures are used to build models that reveal the brain circuits supporting performance in the behavioral test of interest. In this work, we pre-define the networks of interest and reverse the information flow to discover the extent to which a test instrument relies on each brain system contributes to test score. This approach, provides quantitative information on the network-level contributions to task performance in standardized testing. The approach is general, and it can be applied to any cognitive test or clinical or symptom measures. While many of the cognitive tests used in this work have been used for more than 60 years, they have not previously been assessed in this context. This approach provides a method for choosing the test(s) that maximally rely upon the network of interest and an approach to quantitatively determine the extent to which the test reflects the system it was designed to measure.

As is well known and confirmed by the results presented here, none of the cognitive tests fully isolate specific cognitive constructs, and hence they do not rely on a single network for performance. But we now have a framework to evaluate the tests in terms of the brain systems they reflect. Such an approach can be used for further refinements of the tests in the direction of maximizing dependence upon the network of interest while minimizing dependence upon other circuits, potentially leading to more informative cognitive tests.

Composite test scores and cognitive construct network weighting

At the individual network level, we showed that choosing composite test scores from the top-performing tests for a given network led to substantial increases in predictive power (Fig. 3), suggesting that these outside-the-magnet composite tests can provide improved insight into subject network properties.

One factor that emerges in this approach is the role of overall fluid intelligence in these models. The predictive performance for the matrix reasoning test was often the #1 or #2 ranked test in terms of predictive power (as shown in the matrices in Fig. 3), indicating how overall fluid intelligence tends to dominate test performance. This is consistent with factor analyses of cognitive scores showing a primary factor dominated by intelligence^51,52. To better isolate components of cognitive function of interest, it may be necessary to develop tests that dissociate fluid intelligence from other cognitive factors, and the approach presented here provides a means to assess the progress in this direction.

Similarly, assessment of the subscores that contribute to standardized composite scores reveals cases where composite scores may include components that rely upon different circuitry. While subtests are purposefully designed to measure different aspects of cognitive function, the findings here, identifying the circuits they engage, can provide guidance as to whether combining such subtests into a composite score makes sense for brain-behavior modeling. We show, for example, cases where the inclusion of a specific subscore in the composite degrades the predictive model performance. Poor predictive performance implies a weak link between brain functional organization and the composite test score, suggesting the individual subscores don’t impinge on the same brain systems. In the development of these brain-behavior models, this approach then provides a means for assessing the network dependency of each subscore and can be used to guide the formation of composite scores. Subscores in this context should all reflect the same underlying brain circuitry to better assess that circuitry. As the results show, this is not currently the case with some composite scores, noting of course that these scores were not originally designed to interrogate specific brain networks.

We also evaluated combinations of behavioral tests for which the cognitive construct networks exhibit uniquely high predictive power. As shown in Fig. 5, this analysis led to the formation of test combinations of up to n = 2 for the cognitive control network and n ≥ 4 for the language network. In contrast, the remaining four cognitive construct networks did not yield any test combinations that led to both highly and unique predictive power. Several factors may explain this pattern; for instance, there is a general tendency for multiple networks to predict well on similar test combinations, fulfilling the high-prediction criterion but not the uniqueness one, possibly due to general intelligence contributing broadly to performance across tests. Furthermore, the analysis involves only 16 behavioral tests and six cognitive construct networks, leaving limited opportunity for a test combination to be both highly and uniquely predicted by a given network. Running such an analysis with a larger number of behavioral tests and more granular cognitive construct network definitions could potentially lead to more test combinations making the requirement of high predictability and uniqueness. Additionally, none of these tests were developed with imaging data available to assess the brain systems supporting performance nor was that a criterion in test development. Moving forward, new tests may be developed that are better suited to these goals.

Transdiagnostic dataset

We investigate this work in a transdiagnostic sample context testing the notion that such models can be derived in a diverse population and hence have a high chance at generalizability. One of the problems with a less diverse subject sample (such as the original data from the Human Connectome Project²²) is that there is little variation in external measure scores (everyone is at ceiling in the scores) and such samples are not conducive to modeling. While we have previously shown predictive modeling of fluid intelligence in that non-diverse sample^25,29, most of the external cognitive measures are poorly distributed and thus not appropriate for building such models, whereas the more well-distributed behavioral scores derived from a heterogenous population, as used in this study, are better suited for this purpose.

Network definitions

This approach is general and can be applied in the context of any a priori brain networks of interest. Here, we defined the construct networks by first defining the nodes for each construct using Neurosynth³⁸ and then including all edges connected to those nodes (the entire row of the connectivity matrix for each node). The focus of this work is to demonstrate how brain-behavior modeling with a priori-defined network can inform assessments made outside of the magnet, and we make no claim here as to the veracity of the networks used for this demonstration. In practice, the user may define any network(s) of interest for assessing any external test measure(s) of interest. This work should lead to further investigations developing refined network definitions and test measures to better understand brain-behavior relationships. While only six networks were used, additional networks, and/or refinements of the networks could lead to further insights. For example, the attention network can be broken down into multiple sub-attention networks (visual attention, auditory attention, motor attention, etc.⁵³) based on contributions of the network to various input modalities, including visual, auditory, motion, etc. This would lead to a more granular understanding of the role of each brain system in supporting performance on a cognitive test or symptom score.

While here the focus was on cognitive networks, one could also define networks for other domains such as the positive and negative valence domains, social processing, arousal and regulatory systems, and sensorimotor domains and evaluate their role in external measures of these systems. The development of reliable network assessments for this breadth of domains could greatly improve the use of this predictive modeling framework in clinical applications. As shown in Supplementary Fig. 3, we performed this analysis using the Yeo 7-canonical networks⁴⁶, and the results demonstrate that rankings of the contribution of the different networks to task performance can be obtained for each of those networks.

The original form of brain-behavior modeling was aimed at showing that there is a relationship between brain organization and a range of trait measures and that this relationship can identify the networks supporting such traits^25,29. In this work, we defined our networks independently, using Neurosynth to identify the nodes. However, a combination of approaches such as Neurosynth to define initial target nodes and whole-brain predictive modeling^25,29 could yield optimal network definitions. By including all connections for each node (the entire row for the connectivity matrix for that node) identified in Neurosynth for our network definitions we also included connections between the primary nodes and all other nodes. This was motivated by the fact that the primary networks had only a very limited set of nodes (seven to twelve nodes), and the statistical power associated with such small networks was low. However, the differences in the predictive performance across the cognitive construct networks are not driven by network size in the size range that we used. As shown in Fig. 2, larger networks do not systematically predict more accurately, and the ranking of the cognitive construct networks by predictive power differs across the tests. The expanded network definitions, however, likely reduced the differences in predictive power between the networks, and more refined network definitions could potentially lead to more significant differences in the comparisons presented. It should also be noted that when these six cognitive construct networks were combined, they yielded significantly better predictive power to the whole-brain analysis (Supplementary Fig. 5), suggesting that these networks captured the relevant cognitive circuits underlying the behaviors assessed by these instruments while reducing the noise inherent to whole-brain models.

Measures of functional connectivity

Functional connectivity (FC) in this work was estimated as the Pearson correlation between the preprocessed BOLD time series of each pair of brain regions. Although other estimators, such as partial correlation or regularized multiple regression can capture conditional dependencies that Pearson correlation cannot^54–57, Pearson correlation remains the most widely used approach for FC⁵⁸. Therefore, we adopted this conventional measure to ensure comparability with prior studies. Furthermore, alternative estimators are sensitive to free tuning parameters, whereas Pearson correlation is parameter-free, thus, minimizing the parameter variability concern. As the field converges on a standardized metric for estimating FC, our approach should be enhanced with new approaches to quantifying functional connectivity.

Reliability considerations and predictive power

In this work, we have used predictive power as a measure of the contribution of that network to test performance. Several factors influence predictive power performance in this brain-construct modeling framework. In general, the reliability of the functional connectivity maps in an individual plays a role in determining predictive power achievable from such data³⁰, but here, the fMRI data is the same across all conditions and, therefore, does not contribute to the differential rankings across tests. The only factor that changes across these models is the test instrument data or clinical score and the resulting composite scores. The finding of increased predictive power in the modeling indicates that, indeed, better composite test scores can be made when the composite score formation is directed by the predictive power of individual scores.

It is well known that many of the standardized tests do not have high test/retest reliability⁵⁹. The reliability of the brain test modeling is highly dependent upon the reliability of the test input data⁵⁹. Many of these standardized tests have published reliability statistics that can also be integrated into this analysis and considered in the test ranking. For a given test, however, assessing the relative contribution of each network is somewhat independent of the reliability of either the fMRI data or the test data because those factors are consistent for each network.

In addition to test reliability, the discriminant ability of a test is also important in building predictive models of the type considered here. Ideally, in a study population, test scores are distributed across a wide range to facilitate models linking functional connectivity edge strength to performance. Some cognitive tests are designed to distinguish normal cognitive performance from a small minority of subjects that might exhibit deficits. In this scenario, much of the data is at the ceiling, and it is difficult to build predictive models.

An additional consideration is the degree to which a test has a clear brain correlate. In developing predictive models from cross-sectional data, as we have done, there is an implicit assumption that the same brain networks are used by all individuals in supporting test performance. Predictive models may have low power if there are multiple strategies that may be employed to execute a test, and these different strategies employ different brain circuits. Figure 1 shows that some of the tests have poor predictive power and are not suitable for brain-behavior modeling. This could be due to the reliability of the test, or it could be that the tests allow individuals to use different strategies, and hence brain circuits, to complete the test. Different strategies could rely upon a variety of brain circuits, making it difficult to model such test measures.

As previously reported³¹, predictive models can reveal when biases are present in tests, and models may fail for individuals for whom the test was not designed. Such biases may be identified through an analysis of outliers, and factors to consider include sex, education, socioeconomic status, and cultural biases. Many of these are already well-documented for standardized tests^60,61.

Task functional connectivity & state-dependance

The functional connectivity matrices used in this work consisted of concatenated connectivity matrices calculated from both resting-state and continuous performance task data (two resting and six task runs). The inclusion of task-fMRI data was largely motivated by the goal of capturing evoked brain states that are elicited by task engagement. Previous works has shown that the functional connectivity network experiences modulation as a function of task, brain state, development, and aging^8,47,62–76, spanning moment-to-moment fluctuations to long term changes^28,77,78. State-dependent changes in connectivity have also been verified in preclinical models⁷⁹. There is an emerging consensus that brain-behavior modeling with continuous performance tasks lead to higher predictive power through the contributions of task, intrinsic connectivity, and interactions. Additionally, by synchronizing activity across subjects with tasks, and removing unwanted variance associated with the resting state, unconstrained intra-individual variance (noise) is removed⁴⁷. Tasks have also been shown to reduce motion in clinical populations again providing better performance in predictive modeling than resting-state acquisitions^80,81.

Task-state and resting-state functional connectivities share a stable network architecture^82–84. The brain states afforded by task runs represent the evoked parts of this shared stable network structure⁸². This relatively stable component of functional connectivity has been drawn as analogues to the large-scale collection of Hebbian synaptic weights: the continuous recruitment of a set of brain regions results in the changes in the synaptic connections between them, and this brain network of synaptic connections is captured in the resting state functional connectome^55,82,85. Similarly, under different conditions (task and resting), and across the years, each individual can be identified by a stable functional connectivity pattern^25,84,86. The alignment in the ranking of the tests between resting and resting/task states shown in Supplementary Fig. 4 provides support for the hypothesis that resting and task functional connectivity share an intrinsic network.

Within this framework, tasks can induce coactivation of multiple regions of the functional connectome. To mitigate this effect, it has been proposed that task activation effects should be regressed prior to the on-task functional connectivity analysis⁸⁷. This is true in block design studies, but in the continuous performance tasks used here (there is no task/control cycle) the inter-stimulus interval is such that any activation is essentially in the steady-state, thus, there we do not observe the classical BOLD effect commonly thought of when the term task-evoke activation is employed (we instead have a state manipulation as opposed to an evoked activation). Furthermore, in the predictive modeling framework presented in this study, differences in the functional connectivity across participants are measured, thus, the coactivation of multiple regions of the task-state functional connectome is unlikely to confound the analysis. The Supplementary Fig. 4 results rank the tests similarly showing that the use of task-state functional connectivity without coactivation task effect removal in this predictive modeling framework did not skew the results.

It is important to note that all six in-scanner task runs involve visual simulation and are largely not designed to engage other sensory domains, such as auditory processing and motor function. These tasks were chosen to engage the brain along different cognitive constructs of working memory, emotional intelligence, response inhibition, and attention. It should be noted that these are all visually based tasks, and there could be other scanner tasks that may be better suited for modeling specific external measures. As task-based functional connectivity is further studied, more refined networks are defined, and better external measures are developed isolating specific cognitive constructs, we expect to learn more about the relevant circuits as the field advances.

Quantitative scores versus self-report measures

This work makes it clear that some tests do not yield strong predictive models. A notable observation is that cognitive tests that offer quantitative measures, such as accuracies or reaction times, can often generate models with high predictive power. The same is not true of tests that rely upon self-report or even reports of others. Unfortunately, in clinical contexts, such as those involving psychiatric illnesses, much of the information available is derived from self-report, and there is no reaction-time or accuracy-equivalent score for conditions like depression or mania⁸⁸. This is not to imply that tests without a clear brain correlate are not useful, only that they are not useful for brain-behavior modeling. However, the framework presented here provides a brain-based methodology for evaluating such measures, offering a promising avenue for the objective assessment of clinical presentations. Predictive power in this scenario provides a means for assessing when this problem exists, and it can provide further guidance when attempting to design improved measures that reflect specific brain circuits.

Clinical applications of this framework

This approach to reversing the operation of traditional CPM provides a framework for the development of better symptom measures that reflect specific brain circuits. This could have major implications for clinical practice. One can envision using the forward model in CPM to identify a network that reflects changes in symptom scores across patients, and through an iterative forward and reverse modeling approach (using independent datasets) we could refine clinical measures (the symptom scores) and refine the networks associated with the symptoms. The objective quantitative relationship between brain circuitry and disease will lead to more accurate diagnosis, better understanding of the circuits involved, and the development of new treatments, ultimately improving treatment outcomes. This approach could lead to the development of a family of clinical instruments that provide clear insights into what is happening in an individual’s brain directly through the test instruments and without the need for an fMRI scan.

Models measure networks that vary with performance

It’s important to consider what the predictive models in this context measure and what they do not measure. The models do not provide an assessment of all the networks that may be necessary for an individual to perform a task. The models reflect the circuits that vary across subjects as a function of task performance. This is analogous to the problem in task-based fMRI, wherein activations reflect only those regions that change between task and control conditions. In such a scenario, it is well accepted that other regions may be active but at a constant level between the task and control conditions and thus do not appear in the difference maps. In brain-behavior modeling there may be other networks that are relevant to test performance but that don’t vary across individuals as a function of that performance.

Applying the testing-the-tests framework

Reversing the usual forward problem in brain-behavior modeling, we introduce an approach that allows testing-the-tests. Partitioning the human connectome into a priori-defined networks enables a quantitative assessment of the role of each network in supporting test performance, and this, in turn, provides information as to what systems the test scores are reflecting. Models that establish the association between brain connectivity and cognitive test performance or symptom measure offer a framework for the bidirectional exchange of information between the tests and brain data. This information transfer operates in a feedforward manner, allowing external tests to isolate brain circuits, as demonstrated in previous studies, and in a feedback manner, revealing the brain systems that the test scores really reflect. The predictive power of each brain network can provide important information on tests, symptom scores, or other phenotypic measures.

This framework can be used to derive brain-driven composite test scores combining those tests that rely upon the same brain systems supporting performance. It can also be used to assess the contributions of subscores prior to forming composite scores to ensure additive power for each network model. We show that in some cases, subscores that form composite scores in the standard framework currently used, do not reflect the same brain systems and thus yield lower predictive power in brain-behavior modeling. Assessing how subscores associate with specific brain systems in the design of composite scores can lead to novel, outside-of-the-magnet, assessments that better reflect specific brain systems of interest.

This approach now allows standardized tests, employed in psychology for over 60 years, to be assessed in terms of the specific construct networks upon which they rely for task performance. It provides a framework for the development of new tests that better isolate specific networks of interest and/or the development of composite scores that converge on specific circuits. This approach is general and can be applied with any network definitions and to any external measures (tests, clinical measures, and symptom scores) that have clear brain correlates. Future work can use this feedback approach to refine existing tests or develop new ones that more effectively isolate specific brain circuits, and these, in turn, can be used in a feedforward sense to better isolate the brain circuits of interest.

Methods

Dataset

The dataset was acquired at Yale School of Medicine between February 2018 and August 2024. The dataset consists of imaging data, behavioral measures, and clinical scores of 302 subjects (demographic, clinical, and diagnostic information are included in the Supplementary Information Tables 1 and 2). The cohort consists of both healthy subjects and patients (Supplementary Information Table 2).

The imaging data was collected at Yale on a Siemens 3 T Prisma scanner with a 64-channel head coil. A high-resolution 3D anatomical scan (T1-weighted magnetization-prepared rapid acquisition with gradient-echo (MPRAGE) sequence [208 slices acquired in the sagittal plane, repetition time (TR) = 2,400 ms, echo time (TE) = 1.22 ms, flip angle = 8°, slice thickness = 1 mm, in-plane resolution = 1 mm × 1 mm]) was obtained for alignment to common space. The functional data were obtained using a multiband gradient-echo-planar imaging (EPI) sequence (75 slices acquired in the axial-oblique plane parallel to the AC–PC line, TR = 1,000 ms, TE = 30 ms, flip angle = 55°, slice thickness = 2 mm, multiband factor = 5, in-plane resolution = 2 mm × 2 mm).

Participants completed MRI scans, which were followed by a post-scan neuropsychological and self-report battery; the scan and post-scan battery were each approximately two hours in length and were designed to represent a wide cognitive terrain corresponding to the cognitive constructs. We used data from eight separate imaging conditions: two resting-state runs and six task runs - each run being 6 minutes and 24 seconds long. The six task runs included: (1) 2-back (working memory), (2) stop-signal (response-inhibition); (3) card guessing (reward); (4) gradual-onset continuous performance (gradCPT; sustained attention); (5) reading the mind in the eyes (social); and (6) movie watching tasks (the in-scanner runs are described in Supplementary Table 3). The first and last functional scans were resting-state runs for which the participants were asked to rest with their eyes open while a fixation cross was displayed. The participants completed each of the six tasks during the remaining runs, and the Psychtoolbox-3 presented tasks were ordered in a counterbalanced manner across participants. Each task, excluding the movie-watching task, was prefaced with instructions and practice, which were followed by the opportunity for the participant to ask questions about the task. The participants’ responses were recorded on a two-by-two button box, and a fixation cross was shown between the tasks.

Image processing

Skull stripping of the structural scans was done using optiBET, which is an optimized version of the FMRIB’s Software Library (FSL, 6.0.1.) pipeline⁸⁹. SPM12 was used for motion correction⁹⁰. Nonlinear registration of the MPRAGE to the MNI template was performed using BioImage Suite (35)⁹¹, and linear registration of the functional to the structural images was done through a combination of FSL and BioImage Suite (35). The remaining preprocessing steps were performed in BioImage Suite (35), including regression of mean time courses in white matter, cerebrospinal fluid, and grey matter; low-pass filtering (Gaussian filter, σ = 1.55, with equivalent cut off frequency of 0.12 Hz)³¹ and filtering to correct linear, quadratic, and cubic drift; regression of 24 motion parameters. Analyses and visualizations were conducted with BioImage Suite (35) and MATLAB (Mathworks, R2019a). The dataset followed the same image processing procedure as that in work published by Greene³¹ and Rodriguez⁹².

All registered data were visually examined to ensure whole-brain coverage, adequate registration, and the absence of artifacts or other quality issues. Subjects that did not meet the expected imaging criteria were excluded from the study. Subjects who completed all eight fMRI scans whose grand mean frame-to-frame displacement was less than 0.15 mm and whose maximum mean frame-to-frame displacement was less than 0.2 mm were selected for this study. Two participants who were scanned for all or some of the protocols under a slightly shorter scanning time (25 seconds shorter) were not excluded from the study, as the missing time segments of these scans were expected.

Functional connectivity

The Shen268 atlas³⁷ was applied to the preprocessed data, parcellating it into 268 functionally coherent nodes. The mean time courses of each node pair were correlated, and the correlation coefficients were Fisher transformed, generating eight (for the eight in-scanner runs) 268 × 268 connectivity matrices per subject.

Subjects

The study subjects were recruited through community advertisements and referrals made by Yale clinicians. The participant group is composed of clinically and demographically diverse subjects that experience a range of symptom severities. The patient sample was enriched for mental illness and purposefully designed to be trans-diagnostic. Patients experienced a range of symptoms and symptom severity and frequently had multiple psychiatric diagnoses. The participants had a mean age 30.81 years (SD = 10.93), including 170 females and 132 males. Sex was self-reported; balance was sought out, but the distribution was not strictly controlled for. The measurements were taken from distinct samples and the samples were measured once. The dataset was collected under a protocol approved by the Yale Institutional Review Board. Written informed consent for participation and data sharing was collected from all participants. Participants received monetary compensation for their participation. A subset of the dataset was included in work published by Greene³¹ and Rodriguez⁹².

Cognitive constructs & a priori network definitions

The six cognitive constructs considered in this study are attention, perception, declarative memory, language, cognitive control, and working memory. The attention, perception, language, and working memory networks were defined by searching the respective terms (attention, perception, language, working memory) on the Neurosynth website³⁸ and manually selecting the centers of ROIs displayed on the Neurosynth brain/ROI mapping interface. The declarative memory network was similarly defined by searching the term hippocampus on Neurosynth. The Neurosynth coordinates were pulled from a meta-analysis of thousands of fMRI studies (1831 studies for attention, 1278 studies for perception, 1059 studies for declarative memory, 1101 studies for language, and 1091 studies for working memory). The cognitive control network coordinates were extracted from the cognitive control network definition by Cole and Schneider⁹³ rather than through the Neurosynth website, as the latter’s coordinates are based on less than 600 meta-analysis studies and only reveal a very small set of nodes. The Talairach coordinates for each cognitive construct network were converted to nodes of our function-based brain atlas Shen268²⁹ based on overlap of the coordinates. This resulted in 8 nodes for attention, 9 nodes for perception, 10 nodes for declarative memory, 11 nodes for working memory, 7 nodes for language, and 12 nodes for cognitive control construct. Talairach coordinates, and their respective Shen268³⁷ nodes are listed in Supplementary Table 4. In total, the six constructs comprised 47 nodes, of which 10 were shared across networks. Specifically, nodes 166 and 177 overlapped among the attention, cognitive control, and working memory networks, resulting in eight unique nodes overlapping across constructs: [28, 32, 43, 63, 165, 166, 177, 197]. Figure 6 in Supplementary Information displays the cognitive construct nodes in axial slices of the brain. In addition, through the study’s GitHub page⁹⁴, we have shared the NIFTI mask files for the mapping between each specific cognitive construct network and the Shen268 atlas (https://github.com/YaleMRRC/testing_the_tests/tree/main/cognitive_construct_network_masks).

To determine if these networks span the relevant circuitry to capture behavior, we compared the predictive performance of these six networks combined versus the whole brain (Supplementary Fig. 5), In 15 of the 16 behavioral tests, the corresponding construct networks demonstrated greater predictive power than the whole-brain model (Supplementary Fig. 5). These findings suggest that the six cognitive construct networks capture the cognitive processes underlying these scores while reducing the noise present in whole-brain analyses.

Behavioral measures

Most behavioral studies rely on combinations of behavioral tests that are extracted from the same standardized test banks, such as Wide Range Assessment of Memory and Learning (WRAML)⁹⁵, Delis-Kaplan Executive Function Scale (D-KEFS)⁴¹, Wide-Range Achievement Test (WRAT)⁹⁶, etc. These tests are commonly used in cognitive neuroscience studies under the assumption that they activate specific cognitive systems - for instance, it is generally assumed that WRAML’s finger windows task draws on visuospatial memory, immediate (short-term) memory, and attention^95,97–101. However, there is very little literature on the use of functional connectivity to verify that the tests engage the intended brain networks. To quantify the extent to which a given network plays a role in task performance, we assessed the predictive power in connectome-based modeling for each of the six cognitive networks and 16 standardized behavioral tests. The 16 tests are summarized in Table 1. All of the tests in Table 1 are age-normed according to the test manuals. The scaled subscores and composite scores included in Fig. 4 were adjusted using age-normed procedures, while the raw scores were not. Importantly, all of the functional connectivities and behavioral scores underwent regression of five covariates (age, sex, race, English as a first language, and education), thus, there should be no covariate bases embedded in the findings which is reflected in the low correlations between the predicted mean square errors and covariates across the cognitive constricts and behavioral scores (Supplementary Tables 15–24).

Model

We applied kernel ridge regression as our predictive model. Kernel ridge regression is a nonlinear machine learning method that has previously been used for behavior prediction using resting-state and task-based fMRI data^50,102. It implicitly employs a nonlinear mapping Φ(·) that transforms the input space to a high dimensional feature space and builds a linear regression model using the feature vector as shown in Eq. 1.

C = \frac{1}{2} \sum {(y_{i} - w^{T} Φ (x_{i}))}^{2} + \frac{1}{2} λ {∥ w ∥}^{2}

Our predictive model is a variant of CPM^26,29. In the naïve CPM algorithm²⁵, edges that vary as a function of task performance are selected across the whole brain, with no constraints on locations and only a feature selection step to select relevant edges. In this study, because we are interested in individual cognitive construct networks that are defined a priori, the feature selection step is essentially the entire network defined a priori for that particular construct (described in the Cognitive constructs & a priori network definitions section). We used the Shen268 atlas³⁷, which was defined in a separate population of healthy subjects and covered the whole brain (Online Methods, Yale data set). The whole-brain connectivity matrix is found by taking the correlation between each node and the other 267 nodes, resulting in a 268 by 268 connectivity matrix. The construct networks are then extracted from this as the rows of the whole-brain connectivity matrix for the nodes identified for that construct using Neurosynth³⁸ (described in the Cognitive constructs & a priori network definitions section). For example, for the perception cognitive construct network, we identify the nine nodes associated with perception in Neurosynth and form a matrix consisting of the rows for each of those nine nodes from the whole-brain connectivity matrix. Such a matrix was created for each of the eight task runs, and the edges for the eight matrices were concatenated. Thus, the input feature of the perception cognitive construct network consists of 9 (nodes) × 268 (nodes) × 8 (runs) edges. We use these matrices as the input and compute the kernel based on the L2 distance between the matrices (normalized by the number of included edges). The predicted value y^* of a new data x^* is given according to Eq. 2.

y^{*} = \bar{y} {(K + λ I)}^{- 1} κ (x^{*}),

where K is the n-by-n kernel matrix of the training set, and κ(x^*) is the n by 1 kernel vector of the test sample to the training set. We chose the Gaussian kernel for the model, which is defined in Eq. 3.

G (x, y) = e^{- σ d^{2}},

$∥ d (x, y) ∥$ is the L2 distance between the input features (edges within the associated construct or within the whole brain). The L2 distance is calculated and then normalized by the number of edges. The scale parameter σ is selected to be approximately on the same order as the inverse of the mean distance squared between subjects. After normalization, the mean distance squared between subjects is approximately 2, and the scale parameter σ is set to 0.2 for all models.

In all cross-validation iterations, to simplify the model selection procedure without compromising the performance, we set all λ to 1 as it is, on predictive modeling performed on a subset n = 200 participants of the data, the average lambda value for which the maximum predictive power is achieved across the six constructs for the 16 behavioral scores.

It is important to note, for out-of-sample extensions with larger number of subjects, it may be necessary to retune the parameters, as the subject space is more densely sampled, enabling the use of more localized kernels. If the external data size is small, we recommend using the same strategy described above for parameter selection.

To assess the model’s predictive power, we compute the coefficient of determination (R²), which quantifies the proportion of variance in observed behavioral scores explained by the kernel ridge regression predicted scores, across 100 randomizations. In addition, for the analysis in Figs. 1 and 2, we quantified the predictive performance using Pearson correlation (r) between the observed and predicted scores (reported in Supplementary Figs 1 and 2).

The same approach was used when analyzing the predictive model on the whole brain and six cognitive constructs combined. In the case of whole brain prediction, the input feature consists of the edges of the matrix of size 268 (nodes) × 268 (nodes) × 8 (runs). Similarly, when all six cognitive construct networks are combined, the input feature consists of the edges of the 47 (8 nodes for attention + 9 nodes for perception + 10 nodes for declarative memory construct + 11 nodes for working memory + 7 nodes for language + 12 nodes for cognitive control construct – 10 nodes that overlap across the cognitive constructs) × 268 (nodes) × 8 (runs).

To eliminate the confounding effect from demographic covariate factors, we removed age, sex, race, English as a first language, and years of education from both the connectivity matrices and the scores. In the training set, linear regression was applied to individual edges in the matrix, and the edge connectivity was replaced by the residuals of the regression. The same linear regression (using coefficients derived from the training set) was applied to the test scores. As the regression was applied to the training data only, there was no data leakage. The proportion of variance removed by the linear regression varies across the test scores, from less than 5% (cancellation) to ~20% (Boston naming test). Among the five covariates, years of education explained the most variance. With the regression of covariates from both matrices and scores, predictive models were built where the results are orthogonal to these confounding factors. To ensure that the correction method adequately addresses the confounds, we calculated the correlation between the kernel ridge regression prediction errors and the confounds. Supplementary Tables 15–24 show that the prediction errors and the confounds are not correlated.

Composite scores based on grouping individual tests that yield the highest predictive power for individual networks

Composite test scores were formed based on predictive performance for a single network. The dataset was split in a 2:1 ratio for training and testing, across 20 splits. The top behavioral tests that yielded the highest predictive power for that network were found using training sets of n = 200 participants across 100 10-fold cross-validations. The tests were combined into a composite test score using the testing sets (n = 102 participants) for each network and a predictive model with this composite score was constructed for each network. The optimal composite score has the best predictive power to test size ratio. This was determined by ranking the individual tests by predictive power for each cognitive construct network. Starting from the most predictive test, we progressively grouped the tests into composites, adding one test at a time in order of descending predictive power. We conducted sequential one-tailed t-tests to compare whether each test composite had higher predictive power than the best-performing individual test. We then tested whether each composite had significantly greater predictive power than the preceding composite (using one-tailed t-tests). If a composite did not show a significant improvement over the previous one, the previous composite was considered the optimal composite size. We also compared through two-tailed t-tests that the optimal composite has higher predictive power than the control random and worst test composites (which consist of the same number of tests as the optimal composite). For the random condition, ten composites were generated per split, and their predictive power was averaged to yield a single value per split.

Composite scores based on grouping individual tests that yield the highest predictive power and maximize network isolation

The dataset was split in a 2:1 ratio for training and testing, across 20 splits. Composite test scores were constructed based on two criteria: high predictive performance and network specificity. In the training set, for each of the 20 splits, (n = 200 participants), across 100 10-fold cross validations, for two-test composite scores, a composite was selected if the predictive power of its corresponding network ranked within the top 5% among 120 possible composites. Tests were ranked according to predictive power across 20 random data splits (100 iterations each), with the top 5% determined from the aggregated ranking results. Additionally, the predictive power of the cognitive construct network on the two-test composite had to be significantly greater than its predictive power on either of the individual tests comprising that composite. Significance testing was performed using one-tailed t-tests. Network isolation was quantified as the isolated difference, defined as the predictive power of the network on the composite minus the maximum predictive power of the remaining five networks on the same composite. The isolated difference was required to be positive and to rank within the top 5% of all composites. Tests were ranked by isolated difference across 20 randomized data splits (100 iterations per split), and the top 5% were determined from the aggregate ranking across all iterations. For composites including more than two tests, the criteria were that their predictive power had to be significantly greater than that of smaller composites, while maintaining a positive isolated difference. Finally, using an independent testing set (n = 102 participants) for each of the 20 splits, we verified that both the predictive performance and the isolated difference exhibited patterns consistent with those observed in the training data.

Statistical significance testing

Permutation-based null distributions were generated for each of the 16 test scores by randomly shuffling the target labels 1,000 times and computing predictive accuracy for each permutation. Statistical significance was assessed using one-sided permutation tests in which p-values were calculated as the proportion of null accuracies greater than or equal to the observed mean. P-values were adjusted using MATLAB’s (R2019a) implementation of the Benjamini-Hochberg procedure to control the false discovery rate (FDR), limiting the expected proportion of false positives while retaining statistical power. For the analysis in Fig. 1 and Supplementary Fig. 1, multiple-comparison correction was done across n = 16 comparisons for the 16 behavioral tests, and in Fig. 2 and Supplementary Fig. 2, multiple-comparison correction was done across n = 6 comparisons for the 6 cognitive construct networks.

Under the assumption that larger composite tests yield higher predictive power, paired one-tailed t-tests (MATLAB, R2019a) at the 5% level were used to compare predictive performance on composite test scores for 20 fixed random data splits (Figs. 3 and 5). The composite procedure followed a strict sequential rule, stopping when p > 0.05; therefore, multiple-comparison correction was not applied. Two-tailed t-tests (MATLAB, R2019a) at the 5% level were used to compare predictive performance on the optimal composites and the random and worst test composites in Fig. 3. Statistical significance testing for comparisons of predictive power described in the caption of Fig. 4 (subscores) was conducted using MATLAB’s (R2019a) two-sample one-tailed unequal variance t-tests at the 5% significance level (under the assumption that larger composite tests yield higher predictive power), with Bonferroni correction applied to each comparison within each construct to control for multiple comparisons (n = 6 comparisons for Fig. 4a, n = 24 comparisons for Fig. 4b). The two-sample t-test (assuming unequal variances) was used for the analysis in Supplementary Fig. 5, with Bonferroni correction applied to each comparison to control for multiple comparisons (n = 16 comparisons).

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Supplementary information

Supplementary information^{(7.5MB, pdf)}

Reporting Summary^{(111.4KB, pdf)}

Transparent Peer Review file^{(8.9MB, pdf)}

Source data

Source Data^{(9.3MB, xlsx)}

Author contributions

A.S. and R.T.C conceptualized the study, with guidance from X.S. A.S. and X.S. performed the analyses with support from R.T.C and W.L. A.S.G. designed the Yale study with support from R.T.C. and D.S. A.S.G, S.M, F.T. and J.A. collected the Yala dataset. S.W.W, R.B.K., G.S., and V.H.S. supported all clinical aspects of the Yale study. A.S. wrote the manuscript with contributions from R.T.C and X.S., and comments from all authors.

Peer review

Peer review information

Nature Communications thanks Seyul Kwak, Javier Rasero and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. A peer review file is available.

Funding

This work was supported by funding from the National Institutes of Health (MH121095 to D.S. and R.T.C.) and (EB034720 and MH138347 to R.T.C.) and (MH103831 to V.S). This work was also supported by the Gustavus and Louise Pfeiffer Research Foundation (V.S) and by the State of Connecticut, Department of Mental Health and Addiction Services (V.S.), but this publication does not express the views of the Department of Mental Health and Addition Services or the State of Connecticut. The views and opinions expressed are those of the authors.

Data availability

The data are available under restricted access via the NIMH data archive (Collection 3276; https://nda.nih.gov/edit_collection.html?id=3276), with access granted in accordance with standard data use agreements and procedures. The ids corresponding to the n = 302 participants included in this study are listed in the source data file and can be used to access the associated data through the NIMH data archive. Access can be attained by submitting a data access request (https://nda.nih.gov/nda/tutorials/electronic-data-access-request). The lead recipient must hold a principal investigator (PI) role in their eRA Commons account, and the NDA account must be affiliated with an NIH-recognized research institution. Dataset access is usually received within 10 business days. Data access is available for one year. Source data are provided with this paper.

Code availability

MATLAB code (R2018a) to run kernel ridge regression analyses with a demo data set (n = 10 participants) is available on the GitHub page⁹⁴ (https://github.com/YaleMRRC/testing_the_tests). We have made the mapping between each specific construct network and the Shen268 atlas (Supplementary Table 4) available as NIFTI mask files on the GitHub page⁹⁴ (https://github.com/YaleMRRC/testing_the_tests/tree/main/cognitive_construct_network_masks).

Competing interests

In the past two years, G.S. has received grants/research contracts from Johnson & Johnson/Janssen, Michael J Foxx Foundation, Merck, National Institutes of Health, and Patient Centered Outcomes Research Institute. G.S. also served as consultant to Abbvie, Actinogen Medical, Alto Neuroscience, Atai, Allumis, Axsome Therapeutics, Biogen, Biohaven Pharmaceuticals, Boehringer Ingelheim International GmbH, Bristol-Myers Squibb, Clexio, Denovo Biopharma, Douglas Pharmaceuticals, EMA Wellness, Embark, Freedom Biosciences, GH Research, Gilgamesh, Holmusk, Janssen, Lilly, Merck, Neumora, Neurocrine, Newleos, Novartis, Relmada Therapeutics, Seaport Pharmaceuticals, Supernus, Tetricus, and Transcend Therapeutics. G.S. also holds equity in Biohaven Pharmaceuticals, Freedom Biosciences, Gilead, Oui Therapeutics, Relmada, and Tetricus. G.S. is a co- inventor on a US patents (#8,778,979), (#11554117), (#10052318) (#11793794) and (#12090145). Yale University has a financial relationship with Janssen Pharmaceuticals and may receive financial benefits from this relationship. G.S. receives no benefit from this agreement and the University has put multiple measures in place to mitigate this institutional conflict of interest. Questions about the details of these measures should be directed to Yale University’s Conflict of Interest office. The remaining authors declare no competing interests.

Footnotes

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Anja Samardzija, Email: anja.samardzija@yale.edu.

R. Todd Constable, Email: todd.constable@yale.edu.

Supplementary information

The online version contains supplementary material available at 10.1038/s41467-026-73941-0.

References

1.Knapp, T. J. The emergence of cognitive psychology in the latter half of the twentieth century. In: Approaches to Cognition 13–35 (Routledge, 2016).
2.Malim, T. Cognitive processes: attention, perception, memory, thinking and language. (Bloomsbury Publishing, 1994).
3.Barsalou, L. W. Cognitive psychology: An overview for cognitive scientists. (Psychology Press, 2014).
4.Smith, S. M. et al. A positive-negative mode of population covariation links brain connectivity, demographics and behavior. Nat. Neurosci.18, 1565–1567 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Stoyanov, D. et al. Group independent components underpin responses to items from a depression scale. Acta Neuropsychiatrica36, 9–16 (2024). [DOI] [PubMed] [Google Scholar]
6.Genon, S., Eickhoff, S. B. & Kharabian, S. Linking interindividual variability in brain structure to behaviour. Nat. Rev. Neurosci.23, 307–318 (2022). [DOI] [PubMed] [Google Scholar]
7.Tian, Y., Yeo, B. T., Cropley, V. & Zalesky, A. High-resolution connectomic fingerprints: Mapping neural identity and behavior. NeuroImage229, 117695 (2021). [DOI] [PubMed] [Google Scholar]
8.Barron, D. S. et al. Transdiagnostic, connectome-based prediction of memory constructs across psychiatric disorders. Cereb. Cortex31, 2523–2533 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
9.Dubois, J., Galdi, P., Han, Y., Paul, L. K. & Adolphs, R. Resting-state functional brain connectivity best predicts the personality dimension of openness to experience. Personal. Neurosci.1, e6 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Dubois, J., Galdi, P., Paul, L. K. & Adolphs, R. A distributed brain network predicts general intelligence from resting-state human neuroimaging data. Philos. Trans. R. Soc. B: Biol. Sci.373, 20170284 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
11.Strauss, M. E. & Smith, G. T. Construct validity: Advances in theory and methodology. Annu. Rev. Clin. Psychol.5, 1–25 (2009). [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Embretson, S. & Gorin, J. Improving construct validity with cognitive psychology principles. J. Educ. Meas.38, 343–368 (2001). [Google Scholar]
13.Sireci, S. G. The construct of content validity. Soc. Indic. Res.45, 83–117 (1998). [Google Scholar]
14.Fiske, D. W. Convergent-discriminant validation in measurements and research strategies. N. Directions Methodol. Soc. Behav. Sci.12, 77–92 (1982). [Google Scholar]
15.Rosenbaum, P. R. Criterion-related construct validity. Psychometrika54, 625–633 (1989). [Google Scholar]
16.Vaidya, A. R., Pujara, M. S., Petrides, M., Murray, E. A. & Fellows, L. K. Lesion studies in contemporary neuroscience. Trends Cogn. Sci.23, 653–671 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
17.Broca, P. Remarques sur le siège de la faculté du langage articulé, suivies d’une observation d’aphémie (perte de la parole). Bull. et. Memoires de. la Soc. anatomique de. Paris6, 330–357 (1861). [Google Scholar]
18.Wernicke, C. Der aphasische Symptomencomplex: eine psychologische Studie auf anatomischer Basis. (Cohn & Weigert, 1874).
19.D’Esposito, M. & Postle, B. R. The cognitive neuroscience of working memory. Annu. Rev. Psychol.66, 115–142 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
20.Haxby, J. V. Multivariate pattern analysis of fMRI: the early beginnings. Neuroimage62, 852–855 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
21.Coull, J. T. & Nobre, A. C. Where and when to pay attention: the neural systems for directing attention to spatial locations and to time intervals as revealed by both PET and fMRI. J. Neurosci.18, 7426–7435 (1998). [DOI] [PMC free article] [PubMed] [Google Scholar]
22.Van Essen, D. C. et al. The Human Connectome Project: a data acquisition perspective. Neuroimage62, 2222–2231 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
23.Mill, R. D., Ito, T. & Cole, M. W. From connectome to cognition: The search for mechanism in human functional brain networks. NeuroImage160, 124–139 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
24.Sporns, O. The human connectome: a complex network. Ann. N. Y. Acad. Sci.1224, 109–125 (2011). [DOI] [PubMed] [Google Scholar]
25.Finn, E. S. et al. Functional connectome fingerprinting: identifying individuals using patterns of brain connectivity. Nat. Neurosci.18, 1664–1671 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
26.Greene, A. S., Gao, S., Scheinost, D. & Constable, R. T. Task-induced brain state manipulation improves prediction of individual traits. Nat. Commun.9, 2807 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
27.Rosenberg, M. D., Finn, E. S., Scheinost, D., Constable, R. T. & Chun, M. M. Characterizing attention with predictive network models. Trends Cogn. Sci.21, 290–302 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
28.Rosenberg, M. D. et al. Functional connectivity predicts changes in attention observed across minutes, days, and months. Proc. Natl. Acad. Sci.117, 3797–3807 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
29.Shen, X. et al. Using connectome-based predictive modeling to predict individual behavior from brain connectivity. Nat. Protoc.12, 506–518 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
30.Rosenberg, M. D. & Finn, E. S. How to establish robust brain–behavior relationships without thousands of individuals. Nat. Neurosci.25, 835–837 (2022). [DOI] [PubMed] [Google Scholar]
31.Greene, A. S. et al. Brain–phenotype models fail for individuals who defy sample stereotypes. Nature609, 109–118 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
32.Niv, Y. The primacy of behavioral research for understanding the brain. Behav. Neurosci.135, 601 (2021). [DOI] [PubMed] [Google Scholar]
33.Scheinost, D. et al. Ten simple rules for predictive modeling of individual differences in neuroimaging. Neuroimage193, 35–45 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
34.Luo, W., Greene, A. S. & Constable, R. T. Within node connectivity changes, not simply edge changes, influence graph theory measures in functional connectivity studies of the brain. NeuroImage240, 118332 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
35.Rubinov, M. & Sporns, O. Complex network measures of brain connectivity: uses and interpretations. Neuroimage52, 1059–1069 (2010). [DOI] [PubMed] [Google Scholar]
36.Buckner, R. L., Krienen, F. M., Castellanos, A., Diaz, J. C. & Yeo, B. T. The organization of the human cerebellum estimated by intrinsic functional connectivity. J. Neurophysiol.106, 2322–2345 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
37.Shen, X., Tokoglu, F., Papademetris, X. & Constable, R. T. Groupwise whole-brain parcellation from resting-state fMRI data for network node identification. Neuroimage82, 403–415 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
38.Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C. & Wager, T. D. Large-scale automated synthesis of human functional neuroimaging data. Nat. methods8, 665–670 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
39.Ooi, L. Q. R. et al. Longer scans boost prediction and cut costs in brain-wide association studies. Nature, 1–10 (2025). [DOI] [PMC free article] [PubMed]
40.Ooi, L. Q. R. et al. Comparison of individualized behavioral predictions across anatomical, diffusion and functional connectivity MRI. NeuroImage263, 119636 (2022). [DOI] [PubMed] [Google Scholar]
41.Delis, D. C., Kaplan, E. & Kramer, J. H. Delis-Kaplan executive function system. (Psychological Corporation, 2001).
42.Golden, C., Freshwater, S. M. & Golden, Z. Stroop color and word test. (1978).
43.Lippa, S. M. & Davis, R. N. Inhibition/switching is not necessarily harder than inhibition: An analysis of the D-KEFS color-word interference test. Arch. Clin. Neuropsychol.25, 146–152 (2010). [DOI] [PubMed] [Google Scholar]
44.Periáñez, J. A., Lubrini, G., García-Gutiérrez, A. & Ríos-Lago, M. Construct validity of the stroop color-word test: influence of speed of visual search, verbal fluency, working memory, cognitive flexibility, and conflict monitoring. Arch. Clin. Neuropsychol.36, 99–111 (2021). [DOI] [PubMed] [Google Scholar]
45.Lipkin, B. et al. Probabilistic atlas for the language network based on precision fMRI data from> 800 individuals. Sci. Data9, 529 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
46.Yeo, B. T. et al. The organization of the human cerebral cortex estimated by intrinsic functional connectivity. J. Neurophysiol.106, 1125–1165 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
47.Finn, E. S. Is it time to put rest to rest? Trends Cogn. Sci.25, 1021–1032 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
48.Greene, A. S., Horien, C., Barson, D., Scheinost, D. & Constable, R. T. Why is everyone talking about brain state? Trends Neurosci.46, 508–524 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
49.Roth, R. M., Isquith, P. K. & Gioia, G. A. Behavior rating inventory of executive function®--adult version. (Psychological Assessment Resources, 2005).
50.He, T. et al. Deep neural networks and kernel regression achieve comparable accuracies for functional connectivity prediction of behavior and demographics. NeuroImage206, 116276 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
51.Spearman, C. The abilities of man. (1961). [DOI] [PubMed]
52.Spearman, C. “General Intelligence” Objectively Determined and Measured. (1961).
53.Nobre, A. C. & van Ede, F. Attention in flux. Neuron111, 971–986 (2023). [DOI] [PubMed] [Google Scholar]
54.Reid, A. T. et al. Advancing functional connectivity research from association to causation. Nat. Neurosci.22, 1751–1760 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
55.Mill, R. D. et al. Network modeling of dynamic brain interactions predicts emergence of neural information that supports human cognitive behavior. PLoS Biol.20, e3001686 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
56.Peterson, K. L., Sanchez-Romero, R., Mill, R. D. & Cole, M. W. Regularized partial correlation provides reliable functional connectivity estimates while correcting for widespread confounding. Imaging Neuroscience 3, IMAG. a. 162 (2025). [DOI] [PMC free article] [PubMed]
57.Yoo, K. et al. Connectome-based predictive modeling of attention: Comparing different functional connectivity features and prediction methods across datasets. Neuroimage167, 11–22 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
58.Liu, Z.-Q. et al. Benchmarking methods for mapping functional connectivity in the brain. Nature Methods, 1–10 (2025). [DOI] [PMC free article] [PubMed]
59.Nikolaidis, A. et al. Suboptimal phenotypic reliability impedes reproducible human neuroscience. BioRxiv, 2022.2007. 2022.501193 (2022).
60.Warne, R. T., Yoon, M. & Price, C. J. Exploring the various interpretations of “test bias”. Cult. Divers Ethn. Minor Psychol.20, 570–582 (2014). [DOI] [PubMed] [Google Scholar]
61.O’Connor, D. W., Pollitt, P. A. & Treasure, F. P. The influence of education and social class on the diagnosis of dementia in a community population. Psychological Med.21, 219–224 (1991). [DOI] [PubMed] [Google Scholar]
62.Finn, E. S. et al. Can brain state be manipulated to emphasize individual differences in functional connectivity? NeuroImage160, 140–151 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
63.Thiele, J. A., Faskowitz, J., Sporns, O. & Hilger, K. Multitask brain network reconfiguration is inversely associated with human intelligence. Cereb. Cortex32, 4172–4182 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
64.McCormick, E. M., Arnemann, K. L., Ito, T., Hanson, S. J. & Cole, M. W. Latent functional connectivity underlying multiple brain states. Netw. Neurosci.6, 570–590 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
65.Tetereva, A., Li, J., Deng, J. D., Stringaris, A. & Pat, N. Capturing brain-cognition relationship: Integrating task-based fMRI across tasks markedly boosts prediction and test-retest reliability. NeuroImage263, 119588 (2022). [DOI] [PubMed] [Google Scholar]
66.Gbadeyan, O., Teng, J. & Prakash, R. S. Predicting response time variability from task and resting-state functional connectivity in the aging brain. NeuroImage250, 118890 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
67.DeYoung, C. G. et al. Reproducible between-person brain-behavior associations do not always require thousands of individuals. PsyArXiv, (2022).
68.Kurtin, D. L., Scott, G., Hebron, H., Skeldon, A. C. & Violante, I. R. Task-based differences in brain state dynamics and their relation to cognitive ability. NeuroImage271, 119945 (2023). [DOI] [PubMed] [Google Scholar]
69.Tomasi, D. & Volkow, N. D. Network connectivity predicts language processing in healthy adults. Hum. Brain Mapp.41, 3696–3708 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
70.Zhang, H., Meng, C., Di, X., Wu, X. & Biswal, B. Static and dynamic functional connectome reveals reconfiguration profiles of whole-brain network across cognitive states. Netw. Neurosci.7, 1034–1050 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
71.Zhao, W. et al. Task fMRI paradigms may capture more behaviorally relevant information than resting-state functional connectivity. NeuroImage270, 119946 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
72.Finn, E. S. & Bandettini, P. A. Movie-watching outperforms rest for functional connectivity-based prediction of behavior. NeuroImage235, 117963 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
73.Jiang, R. et al. Task-induced brain connectivity promotes the detection of individual differences in brain-behavior relationships. NeuroImage207, 116370 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
74.Gao, S., Greene, A. S., Constable, R. T. & Scheinost, D. Combining multiple connectomes improves predictive modeling of phenotypic measures. Neuroimage201, 116038 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
75.DeYoung, C. G. et al. Beyond increasing sample sizes: Optimizing effect sizes in neuroimaging research on individual differences. J. Cogn. Neurosci.37, 1023–1034 (2025). [DOI] [PubMed] [Google Scholar]
76.Greene, A. S., Gao, S., Noble, S., Scheinost, D. & Constable, R. T. How Tasks Change Whole-Brain Functional Organization to Reveal Brain-Phenotype Relationships. Cell Rep.32, 108066 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
77.Rosenberg, M. D., Finn, E. S., Constable, R. T. & Chun, M. M. Predicting moment-to-moment attentional state. Neuroimage114, 249–256 (2015). [DOI] [PubMed] [Google Scholar]
78.Cutts, S. A., Faskowitz, J., Betzel, R. F. & Sporns, O. Uncovering individual differences in fine-scale dynamics of functional connectivity. Cereb. cortex33, 2375–2394 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
79.Bottom-Tanzer, S. et al. Traumatic brain injury disrupts state-dependent functional cortical connectivity in a mouse model. Cereb. Cortex34, bhae038 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
80.Huijbers, W., Van Dijk, K. R., Boenniger, M. M., Stirnberg, R. & Breteler, M. M. Less head motion during MRI under task than resting-state conditions. Neuroimage147, 111–120 (2017). [DOI] [PubMed] [Google Scholar]
81.Vanderwal, T., Kelly, C., Eilbott, J., Mayes, L. C. & Castellanos, F. X. Inscapes: A movie paradigm to improve compliance in functional magnetic resonance imaging. Neuroimage122, 222–232 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
82.Cole, M. W., Bassett, D. S., Power, J. D., Braver, T. S. & Petersen, S. E. Intrinsic and task-evoked network architectures of the human brain. Neuron83, 238–251 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
83.Krienen, F. M., Yeo, B. & Buckner, R. L. Reconfigurable task-dependent functional coupling modes cluster around a core functional architecture. Philos. Trans. R. Soc. B: Biol. Sci.369, 20130526 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
84.Gratton, C. et al. Functional brain networks are dominated by stable group and individual factors, not cognitive or daily variation. Neuron98, 439–452.e435 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
85.Wig, G. S., Schlaggar, B. L. & Petersen, S. E. Concepts and principles in the analysis of brain networks. Ann. N. Y. Acad. Sci.1224, 126–146 (2011). [DOI] [PubMed] [Google Scholar]
86.Horien, C., Shen, X., Scheinost, D. & Constable, R. T. The individual functional connectome is unique and stable over months to years. Neuroimage189, 676–687 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
87.Cole, M. W. et al. Task activations produce spurious but systematic inflation of task functional connectivity estimates. Neuroimage189, 1–18 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
88.Greene, A. S. & Constable, R. T. Clinical Promise of Brain-Phenotype Modeling: A Review. JAMA psychiatry80, 848–854 (2023). [DOI] [PubMed] [Google Scholar]
89.Sm, S. Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage23, S208–S219 (2004). [DOI] [PubMed] [Google Scholar]
90.Frackowiak, R. S. Human brain function. (Elsevier, 2004).
91.Joshi, A. et al. Unified framework for development, deployment and robust testing of neuroimaging algorithms. Neuroinformatics9, 69–84 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
92.Rodriguez, R. X., Noble, S., Camp, C. C. & Scheinost, D. Connectome caricatures remove large-amplitude coactivation patterns in resting-state fMRI to emphasize individual differences. Nat. Neurosci.28, 2601–2611 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]
93.Cole, M. W. & Schneider, W. The cognitive control network: Integrated cortical regions with dissociable functions. Neuroimage37, 343–360 (2007). [DOI] [PubMed] [Google Scholar]
94.Samardzija, A. et al. Using Connectome-Based Predictive Models to Reveal the Systems Standardized Tests and Clinical Symptoms are Reflecting. testing_the_tests 10.5281/zenodo.19574108 (2026). [DOI] [PMC free article] [PubMed]
95.Sheslow, D. & Adams, W. WRAML: Wide range assessment of memory and learning. (Jastak Associates, 1990).
96.Wilkinson, G. S. & Robertson, G. J. Wide range achievement test 4. (Psychological Assessment Resources, 1993).
97.Williams, D. L., Goldstein, G. & Minshew, N. J. The profile of memory function in children with autism. Neuropsychology20, 21 (2006). [DOI] [PMC free article] [PubMed] [Google Scholar]
98.McQuade, J. D., Murray-Close, D., Shoulberg, E. K. & Hoza, B. Working memory and social functioning in children. J. Exp. child Psychol.115, 422–435 (2013). [DOI] [PubMed] [Google Scholar]
99.Mawjee, K., Woltering, S. & Tannock, R. Working memory training in post-secondary students with ADHD: A randomized controlled study. PloS one10, e0137173 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
100.Williams, D. L., Goldstein, G., Carpenter, P. A. & Minshew, N. J. Verbal and spatial working memory in autism. J. autism developmental Disord.35, 747–756 (2005). [DOI] [PubMed] [Google Scholar]
101.Mealer, C., Morgan, S. & Luscomb, R. Cognitive functioning of ADHD and non-ADHD boys on the WISC-III and WRAML: An analysis within a memory model. J. Atten. Disord.1, 133–145 (1996). [Google Scholar]
102.Chen, J. et al. Shared and unique brain network features predict cognitive, personality, and mental health scores in the ABCD study. Nat. Commun.13, 1–17 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
103.Kaplan, E., Goodglass, H. & Weintraub, S. Boston naming test. (APA PsycTests, 2001).
104.Wechsler, D. Wechsler adult intelligence scale. (Psychological Corporation, 1955).
105.Wechsler, D. Wechsler abbreviated scale of intelligence. (Pearson, 1999).

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary information^{(7.5MB, pdf)}

Reporting Summary^{(111.4KB, pdf)}

Transparent Peer Review file^{(8.9MB, pdf)}

Source Data^{(9.3MB, xlsx)}

Data Availability Statement

[CR1] 1.Knapp, T. J. The emergence of cognitive psychology in the latter half of the twentieth century. In: Approaches to Cognition 13–35 (Routledge, 2016).

[CR2] 2.Malim, T. Cognitive processes: attention, perception, memory, thinking and language. (Bloomsbury Publishing, 1994).

[CR3] 3.Barsalou, L. W. Cognitive psychology: An overview for cognitive scientists. (Psychology Press, 2014).

[CR4] 4.Smith, S. M. et al. A positive-negative mode of population covariation links brain connectivity, demographics and behavior. Nat. Neurosci.18, 1565–1567 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR5] 5.Stoyanov, D. et al. Group independent components underpin responses to items from a depression scale. Acta Neuropsychiatrica36, 9–16 (2024). [DOI] [PubMed] [Google Scholar]

[CR6] 6.Genon, S., Eickhoff, S. B. & Kharabian, S. Linking interindividual variability in brain structure to behaviour. Nat. Rev. Neurosci.23, 307–318 (2022). [DOI] [PubMed] [Google Scholar]

[CR7] 7.Tian, Y., Yeo, B. T., Cropley, V. & Zalesky, A. High-resolution connectomic fingerprints: Mapping neural identity and behavior. NeuroImage229, 117695 (2021). [DOI] [PubMed] [Google Scholar]

[CR8] 8.Barron, D. S. et al. Transdiagnostic, connectome-based prediction of memory constructs across psychiatric disorders. Cereb. Cortex31, 2523–2533 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR9] 9.Dubois, J., Galdi, P., Han, Y., Paul, L. K. & Adolphs, R. Resting-state functional brain connectivity best predicts the personality dimension of openness to experience. Personal. Neurosci.1, e6 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR10] 10.Dubois, J., Galdi, P., Paul, L. K. & Adolphs, R. A distributed brain network predicts general intelligence from resting-state human neuroimaging data. Philos. Trans. R. Soc. B: Biol. Sci.373, 20170284 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR11] 11.Strauss, M. E. & Smith, G. T. Construct validity: Advances in theory and methodology. Annu. Rev. Clin. Psychol.5, 1–25 (2009). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR12] 12.Embretson, S. & Gorin, J. Improving construct validity with cognitive psychology principles. J. Educ. Meas.38, 343–368 (2001). [Google Scholar]

[CR13] 13.Sireci, S. G. The construct of content validity. Soc. Indic. Res.45, 83–117 (1998). [Google Scholar]

[CR14] 14.Fiske, D. W. Convergent-discriminant validation in measurements and research strategies. N. Directions Methodol. Soc. Behav. Sci.12, 77–92 (1982). [Google Scholar]

[CR15] 15.Rosenbaum, P. R. Criterion-related construct validity. Psychometrika54, 625–633 (1989). [Google Scholar]

[CR16] 16.Vaidya, A. R., Pujara, M. S., Petrides, M., Murray, E. A. & Fellows, L. K. Lesion studies in contemporary neuroscience. Trends Cogn. Sci.23, 653–671 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR17] 17.Broca, P. Remarques sur le siège de la faculté du langage articulé, suivies d’une observation d’aphémie (perte de la parole). Bull. et. Memoires de. la Soc. anatomique de. Paris6, 330–357 (1861). [Google Scholar]

[CR18] 18.Wernicke, C. Der aphasische Symptomencomplex: eine psychologische Studie auf anatomischer Basis. (Cohn & Weigert, 1874).

[CR19] 19.D’Esposito, M. & Postle, B. R. The cognitive neuroscience of working memory. Annu. Rev. Psychol.66, 115–142 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR20] 20.Haxby, J. V. Multivariate pattern analysis of fMRI: the early beginnings. Neuroimage62, 852–855 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR21] 21.Coull, J. T. & Nobre, A. C. Where and when to pay attention: the neural systems for directing attention to spatial locations and to time intervals as revealed by both PET and fMRI. J. Neurosci.18, 7426–7435 (1998). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR22] 22.Van Essen, D. C. et al. The Human Connectome Project: a data acquisition perspective. Neuroimage62, 2222–2231 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR23] 23.Mill, R. D., Ito, T. & Cole, M. W. From connectome to cognition: The search for mechanism in human functional brain networks. NeuroImage160, 124–139 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR24] 24.Sporns, O. The human connectome: a complex network. Ann. N. Y. Acad. Sci.1224, 109–125 (2011). [DOI] [PubMed] [Google Scholar]

[CR25] 25.Finn, E. S. et al. Functional connectome fingerprinting: identifying individuals using patterns of brain connectivity. Nat. Neurosci.18, 1664–1671 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR26] 26.Greene, A. S., Gao, S., Scheinost, D. & Constable, R. T. Task-induced brain state manipulation improves prediction of individual traits. Nat. Commun.9, 2807 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR27] 27.Rosenberg, M. D., Finn, E. S., Scheinost, D., Constable, R. T. & Chun, M. M. Characterizing attention with predictive network models. Trends Cogn. Sci.21, 290–302 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR28] 28.Rosenberg, M. D. et al. Functional connectivity predicts changes in attention observed across minutes, days, and months. Proc. Natl. Acad. Sci.117, 3797–3807 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR29] 29.Shen, X. et al. Using connectome-based predictive modeling to predict individual behavior from brain connectivity. Nat. Protoc.12, 506–518 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR30] 30.Rosenberg, M. D. & Finn, E. S. How to establish robust brain–behavior relationships without thousands of individuals. Nat. Neurosci.25, 835–837 (2022). [DOI] [PubMed] [Google Scholar]

[CR31] 31.Greene, A. S. et al. Brain–phenotype models fail for individuals who defy sample stereotypes. Nature609, 109–118 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR32] 32.Niv, Y. The primacy of behavioral research for understanding the brain. Behav. Neurosci.135, 601 (2021). [DOI] [PubMed] [Google Scholar]

[CR33] 33.Scheinost, D. et al. Ten simple rules for predictive modeling of individual differences in neuroimaging. Neuroimage193, 35–45 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR34] 34.Luo, W., Greene, A. S. & Constable, R. T. Within node connectivity changes, not simply edge changes, influence graph theory measures in functional connectivity studies of the brain. NeuroImage240, 118332 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR35] 35.Rubinov, M. & Sporns, O. Complex network measures of brain connectivity: uses and interpretations. Neuroimage52, 1059–1069 (2010). [DOI] [PubMed] [Google Scholar]

[CR36] 36.Buckner, R. L., Krienen, F. M., Castellanos, A., Diaz, J. C. & Yeo, B. T. The organization of the human cerebellum estimated by intrinsic functional connectivity. J. Neurophysiol.106, 2322–2345 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR37] 37.Shen, X., Tokoglu, F., Papademetris, X. & Constable, R. T. Groupwise whole-brain parcellation from resting-state fMRI data for network node identification. Neuroimage82, 403–415 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR38] 38.Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C. & Wager, T. D. Large-scale automated synthesis of human functional neuroimaging data. Nat. methods8, 665–670 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR39] 39.Ooi, L. Q. R. et al. Longer scans boost prediction and cut costs in brain-wide association studies. Nature, 1–10 (2025). [DOI] [PMC free article] [PubMed]

[CR40] 40.Ooi, L. Q. R. et al. Comparison of individualized behavioral predictions across anatomical, diffusion and functional connectivity MRI. NeuroImage263, 119636 (2022). [DOI] [PubMed] [Google Scholar]

[CR41] 41.Delis, D. C., Kaplan, E. & Kramer, J. H. Delis-Kaplan executive function system. (Psychological Corporation, 2001).

[CR42] 42.Golden, C., Freshwater, S. M. & Golden, Z. Stroop color and word test. (1978).

[CR43] 43.Lippa, S. M. & Davis, R. N. Inhibition/switching is not necessarily harder than inhibition: An analysis of the D-KEFS color-word interference test. Arch. Clin. Neuropsychol.25, 146–152 (2010). [DOI] [PubMed] [Google Scholar]

[CR44] 44.Periáñez, J. A., Lubrini, G., García-Gutiérrez, A. & Ríos-Lago, M. Construct validity of the stroop color-word test: influence of speed of visual search, verbal fluency, working memory, cognitive flexibility, and conflict monitoring. Arch. Clin. Neuropsychol.36, 99–111 (2021). [DOI] [PubMed] [Google Scholar]

[CR45] 45.Lipkin, B. et al. Probabilistic atlas for the language network based on precision fMRI data from> 800 individuals. Sci. Data9, 529 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR46] 46.Yeo, B. T. et al. The organization of the human cerebral cortex estimated by intrinsic functional connectivity. J. Neurophysiol.106, 1125–1165 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR47] 47.Finn, E. S. Is it time to put rest to rest? Trends Cogn. Sci.25, 1021–1032 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR48] 48.Greene, A. S., Horien, C., Barson, D., Scheinost, D. & Constable, R. T. Why is everyone talking about brain state? Trends Neurosci.46, 508–524 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR49] 49.Roth, R. M., Isquith, P. K. & Gioia, G. A. Behavior rating inventory of executive function®--adult version. (Psychological Assessment Resources, 2005).

[CR50] 50.He, T. et al. Deep neural networks and kernel regression achieve comparable accuracies for functional connectivity prediction of behavior and demographics. NeuroImage206, 116276 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR51] 51.Spearman, C. The abilities of man. (1961). [DOI] [PubMed]

[CR52] 52.Spearman, C. “General Intelligence” Objectively Determined and Measured. (1961).

[CR53] 53.Nobre, A. C. & van Ede, F. Attention in flux. Neuron111, 971–986 (2023). [DOI] [PubMed] [Google Scholar]

[CR54] 54.Reid, A. T. et al. Advancing functional connectivity research from association to causation. Nat. Neurosci.22, 1751–1760 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR55] 55.Mill, R. D. et al. Network modeling of dynamic brain interactions predicts emergence of neural information that supports human cognitive behavior. PLoS Biol.20, e3001686 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR56] 56.Peterson, K. L., Sanchez-Romero, R., Mill, R. D. & Cole, M. W. Regularized partial correlation provides reliable functional connectivity estimates while correcting for widespread confounding. Imaging Neuroscience 3, IMAG. a. 162 (2025). [DOI] [PMC free article] [PubMed]

[CR57] 57.Yoo, K. et al. Connectome-based predictive modeling of attention: Comparing different functional connectivity features and prediction methods across datasets. Neuroimage167, 11–22 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR58] 58.Liu, Z.-Q. et al. Benchmarking methods for mapping functional connectivity in the brain. Nature Methods, 1–10 (2025). [DOI] [PMC free article] [PubMed]

[CR59] 59.Nikolaidis, A. et al. Suboptimal phenotypic reliability impedes reproducible human neuroscience. BioRxiv, 2022.2007. 2022.501193 (2022).

[CR60] 60.Warne, R. T., Yoon, M. & Price, C. J. Exploring the various interpretations of “test bias”. Cult. Divers Ethn. Minor Psychol.20, 570–582 (2014). [DOI] [PubMed] [Google Scholar]

[CR61] 61.O’Connor, D. W., Pollitt, P. A. & Treasure, F. P. The influence of education and social class on the diagnosis of dementia in a community population. Psychological Med.21, 219–224 (1991). [DOI] [PubMed] [Google Scholar]

[CR62] 62.Finn, E. S. et al. Can brain state be manipulated to emphasize individual differences in functional connectivity? NeuroImage160, 140–151 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR63] 63.Thiele, J. A., Faskowitz, J., Sporns, O. & Hilger, K. Multitask brain network reconfiguration is inversely associated with human intelligence. Cereb. Cortex32, 4172–4182 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR64] 64.McCormick, E. M., Arnemann, K. L., Ito, T., Hanson, S. J. & Cole, M. W. Latent functional connectivity underlying multiple brain states. Netw. Neurosci.6, 570–590 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR65] 65.Tetereva, A., Li, J., Deng, J. D., Stringaris, A. & Pat, N. Capturing brain-cognition relationship: Integrating task-based fMRI across tasks markedly boosts prediction and test-retest reliability. NeuroImage263, 119588 (2022). [DOI] [PubMed] [Google Scholar]

[CR66] 66.Gbadeyan, O., Teng, J. & Prakash, R. S. Predicting response time variability from task and resting-state functional connectivity in the aging brain. NeuroImage250, 118890 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR67] 67.DeYoung, C. G. et al. Reproducible between-person brain-behavior associations do not always require thousands of individuals. PsyArXiv, (2022).

[CR68] 68.Kurtin, D. L., Scott, G., Hebron, H., Skeldon, A. C. & Violante, I. R. Task-based differences in brain state dynamics and their relation to cognitive ability. NeuroImage271, 119945 (2023). [DOI] [PubMed] [Google Scholar]

[CR69] 69.Tomasi, D. & Volkow, N. D. Network connectivity predicts language processing in healthy adults. Hum. Brain Mapp.41, 3696–3708 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR70] 70.Zhang, H., Meng, C., Di, X., Wu, X. & Biswal, B. Static and dynamic functional connectome reveals reconfiguration profiles of whole-brain network across cognitive states. Netw. Neurosci.7, 1034–1050 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR71] 71.Zhao, W. et al. Task fMRI paradigms may capture more behaviorally relevant information than resting-state functional connectivity. NeuroImage270, 119946 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR72] 72.Finn, E. S. & Bandettini, P. A. Movie-watching outperforms rest for functional connectivity-based prediction of behavior. NeuroImage235, 117963 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR73] 73.Jiang, R. et al. Task-induced brain connectivity promotes the detection of individual differences in brain-behavior relationships. NeuroImage207, 116370 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR74] 74.Gao, S., Greene, A. S., Constable, R. T. & Scheinost, D. Combining multiple connectomes improves predictive modeling of phenotypic measures. Neuroimage201, 116038 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR75] 75.DeYoung, C. G. et al. Beyond increasing sample sizes: Optimizing effect sizes in neuroimaging research on individual differences. J. Cogn. Neurosci.37, 1023–1034 (2025). [DOI] [PubMed] [Google Scholar]

[CR76] 76.Greene, A. S., Gao, S., Noble, S., Scheinost, D. & Constable, R. T. How Tasks Change Whole-Brain Functional Organization to Reveal Brain-Phenotype Relationships. Cell Rep.32, 108066 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR77] 77.Rosenberg, M. D., Finn, E. S., Constable, R. T. & Chun, M. M. Predicting moment-to-moment attentional state. Neuroimage114, 249–256 (2015). [DOI] [PubMed] [Google Scholar]

[CR78] 78.Cutts, S. A., Faskowitz, J., Betzel, R. F. & Sporns, O. Uncovering individual differences in fine-scale dynamics of functional connectivity. Cereb. cortex33, 2375–2394 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR79] 79.Bottom-Tanzer, S. et al. Traumatic brain injury disrupts state-dependent functional cortical connectivity in a mouse model. Cereb. Cortex34, bhae038 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR80] 80.Huijbers, W., Van Dijk, K. R., Boenniger, M. M., Stirnberg, R. & Breteler, M. M. Less head motion during MRI under task than resting-state conditions. Neuroimage147, 111–120 (2017). [DOI] [PubMed] [Google Scholar]

[CR81] 81.Vanderwal, T., Kelly, C., Eilbott, J., Mayes, L. C. & Castellanos, F. X. Inscapes: A movie paradigm to improve compliance in functional magnetic resonance imaging. Neuroimage122, 222–232 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR82] 82.Cole, M. W., Bassett, D. S., Power, J. D., Braver, T. S. & Petersen, S. E. Intrinsic and task-evoked network architectures of the human brain. Neuron83, 238–251 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR83] 83.Krienen, F. M., Yeo, B. & Buckner, R. L. Reconfigurable task-dependent functional coupling modes cluster around a core functional architecture. Philos. Trans. R. Soc. B: Biol. Sci.369, 20130526 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR84] 84.Gratton, C. et al. Functional brain networks are dominated by stable group and individual factors, not cognitive or daily variation. Neuron98, 439–452.e435 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR85] 85.Wig, G. S., Schlaggar, B. L. & Petersen, S. E. Concepts and principles in the analysis of brain networks. Ann. N. Y. Acad. Sci.1224, 126–146 (2011). [DOI] [PubMed] [Google Scholar]

[CR86] 86.Horien, C., Shen, X., Scheinost, D. & Constable, R. T. The individual functional connectome is unique and stable over months to years. Neuroimage189, 676–687 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR87] 87.Cole, M. W. et al. Task activations produce spurious but systematic inflation of task functional connectivity estimates. Neuroimage189, 1–18 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR88] 88.Greene, A. S. & Constable, R. T. Clinical Promise of Brain-Phenotype Modeling: A Review. JAMA psychiatry80, 848–854 (2023). [DOI] [PubMed] [Google Scholar]

[CR89] 89.Sm, S. Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage23, S208–S219 (2004). [DOI] [PubMed] [Google Scholar]

[CR90] 90.Frackowiak, R. S. Human brain function. (Elsevier, 2004).

[CR91] 91.Joshi, A. et al. Unified framework for development, deployment and robust testing of neuroimaging algorithms. Neuroinformatics9, 69–84 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR92] 92.Rodriguez, R. X., Noble, S., Camp, C. C. & Scheinost, D. Connectome caricatures remove large-amplitude coactivation patterns in resting-state fMRI to emphasize individual differences. Nat. Neurosci.28, 2601–2611 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR93] 93.Cole, M. W. & Schneider, W. The cognitive control network: Integrated cortical regions with dissociable functions. Neuroimage37, 343–360 (2007). [DOI] [PubMed] [Google Scholar]

[CR94] 94.Samardzija, A. et al. Using Connectome-Based Predictive Models to Reveal the Systems Standardized Tests and Clinical Symptoms are Reflecting. testing_the_tests 10.5281/zenodo.19574108 (2026). [DOI] [PMC free article] [PubMed]

[CR95] 95.Sheslow, D. & Adams, W. WRAML: Wide range assessment of memory and learning. (Jastak Associates, 1990).

[CR96] 96.Wilkinson, G. S. & Robertson, G. J. Wide range achievement test 4. (Psychological Assessment Resources, 1993).

[CR97] 97.Williams, D. L., Goldstein, G. & Minshew, N. J. The profile of memory function in children with autism. Neuropsychology20, 21 (2006). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR98] 98.McQuade, J. D., Murray-Close, D., Shoulberg, E. K. & Hoza, B. Working memory and social functioning in children. J. Exp. child Psychol.115, 422–435 (2013). [DOI] [PubMed] [Google Scholar]

[CR99] 99.Mawjee, K., Woltering, S. & Tannock, R. Working memory training in post-secondary students with ADHD: A randomized controlled study. PloS one10, e0137173 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR100] 100.Williams, D. L., Goldstein, G., Carpenter, P. A. & Minshew, N. J. Verbal and spatial working memory in autism. J. autism developmental Disord.35, 747–756 (2005). [DOI] [PubMed] [Google Scholar]

[CR101] 101.Mealer, C., Morgan, S. & Luscomb, R. Cognitive functioning of ADHD and non-ADHD boys on the WISC-III and WRAML: An analysis within a memory model. J. Atten. Disord.1, 133–145 (1996). [Google Scholar]

[CR102] 102.Chen, J. et al. Shared and unique brain network features predict cognitive, personality, and mental health scores in the ABCD study. Nat. Commun.13, 1–17 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR103] 103.Kaplan, E., Goodglass, H. & Weintraub, S. Boston naming test. (APA PsycTests, 2001).

[CR104] 104.Wechsler, D. Wechsler adult intelligence scale. (Psychological Corporation, 1955).

[CR105] 105.Wechsler, D. Wechsler abbreviated scale of intelligence. (Pearson, 1999).

PERMALINK

Using connectome-based predictive models to reveal the systems standardized tests and clinical symptoms are reflecting

Anja Samardzija

Xilin Shen

Wenjing Luo

Abigail S Greene

Saloni Mehta

Fuyuze Tokoglu

Jagriti Arora

Scott W Woods

Rachel B Katz

Gerard Sanacora

Vinod H Srihari

Dustin Scheinost

R Todd Constable

Abstract

Introduction

Table 1.

Results

Ranking the tests by cognitive construct network weight

Fig. 1. Predictive performance for each cognitive construct network.

Fig. 2. Predictive power of the six cognitive construct networks on each of the 16 behavioral measures.

Testing the tests can be performed with any network of interest

Testing the tests can be performed with any functional connectivity state

Cognitive construct network-driven test combinations yield models with higher predictive power

Fig. 3. Predictive power of cognitive construct network-driven test combinations.

Brain-construct modeling provides a means to evaluate the contributions of subtests

Fig. 4. Brain-construct modeling applied on subtests and composite scores.

Test composite formation based on high prediction and cognitive construct network isolation

Fig. 5. Maximizing predictive power and cognitive construct network isolation on test composites.

Discussion

Assessing what systems test performance relies upon

Brain-based assessment of the systems the tests are testing

Composite test scores and cognitive construct network weighting

Transdiagnostic dataset

Network definitions

Measures of functional connectivity

Reliability considerations and predictive power

Task functional connectivity & state-dependance

Quantitative scores versus self-report measures

Clinical applications of this framework

Models measure networks that vary with performance

Applying the testing-the-tests framework

Methods

Dataset

Image processing

Functional connectivity

Subjects

Cognitive constructs & a priori network definitions

Behavioral measures

Model

Composite scores based on grouping individual tests that yield the highest predictive power for individual networks

Composite scores based on grouping individual tests that yield the highest predictive power and maximize network isolation

Statistical significance testing

Reporting summary

Supplementary information

Source data

Author contributions

Peer review

Peer review information

Funding

Data availability

Code availability

Competing interests

Footnotes

Contributor Information

Supplementary information

References

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases