Feature consistency in transdiagnostic connectome-based models of sustained attention and autism symptoms

Abstract

Sustained attention is an important neurobiological process. Difficulties with attention play a key role in neurodevelopmental disorders, such as attention-deficit/hyperactivity disorder (ADHD) and autism. Here, we identified functional connections consistently associated with sustained attention across datasets, participant populations, and fMRI scan types. We interrogated five transdiagnostic, previously published connectome-based models predicting attention and autistic phenotypes. All models were related to sustained attention, including in samples comprising participants with autism. We found that model similarity was associated with participant characteristics—including age and clinical diagnosis—and predicted behavioral measure. As expected, models predicting attention phenotypes shared more similar features with each other than models predicting autism symptoms. Furthermore, predictive features overlapped more between datasets that included participants of similar ages (i.e., youth vs. adult) and diagnostic status (autism diagnosis vs. no diagnosis). This suggests that functional connectivity patterns predicting individual differences in behavior are phenotype-specific and may vary as a function of age and clinical diagnosis.

Introduction

The ability to pay attention is central to human experience. Whether focusing during conversations, concentrating on other drivers on the highway, or searching for groceries in a crowded supermarket aisle, the capacity to sustain attention influences nearly every waking moment. Perhaps in part because of its ubiquity, ‘sustained attention’ is a broad construct with poorly understood brain correlates¹. A better characterization of the neurobiology of sustained attention is needed.

Beyond its importance in daily life, sustained attention is important in neurology and psychiatry. Sustained attention is associated with numerous clinical conditions, including dyslexia in youth^2,3 and delirium in medically ill patients⁴. Difficulties with sustained attention also play an important role in neurodevelopmental disorders, such as attention-deficit/hyperactivity disorder (ADHD) and autism spectrum disorder (hereafter, ‘autism’). It is estimated that approximately 8% of youth worldwide⁵ have been diagnosed with ADHD, with that number reaching 10.5% of youth in the United States (US)⁶. The prevalence of autism indicates the condition is also pervasive. In the US, approximately 3.2% of eight-year-olds carry an autism diagnosis⁷; worldwide, approximately 1 in 127 individuals have been diagnosed⁸.

Although they are distinct diagnoses, there is evidence that ADHD and autism have a number of behavioral similarities. Patients in both conditions tend to have sensory processing isusues⁹, executive dysfunction¹⁰, irritability¹¹, and metacognitive difficulties, such as performance monitoring⁹. There is also a relationship between autism and attention, such that individuals with greater autistic symptoms tend to have more difficulties with attention^12–14.

Furthermore, there are shared neurobiological correlates between ADHD and autism. Using magnetic resonance imaging (MRI) functional connectivity to measure synchrony between brain regions¹⁵, similarities have been found in the functional networks associated with each condition. For example, heteromodal association cortices—associated with higher-order cognitive functions and comprising networks such as the frontoparietal and default mode networks¹⁶—have been shown to consistently mediate both autistic and ADHD phenotypes^17–20.

Despite the many neurobiological similarities between ADHD and autism, the extent to which sustained attention itself is supported by consistent neurobiological patterns is unclear. For instance, sustained attention is exquisitely sensitive to brain state. Factors including caffeine intake²¹, emotional valence²², and fatigue level²³ have all been shown to alter functional connectivity, as well as performance on attention tasks^24–26. Nevertheless, work in neurotypical subjects suggests there might be a shared functional architecture mediating sustained attention. Across diverse sustained attention tasks, spanning different modalities (e.g., auditory and visual attention conditions), commonalities can be found in brain areas, particularly in visual and heteromodal association cortex^27–30. The same visual and heteromodal networks tend to be involved when probing resting-state conditions for associations with sustained attention phenotypes¹. The consistencies in neurotypical participants leads to a question about the functional connections and networks in autism and ADHD: are there consistent functional connectivity features associated with sustained attention in transdiagnostic samples? Or, despite the similarities, are different features underpinning sustained attention in autism and ADHD?

To address this issue, we interrogated data from five previously published models of sustained attention and autism (Figure 1; Table 1)^{1,18,28,31,32}. The models were selected because they contained a number of similarities allowing comparisons across studies. First, the same functional brain atlas was used³³. Second, all studies used connectome-based predictive modelling (CPM)^34,35, a machine learning method for deriving brain-behavior models from functional connectomes. Third, all models were related to attention, including in samples comprising autism participants. Specifically, all models were generated either by predicting attention phenotypes, shown to generalize to external samples predicting attention phenotypes, or were built using functional data from an attention task.

Figure 1.

Overview of the current paper. A) Study design. The original sample from which each attention model was derived is shown in the oval, with the ovals arranged by increasing age as indicated by the arrow. The fMRI data used to generate the model is shown directly under the dataset name. The Yale youth and ABIDE datasets comprised participants with and without autism (indicated by the purple-pink gradient shown around the ovals, with purple indicating a higher proportion of participants with autism). The adult attention sample and avCPT sample comprised neurotypical participants. Autism phenotypes were used to generate models M1 and M2 (SRS total scores and ADOS total calibrated severity scores, respectively). Attention phenotypes were used to generate models M3, M4, and M5 (gradCPT task performance). Note that the same parent sample (the Yale youth sample) was used to generate the M2 and M3 models. B) The overarching goal of the paper is to determine if a core transdiagnostic, sustained attention network could be derived. C) Specifying how each model relates to attention. The leftmost column specifies the model. In the table, the second column from the left specifies if the model was generated using in-scanner attention task functional data. The third column from the left specifies if the model was generated by predicting an attention phenotype. The rightmost column specifies if the model generalized to predict attention in an external sample. ABIDE, autism brain imaging data exchange; Attn, attention; avCPT, audio-visual continuous performance task. M, model.

Table 1.

Demographics and modelling parameters of original datasets used to generate predictive models.

	ABIDE (M1)	Yale youth autism (M2)	Youth attention (M3)	avCPT (M4)	Adult attention (M5)
Original publication	Lake et al., 201918	Horien et al., 202632	Horien et al. 202231	Corriveau et al., 202528	Rosenberg, Finn et al., 20161
Number of participants (males)	229 (164)	63 (34)	70 (39)	Visualrelevant: 43 (19) Auditoryrelevant: 43 (20)	25 (12)
Number of participants with a neurodevelopmental or psychiatric condition	77 autism	31 total 5 = ADHD 2 = anxiety disorder 20 = autism 4 = BAP	33 total 7 = ADHD 2 = anxiety disorder 20 = autism 3 = BAP 1 = bipolar	-	-
Age in years, mean (st. dev.)	10.45 (1.8)	11.7 (2.8)	11.59 (2.87)	Visualrelevant: 22.40 (4.18) Auditoryrelevant: 22.28 (3.66)	22.79 (3.54)
IQ	113.7 (15.1)	107.8 (15.1)	107.23 (15.93)	-	-
fMRI data	Resting-state	Continuous attention task (gradCPT)	Continuous attention task (gradCPT)	Continuous attention task (avCPT)	Continuous attention task (gradCPT)
Amount of fMRI data	One four minute resting-state run	Two 5:02 gradCPT runs averaged	Two 5:02 gradCPT runs averaged	One 10-min visualrelevant CPT, one 10-min auditoryrelevant CPT	Three 13:44 min gradCPT runs concatenated
Phenotype predicted in original model	SRS total scores39	ADOS total calibrated score43	Performance on attention task (d’)1	Performance on attention task (d’)28	Performance on attention task (d’)1
Number of trials in CPT	-	302	302	500	2700
Prediction performance in original sample	Pearson’s rho between predicted and observed total SRS scores = 0.32, P-value < 5E-10	Spearman’s rho between predicted and observed total ADOS scores = 0.445, P-value = 0.002	Spearman’s rho between predicted and observed total d’ scores = 0.54, P-value = 0.0001	-	Pearson’s rho between predicted and observed total d’ scores = 0.84, P-value = 1.3E-7
Phenotypes successfully predicted in test datasets in original publication	ADHD Rating Scale IV45	Sustained attention task performance (d’)1 SRS scores39	Sustained attention task performance (d’)1	-	ADHD Rating Scale IV45

ABIDE, autism brain imaging data exchange; ADHD, attention-deficit/hyperactivity disorder; BAP, broader autism phenotype; ADOS, Autism Diagnostic Observations Schedule; avCPT, audio-visual continuous performance task; CPM, connectome-based predictive modelling; CPT, continuous performance task; gradCPT, gradual-onset continuous performance task; IQ, intelligence quotient; SRS, social responsiveness scale; TR, repetition time.

To establish the upper bounds of consistency, we leverage a number of further similarities in the samples. Multiple models were derived using similar attention task scanning conditions. Some of the models were validated on the same external samples (Supplemental Figure 1). Further, we capitalize on the fact that two models from separate publications shared ~90% of subjects, but predicted different phenotypes—one predicting attention task performance and the other predicting autism symptoms. We hypothesized that we would find commonalities across all five models. In particular, we predicted the same functional connections and network structure would be observed.

We found that most brain-sustained attention models shared functional connections and received contributions from similar canonical functional networks, although overlap across datasets was not perfect. Models predicting attention phenotypes shared more similar features with each other than models predicting autism symptoms. This finding was exemplified by the fact that two of the only models that did not share connections were from the samples sharing ~90% of subjects, but predicted different phenotypes. We also observed that model similarity was associated with other characteristics, including age and clinical diagnosis. In total, our findings suggest that consistencies in model similarity are associated with numerous participant factors, and connectivity patterns predicting individual differences in behavior tend to be phenotype-specific.

Methods

Overview of the methodological approach

We have two main goals in this paper. First, we assess the consistency of sustained attention brain-behavior relationships at the edge and network levels. Second, we determine factors influencing brain-sustained attention relationships, such as age, diagnosis, and other demographic variables. We use data from five previously published studies (‘publication preregistration’³⁶). Despite differences in the datasets, all models relate to sustained attention, including those derived from samples comprising participants with autism (Figure 1; Table 1).

Description of datasets used to generate predictive models

All datasets were collected in accordance with the institutional review board or research ethics committee at each site. Where appropriate, informed consent was obtained from the parents or guardians of participants. Written assent was obtained from children aged 13–17 years; verbal assent was obtained from participants under the age of 13 years.

Dataset 1, model 1: ABIDE model

The Autism Brain Imaging Data Exchange (ABIDE) dataset comprised individuals with and without autism (n = 229, 65 females; mean age = 10.45 years, st. dev. = 1.8 years; mean IQ = 113.7, st. dev. = 15.1; 77/229 participants were diagnosed with autism; Table 1). Throughout the text, we refer to this sample as the ‘ABIDE model/sample’^37,38. Processing of these data is described elsewhere¹⁸. Resting-state data were used. Social responsiveness scale (SRS)³⁹ total scores (reflective of social difficulties in autism) were the behavior of interest (mean score = 42.4, st. dev. = 40.2).

Dataset 2, model 2: Yale youth autism model

The second dataset comprised 63 subjects (mean age = 11.7 years, st. dev. = 2.8 years; 29 females; mean IQ = 107.8, st. dev. = 15.1); full exclusion criteria and imaging parameters have been published elsewhere³². Twenty of the participants had autism; 11 other participants had a different neurodevelopmental condition (five with ADHD, two with anxiety disorder, and four were classified as belonging to the broader autism phenotype)⁴⁰. Hereafter, we refer to this model/sample as the ‘Yale youth autism model/sample.’ Subjects performed a sustained attention task, the gradual-onset continuous performance task (gradCPT), in the scanner^1,41,42. For a description of the gradCPT, see ‘Functional imaging conditions: sustained attention tasks and resting-state.’ For specific gradCPT task parameters, see Table 1. Autism symptoms were scored using the Autism Diagnostic Observation Schedule-2 (ADOS-2)⁴³; calibrated severity scores were used in the present work for ADOS total score (mean score = 3.1, st. dev. = 3.1).

Dataset 2, model 3: youth attention model

The second dataset was also used to generate a different model. The sample comprised 70 subjects (mean age = 11.59 years, st. dev. = 2.87 years; 31 females; mean IQ = 107.23, st. dev. = 15.9); full details of the sample have been described elsewhere^31,44. Twenty of the participants had autism. Thirteen other participants had a different neurodevelopmental condition (seven with ADHD, two with anxiety disorder, one with bipolar disorder, and three were classified as belonging to the broader autism phenotype)⁴⁰. We refer to this model/sample as the ‘youth attention model/sample,’ but readers should keep in mind the transdiagnostic nature of the dataset. Subjects performed gradCPT in the scanner^1,41,42. Accuracy on the gradCPT is the behavior of interest in this sample (d’; mean score = 2.54, st. dev. = 0.9). Note that because they were drawn from the same parent studies, models 2 and 3 shared 56 subjects.

Dataset 3, model 4: avCPT model

The next dataset has been described elsewhere²⁸. The final sample size was n = 43 (n = 24 females in visual condition, n = 23 females in auditory condition); average age = 22.4 (st. dev. = 4.18). Participants completed an in-scanner 10-minute audio-visual continuous performance task (avCPT; we refer to this model/sample hereafter as the ‘avCPT model/sample’). Accuracy on the avCPT is the behavior of interest in this sample (d’ visual condition: mean score = 3.08, st. dev. = 0.65; d’ auditory condition: mean score = 0.95, st. dev. = 0.60). (For a description of the avCPT, see ‘Functional imaging conditions: sustained attention tasks and resting-state’).

Dataset 4, model 5: adult attention model

The last dataset comprised neurotypical adults (hereafter referred to as the ‘adult attention model/sample’; n = 25, 13 females, mean age = 22.8 years, st. dev. = 3.5 years); full details of this sample are described elsewhere¹. Subjects performed gradCPT in the scanner^1,41,42. Accuracy on the gradCPT is the behavior of interest in this sample (d’; mean score = 2.11, st. dev. = 0.92).

Functional imaging conditions: sustained attention tasks and resting-state

Functional imaging data were obtained while participants in the youth attention sample and the Yale youth sample completed the gradCPT in the scanner. The gradCPT has been described previously^1,41,42. Briefly, participants viewed grayscale pictures of cities and mountains presented at the center of the screen, with images gradually transitioning from one to the next every 1,000 ms. Subjects were instructed to respond with a button press for city scenes and to withhold button presses for mountain scenes. City scenes occurred randomly 90% of the time. Performance was calculated using d’ (sensitivity), the participant’s z-scored hit rate minus z-scored false alarm rate. Subjects in the adult attention sample⁴⁶ also performed the gradCPT in the scanner. The same parameters were used as above, except scene transitions took 800 ms.

Participants in the avCPT sample completed the audio-visual continuous performance task (avCPT)²⁸. The task was completed twice in two separate sessions, so that the visual condition required a button press to frequent (90%) visual stimuli and withhold for infrequent (10%) stimuli (indoor and outdoor scenes, counterbalanced across participants); the same frequency parameters were used in the auditory version of the task (natural and artificial sounds, counterbalanced across participants). Performance was quantified using d’ (sensitivity) as above, the participant’s z-scored hit rate minus z-scored false alarm rate. In ABIDE, functional data comprised resting-state scans as described previously^37,38.

Processing of functional imaging data

Functional data were processed as previously described^28,47–50 and followed the same general approach. This included alignment of functional data to a shared space, regression of covariates of no interest, including a head motion model, the mean signal from white matter and ventricle masks, and mean global signal (see Supplemental Materials, ‘Preprocessing of imaging data’ for full details of preprocessing).

Connectomes were generated using a 268-node atlas³³. The mean time-course of each region of interest (“node”) was calculated, and the Pearson correlation coefficient was calculated between each node pair to achieve a symmetric 268×268 matrix of correlation values representing connections between nodes (“edges”). Nodes were grouped into one of ten canonical functional networks described previously^51,52. Care was taken in each study to ensure artifacts related to head motion were not driving model results (see Supplemental Methods, ‘Minimizing the impact of head motion’ and Supplemental Table 1).

Note that the original avCPT sample comprised 45 visual and 44 auditory runs; we excluded two participants from the visual run and one participant from the auditory runs for missing a significant number of edges (>4,100 / 35,778 total) due to poor coverage. In the remaining subjects, twelve participants in the auditory runs and 18 subjects in the visual runs had incomplete functional coverage. If any subject was missing an edge, we removed that edge from all participants being compared from all samples; the final connectome size was 32,640 edges and 31,626 edges in the auditory and visual runs, respectively.

Accounting for differences in size of the brain-sustained attention predictive models

Connectome-based predictive modelling (CPM; Supplemental Figure 2) was used to generate the models in the current work. Importantly, each component publication conducted numerous controls to ensure that variables like age, sex, head-motion, and other factors were not confounding CPM results^{1,18,31,32,28}. CPM uses cross-validation to determine the functional connections predictive of a given phenotype, resulting in a summary network model. Based on the edges most positively and negatively correlated with behavior, a positive and negative network model is generated, respectively. The summary network model is then used to predict phenotypic outcomes in unseen participants. The summary network model is the output of interest in the present work. Due to the relationship between autism and attention (higher autism symptoms associated with poorer attention^12–14), we considered the networks from ABIDE and the Yale youth autism study in the opposite direction. Specifically, the positive ABIDE and Yale youth autism networks were considered with the negative networks from the avCPT, adult attention, and youth attention samples, and vice-versa.

Based on different modelling parameters in each original study (Supplemental Table 2), each summary network is a different size. To ensure this did not confound results, we reanalyzed the original data used to construct the models in each sample. Edges were correlated with behavior in each sample, ranked according to correlation strength, and the top 1,000 positively and negatively correlated edges were retained for analysis. (This corresponds to the feature selection step in Supplemental Figure 2, box 3, except instead of a feature selection threshold, we have only retained the top 1,000 edges.) Note that 2,000 edges are chosen in total, in line with the original networks studied here, as well as previously published CPM networks of other phenotypes (e.g.,⁵³). To ensure results were not driven by the use of a specific threshold, we also tested 500 edges in each positive and negative network (1,000 total), as well as 2,500 edges in each network (5,000 edges total).

Statistical overlap of edges in predictive models

To assess the probability of networks containing a statistically significant number of shared edges, we performed 1,000 iterations of generating random networks, ensuring the null networks were the same size as the original network from a given sample (see Supplemental Figure 3A for a schematic of the process). We then compared the number of times the random networks resulted in greater than or equal number of overlapping edges in the actual predictive networks. In other words, we calculated the overlap of random network models and determined how many edges were shared by models.

We next assessed the overlap of edges among specific models themselves in a pairwise fashion. That is, we considered two predictive models at a time and asked if the overlap among edges was statistically significant. To do so, we used the hypergeometric cumulative density function (CDF) as previously described^28,36. Briefly, in MATLAB, the pseudocode is P = 1 – hygecdf(x, M, K, n), where x equals the number of overlapping edges between the two models, M equals the total number of edges in the connectome (31,626 edges after removing nodes that were missing from the avCPT visual run), K equals the number of edges in the first predictive model being compared, and n equals the number of edges in the second predictive model being compared. This allowed us to determine empirical P-values of drawing up to x of K possible items in n drawings without replacement from an M-item population. We used the Benjamini-Hochberg procedure⁵⁴ to control for multiple comparisons (20 total, ten comparisons among the positive network and ten among the negative networks).

Quantifying the contribution of network pairs to predictive models

To determine the importance of specific networks in the predictive models, we quantified the number of edges between a given network pair, correcting for network size as described previously^31,55. We calculated null predictive network models for each sample (see Supplemental Figure 3B for a schematic of the process). Specifically, we generated random binary networks, with edge locations randomly generated across the connectome, ensuring the size of the null network matched that of the true predictive network from a given dataset. The process was repeated 5,000 times. We then quantified the number of edges between a given network pair in the null network, again accounting for network size. P-values were obtained by calculating the number of times out of 5,000 that null models had a greater than or equal number of edges as the observed network pair from the predictive model. Given this analysis involved 45 comparisons for a given network per sample (i.e., (10 networks x 10 networks – 10 networks) / 2 = 45 unique network pairs), we use the stricter Bonferroni correction⁵⁶ when reporting significant P-values.

Assessing brain-behavior relationships using representational similarity analysis

To assess the importance of different dataset features on the brain-behavior relationships, we used representational similarity analysis (RSA)⁵⁷. We correlated the connectivity matrices from all participants in a sample with the behavior of interest, resulting in a 31,626×1 vector per dataset. Note we used the visual run of the avCPT sample for our main analysis. After correlating these vectors across samples to obtain a feature importance similarity matrix, we generated model representational similarity matrices based on mean age of the sample, functional run type (task or rest), behavioral measure (attention score or autism symptom score), and percentage of the sample diagnosed with autism. We z-scored the entries in each representational similarity matrix to obtain standardized betas in a multiple linear regression, allowing us to assess the unique variance in the feature importance similarity matrix explained by each predictor:

$${\mathit{\text{Similarity}}}_{\left\{ij\right\}}={\beta }_0+{\beta }_1\mathit{\text{AgeDif}}{f}_{\left\{ij\right\}}^{\left\{\left(z\right)\right\}}+{\beta }_2\mathit{\text{RunTypeMatc}}{h}_{\left\{ij\right\}}^{\left\{\left(z\right)\right\}}+{\beta }_3\mathit{\text{BehaviorMatc}}{h}_{\left\{ij\right\}}^{\left\{\left(z\right)\right\}}+{\beta }_4\mathit{\text{AutismDif}}{f}_{\left\{ij\right\}}^{\left\{\left(z\right)\right\}}+{\epsilon }_{\left\{ij\right\}}$$

where Similarity_{ij} is the Pearson correlation between feature importance vectors for datasets i and j; AgeDiff_{ij} is the absolute difference in mean age between datasets; RunTypeMatch{ij} is a binary indicator of the type of functional data used, either resting-state or task data (1 = same run type, 0 = different); BehaviorMatch_{ij} is a binary indicator of the type of phenotype predicted, either autistic symptoms or attention task performance (1 = same behavioral measure, 0 = different behavioral measure); AutismDiff_{ij} is the absolute difference in percent autism diagnosis between datasets; β₀ represents the intercept; and ε_{ij} is the residual error.

Our main goal with this analysis is not to derive a new model predicting performance; rather, it is to compare the relative importance of each predictor. To ensure results were robust, we repeated analyses using the auditory run of the avCPT sample.

Code and data availability

Preprocessing was carried out using freely available software: https://medicine.yale.edu/bioimaging/suite/. Template scripts used for preprocessing of the functional data are available here: https://github.com/clhorien/transdiagnostic_attn_overlap/tree/main. The functional parcellation is available here: https://www.nitrc.org/frs/?group_id=51. ABIDE data are available here: https://fcon_1000.projects.nitrc.org/indi/abide/. Data from the avCPT sample are available here: https://osf.io/bt2xy/overview. All other data or materials are available from the authors upon request.

Results

Characteristics of previously published predictive models

We began by assessing basic aspects of the previously published predictive models. We analyzed characteristics of the CPM networks derived from each study (i.e., we did not ensure networks were the same size at this stage). The original networks comprised a dense collection of edges spanning the entire brain (Supplemental Figures 4–6). Network size and the number of interhemispheric versus intrahemispheric connections were also assessed. Despite using different modelling parameters (Supplemental Table 2), networks were generally the same size and comprised a similar number of connections within and between hemispheres (Supplemental Figure 4; Supplemental Table 3). Functional networks from across the brain contributed to the models, from visual and somatomotor networks to higher-order association networks, including the default mode, frontoparietal, and medial frontal networks (Supplemental Figures 5–6).

Edge overlap in brain-sustained attention predictive models

We next quantified the number of times individual edges were shared across models. Because of differences in network size across samples, network size was kept consistent across samples. Specifically, we assessed the top 1,000 edges correlated in each direction with the behavior (the ‘positive’ and ‘negative’ networks, respectively; recall that we accounted for autism diagnosis in this procedure). In the positive and negative networks, we observed 408 and 423 edges that were shared by two models (Figure 2A), respectively. Both of these values were statistically significant compared to the overlap found in null models (P-value for positive network = 0.00099; P-value for negative network = 0.00099, corrected for multiple comparisons). When assessing if edges were shared across a greater number of models, 28 edges were shared three times in the positive networks and 35 edges were shared three times in the negative networks (P-value for positive network = 0.00099; P-value for negative network = 0.00099, corrected for multiple comparisons). Notably, no edges were shared across four models in the positive networks; only one edge was shared across four models in the negative networks, which was not statistically significant (P-value = 0.167, corrected for multiple comparisons). No edges were shared across five models in either of the networks. To ensure results were consistent, we repeated analyses using a different number of edges. We observed a similar pattern when testing 500 edges in each network (1,000 total) and 2,500 edges in each network (5,000 total; Supplemental Figure 7).

Figure 2.

Quantifying the occurrence of individual edges across models. A) Violin plots show the number of edges shared across models as a red horizontal line in each plot. The grey violin plot represents null results of the number of shared edges obtained from 1,000 permutations, with the black bar in the violin plot indicating the median of the 1,000 permutations. The positive network is shown in the left column; the negative network, in the right column. Above each plot, ‘k’ corresponds to the edge occurrence across models. P-values are shown at the top of each plot; asterisks (*) indicate statistical significance after multiple comparisons correction. B) Edge overlap. Two matrices are shown: overlap between the top-ranked edges (‘Pos-Pos’, left); overlap between the bottom-ranked edges (‘Neg-Neg’, right). Each cell corresponds to the number of edges shared between models. Dataset from which the model was obtained is indicated along the rows and columns of each matrix. The colorbar is scaled so that fewer shared edges (in terms of absolute number) are darker colors; more shared edges are lighter colors. An asterisk indicates a statistically significant shared number of edges between the two models after correcting for multiple comparisons. ABIDE, autism brain imaging data exchange. Att, attention; Aut, autism; avCPT, audio-visual continuous performance task; Neg, negative network; Pos, positive network.

Given the shared edges, we next assessed overlap when considering the models pairwise (Figure 2B). In the positive network, we observed 8/10 pairwise comparisons that contained a statistically significant number of edges in the positive network (shared number of edges in all cases > 47, all P-values < 0.003, corrected for multiple comparisons). In the negative network, 5/10 pairwise comparisons shared a statistically significant number of edges (shared number of edges in all cases > 42, all P-values < 0.019, corrected for multiple comparisons). Of note, the two models sharing subjects, but predicting different categories of phenotypes, did not share a statistically significant number of edges in the positive or negative networks (the Yale youth autism sample predicting ADOS and the youth attention sample predicting gradCPT task performance; positive network shared edges = 32, P-value = 0.43; negative network shared edges = 20, P-value 0.98), suggesting that edge overlap was not driven by sample overlap.

To ensure findings were robust, we repeated pairwise comparisons with different network sizes. We obtained a similar pattern when testing 500 edges in each network and 2,500 edges in each network (Supplemental Figure 8). Taken together, data in this section indicate that while we did not observe a core edge set across all samples, most models tended to have similar edges when considered pairwise.

Network consistency in brain-sustained attention relationships

The models were next analyzed at the network level, focusing on both within- and between-network pairs. We again assessed the top 1,000 edges correlated positively and the top 1,000 edges correlated negatively with behavior. Because of differences in size in within- and between-network pairs in the connectomes, we corrected for network size in this procedure (Methods). We observed network pairs linking all parts of the brain were represented (Figure 3A). In the positive network, visual, motor, and default mode networks had the highest fraction of edges in the five samples. In the negative network, visual, default mode, and cerebellar networks had the highest fraction of edges. We obtained similar results when testing network sizes of 500 edges in each network and 2,500 edges in each network (Supplemental Figure 9).

Figure 3.

Neuroanatomy of the attention network model. A) Summary matrix for all five datasets. B) Summary matrix for the datasets predicting autistic phenotypes C) Summary matrix for the datasets predicting continuous attention task performance. For all plots: Positive network masks are shown in the top row; negative network masks, in the bottom row. Each cell in a matrix corresponds to the fraction of edges in a given network pair that were included in the given model, after correcting for network size. The colorbars for each matrix are scaled so that a lower fraction of edges corresponds to darker colors; a higher fraction of edges corresponds to lighter colors. Statistically significant network pairs after multiple comparisons correction are denoted with an asterisk (*). ABIDE, autism brain imaging data exchange. Att, attention; Aut, autism; avCPT, audio-visual continuous performance task; Neg, negative network; Pos, positive network. Network labels: MF, medial frontal; FP, frontoparietal; DM, default mode; MT, motor; VI, visual I; VII, visual II; VA, visual association; CO, cingulo-opercular; SB, subcortical; CB, cerebellum.

Across the individual samples, the location of statistically significant network pairs was assessed after accounting for multiple comparisons (see Supplemental Figure 3B for a schematic of the procedure). No discernible pattern emerged regarding the location of statistically significant networks in the individual samples (see Supplemental Figure 10 for the network pairs surviving multiple comparisons correction). Sensorimotor and cerebellar networks were statistically significant, as were higher order networks, including the default mode and frontoparietal networks. Findings were similar when different sized networks were used (500 and 2,500 edges; Supplemental Figure 10A–C).

Despite the broad nature of the models, there was little overlap in terms of which network pairs were statistically significant across the samples as a whole. For example, no network pairs were common across all five samples in the positive or negative networks. When we considered overlap among the datasets predicting autistic phenotypes, we observed three shared network pairs: one in the positive and two in the negative network (Figure 3B). All three comprised connections involving heteromodal association networks. There were two common network pairs when considering the datasets predicting continuous attention performance (Figure 3C). We also assessed pairwise comparisons of shared networks and determined most samples shared at least one statistically significant network. Results were again consistent using the differently sized networks (Supplemental Figure 10D–F). The samples sharing subjects, but predicting different phenotypes (Yale youth autism and youth attention), tended not to share networks.

To confirm specificity of the positive and negative networks, we flipped the sign of each network and assessed overlap. That is, the positive networks of the autism samples (Yale youth autism and ABIDE) were compared with the negative networks of the attention samples (adult attention, youth attention, and avCPT) and vice-versa. We observed no statistically significant shared network pairs in both cases.

The results in this section suggest that sustained attention models have distributed network representations. Although no core set of networks was observed, some degree of overlap was detected when we grouped by phenotype.

Exploring the factors impacting brain-sustained attention relationships

In the previous sections, we found that although overlap was not absolute, some of the models shared features. A natural follow-up question is if there are meaningful differences between the datasets—such as demographic and phenotypic factors—that are associated with model similarity. Hence, we conducted analyses assessing brain-sustained attention relationships using unthresholded data and RSA. One of the prime benefits of using RSA⁵⁷ is the investigation of whole-brain patterns mediating sustained attention, even those below the surface of what survives significance thresholding used in feature selection. Predictors including the mean age of the sample, functional run type (task or rest), behavioral measure (attention score or autism symptom score), and percentage of the sample that had an autism diagnosis were assessed. We observed age and autism diagnosis were associated with similarity between brain-sustained attention vectors, albeit in different directions (P-values < 0.05; see Table 2 for regression coefficients and confidence intervals). Run type and behavioral measure were not statistically significant.

Table 2.

Results of RSA using the original data, unthresholded in each study. Shown is the standardized (z-scored) regression coefficient, along with the 95% confidence intervals (CI).

	Regression coefficient	Lower bound 95% CI	Upper bound 95% CI
Age	1.064*	0.421	1.71
Run type	−0.0467	−0.1399	0.0466
Behavioral measure	0.0053	−0.0737	0.0842
Autism diagnosis	−1.1269*	−1.7621	−0.4916

Asterisks (*) indicates the regression coefficient is statistically significant (P < 0.05).

Lastly, we examined pairwise correlations among the datasets. The first step in RSA is calculating a similarity matrix of edge-sustained attention vectors across datasets through the use of correlation. We compared the similarity among the samples to that observed from null models. Using the visual run of the avCPT task, we observed 4/10 correlations among datasets were positively correlated (Figure 4; Supplemental Table 4; all Pearson correlations > 0.13; all P-values < 0.005, corrected for multiple comparisons). All of the positive correlations were between samples that had congruent phenotypes (i.e., when both predicted autism symptoms or attention task performance in the original sample). One of the correlations among brain-sustained attention vectors was statistically significantly negatively correlated. This was among phenotypes that were incongruent (Yale youth autism and adult attention sample; Pearson correlation = −0.14; P-value = 0.005, corrected for multiple comparisons; Figure 4).

Figure 4.

Relationship of brain-sustained attention across samples A) Histograms showing the observed relationship (in red) between the brain-sustained attention vectors (i.e., the Pearson correlation of the brain-sustained attention vectors, the datasets compared are indicated in the rows and columns). Null values for the histograms are shown in blue. In the upper left of each plot, the Pearson correlation coefficient between the datasets in each row and column is shown, along with the P-value. B) Similarity of edge-behavior vectors across datasets. Each bar represents the Pearson correlation coefficient (y-axis) between the datasets indicated on the x-axis. Behaviors that are both related to attention or autism (‘Congruent phenotypes’) are indicated in salmon color; behaviors that are not related between datasets (‘Incongruent phenotype’), in which one sample comprises an autism phenotype with the other comprising an attention phenotype, are indicated in green. In both plots, asterisk (*) indicates statistical significance after correction for multiple comparisons. ABIDE, autism brain imaging data exchange dataset; avCPT, audio-visual continuous performance task dataset.

To ensure results were robust, we also tested similarity of the edge-sustained attention vectors when the auditory condition from the avCPT sample was used instead of the visual condition (recall the avCPT sample comprised two separate conditions, one visual condition and one auditory condition). Findings were consistent (Supplemental Table 4). In sum, the factors modulating brain-sustained attention relationships are complex, though variables such as age, diagnosis, and phenotype appear to be important from preliminary analyses.

Discussion

In the present work, we attempted to identify consistent functional connectome-based features associated with sustained attention. Using data from five transdiagnostic, previously published functional connectivity-based brain-behavior predictive models, we found that most sustained attention models contained similar functional connections and resting-state functional networks. Factors including participant age, clinical diagnosis, and predicted behavior were associated with model similarity. As expected, models predicting sustained attention were more closely related with each other than models predicting autism symptoms. In sum, data suggest that patterns predicting individual differences in behavior are phenotype-specific and may vary as a function of age and clinical diagnosis.

Distributed, transdiagnostic models and the search for a core architecture

We did not derive a core sustained attention network across the five samples. The result is notable given the many similarities among the datasets and that the models have generalized in similar (or in many cases the same) datasets. We also reiterate that the two models sharing 90% of subjects did not contain overlapping edges or networks. One interpretation of the lack of overlap is that despite being broadly related to sustained attention, models tend to be specific to the construct they were built to predict. They do not appear to be general across a range of associated phenotypes.

Another explanation for the lack of strict overlap relates to the distributed nature of the models. The models comprised a dense collection of edges spanning the entire brain, with inputs involving nearly every within- and between-network pair across subcortical, cortical, and cerebellar networks. The multivariate nature of the models echoes results in the functional connectivity reliability literature. In particular, individual edges do not tend to have high reliability⁵², but multivariate measures of reliability tend to be higher^58,59. In the current study, this might explain that broad, qualitative similarities can be observed—most models tended to comprise a high fraction of edges in visual and heteromodal networks. The more similar the phenotypes involved in the brain-behavior prediction, the more similar the coarse similarities become.

Alternatively, another reason for the lack of consistency might be due to the transdiagnostic samples. Previous studies observed overlap in datasets comprised solely of neurotypical participants²⁸. The two samples comprising solely neurotypical participants in the present paper shared a statistically significant number of edges and networks at every threshold studied. There are functional connectivity differences that can be reliably used to differentiate autism cases from controls (reviewed in⁶⁰), potentially complicating the search for a core set of features.

Neurobiological patterns observed in transdiagnostic sustained attention network models

Although perfect overlap was not obtained, neurobiological patterns emerged. In the positive network, visual, motor, and default mode networks had the highest fraction of edges in the five samples. In the negative network, visual, default mode, and cerebellar networks had the highest fraction of edges. When we considered overlap among the datasets predicting autistic phenotypes, we observed three shared networks. All three comprised connections involving visual or heteromodal association networks. When we assessed commonalities among the datasets predicting attention phenotypes, we observed two shared networks, involving visual, motor, and heteromodal networks. One interpretation of these findings is given the cognitive demands associated with sustaining attention^61,62, coordination is required between sensory networks and networks mediating higher-order functions, like heteromodal networks. In addition, the fact that connections linking visual and motor networks emerged as consistently important in the attention samples is probably due to the in-scanner button presses associated with the gradCPT.

In general, the finding that sensorimotor and heteromodal networks are important in sustained attention models is reminiscent of the sensorimotor-association (SA) axis⁶³. Individual differences in the SA axis have previously been linked to differences in complex phenotypes like intelligence⁶⁴ and conditions like autism^65–67. That we used a fully data-driven approach and still observed SA axis importance speaks to the potential ‘omnipresence’ of the axis in the connectome⁶⁴. In addition, our finding that the cerebellum is important in the negative network is intriguing. Previous work⁶⁸ suggests that the cerebellum might be playing a key role in mediating the complex interplay between association networks responsible for higher-order functions, like attention. Due to the well-known effects of the cerebellum coordinating movement, however, and the button pressing aspect of gradCPT, more work is needed to better understand the role in sustained attention.

Factors impacting brain-sustained attention relationships

Using RSA, age and autism diagnosis were associated with similarity between brain-sustained attention vectors. Specifically, we observed a positive association between the similarity of brain-sustained attention vectors and age. We observed a negative association between the similarity of brain-sustained attention vectors and autism diagnosis. That is, the more similar datasets were with respect to autism diagnosis, the less similar their groupwise brain-sustained attention vectors were. Developmental changes in the connectome most likely account for the relationship with age^16,69. The complex relationship between autism symptoms and the impact on the connectome possibly explain the inverse relationship observed here⁷⁰. Furthermore, there are likely distinct neurobiological subtypes of autism that do not cleanly align with phenotypic profiles^71,72, meaning that similar autism symptoms do not always correlate with more similar brain networks.

We did not observe run type (rest or task) or behavioral measure (attention score or autism symptom score) to be statistically significant in RSA. Nevertheless, we did find that when phenotypes were similar, connectomes tended to be more similar (i.e. Figure 4B) and vice-versa. In other words, if two samples predicted autism phenotypes, or both predicted performance on gradCPT, their brain-behavior vectors tended to be more similar. One interpretation is that while a variable like behavioral measure does not perfectly modulate brain-sustained attention relationships across all of the samples, there are still instances in which two vectors will be similar because they predicted similar phenotypes.

Another possibility for the seemingly discrepant findings regarding the effect of behavioral measure is that there are other confounding factors that were unmodelled. We restricted analyses to demographic features that were common to all five datasets. Future work in prospective studies could more rigorously assess other variables. Given the importance of other sociodemographic factors in brain-behavior models^48,73, particular focus could be placed on influences such as like socioeconomic status⁷⁴, air pollution⁷⁵, and other environmental exposures (e.g.⁷⁶). Our results indicate that although there appear to be some factors associated with a core sustained attention neurobiology, the relationship is complex and much important work remains to be conducted.

Limitations and future directions

There are a number of limitations and future directions to note. We assessed predictive models made with CPM. Future work could study other modelling methods. In addition, the published models here all relied on edge selection to generate binary network masks. We have accounted for this by ensuring all networks were of the same size, as well as returning to the original unthresholded data of each study. Nevertheless, there are alternative ways to summarize model weights^77–81 other than the feature selection method described here. In fact, recent work suggests that edges often ignored by feature selection still retain predictive utility⁸². Other groups could investigate the extent to which other feature selection methods are consistent and offer interpretable neurobiological insights.

The samples used in the original studies are somewhat small, particularly when compared with samples like the Adolescent Brain Cognitive Development (ABCD)⁸³ Study or the UK-Biobank⁸⁴. Assessing if commonalities can be found from other imaging modalities, such as structural predictive models,⁸⁵ remains an area of further study. There has been growing interest in using precision functional mapping methods⁸⁶. Future work could examine consistencies in predictive models generated using longer scan times.

Conclusion

We have characterized neurobiological trends in predictive models of sustained attention in transdiagnostic samples comprising autism and ADHD participants. We observed functional connectivity patterns mediating sustained attention have some consistencies in neurobiology, with factors like age, autism diagnosis, and phenotype associated with model similarity. Our results underscore the importance of searching for consistent markers of transdiagnostic sustained attention phenotypes.

Highlights.

• We interrogated five previously published, transdiagnostic models of sustained attention and autistic phenotypes

• Functional connectome-based models shared edges and networks

• Phenotype, age, and diagnosis were associated with model similarity