Brain parcellation selection: An overlooked decision point with meaningful effects on individual differences in resting-state functional connectivity

Abstract

Over the past decade extensive research has examined the segregation of the human brain into large-scale functional networks. The resulting network maps, i.e. parcellations, are now commonly used for the a priori identification of functional networks. However, the use of these parcellations, particularly in developmental and clinical samples, hinges on four fundamental assumptions: (1) the various parcellations are equally able to recover the networks of interest; (2) adult-derived parcellations well represent the networks in children’s brains; (3) network properties, such as within-network connectivity, are reliably measured across parcellations; and (4) parcellation selection does not impact the results with regard to individual differences in given network properties. In the present study we examined these assumptions using eight common parcellation schemes in two independent developmental samples. We found that the parcellations are equally able to capture networks of interest in both children and adults. However, networks bearing the same name across parcellations (e.g., default network) do not produce reliable within-network measures of functional connectivity. Critically, parcellation selection significantly impacted the magnitude of associations of functional connectivity with age, poverty, and cognitive ability, producing meaningful differences in interpretation of individual differences in functional connectivity based on parcellation choice. Our findings suggest that work employing parcellations may benefit from the use of multiple schemes to confirm the robustness and generalizability of results. Furthermore, researchers looking to gain insight into functional networks may benefit from employing more nuanced network identification approaches such as using densely-sampled data to produce individual-derived network parcellations. A transition towards precision neuroscience will provide new avenues in the characterization of functional brain organization across development and within clinical populations.

1. Introduction

The human brain is segregated into a series of large-scale functional networks comprised of widely distributed regions. Exploration of these functional networks has been conducted using an array of techniques including clustering methods such as Gaussian mixture models (Lashkari et al., 2010; Yeo et al., 2011), meta-analytic connectivity methods (Eickhoff et al., 2011; Power et al., 2011), edge detection methods (Gordon et al., 2016; Laumann et al., 2015), multi-modal methods (Glasser et al., 2016), among many others (Schaefer et al., 2018; Baldassano et al., 2015; Blumensath et al., 2013; Smith et al., 2009). This work has given rise to numerous brain parcellation schemes that detail the specific brain regions that comprise each of the functional networks. Though the primary goal of these parcellations is to reveal brain organization, cognitive, developmental and clinical research has frequently used the parcellations as a means of identifying these networks in subjects to examine individual differences in the functional organization of the brain based on cognition (Lopez et al., 2019; Murphy et al., 2020), age (Jalbrzikowski et al., 2019; Lopez et al., 2019; Satterthwaite et al., 2013; Sylvester et al., 2018) or the presence of psychopathology (Fan et al., 2019; Lydon-Staley et al., 2019; Reggente et al., 2018; Yu et al., 2019). Brain parcellations are used to extract data from a set of parcels, or brain regions, that comprise the pre-identified functional networks. These data are then used to examine network-specific properties such as within-network connectivity (Fan et al., 2019; Karcher et al., 2019; Lydon-Staley et al., 2019; Yu et al., 2019), or graph theory metrics such as global efficiency and modularity (Bullmore and Sporns, 2009; Grayson and Fair, 2017; Stumme et al., 2020) to draw conclusions about global properties of brain function and individual differences in network properties as a function of development or across clinical groups. The widespread adoption of these parcellations in cognitive neuroscience broadly (Geerligs et al., 2015; Finc et al., 2020; Murphy et al., 2020; Weis et al., 2020), and specifically within developmental (Baum et al., 2020; Karcher et al., 2019; Lopez et al., 2019; Tooley et al., 2020) and clinical research (Fair et al., 2013; Xia et al., 2018; Yerys et al., 2019) has allowed for extensive exploration of individual differences in functional brain organization.

The application of brain parcellations is now commonplace. However, this practice hinges on four major assumptions (Fig. 1) that, to our knowledge, have never been evaluated empirically. The first assumption is that the networks of interest are well recovered in the data extracted from a parcellation, such that the extracted data can be used to construct network-specific measures. Furthermore, as most studies rely on a single parcellation, it is assumed that the various parcellations are equally able to capture or recapitulate the networks of interest. Given that adult-derived parcellations are frequently applied to data from developmental samples (Alarcón et al., 2018; Fair et al., 2013; Jalbrzikowski et al., 2019; Lopez et al., 2019; Satterthwaite et al., 2013; Sylvester et al., 2018; Yerys et al., 2019), a second assumption is that these parcellations reflect the topography of functional networks in children, and thus that the networks of interest will be accurately captured by these parcellations in developmental samples. Third, many of the available parcellations share highly similar labeling schemes, where networks that share similar spatial extents across the various parcellations also bear the same network labels (e.g., “default network ”, “dorsal attention network ”, etc.). This shared labeling has led to the assumption that the networks that are represented across the parcellations can be considered analogous. In other words, the default network identified in parcellation scheme A is the same network as the default network identified in parcellation scheme B. Thus, the parcellations are assumed to produce reliable measurement of network properties such as average within network connectivity. Finally, though some studies replicate results using a series of parcellations (Finc et al., 2020 Finc et al., 2021; Geerligs et al., 2015; Luppi et al., 2021; Tooley et al., 2020; Xia et al., 2018), a majority of studies only employ a single parcellation. This practice presumes that the various parcellations can be considered interchangeable, such that parcellation selection does not have an impact on the results obtained.

Fig. 1.

Four major assumptions underlie the use of a priori brain parcellations to study individual differences in functional brain organization in developmental and clinical research.

In the present study, we examine these four assumptions. To do so, we selected a set of parcellation schemes commonly used (Glasser et al., 2016; Gordon et al., 2016; Power et al., 2011; Schaefer et al., 2018; Yeo et al., 2011). We were also interested in assessing the extent to which the meta-analytic platform NeuroSynth is able to produce sensible networks maps. NeuroSythn.org allows researchers to identify task-evoked activity mappings (Yarkoni et al., 2011; Lieberman and Eisenberger, 2015) using keywords. As researchers often refer to the primary functional networks when discussing task-evoked activity, primary functional networks have also begun to be identified using NeuroSynth’s meta-analytic approach (Franzmeier et al., 2017; Wang et al., 2020). We were interested in examining how well the NeuroSynth platform is able to identify functional networks, so included maps of four primary networks of interest (the default network, the control network, the dorsal attention network, and the salience network) derived using NeuroSynth.org.

Using the eight selected parcellation schemes, we examined the following questions: (1) Are the parcellations equally able to recover the networks of interest (i.e. does the parcellation-extracted data recapitulate the expected correlational network structure defined by the parcellations)? (2) Do these adult-derived parcellations well represent the networks in children’s brains? (3) Are network properties, such as within-network connectivity, reliably measured across parcellations? (4) Does the parcellation selected impact the results with regard to individual differences in network properties? To evaluate the fourth assumption, we examine three common questions about individual differences in functional brain organization, including whether functional connectivity within networks of interest vary as a function of age, environmental experience and cognitive ability. Here, we focus on poverty as a measure of environmental experience. We examine these four assumptions in two large samples of children and adolescents, focusing on four primary networks of interest that are widely studied and derived using parcellation schemes: the default, control, dorsal attention, and salience networks.

2. Methods

We implemented all analysis in two large samples of children and adolescents that acquired information on age, environmental experiences (e.g., poverty), and cognitive ability.

Sample 1.

A sample of 160 youth aged 8–17 were recruited to participate from the Seattle area between January 2015 and June 2017. Of these, 38 were excluded for the following reasons: fell asleep during scan (4), had incomplete resting-state scan (5), failed to successfully complete preprocessing due to insufficient data (22), or other technical issues with the scan (7). The final analytic sample consisted of 122 subjects age 8–17 (M = 12.8, SD = SS2.64; 59 male) with good quality resting-state functional magnetic resonance imaging (rsfMRI) data (6 min scans). The data were collected as part of a larger study investigating the effects of childhood maltreatment on brain structure and function. Sixty-four subjects in the final sample had experienced maltreatment (Weissman et al., 2020). We selected this sample due to the wide variation in age, environmental experience, and cognitive ability —factors that are the focus of individual difference analysis used to examine the fourth study question. The sample provides an excellent opportunity to examine whether parcellation selection influences the magnitude of individual differences observed as a function of these characteristics.

Sample 2.

We used data from the Pediatric Imaging, Neurocognition, and Genetics (PING) study. The PING repository is publicly available to investigators through the PING Portal (http://pingstudy.ucsd.edu), and contains multimodal neuroimaging data, developmental histories, and behavioral and cognitive assessments of 1493 children and adolescents between the ages of 3 and 21. A total of 643 participants had rsfMRI data. A total of 313 subjects were excluded due to inadequate scan time or quality. The final sample consisted of 330 participants ages 3 – 21(M = 14, SD = 5; 153 male) with good quality rsfMRI data (6 min scans). For a detailed description of the PING data collection and repository see Jernigan et al. (2016). We selected this sample due to the large size and wide age range —to allow us to examine whether parcellation scheme influences the magnitude of individual differences in network properties as a function of age.

Imaging acquisition.

Sample 1. Scanning was performed on a 3T Phillips Achieva scanner at the University of Washington Integrated Brain Imaging Center using a 32-channel head coil. T1-weighted MPRAGE volumes were acquired (repetition time = 2530 ms, TE = 3.5 ms, flip angle = 7°, FOV = 256 × 256, 176 slices, in-plane voxel size = 1 mm ³) for co-registration with fMRI data. Blood oxygenation level dependent (BOLD) signal during functional runs was acquired using a gradient-echo T2 ^∗ -weighted echo planar imaging (EPI) sequence. Thirty-seven 3 mm thick slices were acquired sequentially and parallel to the AC-PC line (TR = 2 s, TE = 25 ms, flip angle = 79°, inter-slice gap = 0.6 mm, FOV = 224 × 224 × 132.6, matrix size = 76 × 74). Prior to each scan, four images were acquired to allow longitudinal magnetization to reach equilibrium. Sample 2.

A standardized multiple-modality high-resolution structural MRI protocol was implemented at the nine Pediatric Imaging, Neurocognition, and Genetics (PING) sites and 12 scanners: Johns Hopkins University (Philips – Achieva); Massachusetts General Hospital (Siemens – TrioTim 1); Massachusetts General Hospital (Siemens – TrioTim 2); University of California at Davis (Siemens -TrioTim); University of California at Los Angeles (Siemens -TrioTim 1); University of California at Los Angeles (Siemens -TrioTim 2); University of California at San Diego (GE – Signa); University of California at San Diego (GE – Discovery); University of Hawaii (Siemens -TrioTim); University of Massachusetts (Philips – Achieva); Weill Cornell Medical College (Siemens -TrioTim); Yale University (Siemens -TrioTim). The protocol included a conventional three-plane localizer, a sagittal 3D inversion recovery spoiled gradient echo T1-weighted volume optimized for maximum gray/white matter contrast (echo time = 3.5 ms, repetition time = 8.1 ms, inversion time = 640 ms, flip angle = 8°, receiver bandwidth = ± 31.25 kHz, FOV = 24 cm, frequency = 256, phase = 192, slice thickness = 1.2 mm). The relaxation rates and scanner-specific gradient coil nonlinear warping were measured, and the latter corrected for, for all scanners, from all sites. Sagittal 3D cube T2-weighted volume, gradient echo-planar imaging scans and associated calibration scans for resting state functional MRI were also collected at all sites. Prospective motion correction for non-diffusion imaging acquisition was implemented for a majority of the data collected in San Diego and Honolulu. Scanning durations for each sequence of the protocol were T1 for 8:05, T2 for 4:25. For a detailed account of the Sample 2 imaging data please see (Fjell et al., 2012; Jernigan et al., 2016) and for site-specific scanner protocols please see https://nda.nih.gov/edit_collection.html?id=2607.

Resting-state fMRI preprocessing.

The rsfMRI data, from both Samples 1 and 2, were preprocessed following guidelines for optimal reduction of the influence of motion artifact from Ciric et al. (2017) with the pipeline implemented using Make, a software tool that allows for the integration of multiple software packages (Askren et al., 2016). The first four volumes of the resting state run were discarded. We then registered the timeseries to the middle volume using FSL MCFLIRT (Jenkinson et al., 2002). Linear, and non-linear transformations were estimated for registering each subject’s resting state timeseries to their T1 image, from the T1 to a sample-specific template, and from that template to Montreal Neurological Institute (MNI). Anatomical co-registration of the functional data with each participant’s T1-weighted image and normalization were performed using Advanced Normalization Tools (ANTs) software, version 2.1.0, because of superior registration within pediatric samples (Avants et al., 2011). We then performed slice-timing correction (using FSL slicetimer) (Jenkinson et al., 2002), replaced outlier voxel-values using AFNI 3dDespike (Cox, 1996), and Gaussian spatial smoothing using a 6 mm-FWHM smoothing kernel (FSL SUSAN) (Smith and Brady, 1995). Next, we used AFNI 3dDeconvole (Cox, 1996) to regress nuisance variables from the timeseries. These variables included a regressor for volumes with framewise displacement > 0.5 mm, for which the derivative of variance in BOLD signal across the brain (DVARS) exceeded the upper fence (above 75th percentile + 1.5 × inter-quartile range), or for which signal intensity was more than 3 SD from the mean. We included a total of 18 other nuisance regressors, including six motion parameters and their six derivatives, signal from CSF and white matter (extracted prior to smoothing) and their derivatives, and the mean signal across all voxels in the brain (i.e., global signal) and the derivative. We did not use the recommended additional 18 regressors (representing the squared term of each of the 18 regressors included in the model) (Ciric et al., 2017) because we did not have the effective degrees of freedom necessary, with a substantial proportion of participants in both samples failing preprocessing when these additional 18 regressors were included. We opted to retain more data using the best possible strategy the data would allow while retaining the largest number of subjects possible in each sample. This choice is unlikely to have a meaningful impact on our results, because our goal was simply to evaluate whether differences emerged in network properties as a function of which of the 8 parcellation schemes was used. The data were then bandpass filtered using AFNI’s 1dBport to retain frequencies in the range 0.01 Hz < f < 0.8 Hz. Subjects with at least 4.5 min of data following preprocessing were retained. The median percent of timepoints flagged in Sample 1 was 7.5% and the range was 0 −23 %. The median percent of timepoints flagged in Sample 2 was 8.6% and the range was 0 −29 %.

Parcellation selection.

We selected eight parcellations schemes commonly used in developmental and clinical research, including the Power 2011 scheme (Power et al., 2011), the Yeo 2011 7-network parcellation (Yeo et al., 2011), the Gordon 2016 parcellation (Gordon et al., 2016), the Glasser 2016 parcellation (Glasser et al., 2016), and the three of the Schaefer 2018 parcellation schemes (100, 200, and 400 parcel schemes) (Schaefer et al., 2018) (Fig. 2). We also included network schemes derived from the NeuroSynth platform (Yarkoni et al., 2011). To create the NeuroSynth mappings, we downloaded the forward inference(uniformity test) meta-analytic term maps from the four networks of interest: the default network, the control network, the dorsal attention network and the salience network. Maps were identified on www.neurosynth.org using the following key word searches: “default mode network ”, “control network ”, “dorsal attention network ”, and “salience network ”. For each of the four maps obtained, we identified peaks within the clusters of activity in each map and drew 4 mm spheres around the peaks, thus creating volume-based masks that could then be used to extract timeseries data. We acknowledge that these parcellations were generated using different analytical approaches, however the focus of this paper is not in examining the validity of any one of those approaches over another. Instead, we are interested in examining how these parcellations, once established, compare when applied as a priori network schemes.

Fig. 2.

The eight parcellation schemes selected for evaluation: Glasser 2016, Gordon 2016, NeuroSynth (keywords: “default network ”, “control network ”, “dorsal attention network ”, “salience network ”), Power 2011, Schaefer 2018 (100, 200, 400), Yeo et al. (2011). The legends detail the four networks of interest selected for examination: default, control, dorsal attention, salience.

Networks of Interest.

For the purposes of this evaluation, we focused on four networks of interest: the default network (DN), the fronto-parietal control network (CN), the dorsal attention network (DaN) and the salience network (SaN). We elected to focus on these four networks, as they have received the most attention in developmental and clinical work. Furthermore, these four networks are included in each of the eight parcellations and are composed of at least two or more parcels in each parcellation – a necessary property to evaluate within-network connectivity (i.e., to calculate the correlations between the regions in a network). The auditory, visual, and somatomotor networks did not fit this specification in the Yeo parcellation and have been studied less frequently in clinical and developmental work. As a result, we did not include the auditory, visual, or somatomotor networks in our analyses. As seen in Fig. 2, the default, control, and dorsal attention networks are labeled similarly across the eight parcellations. It is important to note that what we are calling the “salience network ” is the least consistently labeled network, with “cingulo-opercular ” and “ventral attention ” being other names used to describe a network that is generally comprised of regions including the anterior insula, dorsal anterior cingulate cortex, and other regions in dorsomedial prefrontal cortex. Please see Uddin et al. (2019) for discussion of how inconsistent network nomenclature may be impeding clear synthesis of findings, along with a proposed universal taxonomy. All analyses focus on these four networks of interest as a means of evaluating the eight parcellations.

Parcellation registration and timeseries extraction.

Following preprocessing, each of the eight parcellation schemes were registered to each subject’s resting-state functional data. For the two volume-based schemes (Power 2011 and NeuroSynth), registration of each parcellation was conducted using ANTs (using the antsApplyTransforms function) to take the parcellation from MNI space to a study specific template, then to native T1 space, and finally to the native resting-state functional space (Avants et al., 2011). After each volume-based scheme was registered to each subject’s functional native space, the mean BOLD timeseries of each parcel in each scheme was extracted using the FSL fslmeants function. For the six surface-based parcellations (Yeo et al., 2011; Gordon et al., 2016; Glasser et al., 2016, and Schaefer et al., 2018 [100, 200, 400]), subject’s native resting-state functional data was first projected onto the surface (using the FreeSurfer bbregister and mri_vol2surf commands). Each parcellation scheme was then registered to the subject’s surface space using the FreeSurfer mri_surf2surf function. Finally, the timeseries for each parcel was extracted using the FreeSurfer mri_segstats function. All following analyses were conducted using the extracted timeseries data.

2.1. Data analysis

Code for all analyses is available at https://osf.io/bjde4/. Data are not publicly available, but are available from the authors upon request.

Method to address question 1:

Are the parcellations equally able to recover the networks of interest?

When a priori parcellations are used, it is assumed that the networks of interest are well represented in the timeseries data extracted and that the various parcellations are equally able to represent the networks of interest. To evaluate this assumption, we employed a technique that allows one to assess how well data organically clusters into pre-defined groupings that align with the functional networks of interest. The technique relies on assessing the difference in goodness of fit between two versions of a multi-dimensional scaling (MDS) procedure: unconstrained and constrained MDS as implemented in the R SMACOF package (version 2.1.0; De Leeuw and Mair, 2009;) in R version 4.0.0 (R Core Team, 2020). We briefly describe MDS and explain how unconstrainted and constrained MDS can be used in conjunction to assess the extent to which the networks of interest are represented in the timeseries data extracted from the various parcellation schemes.

Multi-dimensional scaling.

MDS is an exploratory unsupervised learning technique that allows the correlations between a set of items in a multivariate dataset to be visualized using a spatial configuration (Borg and Groenen, 2005; Borg et al., 2018). The primary result of an MDS analysis is a graphical plot, or configuration, that spatially illustrates the similarity of items, such that highly correlated items are plotted near one another in space, and items that are negatively correlated are spatially distant. If two items appear in the exact same location in space in the MDS configuration, the items are perfectly correlated. See Fig. 3A for a visualization of an MDS configuration, in which the similarity between parcel timeseries extracted using the Yeo et al. (2011) for a single subject is visualized. To conduct an MDS analysis using a correlation matrix, the matrix is first converted, using the equation below, into a dissimilarity matrix, which is the required input for MDS function in the smacof package.

$$d(x,y)=\sqrt{1-r(x,y)}$$

Fig. 3.

A: Unconstrained MDS solution for the timeseries extracted from the Yeo et al. (2011) parcellation for the four networks of interest for a single subject. In this configuration, the timeseries data clearly separate out into the networks of interest. B: The constrained MDS solution for the timeseries extracted from the Yeo et al. (2011) parcellation for the four networks of interest for a single subject. Though the stress value for the constrained MDS solution increases, the stress score is not highly impacted when the network constraints are added in the constrained MDS, as the network groupings were already present in the unconstrained solution. C: The unconstrained MDS solution for the timeseries extracted from the NeuroSynth scheme for the four networks of interest for a single subject. In this configuration the timeseries data do not clearly separate out into the networks of interest. D: The constrained MDS solution for the timeseries extracted from the NeuroSynth scheme for the four networks of interest for a single subject. Even within the constrained MDS solution, the NeuroSynth data fail to clearly separate, which is reflected in the large stress score. The large difference in stress scores between the unconstrained and constrained MDS solutions for the NeuroSynth scheme suggests that the networks of interest are poorly represented in the extracted timeseries data for this subject.

Here, r (x,y) denotes the Pearson correlation coefficient between vectors (here parcel timeseries) x and y. By converting the correlation matrix (where large values indicate similarity) into a dissimilarity matrix (where small values indicate similarity), the relationship between two variables(x,y) is thus represented by a plottable proximity or distance: d (x,y). The dissimilarity matrix is then used to solve for a spatial configuration in low-dimensional space (typically 2 dimensions for ease of visualization) where the distance between the points in the configuration solution are as close as possible to the given input dissimilarities (proximities) (Mair, 2018). The minimization (goodness of fit) function used to identify the ideal configuration solution is known as stress. Its value (the stress value) quantifies the extent to which the distances between items in the configuration plot are equal to the distances in the original input matrix. In an ideal solution (a stress value of 0), there would be no difference between the distances between points in the configuration plot and the input distance matrix. There are various types of MDS techniques, the most common being unconstrained MDS (Mair, 2018), where no structural constraints are imposed on the configuration solution. Unconstrained MDS produces the lowest possible stress value (i.e. produces the configuration in which the distances between items in the plot most closely resemble the proximities between items in the given input dissimilarity matrix). On the other hand, constrained MDS allows the user to restrict the configuration solution using a priori groupings; items that belong to the same a priori grouping are forced to be spatially close together (Borg and Lingoes, 1980; De Leeuw and Heiser, 1980; Heiser and Meulman, 1983; Mair, 2018). This is done by sectioning the configuration space into the number of a priori groupings and constraining all items from each group to fall somewhere within their designated area. As constrained MDS does not allow for a free solution, the stress value will necessarily be larger.

As noted above, when a priori parcellations are used, it is assumed that the networks of interest are represented in the timeseries data extracted. We based our analysis in the present study on the idea that if the network groupings are indeed represented in the extracted timeseries data, the data from each network should cluster together in an unconstrained MDS solution. Fig. 3A and C below illustrate the unconstrained MDS configuration plots for a single subject’s timeseries data extracted from the Yeo et al. (2011) parcellation and the NeuroSynth maps for the four networks of interest. By examining the unconstrained plots one can see that timeseries data from the Yeo et al. (2011) parcellation organically separate into the four networks of interest. However, the timeseries data from the NeuroSynth mappings fail to separate into identifiable network groupings. We can quantify how well the timeseries data cluster into the networks of interest by also fitting a regionally constrained MDS solution. In the constrained MDS, the timeseries from regions within a given network (aka. grouping), as defined by each parcellation, are forced to fall within the same constrained portion of the MDS space (eg. SaN regions timeseries in the top left quadrant, DN regions timeseries in the top right quadrant, DaN regions timeseries in the bottom left, and CN regions timeseries in the bottom right). This is conceptually equivalent to a cluster of nodes within a network model.

The change in stress values (i.e., goodness of fit) between the unconstrained and constrained solutions can then be assessed. If the network groupings are indeed represented in the data, the forced clustering of the data in the constrained MDS solution, will only have a small effect on the stress value, as the data will already naturally fall into distinct spatial groupings. Therefore, the difference in the stress value between the unconstrained and constrained MDS should be small. If, however, the timeseries data do not naturally cluster into the network groupings, the stress values for the constrained MDS will be significantly higher and will thus result in a large change in stress (or goodness of fit) between the unconstrained and constrained MDS solutions. We can therefore use the difference in the stress values between unconstrained and constrained MDS solutions to assess how well the networks of interest are represented in timeseries data extracted from a given parcellation. Fig. 3B and D illustrate the constrained MDS solutions for the Yeo et al. (2011) parcellation and NeuroSynth schemes for the same single subject as the unconstrained MDS solutions. As seen in the figure, the Yeo et al. (2011) constrained MDS solution leads to a stress value of 0.212, and thus a stress difference of 0.054, versus the much higher stress difference of 0.235 between NeuroSynth’s unconstrained and constrained solutions. This suggests that for this subject, the Yeo et al. (2011) parcellation is notably better than the NeuroSynth scheme at representing the networks of interest. The metric of stress difference can therefore be used to determine how well the networks of interest are represented in the timeseries data extracted from a given parcellation.

To evaluate whether the networks of interest are equally represented in the timeseries data extracted, we calculated the stress difference between a constrained and an unconstrained MDS solution for each of the eight parcellations, for each subject. Note that to solve for an MDS solution, an initial starting configuration is required that is then optimized. We elected to use a Torgerson configuration (classical MDS) as the initial starting configuration for the unconstrained MDS (instead of a random start), as this initial start generally leads to among the lowest stress value and further produces non-stochastic MDS solutions (Borg and Mair, 2017; Mair et al., 2016). We then fit a mixed-effects model to examine whether stress differences (dependent variable) varied significantly across parcellations (independent variable), with observations nested within subject (ie. subject ID was associated with a random effect) (see R model syntax below).

$$\text{Stress}.\text{Difference}\sim \text{Parcellation}+\left(1\right|\text{SUB}\_\text{ID})$$

All mixed-effects modeling was performed using in the lme4 package version 1.1.23 (Bates et al., 2015) in R version 4.0.0 (R Core Team, 2020).

Although MDS has been used to examine resting-state networks (Gratton et al., 2018), it has not been used frequently as a method for quantifying cortical networks. As such, we conducted a second analysis using more commonly used graph theory approaches to confirm these results, focusing specifically on network modularity. By examining the difference between the modularity of a data-driven clustering approach (walk trap) and modularity when the a priori network groupings are applied, we were able to evaluate the extent to which the a priori networks are observed in the extracted data using an alternative approach. See supplemental materials for additional details on these analyses.

Method to address question 2:

Do these adult-derived parcellations well represent the networks in children’s brains?

All parcellation schemes to date have been developed in large adult samples. Applying these parcellation schemes in developmental samples therefore assumes that the networks of interest are represented in the same way and to the same extent in children as in adults. To evaluate this assumption, we expanded the hierarchical linear model used to examine stress difference by parcellation (described above) by adding an age term, as well as interaction term for age by parcellation. To test if the interaction term was significant, we compared this expanded model to one in which age was included, but not allowed to interact. This comparison was done using a likelihood ratio test (see R model syntax below).

$$\text{Model}1:\text{Stress}.\text{Diff}\sim \text{Age}+\text{Parcellation}+(1|\text{SUB}\_\text{ID})$$

$$\text{Model}2:\text{Stress}.\text{Diff}\sim \text{Age}*\text{Parcellation}+(1|\text{SUB}\_\text{ID})$$

If the adult-derived parcellations can capture the networks in children and adolescence equally well as in adults, we would not expect to see a change in stress difference across the age range examined for any of the parcellations.

Methods to address question 3:

Are network properties, such as within network connectivity, reliably measured across parcellations?

One of the primary measures of network functional properties examined in developmental and clinical research is within-network functional connectivity (Alarcón et al., 2018; Jalbrzikowski et al., 2019; Lopez et al., 2019; Satterthwaite et al., 2013; Sylvester et al., 2018; Fan et al., 2019; Lydon-Staley et al., 2019; Reggente et al., 2018; Yu et al., 2019). We therefore examined whether the parcellation schemes produced reliable measures of functional connectivity within each of the four networks of interest (SN, CN, DN, DaN). To calculate within-network functional connectivity, Fisher’s z -transformed Pearson’s correlation coefficients were computed between all homotopic pairings (intra-hemispheric regions) of parcels within a given network and then averaged. Only homotopic correlations were used in the calculation of average functional connectivity to avoid inflated estimates driven by high correlations between inter-hemispheric medial structures, such as the posterior cingulate cortex (Ciric et al., 2017). By averaging all homotopic pair-wise correlations within each network, we were able to obtain a single measure of average functional connectivity for the four networks of interest in each parcellation, for each subject.

We used mixed-effects modeling to assess whether the mean level of within network functional connectivity was significantly different across the eight parcellations. Four models were estimated, one for each of the four networks of interest, with average network functional connectivity as the dependent variable, parcellation as the independent variable, and observations nested within subjects (see R model syntax below).

$$\text{Network}.\text{Connectivity}\sim \text{Parcellation}+(1|\text{Sub}\_\text{ID})$$

Models were also conducted excluding parcellations that failed to recover the networks of interest (ie. failed assumption 1 above), to confirm that such parcellations are not driving results. We additionally examined whether parcellation characteristics such as number of nodes within network, the extent of surface covered by parcellation, and data quality (i.e., motion) influenced connectivity scores. Please see the supplemental materials for additional details.

We then examined the extent to which subjects maintained their rank order of within-network connectivity across the parcellations (ie. examined the inter-parcellation consistency of network connectivity scores). To evaluate this consistency across parcellations, between-parcellation intra-class correlation coefficients (ICC) estimates of within-network connectivity were calculated for each of the networks of interest, using the psych package version 2.0.9 (Revelle, 2020), based on a single rater/measurement, consistency, two-way mixed effects model (Koo and Li, 2016). Specifically, we extracted the mean squared error (MS _E) and the mean squared row (MS _R) from the Psych ICC function output and used those values to calculate the ICCs using the model below (Koo and Li, 2016).

$$\frac{M{S}_R-M{S}_E}{M{S}_R+(k-1)M{S}_E}$$

Given the poor performance of Neurosyth in the analyses described above, as well as the uncorrelated nature of the Neurosynth functional connectivity estimates with the other parcellations, the Neurosynth parcellation was also excluded from this analysis. ICC estimates were bootstrapped using the boot package version 1.3.25 (Canty and Ripley, 2020) such that 95% confidence intervals (the percentile method) could be calculated. Note that during bootstrapping, the same 5000 samples were used to calculate each network’s consistency estimates. Furthermore, to examine whether a given network was more reliable across the parcellations than the others, the differences between network consistency (ICC) estimates were calculated between each of the possible network pairings (SaN –CN, SaN-DN, SaN-DaN, CN-DN, CN-DaN, DN-DaN); eg. SaN ICC (0.70) – DN ICC (0.50) = difference of 0.2. This allowed for the examination of whether, for example, the salience network produced more reliable estimates of functional connectivity than the control network or the default network. 95% confidence intervals for between-network ICC differences were calculated by subtracting each network’s bootstrapped ICCs (5000 samples) from all possible network pairings (SaN –CN, SaN-DN, SaN-DaN, CN-DN, CN-DaN, DN-DaN.

Method to address question 4:

Does the parcellation selected impact the results with regard to individual differences in network properties such as within network connectivity?

A primary focus in developmental and clinical work that employs parcellation schemes is examining individual difference in network connectivity as a function of variables of interest, such as age (Baum et al., 2020; Jalbrzikowski et al., 2019; Lopez et al., 2019), the presence of mental health problems (Franzmeier et al., 2019; Kebets et al., 2019; Fatt et al., 2020; Yerys et al., 2019), environmental experiences —such as poverty (Tooley et al., 2020; Chan et al., 2018), or in relation to specific cognitive capacities, such as intelligence or executive functioning (Reineberg and Banich, 2016; Finc et al., 2020; Murphy et al., 2020; Alarcón et al., 2018). Studies generally rely on a single parcellation scheme to estimate functional connectivity, with a few notable exceptions (Finc et al., 2020; Tooley et al., 2020; Xia et al., 2018; Shafiei et al., 2019). Given that few studies examine whether results replicate when using other parcellations, the assumption that the parcellations are interchangeable and will produce equivalent results has not been empirically evaluated.

To do so, we examined the extent to which parcellation selection impacts the results of three distinct hypotheses that parallel questions commonly explored in research on individual differences in functional connectivity. First, we examined the whether the association between functional connectivity and age varies as a function of parcellation (in Sample 2, which had a wide age range of 3–21 years). We examined age effects for all four of the networks of interest. We conducted a model comparison between three mixed-effects models (with observations nested within subjects) examining the association between within-network connectivity (dependent variable) and age (independent variable). In the first model no term for parcellation was included. In the second model a term for parcellation was added, and finally in the third model an interaction term between age and parcellation was also added. We then conducted a likelihood ratio test to examine if a model with an interaction term for parcellation was a better fit. If parcellation selection does not impact the results of a given hypothesis, a model without an interaction term for parcellation should fit as well as a model that includes an interaction term (see R model syntax below).

$$\text{Model}1:\text{Connectivity}\sim \text{Age}+(1|\text{Sub}\_\text{ID})$$

$$\text{Model}2:\text{Connectivity}\sim \text{Age}+\text{Parcellation}+\left(1|\text{Sub}\_\text{ID}\right)$$

$$\text{Model}3:\text{Connectivity}\sim \text{Age}+\text{Parcellation}+\text{Age}:\text{Parcellation}+\left(1|\text{Sub}\_\text{ID}\right)$$

Second, we examined whether the association between network connectivity and living in a family below the poverty line varies as a function of parcellation (in Sample 1 only, as Sample 2 did not include measures of family size that are necessary to estimate poverty status). Poverty was defined as those subjects whose family income was below the poverty line for a family of that size, based on Census thresholds in the year the study was conducted. We estimated a similar mixed-effects model comparison as described and illustrated above for age above, but with poverty modeled as the independent variable and default connectivity as the dependent. Finally, we also examined whether the association between control network connectivity and executive function (inhibition) varies as a function of parcellation (in Sample 1). Inhibition was measured using a standardized task —the arrows inhibition task from the NEPSY-II (Brooks et al., 2009), which assesses inhibition of an automatic response. Participants viewed multiple rows of black and white arrows pointing either up or down. In the baseline trial, participants were asked to say the direction that each arrow was pointing. In the inhibition trial, participants were asked to say the opposite direction that each arrow was pointing. The time taken to complete the baseline trial was subtracted from the time required to complete the inhibition trial. Larger latencies indicate worse inhibitory control. We conducted a similar mixed-effects model comparison as described above for age and poverty, but with inhibition modeled as the independent variable and control connectivity as the dependent variable.

3. Results

Results Question 1:

Are the parcellations equally able to recover the networks of interest?

We used a multi-dimensional scaling technique to assess the extent to which the networks of interest are recovered in data extracted from the eight parcellations evaluated. Fig. 4A illustrates the stress values for the constrained and unconstrained MDS solutions in Sample 1, which makes apparent the relative change in stress between the two models for each parcellation. As seen in Fig. 4B, the mean stress difference (constrained MDS solution stress – unconstrained MDS solution stress) is significantly different across the eight parcellations evaluated (F(7, 847) = 230.14, p < 0.001) in Sample 1. The NeuroSynth mapping exhibits the highest overall stress difference, indicating that this method of timeseries extraction is least able to reconstruct the networks of interest. The other seven parcellations produce values of stress difference that are much more similar than NeuroSynth, however the stress difference remained significantly different across the remaining 7 parcellations when NeuroSynth was excluded from the analysis (F (6, 726) = 4.26, p < 0.001). Given that the unconstrained stress value will always be smaller than the constrained MDS stress value, we did not test whether stress differences for each parcellation alone are different from zero (i.e. a onesample t -test). Similar results were observed within Sample 2 (Fig. 4C and D). There was a significant difference in stress difference across the eight parcellations (F (7, 2303) = 544.41, p < 0.001), and NeuroSynth exhibited a notably larger stress difference relative to the other seven parcellations. Again, results remained significant when NeuroSynth was excluded from the analysis (F (6, 1974) = 21.6, p < 0.0001).

Fig. 4.

A: The stress values of the unconstrained (light gray) and constrained (dark gray) MDS solutions for each parcellation (Sample 1). The stress difference (between unconstrained and constrained MDS) is illustrated by the dashed turquoise line. B: The stress difference (constrained stress minus unconstrained stress) for each parcellation examined (Sample 1). C: The stress values of the unconstrained (light gray) and constrained (dark gray) MDS solutions for each parcellation (Sample 2). The stress difference (between unconstrained and constrained MDS) is illustrated by the dashed turquoise line. D: The stress difference (constrained stress minus unconstrained stress) for each parcellation examined (Sample 2).

We found consistent results when using the additional modularity difference approach (see Fig. S2) as when using the unconstrained versus constrained MDS stress difference approach illustrated above, where the parcellations exhibit significantly different modularity differences, F (7, 843) = 652, p < 0.001. Specifically, the NeuroSynth mapping exhibits the highest overall modularity difference, confirming that this method of timeseries extraction is least able to reconstruct the networks of interest. Like in MDS approach, the other seven parcellations produce values of modularity difference that are much more similar.

Results Question 2:

Do these adult-derived parcellations well represent the networks in children’s brains?

To assess whether the functional networks identified in parcellation schemes developed in adults are well-represented in data from children, we examined whether the stress difference ––which estimates how well the extracted data recapitulate the networks –– varies as a function of age. If stress difference decreases as a function of age, this would suggest that the networks of interest are less identifiable within data from children than adults. Results in Sample 1 suggest that the association of the stress difference with age varies by parcellation (χ²[7] = 22.2, p = 0.0023, ΔAIC = −8.3) (Fig. 5A). This overall result was driven by the NeuroSynth parcellation, which showed an significant increase in stress difference with age relative to the grand mean (b = 0.0034, t (840) = 3.662, p = 0.00026). Given that NeuroSynth was found to perform poorly in terms of network recapitulation (Fig. 4D) and also exhibited a significantly different age effect, we ran the analysis again after excluding NeuroSynth. When we repeated the model comparison described above, the age by parcellation interaction was no longer significant (χ²[6] = 11.6, p = 0.072, ΔAIC = 0.4). Furthermore, a model with a main effect for age did not fit significantly better than a model without age (χ²[1] = 1.9, p = 0.17, ΔAIC = 0.1). Similar results were also observed within Sample 2, where a model with a main effect for age also did not fit significantly better than a model without age included (χ²[1] = 9.6, p = 0.21, ΔAIC = 5) (Fig. 5B). This suggests that, excluding NeuroSynth, the functional networks in the parcellations derived from adult data were similarly represented in pediatric data.

Fig. 5.

A: Stress difference by age by parcellation for the Sample 1 B: Stress difference by age by parcellation for the Sample 2. With the exception of NeuroSynth, stress difference did not significantly vary by age in either sample.

Results Question 3:

Are network properties, such as within network connectivity, reliably measured across parcellations?

Measures of within-network functional connectivity varied significantly by parcellation for each of the four networks assessed: salience (F (7, 850) = 76, p < 0.0001), control (F(7, 850) = 93, p < 0.0001), dorsal attention (F (7, 850) = 92, p < 0.001), and default (F (7, 850) = 330, p < 0.0001). As seen in Fig. 6A, the salience network exhibited the least variability in within-network connectivity across the eight parcellations, followed by the control network (Fig. 6B). The default network (Fig. 6C) and the dorsal attention network (Fig. 6D) exhibited highly variable within-network connectivity, with NeuroSynth producing the lowest scores and Yeo et al. (2011) producing the highest. All of these results were consistent in Sample 2, with salience (F (7, 2300) = 360, p < 0.0001), control (F (7, 2300) = 390, p < 0.0001), default (F (7, 2300) = 720, p < 0.0001), and dorsal attention (F (7, 2300) = 340, p < 0.001) networks all showing significant differences in average functional connectivity across the eight parcellations assessed. A similar pattern of variability in connectivity measures across the parcellations was also observed, with the default and dorsal attention networks exhibiting the most variability, followed by the control and then the salience network. The pattern of results in both Samples 1 and 2 is the same when NeuroSynth is excluded from the analysis (p < 0.0001 for all comparisons). We conducted additional analyses to examine whether parcellation characteristics, such as the number of nodes (i.e. parcels) in a network, extent of surface coverage, or data quality (i.e., motion) influenced estimates of within-network connectivity. A full discussion of these results is in the Supplement. Briefly, functional connectivity was negatively associated with the number of nodes in a network, b = −0.00149, t(28.8) = −2.12, p = 0.0424, (see Fig. S3A), and varied significantly as a function of parcellation surface coverage, F (3300) = 110, p < 0.0001, (see Fig. S3B). In contrast, framewise displacement —a metric of data quality —was not associated with functional connectivity, b = −0.0817, t (120) = −0.993, p = 0.323 (see Fig. S3C).

Fig. 6.

Measures of average functional connectivity by parcellation for each of the four networks of interest. Connectivity varied significantly across the parcellations in all the networks assessed, including (A) salience (B) control (C) default and (D) dorsal attention in both Sample 1 (dark color) and Sample 2 (light color). gray dashed line represents the grand mean connectivity (Sample 1) for each of the networks.

We also examined whether the various parcellations produce reliable measures of within-network connectivity, which would imply that subjects maintain consistent rank order in functional connectivity within a particular network across the parcellations examined. To examine the internal consistency of the parcellation-derived connectivity scores, we calculated reliability (ICC) measures for each of the networks of interest. Note that, given the poor performance on NeuroSynth in the analyses described above, NeuroSynth was excluded from this evaluation. As seen in Fig. 7A, the salience network produced the most reliable scores with an ICC of 0.70, suggesting moderate consistency across parcellations (Koo and Li, 2016). As illustrated in Fig. 7C, functional connectivity estimates within the salience network were significantly more stable across parcellations than the other three networks (SaN –CN: 95% CI [0.00990 0.159]; SaN-DN: 95% CI [0.117 0.298]; SaN-DaN: 95% CI [0.139 0.335]). The control network’s consistency across parcellations was significantly higher than both the default (CN-DN: 95% CI [0.0358 0.212]) and dorsal attention (CN-DaN: 95% CI [0.0432 0.261]) networks. However, with an ICC of 0.62, the control network still exhibited relatively poor consistency. The default (ICC = 0.5) and dorsal attention (ICC = 0.47) networks produced even lower consistency estimates of within-network function connectivity, that did not differ significantly from one another (DN-DaN: 95% CI [ −0.0823 0.140]). Fig. 7A also displays correlation matrices of within-network connectivity values across the parcellation schemes.

Fig. 7.

Consistency of functional connectivity across the parcellations for each of the networks of interest (salience, control, default, dorsal attention) in Sample 1. NeuroSynth was excluded from this assessment, given its poor performance in the preceding analyses. A: Correlation matrices of functional connectivity by parcellation for each of the networks of interest. B: Consistency estimate for each network of interest (across the parcellations examined) and bootstrapped confidence intervals. C: Differences in the consistency of connectivity between networks, illustrating that some networks exhibit more consistent connectivity scores across the parcellations (salience and control networks).

Similar results were found in Sample 2, where the salience network was found to be the most reliable (ICC = 0.69), followed by the control (ICC = 0.62), default (ICC = 0.5), and dorsal attention (ICC = 0.32) networks (Fig. 8). The only difference obtained in Sample 2 was that the default network was significantly more reliable than the dorsal attention network (DN-DaN: 95% CI [0.103 0.258]).

Fig. 8.

Consistency of functional connectivity across the parcellations examined for each of the networks of interest (salience, control, default, dorsal attention) in Sample 2. A: Correlation matrices of functional connectivity by parcellation for each of the networks of interest. B: Consistency estimate for each network of interest (across the parcellations examined) and bootstrapped confidence intervals. C: Between-network consistency difference scores.

Results Question 4:

Does the parcellation selected impact the results with regard to individual differences in network properties such as within-network connectivity?

Finally, we also assessed the extent to which parcellation selection impacts the interpretation of putative results regarding individual differences in functional connectivity as a function of age, poverty, and cognitive function. Specifically, we examined the whether the association of network connectivity with age (Sample 2), poverty (Sample 1), and executive function (Sample 1) varied significantly as a function of parcellation. We examined these associations using each of eight parcellation schemes, and found that for each of the relationships assessed, the effects varied significantly as a function of parcellation.

The eight parcellation schemes examined produced varying results in the association between network connectivity and age (Fig. 9A). We found that a model that includes an interaction term for age by parcellation fit better for three of the four networks of interest (SaN, CN and DN), suggesting that the association between network connectivity and age varied significantly as a function of parcellation for these three networks. For the salience, control network, and default networks, a model that included an interaction term for age by parcellation fit significantly better than a model without (SaN: χ²[7] = 84.7, p < 0.0001, ΔAIC = 70.7; CN: χ²[7] = 27.2, p < 0.001, ΔAIC = 13.2; DN: χ²[7] = 24.9, p < 0.001, ΔAIC = 10.9). In contrast, model fit did not improve for the dorsal attention network when adding an interaction term for age by parcellation (DaN: χ²[7] = 7.7, p = 0.360, ΔAIC = −6.3). These finding suggest that the association between age and network connectivity varies meaningfully across parcellations for three of the four networks. We conducted a follow-up simple slopes analyses for the association between age and network connectivity (for SaN, CN and DN) for each of the parcellations examined to assess whether there is an identifiable pattern in the effects (Aiken et al., 1991). As seen in Fig. 9A (and illustrated in greater detail for the default network in Fig. 9B and C), there was no consistent pattern in terms of which parcellations produce significant associations. This follow-up simple slopes analysis additionally illustrates what would happen if eight independent researchers each selected one of the eight parcellations and conducted a single hypothesis test (i.e., of the association between age and functional connectivity) using the currently accepted approach in the field, which is to use a single parcellation scheme without testing the robustness of effects to parcellation selection. As demonstrated in Fig. 9, when used in this way, use of different parcellations would lead to different conclusions about the association of age with functional connectivity.

Fig. 9.

Examining the association between age and functional connectivity as a function of parcellation (Sample 2). A: Results of a model comparison evaluating whether the addition of an interaction for age by parcellation results in a better fit than a model with no interaction term for each of the networks of interest. The association between age and network connectivity varies significantly as a function of parcellation for the salience, control, and default networks, but not for the dorsal attention network. For networks with a significant interaction of age and parcellation, we provide beta estimates and p-values for the simple slopes of the association between age and functional connectivity for each of parcellation. Greater details on these associations are provided for the default network in panels B,C. B: Figure depicts substantial variability in the point estimates and confidence intervals for the association of age with default connectivity for each of the parcellations. Significant results would have been obtained if an independent researcher had selected either the Glasser, NeuroSynth or Yeo, but not one of the other five parcellations. C: Plots of the association between age and default connectivity for each of the eight parcellations.

The association between functional connectivity and poverty also varied significantly as a function of parcellation for the default network (Fig. 10), but not the other three networks (no significant interaction or main effects). A model examining the association between default connectivity and poverty with an interaction term for parcellation by poverty fit significantly better than one that did not include this interaction (χ²[7] = 30.6, p < 0.001, ΔAIC = 16.6) As detailed in Fig. 10B, a follow-up simple slopes analysis revealed that a significant negative association was observed between poverty and default connectivity in three parcellation schemes, and no association in five parcellation schemes. While this could be an issue of differing degrees of consistency and therefore statistical power across the various parcellations, we would expect to see variability in the confidence bands in such a case, rather than in the point estimates. However, as seen in Fig. 10A, there is notable variability in both the point estimates, as well as in the confidence in those estimates.

Fig. 10.

Examining the association of poverty with default network connectivity as a function of parcellation in Sample 1. A: The point estimates for the association of poverty with default connectivity for each of the parcellations examined. There is notable variability in the point estimates, as well as the confidence in those estimates. B: The parcellation-specific associations between poverty and default connectivity. Significant results would have been obtained if an independent researcher had selected either Gordon, Yeo or Schaefer 400, but not in the other five parcellations.

Finally, the eight parcellation schemes examined also produced varying results in the association between control network connectivity and performance on an inhibitory control task (Fig. 11A). A model that included an interaction term for parcellation by inhibition performance fit significantly better than one that did not (χ²[7] = 32.1, p < 0.001, ΔAIC = 18.1). A simple slopes analysis revealed that significant results would be obtained when employing any of the three Schaefer parcellations, and null results would be obtained for the other five parcellations. As in the other two analyses detailed above, there was notable variability in the beta estimates, as well as in the confidence in those estimates.

Fig. 11.

Examining the association of executive function (performance on an inhibitory control task) and control network connectivity as a function of parcellation in Sample 1. A: The beta estimates for the relationship association of inhibition with control connectivity for each of the parcellations examined. There is notable variability in the beta estimates, as well as the confidence in those estimates. B: The parcellation-specific relationships between inhibition and control connectivity. Significant results would have been obtained if an independent researcher had selected any of the Schaefer parcellations, but not the other five parcellations.

4. Discussion

In the present study we examined four major assumptions made when applying brain parcellation schemes to identify resting-state functional networks. We found that the primary networks identified in the parcellations are equally well represented in the data extracted using the various schemes (with the exception of NeuroSynth). Furthermore, we found that networks of interest were well represented in children’s data, despite the use of adult-derived parcellations for network identification. However, the various parcellation schemes did not produce reliable measures of within-network functional connectivity, and parcellation selection had meaningful effects on the results of hypothesis tests examining individual differences in functional connectivity as a function of age, poverty, and cognitive function.

Network recapitulation.

With the exception of the NeuroSynth-derived maps, the other seven parcellations performed similarly at recapitulating the networks of interest in the parcellation-extracted resting-state functional connectivity data. This suggests that various parcellations are able to capture network-level functional brain organization, when applied as a priori schemes. NeuroSynth is clearly divergent from the other seven schemes in terms of methodological approach, as it relies on meta-analytic synthesis of task-based activity identified through keywords used cognitive neuroscience work (Yarkoni et al., 2011). Though the use of NeuroSythn maps to identify primary functional networks is less common, we were interested in examining the extent to which the meta-analytic patterns of task-based activity discussed as the primary functional networks (ie. the “default network ”) actually hold up against parcellations derived using resting-state data. As indicated by our findings, the resting-state network maps derived using NeuroSynth fail to recapitulate the networks of interest. This suggests that though researchers label or discuss specific patterns of task-based activity as falling into the now canonical functional networks, there is a clear separation between the networks identified at rest from the patterns of activity seen in task-based analysis. While tasks may recruit areas that fall within canonical functional networks, task demands may also lead to activation of other regions that may not necessarily belong to the same functional networks. NeuroSynth failing to recapitulate the networks of interest calls into question the overlap between patterns of activity observed in task and networks derived using connectivity-based parcellation approaches. These findings further call for careful attention to be paid to labeling task-based activity patterns as the canonical functional networks. There has been work to identify overlap between task-based activity and functional connectivity in low-level tasks such as finger tapping (Gordon et al., 2016, 2017; Laumann et al., 2015), and recent work has extended these findings to higher-order cortical networks that appear to be preferentially recruited for various complex functions (Braga et al., 2020; DiNicola et al., 2020). However, it is important to note that the tasks used to evaluate network recruitment were specifically designed to dissociate network functioning. Tasks with more complex demands that were not designed to intentionally dissociate network functioning are likely to lead to less clear preferential network recruitment. This suggests that very careful attention should be paid when making conclusions about network recruitment during tasks that are examining broad cognitive abilities, as opposed to tightly designed tasks aimed at eliciting network specific recruitment. There is much work to be done in exploring the recruitment of cortical association networks during complex cognitive tasks.

The use of adult-derived parcellations in developmental samples.

The application of adult-derived brain parcellations in developmental samples presumes that these parcellation schemes are representative of the primary functional networks in children. By examining the extent to which network recapitulation varied as a function of age and parcellation, we were able to assess the validity of employing these parcellations in developmental samples. With the exception of NeuroSynth, the parcellations show no significant change in stress difference with age in either sample, including one with a wide age range (3–21 years). These findings suggest that the networks of interest in parcellations derived from adult data are represented equally well in resting-state data from children as adults. These findings have clear methodological and conceptual implications with regard to understanding the development of functional networks. Had the extent of network recapitulation varied significantly with age, it would have suggested that the boundaries of the primary networks defined in adult-derived parcellations are not exemplary of the networks in children. Such a finding would suggest that the functional networks undergo drastic changes in spatial layout and extent across development. However, we found this not to be the case, suggesting that functional networks observed in adults are generally established early in development and closely resemble the networks observed in adults.

While our finding revealed that adult-derived parcellations generally reflect the networks in children’s brains, our work is constrained by the use of rigid group-derived parcellations to define the networks. Recent developmental work has found that the functional networks do exhibit subtle yet meaningful changes in topography through development (Cui et al., 2020). The goal of our work was to examine whether the general network structure in children could be captured by adult derived parcellation schemes, which we found to be largely the case. However, we are likely to have missed subtle changes that will likely prove to be meaningful in understanding how the functional networks change across development. Additionally, the application of a parcellation assumes that a network’s topography is exactly the same across all subjects. Recent precision neuroscience work in densely-sampled adults has begun examining the primary functional networks by deriving individual-specific parcellations (Braga et al., 2019; Braga and Buckner, 2017; Gordon et al., 2017; Gratton et al., 2018; Kraus et al., 2021; Laumann et al., 2015) and has revealed that topographies of association networks are highly idiosyncratic across individuals (Hill et al., 2010a; Laumann et al., 2015; Kong et al., 2019). By relying on group-averaged parcellations, we miss the opportunity to examine how this idiosyncratic network organization and topography emerges and how environmental experience and other factors may shape this development. In short, though we did not find significant change in network structure across development, suggesting that the adult derived parcellations can capture general network structure in children, much could be gained by moving away from the rigid use of a priori parcellations to define these networks.

Consistency of network-specific measures derived from parcellation-extracted data.

Although the various parcellations performed similarly at recapitulating the networks of interest, measures of within-network functional connectivity varied meaningfully across parcellations, particularly for the default and dorsal attention networks. Several parcellation characteristics influenced functional connectivity estimates, including number of nodes and degree of coverage of the cortex. Furthermore, estimates of within-network connectivity had moderate to poor consistency across the parcellations. This means that participants with relatively high functional connectivity within a particular network relative to others in a sample in one parcellation scheme would not necessarily have high connectivity in that same network in another parcellation scheme. This finding is concerning, as it suggests that parcellation choice meaningfully influences estimates of functional connectivity.

The salience network was the least consistently labeled network across the various parcellations, but produced the most reliable connectivity estimates, followed by the control network. The default and dorsal attention networks exhibited the lowest consistency. The lack of consistency within the default and dorsal attention networks was largely driven by the anticorrelated/uncorrelated nature of connectivity estimates from the Glasser parcellation (Glasser et al., 2016), which was derived using multi-modal imaging data unlike the other parcellations derived using rsfMRI only. This suggests that the “default ” and “dorsal attention ” networks identified within the Glasser parcellation should not be considered synonymous to those networks sharing the same labels across the other parcellations examined. These findings suggest caution is advised when selecting parcellations and when using the results of a given study to draw generalized conclusions about “canonical ” functional networks. While there does appear to be some consistency in the general extent of networks identified in parcellations developed using various methodological approaches, our results suggest that the variance in extent of the networks defined in each parcellation can have significant impact on connectivity measures derived from parcellation-extracted data. Furthermore, these findings suggest caution is warranted when drawing parallels between networks identified across the various parcellations, suggesting a “ground truth ” of brain organization has yet to be established from our current parcellation practices.

A major limitation of the parcellations employed is that they are derived from group-averaged data. Recent work in densely-sampled individuals demonstrates that many of the functional networks identified in group-averaged data are comprised of distinct yet highly interdigitated networks that support divergent functional abilities (Braga et al., 2019; Braga and Buckner, 2017; DiNicola et al., 2020). These findings raise important questions about the utility of group-averaged data for future work on functional brain organization and suggests that studies examining individual differences such organization may be well served by moving away from the application of a priori parcellations and toward the use of individual-level parcellations. Though such methods require more data from each individual, the knowledge gained from such approaches is likely to be illuminating.

Impact of parcellation selection on results.

In developmental and clinical research, it is common practice to conduct analyses only using a single parcellation scheme. This assumes that parcellation selection does not have an impact on the interpretation of the results. Our findings challenge this assumption. A range of analyses focused on differences in functional connectivity by age, poverty, and cognitive function, revealed that parcellation selection had a significant and meaningful impact on the interpretation of results. For each of the associations examined, roughly three of the eight parcellations produced significant results, while the other five produced null results. Furthermore, we observed meaningful variability not only in the statistical significance of the estimates, but also in the point estimates. These findings suggest that parcellation selection can have a significant influence on the results obtained in studies examining individual differences in resting-state functional connectivity, particularly if only a single parcellation is used. Given this, we caution researchers from relying on a single parcellation in their analysis pipeline unless they have strong theoretical reason to do so. Furthermore, we would recommend employing a series of schemes and potentially even performing a specification curve analysis (Del Giudice and Gangestad, 2021; Simonsohn et al., 2015) to confirm the robustness of results. There are analysis pipelines, such as the XCP pipeline (Ciric et al., 2018), built to be compatible with fmriPREP (Esteban et al., 2019) output and the Brain Imaging Data Specification (Gorgolewski et al., 2016), that make such replications feasible by providing researchers with the option of outputting timeseries extracted from multiple parcellation schemes automatically. By conducting such replications, researchers can be confident that their results are not contingent on parcellation selection.

Furthermore, large population-based samples such as the Adolescent Brain and Cognitive Development (ABCD) study (Casey et al., 2018) offer an opportunity to study not only development and individual differences in network connectivity, but also to evaluate the impact of parcellation choice on results in very large samples, as well as confirm robustness of results to parcellation selection. However, the ABCD study currently only offers the resting-state time series data extracted using the Gordon 2016 parcellation in their tabulated data. Therefore, individual researchers are not easily able to additionally examine the effect of parcellation choice on their findings. The fact that the ABCD study has only released data extracted from a single parcellation makes clear that the assumption in the field is that parcellation selection does not meaningfully impact results. Given our findings, that parcellation selection can/does matter, we suggest that future releases of the ABCD resting-state data include data extracted from additional parcellation schemes.

5. Conclusion

We examined a series of assumptions made when a priori brain parcellation schemes are used to identify the canonical functional networks and examine individual differences in functional connectivity. We found that the networks of interest were equally well recovered in data extracted using a series of eight parcellations, with the exception of NeuroSynth. Furthermore, the networks of interest were equally well represented in pediatric data as in adults. However, within-network functional connectivity showed notable variability and poor consistency across the parcellations examined for each of the four networks. Furthermore, parcellation selection meaningfully impacted the magnitude and significance of associations between functional connectivity and age, poverty, and cognitive function. Our findings suggest that work that depends on a priori parcellations for network identification may benefit from the use of multiple schemes to confirm the robustness and generalizability of results. Furthermore, researchers looking to gain insight into functional networks my benefit from employing more nuanced network identification approaches such as using densely-sampled individual data to produce individual-derived network parcellations. Indeed, recent work examining idiosyncratic network topography, as opposed to within-network connectivity, has illustrated meaningful individual differences in topography associated with cognition and behavior (Bijsterbosch et al., 2018; Kong et al., 2019). A transition towards precision neuroscience in cognitive work in general, and in developmental and clinical work specifically, is likely to improve the characterization of functional brain organization, and links with cognition and behavior, in a way that is unavailable when methods are confined to group averaged approaches.

Data and code availability statement

Analysis code is available at: https://osf.io/bjde4/.

Data: Available upon request.