A framework For brain atlases: Lessons from seizure dynamics

Abstract

Brain maps, or atlases, are essential tools for studying brain function and organization. The abundance of available atlases used across the neuroscience literature, however, creates an implicit challenge that may alter the hypotheses and predictions we make about neurological function and pathophysiology. Here, we demonstrate how parcellation scale, shape, anatomical coverage, and other atlas features may impact our prediction of the brain’s function from its underlying structure. We show how network topology, structure–function correlation (SFC), and the power to test specific hypotheses about epilepsy pathophysiology may change as a result of atlas choice and atlas features. Through the lens of our disease system, we propose a general framework and algorithm for atlas selection. This framework aims to maximize the descriptive, explanatory, and predictive validity of an atlas. Broadly, our framework strives to provide empirical guidance to neuroscience research utilizing the various atlases published over the last century.

1. Introduction

How we define anatomical brain structures and relate those structures to the brain’s function can either constrain or enhance our understanding of behavior and neurological diseases (Bohland et al., 2009; Dickie et al., 2017; Klein and Tourville, 2012; Mandal et al., 2012). Discoveries by scientists like Carl Wernicke and Pierre Paul Broca, who mapped specific brain regions to speech function, in addition to case studies from Phineas Gage and H.M., who lost specific brain regions with resultant changes in brain function and behavior, exemplify how brain structure and function are fundamentally linked (Barker, 1995; Beal et al., 2015; Van Horn et al., 2012). Properly labeling brain structures is paramount for enabling scientists to effectively communicate about the variability between healthy individuals and about the regions involved in neurological disorders (Mazziotta et al., 2001). Yet, no consensus has been reached on the most appropriate ways to label and delineate these regions, as evident by the wide variety of brain maps, or atlases, defining neuroanatomical structures (Evans et al., 2012).

In common usage, an atlas refers to a “collection of maps” National (2022) that typically defines geo-political boundaries and may include coarse borders (continental), fine borders (city), and anything in between (country; Fig. 1 a, left). Borders (National, 2022) are usually consistent across atlases of the world. In contrast, atlases of the brain are not consistent. Four separate atlases (Fig. 1 a, right) may define the superior temporal gyrus differently. For example, approximately ninety percent of the anterior superior temporal gyrus in the Harvard-Oxford atlas (Makris et al., 2006) overlaps with the posterior superior temporal gyrus in the Hammersmith atlas(Hammers et al., 2003). Atlases may also differ in other ways, including parcellation size, neuroanatomical coverage, and complexity of brain region shapes. For instance, the Yeo atlas (Thomas Yeo et al., 2011) contains 7 or 17 parcels while the Schaefer atlases (Schaefer et al., 2018) may have between 100 and 1000 parcels. Complicating matters further, atlases can differ in their intended use. The MMP atlas (Glasser et al., 2016) was intended for surface-based analyses (Coalson et al., 2018), yet a volumetric version (without sub-cortical structures) was independently created and used in connectivity studies (Wu et al., 2019). The plethora of available atlases poses a problem for reproducibility in studying healthy and diseased populations and for metanalyses describing the involvement of different regions of the brain in various diseases. This has been termed the Atlas Concordance Problem (Bohland et al., 2009).

Fig. 1. Many brain atlases are available in the neuroscience literature.

| a, In common usage, an atlas refers to a “collection of maps”National (2022) that defines geo-political borders at different scales. Although borders(National, 2022) are usually consistent across atlases of the world, they are typically not consistent across atlases of the brain. Four separate atlases (left-to-right: CerebrA, AAL, Hammersmith, Harvard-Oxford) may define the superior temporal gyrus differently. The lack of consistency across these labels poses a problem for reproducibility in cognitive, systems, developmental, and clinical studies, as well as metanalyses describing the involvement of different regions of the brain in various diseases (Bohland et al., 2009). This challenge has been previously referred to as the Atlas Concordance Problem. b, Atlases can have varying features (see also Table 1). c, Thus, all current connectivity studies in neuroscience may not accurately reflect some fundamentally “true” architecture. For example, atlases with either large or small parcels may affect the structural connectivity matrices that are used to define the ”true” network architecture of the brain, and subsequently that are used to test hypotheses or make predictions about the brain. d, When combined with white matter tracts reconstructed from diffusion MRI, atlases can be used to measure how different regions of the brain are structurally connected (i). Similarly, intracranial EEG (iEEG) implants can record neural activity to measure how different regions of the brain are functionally connected (ii). Technologies such as fMRI, MEG, and many others can also measure functional connectivity. The statistical similarity between structural and functional connectivity measurements can be calculated (e.g., structure–function correlation; SFC). Such estimates have been used to better understand the pathophysiology of disease. In this study, we evaluate how the varying atlases may alter the power to test a specific hypothesis about the brain’s structure–function relationship in epilepsy.

In the present study, we perform an extensive evaluation of the available atlases in the neuroscience literature (Table 1) by examining the effect of varying features such as parcellation size, coverage, and shape (Fig. 1 b) on structural connectivity (Fig. 1 c). We also examine how atlas choice changes structural network topology by measuring structure–function correlation (SFC) using an atlas-independent measure of functional connectivity (Fig. 1 d). We utilize a total of 55 brain atlases, including many routinely used in common neuroimaging software. Note the important distinction between the terms atlas, template, and stereotactic space (Evans et al., 2012) (see Fig. S1). We found that different atlases may alter the power to test a hypothesis about epilepsy pathophysiology that seizures propagate through the underlying structural connections of the brain. This hypothesis has been previously supported in prior research(Ashourvan et al., 2021; Proix et al., 2017; Shah et al., 2019; Wirsich et al., 2016).

Table 1. Atlases.

|Atlas sources are detailed in Table S1 and abbreviations are in the glossary. S: Structurally defined atlas; F: Functionally defined atlas; M: Multi-modally defined atlas; V: A variably defined atlas that may be structurally, functionally, or multi-modally defined; ROI: region of interest; HCP: Human connectome project dataset (Van Essen et al., 2013); MS: multiple sclerosis.

In the context of our experimental design, we propose a new framework outlining how to appropriately choose an atlas when designing a neuroscience experiment. This framework is derived from historical foundations for assessing the validity and effectiveness of animal models (Willner, 1984), network models (Bassett et al., 2018), and psychometric tests (Association, 1954), which try to maximize the (1) descriptive, explanatory, and (3) predictive validity (Bassett et al., 2018) of a model. Atlases are a tool for investigators to test for causality and to make predictions about the brain. Thus, this framework incorporates a short discussion on explanatory modeling and predictive modeling, each with different goals (”To Explain or to Predict?”(Shmueli, 2010)). A one-size-fits-all approach may not exist for selecting an atlas, nor should it (Salehi et al., 2020); while there is one Planet Earth with a single atlas for a particular use (e.g., an atlas of the geo-political borders for a given point in time), there are many brains, with anatomical and functional variability across populations and species(Salehi et al., 2020). We hope our framework provides empirical guidance to neuroscience research utilizing the various atlases published over the last century.

2. Results

2.1. Clinical data

Forty-one individuals (mean age 34 ± 11; 16 female) underwent High Angular Resolution Diffusion Imaging (HARDI), composed of thirteen controls (mean age 35 ± 13; 6 female) and twenty-eight drug-resistant epilepsy patients (mean age 34 ±11; 12 female) evaluated for surgical treatment. Of the twenty-eight patients, twenty-four were implanted with stereoelectroencephalography (SEEG) and four with electrocorticography (ECoG). Ten SEEG patients (mean age 34 ±8; 4 female) had clinical seizure annotations, and the first seizure from each patient (mean duration 81 s) without artifacts was selected for SFC analyses. Patient and control demographics are included in Table S2.

2.2. Atlas morphology: sizes and shapes

We hypothesized that atlas morphological properties, including size and shape (Fig 2), affect SFC. To test this hypothesis, we first quantified the distributions of parcellation sizes (Fig 2 a) and shapes (Fig 2 b) in various atlases. These results exemplify the diversity of atlas parcellation morphology. Fig 2 c shows a comparison of individual parcellation volumes and sphericities. The remaining atlases are shown in Fig. S2. In contrast to standard atlases, random atlases have constant sphericity with respect to volume size. Note that the distribution of parcellation shapes (i.e. sphericity) is similar across parcellation sizes in random atlases and their parcellations may not represent true anatomical or functional boundaries. Thus, random atlases allow us to study how parcellation scale affects network structure and SFC while keeping the effect of shape constant. Crucially, random atlases also allow us to explore if accurate and precise anatomical boundaries are essential in some experimental designs(Albers et al., 2021).

Fig. 2. Atlas morphology: sizes and shapes.

| a, Volume distribution of atlas parcellations demonstrating the diversity of parcellation sizes. b, Parcellation sphericity distributions illustrating how the shapes of different parcellations may not be uniform. c, Volumes versus sphericity showing how some atlas parcellations may be small and spherical, while others may be large and non-spherical. This illustrates the non-uniformity in atlas parcellations. d, Volumes and sphericity of random atlases showing the uniformity of sphericity with changing volumes. Random atlases allow us to study (1) the effect of parcellation scale without the confound of shape effects and (2) the need for accurate anatomical boundaries to test a hypothesis about the structure–function relationship in the brain at seizure onset. Numbers in legend represent the number of parcellations for each random atlas. Remaining atlases are in Fig. S2.

2.3. Varying atlases affect structural network topology

Although the morphology of atlas parcellations is diverse, we aimed to investigate how these morphological characteristics (particularly parcellation scale) affect structural network topology (Fig 3). Networks are the basis upon which we compute SFC, and not necessarily morphological characteristics, therefore, we measured how network density, mean degree, characteristic path length, mean clustering coefficient, and small worldness change as a function of parcellation scale (Fig. 3 a). We found that the change in these network measures are congruent between standard and random atlases and previous studies (Zalesky et al., 2010). We also show that mean density, a global network measure, is similar between our control (N=13) and patient (N=28) cohorts (Fig. 3 b).

Fig. 3. Structural network differences between atlases.

| a, Density, mean degree, mean clustering coefficient, characteristic path length, and small worldness were calculated for structural connectivity networks. A subset of atlases is shown. Remaining atlases studied are shown in Fig. S3. The average parcellation volume was calculated for each atlas and the corresponding network measure was graphed as the mean of all subjects (N=41; 13 controls, 28 patients). b, Controls and patients were not significantly different in density for the AAL2 atlas (Mann-Whitney U test), illustrating that global structural network measures are similar between cohorts. However, specific edge-level connections between cohorts may be different, and characterizing these differences is out of the scope of this manuscript. Controls and patients were separated and shown in Fig. S4. Network measures using different threshold are shown in Fig. S5.

2.4. Varying atlases affect SFC: Single subject

Fig 4 illustrates an overview of how SFC is calculated. Structure is measured with high angular resolution diffusion imaging (HARDI) and function is measured with SEEG electrode contacts. Structural connectivity matrices are generated based on the atlas chosen (Fig. 4 a) and functional connectivity matrices are generated based on broadband (1 – 127 Hz) cross-correlation of neural activity between the electrode contacts in widows of time (Fig. 4 b, see Methods section on ”Functional Connectivity Network Generation”). Thus, the structural network is static while the functional network is computed across time. The connectivity matrices shown are example data from a single patient, sub-patient07. Functional connectivity matrices are shown for 6 hours before seizure onset, 90 s before seizure onset (t = −90), 40 s after seizure onset (t = 40), 88 seconds after seizure onset (seizure duration = 89 s), and 180 seconds after seizure onset (91 s after seizure termination). Each functional connectivity matrix time window was correlated to each structural connectivity matrix, yielding a SFC at each time window (Fig. 4 c). Each point represents the structural edge weight between two brain regions and their corresponding functional connectivity edge weight in broadband cross-correlation. A line of best fit is shown for visualization, and r values represent Spearman rank correlation for that time point. SFC was graphed for all time points during the interictal, preictal, ictal, and postictal periods for this patient in Fig. 4 d.

Fig. 4. Structure–Function correlation in a single patient using different atlases.

| a, Example atlases and structural connectivity matrices. b, Functional connectivity matrices are computed from SEEG recordings during the interictal, preictal, ictal, and postictal periods. During each period, the SEEG data is binned into non-overlapping windows (the vertically stacked matrices) to create time varying representations of functional connectivity. Broadband cross correlation matrices are shown for sub-patient07 at 6 h before seizure onset, 90 s before seizure onset, 40 s after seizure onset (t = 40), 88 s after seizure onset (seizure duration = 89 s), and 180 seconds after seizure onset (or 91 s after seizure termination). c, Each functional connectivity matrix is correlated to a structural connectivity matrix of a given atlas. Spearman Rank Correlation is measured between all time points and all atlases for each patient. Lines of best fit are for visualization purposes only. d, SFC is graphed at each time point for four example standard atlases (Hammersmith, Craddock400, AAL2, and CerebrA), and four example random atlases (30, 100, 1k, and 10k parcellations). SFC increases during seizure state for some standard atlases (Craddock 400, AAL2, and CerebrA atlases). This result follows previous SFC publications with ECoG (Ashourvan et al., 2021; Shah et al., 2019). However, SFC does not increase for the Hammersmith atlas. These findings highlight that the power to detect a change in the structure–function correlation at seizure onset, and thus the ability to probe the hypothesis that seizure activity is correlated to brain structure, may be reduced using some atlases. The use of different atlases may contradict previous studies.

Four example standard and random atlases are graphed. We show that SFC increases during the ictal state for many atlases (CerebrA, AAL2, Craddock 400), but not all atlases (Hammersmith). The increase in SFC during seizures follows previous SFC studies using ECoG (Ashourvan et al., 2021; Shah et al., 2019). Similarly, SFC increases for a subset of random whole-brain atlases. While parcellation scale may affect SFC, it is not the only feature affecting SFC - the Hammersmith and AAL2 atlases have similar parcellation scales yet diverging neuroanatomical properties and SFC dynamics. These findings highlight inference from one type of atlas may suggest that seizure activity is not correlated to brain structure, contradicting previous studies (Shah et al., 2019).

2.5. Varying atlases affect SFC: Multiple subjects

Fig 5 shows SFC for ten standard atlases and five random atlases using SEEG broadband cross-correlation metrics averaged across all patients with clinically annotated seizures (N = 10). The AAL2 atlas shows a statistically significant increase in SFC from preictal to ictal periods (p < 0.05 by Wilcoxon signed rank test after Bonferroni correction for 55 tests). This change from preictal to ictal SFC is denoted Delta SFC. Using the AAL2 atlas, this finding supports the hypothesis that seizure activity propagates and spreads via axon tracts making up the underlying structural connectivity of the brain (Ashourvan et al., 2021; Shah et al., 2019). SFC was similarly calculated for random whole-brain atlases. A notable finding is that during the interictal period, resting state SFC (rsSFC) increases at larger number of parcellations (i.e. smaller parcellation volumes). We show that rsSFC is observably affected by parcellation scale when plotting the random atlases in Fig 5 (bottom row). These findings may be concerning given that the inherent structure-function relationship in the brain is not necessarily changing at resting state, but its measurement is greatly affected by atlas choice alone.

Fig. 5. Structure–Function Correlation in multiple patients using different atlases.

| SFC for ten standard atlases and five random atlases using SEEG broadband cross-correlation matrices averaged across all patients with clinically annotated seizures (N = 10). Resting state SFC (rsSFC) is the SFC during the interictal period. The change from preictal to ictal SFC is ΔSFC. SFC was similarly calculated for random atlases and shows that rsSFC and ΔSFC may change with parcellation scale. These findings may be concerning given that the inherent structure—function relationship in the brain is not necessarily changing at resting state, but its measurement is greatly affected by atlas choice alone.

2.6. Varying atlases affect resting state SFC and ΔSFC

Resting state SFC (rsSFC) and the change in SFC (ΔSFC) from preictal to ictal periods are affected by parcellation scale (Fig 6). Fig. 6 a shows how rsSFC decreases with larger average parcellation volumes (moving left to right). A large average parcellation volume for a given atlas generally means there is a fewer number of total parcellations (e.g. the MNI structural atlas has a large average parcellation volume given only nine parcellations). In contrast, Fig. 6 b shows ΔSFC increases with larger parcellation volumes (moving left to right). Broadly, ΔSFC may be interpreted as the change in SFC with respect to a disease (e.g. a seizure) and non-disease states. This change metric has been used to characterize and make inferences in many neurological disorders (Cocchi et al., 2014; Sathian and Crosson, 2015). Only a subset of atlases show a change in SFC at seizure onset (Fig. 6 c). These results exemplify that either overly coarse or fine parcellations may not adequately capture the underlying SFC of the brain or its dynamics with relation to a neurological disease.

Fig. 6. The power to test a hypothesis about epilepsy pathophysiology changes depending on atlas choice.

| a, Resting state SFC (rsSFC) decreases with larger parcellation volumes (moving left to right). Random atlases are shown in blue, and select standard atlases are shown in red. Points represent the average across all patients, and bands represent 95% confidence intervals. b, ΔSFC increases with larger parcellation volume (moving left to right). Broadly, DeltaSFC may be interpreted as the change in SFC with respect to disease (e.g. a seizure) and non-disease states, and this change has been used to characterize and make inferences on many neurological diseases. These results exemplify that parcellations that are either too coarse (large volumes) or too fine (small volumes) may not adequately capture the underlying SFC of the brain or its dynamics with relation to a neurological disease. c, A subset of atlases show a difference in preictal and ictal SFC. d, The effect size between preictal and ictal SFC is calculated for all 55 atlases used in this study. Many atlases commonly used in the neuroscience literature have comparable effect sizes to random atlases. The standard atlases with the greatest effect size (and thus power) are the Harvard-Oxford and AAL3 atlases. These atlases outperform many random atlases (where anatomical boundaries are not followed) and may indicate that their parcellation scheme captures the structure–function relationship in the brain at seizure onset with DTI and iEEG.

2.7. Atlas choice affects the power to test a hypothesis

The effect size between preictal and ictal SFC is calculated for all 55 atlases used in this study (Fig. 6 d). Cohen’s d and the difference between the mean ictal and mean preictal SFC are shown. Atlases are ordered by Cohen’s d.

We found that different atlases may alter the power to test the hypothesis about epilepsy pathophysiology that seizures propagate through the underlying structural tracts of the brain, measured with diffusion MRI. This hypothesis has been previously supported in prior studies(Ashourvan et al., 2021; Proix et al., 2017; Shah et al., 2019; Wirsich et al., 2016)

Many atlases commonly used in the neuroscience literature have comparable effect sizes to random atlases (where anatomical boundaries are not followed). The standard atlases with the greatest effect size (and thus power, given equal significance levels and sample sizes) are the Harvard-Oxford and AAL3 atlases. These atlases outperform many random atlases and may indicate that their parcellations may adequately capture the structure–function relationship in the brain. These atlases may capture the “true” structural network architecture (see Fig. 1 c) because these network architectures better differentiate and are more correlated to functional changes seen at seizure onset.

Despite the effect sizes of the Harvard-Oxford and AAL3 atlases, however, there may not be a “true gold standard” atlas or parcellation scheme given that resolution is more critical than the exact border location of parcels(Albers et al., 2021), there may be no single functional atlas for an individual across all brain states(Salehi et al., 2020), and many standard atlases yield similar effect sizes to randomly generated atlases (this study).

3. Discussion

In this study, we performed an extensive evaluation of the available structural, functional, random, and multi-modal atlases in the neuroscience literature (Table 1). We detailed morphological (Fig 2) and network (Fig 3) differences between these atlases. We showed the effect of atlas choice on the measurement of structure-function correlation (SFC) in epilepsy patients (Fig 4 and Fig 5). We also showed how various atlases may affect the power to test a hypothesis about seizure propagation (Fig 6). This work has implications for investigators because the ability to test hypotheses and make predictions about the brain’s function may depend on atlas choice. In light of our study using an extensive list of available brain atlases, we propose a general framework below for evaluating and selecting an atlas (Fig 7).

Fig. 7. A Framework for brain atlases.

| a, Which atlas should be chosen for a study? We propose a framework that helps select an atlas in the context of its descriptive, explanatory, and predictive validity. Descriptive validity means the features of an atlas appropriately resembles the experimental system. An atlas is also a tool to solve a variety of problems in neuroscience. It may be used as part of a methodology to explain causality (explanatory validity), or it may be used to make predictions (predictive validity). These two goals are distinct, and the differences between explanation and prediction ”must be understood for progressing scientific knowledge”Shmueli (2010). These aspects (to explain or to predict) should be considered when selecting an atlas. b, Non-mutually exclusive atlas features related to descriptive validity. c, A list of questions to consider when choosing an atlas. Gray lines connect related questions. d, An algorithm for atlases selection a priori and post hoc. Please see the main text for further details.

3.1. A framework For brain atlases

Various publications have highlighted the Atlas Concordance Problem (Bohland et al., 2009; Dickie et al., 2017; Evans et al., 2012; Mandal et al., 2012), curated several atlases in freely accessible databases (Gorgolewski et al., 2015; Lawrence et al., 2021), and made arguments for why specific atlas features (Fig. 7 b) may be superior in certain situations (Alexander et al., 2017; Brennan et al., 2019; Cabezas et al., 2011; Caspers et al., 2013; Coalson et al., 2018; Diedrichsen et al., 2009; Salehi et al., 2020). There have been great efforts to publish accurate and precise parcellations as seen with an exponential rise in atlas-related publications over the last three decades (Fig. S8). However, none have found a general solution to the underlying problem: Does atlas choice matter?

We provide a framework that allows us to determine if the choice of an atlas is appropriate in the context of its (1) descriptive, (2) explanatory, and (3) predictive validity(Bassett et al., 2018). This framework is borrowed from the logic for assessing network models(Bassett et al., 2018), animal models, (Belzung and Lemoine, 2011; Willner, 1984), and psychometric tests (Association, 2014; 1954), where assessment of these models with standard statistical model-selection methods is particularly challenging. Thus, theoretical constructs already formulated in other fields may provide guidance.

Descriptive validity of an atlas refers to an atlas that appropriately resembles the system in which we work. In other words, it has “face value”Willner (1984). An atlas should include features (Fig. 7 b) relevant to the study (e.g., parcellations containing subcortical structures relevant to epilepsy). Importantly, the descriptive validity of an atlas also relates to the modality scale we use to measure the brain - for example, DWI and fMRI at the macroscale (Sporns et al., 2005), iEEG and tracers at the meso scale (Fornito et al., 2016), and microscopy at the microscale (Sporns, 2011). It is important to select a parcellation scale that resembles the measurement modality resolution (Fig. 6 a). When correlating DWI with iEEG in our study at larger parcellation sizes, we lose our ability to discern precise anatomical locations that are structurally and functionally related (Fig 6 b. Similarly at smaller parcellation sizes (tending to voxel resolution), we may not capture the “true” structural network architecture (Fig. 1 c), and thus we lose our ability to capture structure–function relationship changes at seizure onset.

An atlas is a tool to tackle a wide variety of problems in neuroscience. It may be part of a methodology to explain causality (explanatory validity) or it may be part of a methodology to make predictions (predictive validity). These two goals are distinct, and the differences between explanation and prediction “must be understood for progressing scientific knowledge” as described in “To Explain or Predict?” by Shmueli, 2010(Shmueli, 2010). In the context of building scientific models, a model with a high explanatory ability may not have a high predictive ability.

Similar to models, atlases are also part of a scientific methodology to (1) explain how the brain functions or (2) predict new observations (i.e., they are one part of the overall methodological pipeline to test hypotheses or make predictions about the brain - for studies using atlases). Thus, atlases are tools. An atlas may be suitable for hypothesis testing, for example, because it includes subcortical structures like the hippocampus (also high descriptive validity) to support a hypothesis about seizure propagation through subcortical structures. Intuitively, without subcortical structures, it would be impossible to test hypotheses about subcortical structures. Less intuitively, explanatory validity of an atlas may also relate to the power to test hypotheses, which we show in our study. Some atlases may not be suitable for scientific inquiry because they provide little statistical power to detect differences in disease states, for example, to detect changes in SFC at seizure onset (Fig. 6 b). It may be impossible to accurately predict power using an atlas before conducting a study, however, other studies asking similar questions using similar atlases may provide reasonable estimates of effect sizes (our study has similar effect sizes to a previous study(Shah et al., 2019)). Power may also depend on the accuracy of anatomical boundaries, or in our study, other atlas features such as parcellation scale and configuration (Fig. 6 d). For example, the Harvard-Oxford and AAL3 atlases have similar parcellation configurations and similar power.

Some atlases may or may not be not be suitable for making predictions about new or future observations about the brain. For example, many network properties change with atlas choice (Fig. 3), and thus it is reasonable to suspect model prediction outputs may change with respect to the atlas used to build and train such models. Importantly, the exclusion of some anatomical structures, like white matter or the cerebellum in some atlases, may affect the training data used to build predictive models. In our study, a translational goal is to predict functional seizure activity from structural data. SEEG records activity from both gray matter and white matter; however, recent studies have shown that white matter functional recordings may provide different information than gray matter(Greene et al., 2021; Mercier et al., 2017; Revell et al., 2021; Young et al., 2019). Thus, excluding some anatomical labels may affect model predictions. Another example is the use of network models to predict spread, such as α-synuclein across the brain connectome (Henderson et al., 2019). Without the incorporation of all brain structures related to α-synuclein spread, models to predict and monitor spread may be inaccurate.

3.2. Are accurate anatomical or functional parcellations needed?

During the course of conducting this study, and while undergoing peer review, other atlases with more accurate or relevant parcellations to the study’s population were published in different areas of neuroscience(Callaway et al., 2021; Doucet et al., 2021; Huang et al., 2021; Joglekar et al., 2021; Lewis et al., 2021; Muñoz-Castañeda et al., 2021; 2021; Syversen et al., 2021; Wang et al., 2021; Zhu et al., 2021). Here, we cautiously propose a question: Are efforts to publish more atlases created with different algorithms or slightly modified parcellations from existing atlases providing any advantages over already existing atlases? Naturally, accurate and precise parcellations are needed when probing specific hypotheses about exact structures that depend on accurate segmentation of such structures (particularly at the sub-field or cellular level); however, few studies compare an atlas to a null atlas (one with randomly generated parcellations). Studies that do are Gordon et al. 2016Gordon et al. (2016)and Lewis et al. 2021Lewis et al. (2021).

In this study, we show that random atlases provide similar power to detect differences in SFC between preictal and ictal states (Fig. 6 d). Indeed, it is difficult or nearly impossible to evaluate a newly proposed atlas, given that the performance metrics to evaluate an atlas may be infinite (given infinite experimental designs). Only one such metric, SFC, was used in this study. But given new deep learning methods and other computationally expensive methods using trained classifiers for segmentation, existing atlases may be adequate for labs with limited funding resources, trained personnel, and access to GPUs. These labs may still be capable of answering important questions in neuroscience.

3.3. Which atlas should be used for my study?

One of the most difficult challenges as scientific investigators is to make optimal methodological decisions to discover useful findings for the scientific community. Selecting an atlas is one such decision we may make in some of our studies. We realize the framework provided above may be abstract to some readers; we also provide a concrete list of questions to consider when choosing an atlas (Fig. 7 c) for a neuroimaging study. However, in conducting this study, we also found that researchers may face three problems when choosing an atlas (Fig. 7 d) and these problems are worth further discussion. The first two problems are in selecting an atlas a priori, or before conducting a study. They deal with selecting one or a few atlases to preserve power, or in selecting a standard set of atlas to publish public data for other researchers to use. The third problem is the issue of conflicting results between two atlases and what to do after a study is conducted (post hoc). We provide a further discussion on these problems below.

3.4. Considerations in selecting one or a few atlases

Selecting one atlas may preserve power and avoid a multiple comparisons problem by testing every atlas. Selecting an additional atlas may also be chosen to confirm the robustness of results. In these cases, a balance of time, availability of tools, and atlas features logical for your study as outlined in Fig. 7 a-c need to be considered. For example, if a custom atlas is used, how will that affect replicability and meta analysis in the long-run for the field? What are the atlas features needed (such as scale and coverage of regions)? What are the computational costs and personnel training needed to use particular atlases? (See questions in Fig. 7 c).

3.5. Considerations in selecting a standard set of atlases

When publishing results and/or making data publicly available for other investigators to use, another approach is to select a set of atlases based on the perceived needs of other investigators, atlas features covered, prevalence of atlases used in the literature (Fig. S9a), and dummyTXdummy-(the prevalence of “turn-key” neuroimaging software that incorporate these atlases (Fig. S9b). Studies are emerging with data publicly available for use based on one or a few select atlases(Royer et al., 2021; Sinha et al., 2021). Many turn-key neuroimaging software also inevitably have to make the decision to employ a set of atlases to meet the needs of many researchers. A problem may arise, however, when other researchers need the published data at other atlas resolutions or with other structures. And unfortunately, the value of the data may be lessened and the effort put in by the publishing researchers may be in waste if this happens. What may help with the atlas concordance problem is perhaps a “standard set” of atlases – a set to benchmark studies across the neuroimaging field. Furthermore, turn-key tools like FreeSurfer, QSIprep, DSI-studio, FSL, and many others may benefit from a standard set of incorporated atlases that captures enough features useful to the majority of the neuroscience community, even if not every available atlas is included. Based on our exhaustive search of atlases in the neuroimaging literature, the ability to collect them for use in a single study, the prevalence of certain atlases already in-use (Fig. S9a), and the prevalence of neuroimaging software (Fig. S9b) we propose an initial set of atlases (Fig. 7 d).

The AAL atlas is one of the most commonly used volumetric atlases (Fig. S9a), and along with the Harvard-Oxford atlas, may provide complimentary results when published together. The Brainnetome atlas(Fan et al., 2016) is another structural atlas at a finer resolution, having gained popularity since its introduction in 2016. The Destrieux and DKT atlases are also structural atlases, and already incorporated into one of the most commonly used neuroimaging software, FreeSurfer (https://surfer.nmr.mgh.harvard.edu). FreeSurfer provides surface-based registration, which may more accurately label cortical structures than volumetric registration (Fig. S6). Accurate segmentation of sub-cortical structures may also be acquired from FSLPerlaki et al. (2017) (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki). In addition, the MMP, or “Glasser” atlas was created from multi-modal imaging data. A commonly used atlas provided at different scales is Schaefer atlases provide, however, it does not include subcortical structures.

Random atlases may also provide robust conclusions by allowing researchers to manipulate the resolution, size, and shape of parcellations and iterate over many atlases. Although random parcellations may forgo accuracy because they do not follow true anatomical boundaries, these atlases may still provide similar conclusions to other standard atlases with the added benefit of permuting results over many atlases (Fig 6). An alternative to random atlases is to divide or combine the parcellations of another standard atlas (a “derived” atlas in Fig. 7 d. For example, the AAL 600 is derived from the AAL atlas in which its parcellations are further sub-divided using a specified algorithm. Parcellations may also be sub-divided randomly.

3.6. Considerations in conflicting results between atlases

When more than one atlas is used, results may conflict. We define conflicting results as two different atlases giving alternating predictions (e.g., good vs poor outcomes, increase in SFC rather than decrease in SFC) or support alternating working hypotheses (e.g., the temporal lobe is involved in one atlas, but another atlas highlights the involvement of the frontal lobe in the pathophysiology of a disease). We do not mean that conflicting results arise due to lack of statistical power (e.g., one atlas gives a p-value of 0.06 and another atlas 0.04).

One way to understand if the observed effect is not an artifact of the atlas choice is to select a few atlases with varying features and figure out what is causing the conflict. Unfortunately, there may be no other way given that every study will have different parameters and measurements to know what gives rise to conflicting results. In the matter where conflicting results arise due to atlas selection, then it may troubleshooting may be needed to understand what gives rise to the conflict (surface vs volumetric registration, parcellation scale, missing relevant structures, etc.). Fortunately, however, most atlases in this study affect power rather than conflicting results (Fig. 6 d. We hope this discussion, our study, and our figures provide insight to others.

3.7. Limitations

Our study is not without limitations. A major limitation is that we did not evaluate atlases in a diverse set of experimental systems, but rather limited our analysis to a contemporary topic in epilepsy using SEEG implantations and to a study of the structure–function of the brain, potentially appealing to a wider audience. The question we were trying to answer (“Which atlas should we use?”) is a difficult problem to solve, given that it would be impossible to evaluate all atlases in all experimental designs. We attempted to generalize a framework given our findings after an extensive search for, and curation of, available neuroimaging atlases.

We also did not perform a feature selection analysis post-hoc to maximize Delta SFC at seizure onset; rather, we performed a comprehensive evaluation of many atlases to set a general framework and describe the nuances between the different atlases and their features. Ideally in our study, we required a whole-brain, volumetric atlas that covered the implanted SEEG electrode contacts. No such atlas existed. We opted for combining different atlases or developing randomly parcellated atlases used in previous publications (Mišić et al., 2015; Zalesky et al., 2010). However, no general framework existed to determine which atlas should be used or clearly outlined the feature space of these atlases. We had no formal basis for how changing an atlas could change our results and eventual goal for translating network models to better treat epilepsy patients.

Another limitation, we assume a change in SFC supports the hypothesis that seizures harness the underlying structural connectome of the brain (along with support from prior literature (Ashourvan et al., 2021; Betzel et al., 2019; Shah et al., 2019)). We may be biasing our results to select an atlas that maximizes Delta SFC. However, we wish to select a methodology that allows us to measure any change in brain state that accompanies seizure onset (explanatory validity), permitting us to probe epilepsy biology and understand the processes that govern seizure spread.

An additional limitation concerns the effect of parcellation volume on SFC. In probing this effect across our random atlases and atlases used in the literature, we did not perform controlled experiments to separate the effects of parcellation size from parcellation N (number of parcellations). A future experiment could fix the number of parcellations while changing parcellation volume (or vice versa). This would allow us to test whether parcellation volume or N drives changes in SFC. However, this was outside the scope of our study.

Our goal was to highlight the importance of selecting an appropriate atlas from an array of possibilities, using a data-driven, validated experimental paradigm(Shah et al., 2019). We acknowledge new studies that show that streamline counts may not completely reflect the underlying diffusion data(Smith et al., 2015); however, comparing such techniques were outside the scope and goal of our focused study. We also note that few patients had lesions on imaging. Misalignment due to non-linear distortion may add noise to our data; however, few patients had lesions. Our study was not conducted to necessarily make the claim that SFC changes exist in the brain at seizure onset, but rather to show how varying atlases may change SFC.

Finally, our analysis relies on the assumption that an atlas approach must be used to quantify SFC and does not consider an atlas-agnostic approach nor if such an approach is appropriate. To study SFC using networks, both structural and functional networks must have nodes representing the same entity - neuroanatomical structures. The atlases defining anatomical structures (whether they are functionally, histologically, genetically, procedurally, multi-modally, or randomly defined) are the link between structural connectivity and functional connectivity measurements of the brain. To study SFC, we must rely on the neuroanatomical structures defined by an atlas, then localize electrodes to these regions and correlate the structural measurements (e.g., streamlines, fractional anisotropy, mean diffusivity) with functional measurements (e.g., cross-correlation, coherence, mutual information). Fundamentally, we are defining the nodes of the brain in advance, which can alter our results; a more comprehensive discussion on defining the nodes of the brain are in Fornito et al., 2016 and Bijsterbosh et al., 2017 Bijsterbosch et al. (2017); Fornito et al. (2016).

3.8. Conclusion

The publication of atlases and their distribution across neuroimaging software platforms has risen exponentially over the last three decades. Our study illustrates the critical need to evaluate the reproducibility of neuroscience research using atlases published alongside tools and analysis pipelines already established in the neuroscience community (e.g., FreeSurfer, DSI studio, FSL, SPM, QSIprep, fMRIprep, MRIcron, ANTs, and others).

4. Materials and methods

4.1. Human dataset

MRI data was collected from forty-one individuals, including thirteen healthy controls and twenty-eight drug-resistant epilepsy patients at the Hospital of the University of Pennsylvania. Twenty-four patients underwent stereoelectroencephalography (SEEG) implantation and four underwent electrocorticography (ECoG) implantation. Ten of the SEEG patients had clinically annotated seizures and were used for SFC analyses. Inclusion criteria consisted of all individuals who agreed to participate in our research scanning protocol, and (if they had implantations) allowed their de-identified intracranial EEG (iEEG) data to be publicly available for research purposes on the International Epilepsy Electrophysiology Portal (https://www.ieeg.org) Kini et al. (2016); Wagenaar et al. (2013). Seizure evaluation was determined via comprehensive clinical assessment, which included multimodal imaging, scalp and intracranial video-EEG monitoring, and neuropsychological testing. This study was approved by the Institutional Review Board of the University of Pennsylvania, and all subjects provided written informed consent prior to participating. See Table S2 for subject demographics.

4.2. Structure

Methods and pipelines for structural connectivity generation and analysis are described in the following sections. Specific GitHub files and code are included where applicable.

4.3. Imaging protocol

Prior to electrode implantation, MRI data were collected on a 3T Siemens Magnetom Trio scanner using a 32-channel phased-array head coil. High-resolution anatomical images were acquired using a magnetization prepared rapid gradient echo (MPRAGE) T1-weighted sequence (repetition time = 1810 ms, echo time = 3.51 m, flip angle = 9, field of view = 240 mm, resolution = 0.94×0.94×1.0 mm³ ). High Angular Resolution Diffusion Imaging (HARDI) was acquired with a single-shot EPI multi-shell diffusion-weighted imaging (DWI) sequence (116 diffusion sampling directions, b-values of 0, 300, 700, and 2000 s/mm², resolution = 2.5×2.5×2.5 mm³, field of view = 240 mm). Following electrode implantation, spiral CT images (Siemens) were obtained clinically for the purposes of electrode localization. Both bone and tissue windows were obtained (120 kV, 300 mA, axial slice thickness = 1.0 mm)

4.4. Diffusion weighted imaging (DWI) preprocessing

HARDI images were subject to the preprocessing pipeline, QSIPrep, to ensure reproducibility and implementation of the best practices for processing of diffusion images (Cieslak et al., 2021). Briefly, QSIPrep performs advanced reconstruction and tractography methods in curated workflows using tools from leading software packages, including FSL, ANTs, and DSI Studio with input data specified in the Brain Imaging Data Structure (BIDS) layout.

4.5. Structural network generation

DSI-Studio (http://dsi-studio.labsolver.org, version: December 2020) was used to reconstruct the orientation density functions within each voxel using generalized q-sample imaging with a diffusion sampling length ratio of 1.25 (Fang-Cheng et al., 2010). Deterministic whole-brain fiber tracking was performed using an angular threshold of 35 degrees, step size of 1 mm, and quantitative anisotropy threshold based on Otsu’s threshold (Otsu, 1979). Tracks with length shorter than 10 mm or longer than 800 mm were discarded, and a total of 1,000,000 tracts were generated per brain. Deterministic tractography was chosen based upon prior work indicating that deterministic tractography generates fewer false positive connections than probabilistic approaches, and that network-based estimations are substantially less accurate when false positives are introduced into the network compared with false negatives (Zalesky et al., 2010). To calculate structural connectivity, atlases listed in Table 1 were used. Structural networks were generated by computing the number of streamlines passing through each pair of structural regions in each specific atlas. Streamline counts were log-transformed and normalized to the maximum streamline count, as is common in prior studies (Bonilha et al., 2015; Park et al., 2017; Taylor et al., 2018; Wirsich et al., 2016). GitHub: packages/imaging/tractography/tractography.py

4.6. Atlases

Atlas descriptions and sources used in this study are found in Table S1. The 55 atlases used are listed explicitly in the reporting of effect sizes in Fig. 7 d. All atlases were sourced in MNI space and if not already, resliced to dimensions 182×218×182. Atlases were linear and non-linear registered to T1w subject space using the ICBM 2009c Nonlinear Asymmetric template (Fonov et al., 2011) and FSL flirt and fnirt(Jenkinson et al., 2012).

We also included three atlases registered using surface-based approaches. These atlases (the DKT, DK, and Destrieux atlases) are output from FreeSurfer’s recon-all pipeline(Dale et al., 1999). Many neuroimaging studies and software use volumetric approaches for registration(Coalson et al., 2018), yet surface-based approaches may yield more accurate labeling of the cortical surface (Fig. S6). The DKT40 atlas referred in this study is the surface version, while the DKT31 OASIS is the publicly available volumetric version (see Table S1).

In addition to published standard atlases above, we used whole-brain random atlases. A limitation of standard atlases is that they may not have anatomical definitions for all regions of the brain, and therefore, implanted electrodes may not be assigned properly to a region. This limitation was the impetus of our study (i.e., selecting an appropriate atlas for SEEG electrode localization and quantifying SFC). Whole-brain random atlases, in contrast, provide coverage to all implanted electrodes. They allow for the ability to change some morphological properties (i.e. parcellation size), while keeping other morphologies the same (e.g., parcellation shape; Fig. 2 d). However, a limitation of random atlases is that their regions may not represent true anatomical or functional boundaries. Random atlases were built in the ICBM 2009c Nonlinear Asymmetric template space and covered all voxels, excluding those labeled as CSF or outside the brain. To fill these points, a pseudo grassfire algorithm was applied (Zalesky et al., 2010). Briefly, N points representing the number of parcels of the atlas were randomly chosen as seed points. These seed points were iteratively expanded in all six Cartesian directions until all points were covered by one of the initial N seeds. After each iterative step, the smallest volume region expanded first. Random atlases created were of N equal to 10, 30, 50, 75, 100, 200, 300, 400, 500, 750, 1000, 2000, 5000, and 10,000 parcels. Five permutations for each N were created. GitHub code to generate random atlases: packages/imaging/randomAtlas/randomAtlasGeneration.py

4.7. Atlas morphology: volume and sphericity

Atlas morphological measurements included parcellation size (volume) and shape (sphericity) (Fig 2). Parcellation volume was calculated as the number of voxels in an parcel and log10 transformed. Parcellation sphericity was calculated as the ratio of the surface area of a sphere with an equal volume of the parcellation to the actual surface area of the atlas parcellation. Under this definition, sphericity is bounded from 0 to 1 where 1 is a perfect sphere. For reference, a perfect cube and a hemi-sphere have a sphericity of 0.8 and 0.7 respectively. GitHub: packages/imaging/regionMorphology/regionMorphology.py

4.8. Structural network measures

We characterized the structural network topology of 52 atlases (Fig 3 and Fig. S3). The three surface-based atlases (DKT40, DK, and Destrieux atlases output from the FreeSurfer recon-all pipeline(Dale et al., 1999)) were excluded from analyses of Fig 3 and Fig 3 because they were individually registered to each subjects’ T1w image. To quantify network topology, we examined density, mean degree, mean clustering coefficient, characteristic path length, and small worldness. Connectivity matrices were first binarized, using a threshold of 0, and a distance matrix was computed. The same binarization process and threshold was used across all atlases. The distance of any nodes that were disconnected from the main graph was set to the maximum distance between any pair of nodes in the main graph. Density, mean degree, clustering coefficient, and characteristic path length were then calculated on the binary, undirected graphs. Small worldness was calculated as the s-ratio where s = g/I and is the ratio of the average, normalized clustering coefficient, C, to the normalized characteristic path length, I. g = CG/CR and l = 1G/lR where G is the graph of interest and R represents a “random’ graph that is equivalent to G. To approximate the equivalent random graph R due to intractable computational costs (Maslov, 2002), a well-known analytical equivalent CR = d/N and IR = log N/log d were used, where d denotes average nodal degree. All network measures were calculated using the Brain Connectivity Toolbox for Python. GitHub: papers/brainAtlas/Script_05_structure_02_network_measures.py

4.9. Function

Methods and pipelines for functional connectivity generation and analysis are described in the following sections. Specific GitHub files and code are included where applicable.

4.10. Intracranial EEG acquisition

Stereotactic Depth Electrodes were implanted in patients based on clinical necessity. Continuous SEEG signals were obtained for the duration of each patient’s stay in the epilepsy monitoring unit. Intracranial data was recorded at either 512 or 1024 Hz for each patient. Seizure onset times were defined by the unequivocal onset (Litt et al., 2001). All annotations were verified and consistent with detailed clinical documentation. If a patient had more than one seizure annotated, the first seizure longer than 30 seconds without artifacts was used.

4.11. Electrode localization

In-house software (Azarion et al., 2014) was used to assist in localizing electrodes after registration of pre-implant and post-implant neuroimaging data. All electrode coordinates and labels were saved and matched with the electrode names on IEEG.org. All electrode localizations were verified by a board-certified neuroradiologist (J.S.). Electrode contact assignment to atlas region assignment was performed by rounding electrode coordinates (x,y,z) to the nearest voxel and indexing the given atlas at that voxel in the same space as the patient’s T1w image. Electrodes that fell outside the atlas of interest were excluded from subsequent analysis. Please see Fig. S10 for visualization. We also show the percentage of contacts assigned a region given an atlas (Fig. S7) GitHub: packages/atlasLocalization/atlasLocalization.py

4.12. Functional connectivity network generation

Functional connectivity networks were generated from four periods: interictal, preictal, ictal, and postictal. (1) The interictal period consisted of the time approximately 6 hours before the ictal period. (2) The preictal period consisted of the time immediately before the ictal period. (3) The ictal period consisted of the time between the seizure unequivocal onset and seizure termination. (4) The postictal period consisted of the time immediately after the ictal period. Interictal, preictal, and postictal periods were 180 seconds in duration. Following removal of artifact-ridden electrodes, SEEG signals inside either GM or WM for each period were common-average referenced to reduce potential sources of correlated noise (Ludwig et al., 2009). Next, each period was divided into 2 s time windows with 1 s overlap (Khambhati et al., 2017; 2016; 2015; Kramer et al., 2010). To generate a functional network representing broadband functional interactions between SEEG signals (Fig. 4 b), we carried out a method described in detail previously (Khambhati et al., 2016; Shah et al., 2019). Namely, signals were notch-filtered at 60 Hz to remove power line noise, low-pass and high-pass filtered at 127 Hz and 1 Hz to account for noise and drift, and pre-whitened using a first-order autoregressive model to account for slow dynamics. Functional networks were then generated by applying a normalized cross correlation function ρ between the signals of each pair of electrodes within each time window, using the formula:

$${\rho }_{xy}=\underset{\tau }{\max }\left[\frac{1}{T}\sum _{t=1}^T\frac{\left[x_k\left(t\right)-{\overline{x}}_k\right]\ast \left[y_k\left(t+\tau \right)-{\overline{y}}_k\right]}{{\sigma }_{x_k}{\sigma }_{y_k}}\right]$$

where x and y are signals from two electrodes, k is the 2 s time window, t is one of the T samples during the time window, and τ is the time lag between signals, with a maximum lag of 0.5 s. Here, σ represents the standard deviation of the signal. Note that functional connectivity measurements were also calculated for coherence and zero time-lag Pearson and Spearman rank correlations with associated p-values in defined frequency bands reviewed in Newson and Thiagarajan 2019 (Newson and Thiagarajan, 2019), but were not analyzed or used in hypothesis testing in the study. For data, available data, please see “Data availability and Reproducibility” section below. Networks are represented as fully-weighted connectivity matrices. GitHub Code: GitHub:code/tools/echobase.py

4.13. Structure–function correlation

To quantify the relationship between structure and function in the epileptic brain, we computed the Spearman rank correlation coefficient between the edges of the structural connectivity network and the edges of the functional connectivity networks (Fig. 4 c). To avoid redundancy given the symmetric nature of the matrices, only the upper triangle was analyzed. In brief, the structural connectivity network, representing normalized streamline counts between each atlas region, was first down sampled to only include regions that contained at least one SEEG contact Fig. S10. This gave one static representation of structural connectivity. In the case where multiple electrodes fell in the same atlas region, a random electrode was selected to represent the functional activity of that neuroanatomically defined region. Next, for every time-window of the functional network, the functional network edges were correlated with the down sampled, static structural network edges. This resulted in a structure–function correlation time series. Note that atlases with very small region volumes included more electrodes for SFC calculation. Electrodes that did not localize to an atlas were excluded from analysis. To average the SFC for all patients and each atlas (Fig 5), SFC time-series was resampled to 100 s for each period and each sample was averaged together. GitHub code: packages/eeg/echobase/echobase.py

4.14. Rssfc and Delta SFC

Resting-state SFC (rsSFC) was defined as the SFC during the interictal period, approximately 6 hours before the ictal period. The mean SFC of that period was computed. Delta SFC was defined as the change in the mean SFC from the preictal to the ictal period (Fig 5 top left panel). rsSFC and Delta SFC was calculated for each atlas (Fig 6).

4.15. Statistics

Preictal and ictal SFC for each atlas were compared using effect sizes across the 55 atlases shown in Fig. 6 d. Cohen’s d and the difference between preictal and ictal SFC was calculated.

4.16. Data availability and reproducibility

All code files used in this manuscript are available at https://github.com/andyrevell/revellLab. All de-identified raw and processed data (except for patient MRI imaging) are available for download by following the links on the GitHub. Raw imaging data is available upon reasonable request from Principal Investigator K.A.D. iEEG snippets used specifically in this manuscript are also available, while full iEEG recordings are publicly available at https://www.ieeg.org. The Python environment for the exact packages and versions used in this study in contained in the environment directory within the GitHub. The QSIPrep docker container was used for DWI preprocessing.