Connectomic consistency: a systematic stability analysis of structural and functional connectivity

Abstract

Objective.

Connectomics, the study of brain connectivity, has become an indispensable tool in neuroscientific research as it provides insights into brain organization. Connectomes are generated using different modalities such as diffusion MRI to capture structural organization of the brain or functional MRI to elaborate brain’s functional organization. Understanding links between structural and functional organizations is crucial in explaining how observed behavior emerges from the underlying neurobiological mechanisms. Many studies have investigated how these two organizations relate to each other; however, we still lack a comparative understanding on how much variation should be expected in the two modalities, both between people and within a single person across scans.

Approach.

In this study, we systematically analyzed the consistency of connectomes, that is the similarity between connectomes in terms of individual connections between brain regions and in terms of overall network topology. We present a comprehensive study of consistency in connectomes for a single subject examined longitudinally and across a large cohort of subjects cross-sectionally, in structure and function separately. Within structural connectomes, we compared connectomes generated by different tracking algorithms, parcellations, edge weighting schemes, and edge pruning techniques. In functional connectomes, we compared full, positive, and negative connectivity separately along with thresholding of weak edges. We evaluated consistency using correlation (incorporating information at the level of individual edges) and graph matching accuracy (evaluating connectivity at the level of network topology). We also examined the consistency of connectomes that are generated using different communication schemes.

Main results.

Our results demonstrate varying degrees of consistency for the two modalities, with structural connectomes showing higher consistency than functional connectomes. Moreover, we observed a wide variation in consistency depending on how connectomes are generated.

Significance.

Our study sets a reference point for consistency of connectome types, which is especially important for structure-function coupling studies in evaluating mismatches between modalities.

1. Introduction

Within the last two decades, connectomics, the study of brain connectivity, has emerged as an effective way of analyzing structural and functional connectivity of the brain where the brain network is modeled as a graph with nodes representing the brain regions and weighted edges representing the strength of connectivity between region pairs [60]. Structural connectivity is obtained using fiber tracking of diffusion MRI (dMRI) to estimate the macroscopic axon fiber bundles that connect pairs of brain regions [25]. Functional connectivity of the brain, on the other hand, is generally calculated as the temporal correlation between the activation of brain regions, which is commonly obtained using resting state functional MRI (rs-fMRI) [19]. Understanding the relationships between structural and functional connectivity of the brain is essential to explain how human behavior emerges from the neurobiological organism [36].

Analysis of connectomes have brought to light several significant neuroscientific findings regarding the network organization of the structural [67] and functional [22] connectivity of the brain, as well as how the two organization types relate to each other [42]. Compounded with the fact that both dMRI and resting state-fMRI (rs-fMRI) have their own inherent limitations [9, 53], such as noise and the indirect relation between the measurements and the ensuing model [3], connectomes of both modalities were shown to have significant differences depending on how they are generated [32, 59]. In generation of structural connectomes from dMRI data, for example, the two commonly used tracking approaches (deterministic or probabilistic tracking) are known to result in connectomes of varying densities and organizations [7]. Since such differences might potentially be a source of conflicting results in connectomics studies affecting interpretability of research outcomes, a characterization (quantification and evaluation) of variation in both structural and functional connectivities, is necessary.

Such an analysis is especially important for structure-function coupling analysis, which assumes that functional activations emerge from the structural pathways connecting brain regions [26]. Although functional connectivity in the brain is shown to be partially predictable by structural connectivity [21, 42], for example, the contribution of connectivity variations inherent to the modalities on limiting structure-function coupling is still unclear. Connectivity variation across samples within and between modalities also poses a challenge in the investigation of certain diseases such as traumatic brain injury (TBI), which is heterogeneous in terms of mechanism and pathology (mild/moderate/severe TBI), and manifest itself primarily on structure with ensuing outcomes on function [31]. In the recovery phase of the mild TBI, for example, brain function is completely recovered in many cases while structural recovery is not fully achieved [15]. Consequently, a population study in recovery of mild TBI would benefit from an analysis of consistency variation to properly evaluate the disparity in the recovery of structural and functional connectivity [41].

We define this concept of variation across connectomes as consistency. When evaluated for a single subject longitudinally, consistency elaborates on the change of the brain’s network organization over time. On the other hand, when evaluated across subjects cross-sectionally, it reveals the extent of commonalities in the brain’s network topology among people. Recent years have seen several studies evaluating the connectomic consistency directly [37, 43], or indirectly while investigating test-retest reliability [72] or connectomic fingerprinting [27]. Although consistency has been evaluated for structure [44] and function [24, 37] of the brain separately, evaluation of consistency in the two modalities on the same dataset is still lacking. In order to alleviate the need for a comprehensive assessment of variation in connectomes as a result of various factors, we present an extensive consistency analysis of structural and resting state functional connectomes that are derived from clinical dMRI and rs-fMRI scans.

In our analysis, we evaluated consistency of connectomes both for a single subject examined longitudinally and across a large cohort of subjects cross-sectionally. We assessed consistency in both structural and functional connectomes across three parcellation schemes and various thresholds for pruning weak edges. In further analysis of structural consistency, we compared structural connectomes generated by probabilistic and deterministic tracking algorithms through various edge weighting schemes. In investigating consistency for functional connectomes further, we evaluated positive, negative, and whole functional connectivities separately. We evaluated consistency both at the levels of individual edges using correlation and at the level of network topology via graph matching accuracy. We also examined consistency of structural connectomes that are generated using shortest path and weighted communicability, two commonly applied communication schemes in the study of brain network organization.

2. Data and methods

2.1. Datasets

In this study, the publicly available MyConnectome [47] and Philadelphia Neurodevelopmental Cohort (PNC) [51] datasets were used for evaluating the consistency of structural and resting state functional brain connectivity data, where the first dataset facilitated investigating within-subject consistency across time while the second dataset provided insights on the between-subject consistency cross-sectionally.

MyConnectome data contains 106 sessions of diffusion weighted MRI (dMRI), resting-state functional MRI (rs-fMRI), and task fMRI scans of a 45 years old healthy male recorded within an 18 month period. In this study, data from 16 sessions acquired within a 5 month period with both dMRI and rs-fMRI data acquired on the same day, were used. All of the structural and functional imaging data used in this study was acquired on the same Siemens 3 T Skyra scanner using a 32-channel head coil. A single anatomical T1-weighted image was acquired using an MP-RAGE sequence with TR/TE = 2400/2.14 ms at an isotropic 0.8 mm resolution. The rsfMRI data were collected using a multi-band Stejskal-Tanner EPI sequence (TR/TE = 1160/30 ms, flip angle = 63 degrees) with a voxel size of 2.4 2.4 2.0 mm across 512 volumes obtained in a 10 min scan. The dMRI data were acquired using a multi-band Stejskal-Tanner EPI sequence, with opposite gradient readout options (L > R, and L < R) in order to enable distortion correction. The data were acquired at high resolution of 1.7 mm isotropic voxel size with a TR/TE = 5000/108 ms, in 60 directions with two shells (b = 1000 s mm⁻² and 2000 s mm⁻²) of 30 directions each and four b = 0 volumes.

The PNC dataset consists of dMRI and rs-fMRI scans of 1445 unique subjects aged 8–22 years. For our analyses, participants with poor structural and functional imaging data quality or a history that suggested potential abnormalities of brain development were excluded. In this study, the rs-fMRI and dMRI images of 812 healthy subjects aged 8–22 years (mean = 15.64, SD = 3.28 years), including 341 males and 471 females, were considered. All data were acquired on the same Siemens 3 T Verio scanner, using a 32-channel head coil. For each subject, a T1-weighted anatomical scan was acquired, along with the rs-fMRI and dMRI scans. The T1-weighted MP-RAGE sequence was acquired at a resolution of 1 mm isotropic voxel size with a TR/TE = 1810/3.5 ms. The rs-fMRI data were acquired with 3 mm isotropic voxels at a TR/TE = 3000/32 ms, for 124 volumes where the scan time was 6 min 18 s. The dMRI data were acquired with a 2 mm isotropic voxel size, at a TR/TE = 8100/82 ms in 64 diffusion directions with a b = 1000 s mm⁻² and seven b = 0 volumes.

2.2. Preprocessing

2.2.1. T1 preprocessing

Every T1 image in the PNC dataset and the single anatomical T1 image from the MyConnectome Project underwent bias correction using the N4BiasCorrection tool from ANTs [65] followed by skull stripping using Muse [16].

2.2.2. Resting state—functional MRI preprocessing

In order to calculate functional connectomes, both the PNC and MyConnectome rs-fMRI data were first preprocessed for motion correction [71]. The first 6 volumes of the motion-corrected data were subsequently removed, and the timeseries were slice-time corrected and co-registered to their corresponding processed anatomical MP-RAGE image. Spikes in intensity across volumes were then estimated, volumes with excessive spiking were excluded from further analysis, and signals obtained from voxels containing non-gray matter tissue were regressed out.

2.2.3. Diffusion MRI preprocessing

To calculate the structural connectomes, the dMRI data from both datasets were denoised [34], corrected for motion and eddy current distortion [2], bias-corrected [65], and finally skull stripped. MyConnectome data were further corrected for EPI distortion as the data were acquired with opposite gradient readout options [55]. A standard tensor model was then fit to the single-shell PNC data, and the b = 1000 shell of the MyConnectome data using DIPY to calculate fractional anisotropy (FA) maps, which were used for registrations [20].

2.3. Atlases and registrations

In order to evaluate the effect of parcellation on connectomic consistency, structural and functional connectomes were calculated at three parcellation scales of the Schaefer atlas with additional 18 subcortical regions [52]. The three parcellations, here-after referred to as Schaefer 118, Schaefer 218, and Schaefer 318 based on the number of regions, were defined in MNI space and had to be registered to each subject.

For each dataset, the T1 image was used as the target of registration for both the labels defined in MNI space, and the data defined in dMRI space. Using a deformable registration from ANTs [4], we computed warps from template space (MNI) to a given subjects’ T1 image. Additionally, registrations were computed between a subject’s T1 image and the processed FA map, using two pairs of fixed and moving images (T1-FA, and T1-B0). It is important to note that in the MyConnectome dataset, having only a single anatomical T1 image, all 16 dMRI sessions were registered to this T1, which was also the target of registration from MNI space. Using this registration pipeline, labels defined in MNI space were brought to diffusion and anatomical T1 spaces.

2.4. Connectomes

2.4.1. Resting-state functional connectomes

After rigid registration of the preprocessed BOLD data to the T1 [30], timeseries were extracted using the atlases defined in the T1 space. Pearson’s correlation was then calculated between timeseries of each pair of ROIs and Fisher’s z-transform was applied to the resulting matrices to obtain functional connectomes.

2.4.2. Structural connectomes

In order to evaluate the impact of different tracking algorithms on connectomic consistency, deterministic (SD_STREAM) and probabilistic (IFOD-2) tractography were separately used to calculate connectomes [62, 63]. Both methods were run using Anatomically Constrained Tractography (ACT) [56] in mrtrix3, using both available b-value shells from the MyConnectome dataset, and the single-shell acquisition of the PNC cohort. The tractography options (min/max tract lengths = 5/400 mm, step size = 1, angle = 60°, and streamline count = 10 M) were identical for both tracking methods. Seeding for tractography across both algorithms was performed at the gray matter-white matter interface. Weights were then calculated for each streamline using SIFT2 [58], in order to ensure that connectivity information is reflective of the underlying biology, and not a byproduct of reconstruction inadequacies during tractography. Base structural connectomes were generated by designating the connectivity between pairs of ROIs as the total number of streamlines using the streamline weights as calculated using the SIFT2 method. To evaluate the effect of scaling on consistency, a common practice in connectomics for determining strength of pairwise relationships between regions [59, 74], two further connectomes were derived from the base connectome by i) log scaling the streamline counts and ii) scaling each streamline count by the combined volume of its ROI pair.

2.5. Consistency measures

Connectomic consistency is calculated as a single score for a dataset reflecting the variation of connectivity across samples, where the dataset might consist of connectomes of the same person acquired across time or of different people. In order to calculate consistency of a set of connectomes, we first calculated the average similarity of each connectome relative to the rest of the dataset. We then considered the mean of these similarity scores to quantify connectomic consistency, where higher values indicate that the generated connectomes are consistent across the dataset.

Since connectomic consistency is inherently contingent upon the choice of similarity measure, we evaluated similarity using two measures that are used in state of the art connectomic analysis, which highlight different aspects of connectivity in a brain network. We first considered Pearson’s correlation (denoted r in the text), which is commonly used to assess similarity between connectomes [1, 25]. For each subject, we obtained a vector representation by using the values in the upper triangle of connectomes. We then used Pearson’s correlation between these vectors as similarity between connectomes, which is a value in [−1,1] with larger values indicating higher similarity.

As an alternative to correlation as a measure of consistency, we considered matching accuracy (denoted MA in the text) which we recently proposed as a similarity measure for connectomes [42]. Matching accuracy is a measure driven from the solution to the graph matching problem, where the goal of matching is to find a mapping between the nodes of two given graphs by coupling nodes which resemble one another, while considering the overall connectivity structure of the network [40]. In order to achieve this goal, we formulated graph matching as the following combinatorial optimization problem which is known as the linear assignment problem: given two sets of nodes A and B, and a cost function $c:A\times B\to ℝ$ determining the cost of assigning each node in A to a corresponding node in B, the aim is to calculate a one-to-one mapping f : A → B between the nodes of the two sets by minimizing $\sum _{a\in A}c(a,f(a))$. In the context of connectomics, we regarded the assignment cost between nodes as the Euclidean distance between the k-dimensional feature vectors of nodes which encode the connectivity of a node relative to the rest of the nodes in a parcellation with k ROIs. We solved the optimization problem by using the Hungarian algorithm [33]. In the resulting mapping, we regarded the percentage of nodes that were correctly mapped to their counterparts as the matching accuracy between the two connectomes, which is a value in [0,100] with larger values indicating higher similarity.

Although the two measures quantify the consistency of the networks encoded across connectomes, the two measures differ in their utilization of the network structure. Pearson’s correlation can be considered to be a measure that is sensitive to local similarity, as it compares a narrow window of corresponding edges between two connectomes while ignoring the topology of the network. On the other hand, graph matching accuracy quantifies the similarity of the network topology of two connectomes by considering the connectivity signatures of all regions while determining the optimal mapping between their nodes. Briefly, Pearson’s correlation quantifies the consistency of connections locally, whereas graph matching accuracy quantifies the consistency of the network topology.

2.6. Experimental setup

In our analysis, we carried out a comprehensive set of experiments that can broadly be categorized into four groups.

1. In order to obtain a general view allowing a comparison between modalities, we first made an overall consistency evaluation of structural and functional connectomes. In our analysis of structural connectomes, we considered deterministic and probabilistic connectomes separately, while in functional connectomes, we evaluated the full functional connectome, as well as positive and negative functional connectivity by removing all negative and positive edges, respectively.

2. We then evaluated the consistency of structural connectomes that are obtained through two common post-processing steps of (i) scaling edges [59, 74] and (ii) calculating ‘traffic patterns’ [21]. Among edge scaling schemes, we compared unscaled connectomes to those scaled by the log function and by node volume. As studies investigating structure-function coupling in the brain mainly rely on connectivity maps that are obtained by applying certain traffic schemes over the structural connectivity of brain regions, we compared the consistency of connectomes obtained through two prominent communication schemes to that of direct connectivity between regions. Being the most commonly applied traffic pattern in brain studies, the first communication scheme we evaluated was the weighted shortest path [26], which assumes that communication in the brain occurs through the shortest path between node pairs. To complement this scheme, we also considered weighted communicability [17, 42] which assumes that communication occurs not only through the shortest path, but additionally through suboptimal routes.

3. We further evaluated the consistency of structural connectomes after removing the weakest edges to achieve various target network densities, a commonly applied post-processing method over structural connectomes to remove spurious fibers generated by tracking algorithms [13, 69].

4. Finally, we evaluated the consistency of positive, negative, and full functional connectomes after removing edges lower than various thresholds, which is a common method to remove noise in functional connectivity maps [68].

Throughout our analyses, we investigated connectomic consistency at three parcellations of the Schaefer atlas that we expanded with 18 subcortical ROIs (118, 218, and 318 ROIs), over MyConnectome and PNC datasets, using both correlation and matching accuracy as the connectomic consistency measures.

3. Results

3.1. Comparison of consistency in structural and functional connectomes

In order to get an overall view of connectomic consistency, we first compared the consistency of structure and function on two datasets using the two similarity measures (figure 1). Our results highlighted a significantly higher consistency in structure relative to function across all testing conditions. Focusing on consistency across datasets, we observed that both structural and functional connectivity is more consistent in the longitudinal MyConnectome data (in which the same subject is repeatedly scanned) relative to the cross-sectional PNC data (where all subjects are different). We further observed a reduced consistency both in structure and function with the increase in resolution of parcellation. Comparing tracking methods, we observed that probabilistic connectomes demonstrate higher consistency relative to deterministic connectomes. In comparing functional connectivity types, we observed that positive functional connectivity has higher consistency than full functional connectivity, where the difference is larger over PNC dataset relative to MyConnectome dataset. We also observed that negative functional connectivity has the lowest consistency across all connectivity types with a large margin of difference. Finally, investigating the effect of similarity measures, we observed that both correlation and matching accuracy demonstrated the same consistency patterns in ordering of consistency scores across experiments. Additionally, we noted a variation in the magnitude of consistency differences across modalities with matching accuracy demonstrating a smaller difference of consistency across structural connectivity types and positive and full connectivity, relative to correlation. In consistency of negative functional connectivity, however, matching accuracy demonstrated a larger consistency difference relative to other connectivity types.

Figure 1.

Comparison of structural and functional connectomic consistency. Top row shows correlation-based consistency results which highlight stability at the level of individual connections, whereas the bottom row shows matching accuracy based consistency results highlighting the stability of the network topology. Left column demonstrates consistency of structure and function on the same subject over time, while the right column demonstrates the consistency across a large healthy cohort. (Error bars indicate standard deviation in consistency scores.)

3.2. Consistency of structural connectomes

Investigating the consistency of structural connectivity in further detail (figures 2 and 3), we observed that connectomes generated through probabilistic tracking were more consistent than those generated through deterministic tracking across all testing conditions. In line with the results of the previous experiment (section 3.1), we observed that consistency is lower at finer parcellations across both datasets, and higher across the MyConnectome dataset relative to the PNC dataset. Additionally, we noted that the standard deviation of consistency scores is larger in the PNC dataset relative to that of the MyConnectome dataset. Comparing consistency across communication patterns, we observed that connectomes obtained by applying the weighted communicability scheme over direct connectivity maps achieved highest consistency, followed by direct connections and finally shortest path. However, we noted that the effect of communication patterns on consistency was greater in the PNC dataset compared to that of the MyConnectome dataset. When investigating the effect of edge normalization, we observed that scaling of edges leads to a lower consistency in most of the cases when consistency is measured with correlation. On the other hand, we observed a greater consistency with scaling in matching accuracy-based consistency. Interestingly, log scaling achieved a lower consistency than scaling by node volume in all but shortest path connectomes when the consistency is measured by correlation, whereas the reverse was observed in matching-accuracy based consistency with log scaling achieving a higher consistency than scaling by node volume.

Figure 2.

Consistency of connections in structural connectivity maps of deterministic and probabilistic tracking as quantified by Pearson’s correlation. Consistency is evaluated for two communication patterns in addition to direct connectivity, across two scaling schemes along with no scaling, over the two datasets. (Error bars indicate standard deviation in consistency scores. See figure 3 for consistency of network topology for the same testing conditions.).

Figure 3.

Consistency of network topology in structural connectivity maps of deterministic and probabilistic tracking as quantified by matching accuracy. Consistency is evaluated for two communication patterns in addition to direct connectivity, across two scaling schemes along with no scaling, over the two datasets. (Error bars indicate standard deviation in consistency scores. See figure 2 for consistency of connections for the same testing conditions.)

3.3. Consistency of deterministic and probabilistic structural connectomes with thresholding

Furthering our investigation on structural connectomes, we next evaluated the effect of thresholding on consistency across a range of densities (figure 4), by removing lower weighted edges to set the nonzero edge density of the connectome at a certain level. In this experiment, we investigated direct connectivities (i.e. traffic patterns are not applied) without any scaling, over three parcellations, and across two datasets by using the two consistency measures. Noting that the mean density across connectomes were recorded as shown in table 1 before any thresholding, we did not observe any significant change in consistency until the thresholding removed the majority of the edges which resulted in reduced consistency. To give a specific example, noting that densities of connectomes of MyConnectome and PNC datasets were (85.37%, 91.38%) for probabilistic and (35.18%, 45.11%) for deterministic tracking over Schaefer 118 parcellation, we observed that the consistency did not show any significant change until the threshold of 10% density, which then rapidly reduced with decreasing density. We also observed that this critical threshold value was lower over finer parcellations, such as the Schaefer 318 parcellation where consistency did not change until the density was reduced below 5%. However, we noted that the structural consistency at the highly sparse density of 4% (e.g. with the probabilistic structural consistency scores of (r = 0.96, MA = 93.65%) on MyConnectome and (r = 0.81, MA = 85.74%) on PNC over Schaefer 118 parcellation) was still higher than the consistency of functional connectivity, (e.g. (r = 0.73, MA = 83.73%) on MyConnectome and (r = 0.41, MA = 34.85%) on PNC over Schaefer 118 parcellation). We further observed that, compared to correlation, matching accuracy based consistency was affected less by thresholding for higher densities. However, it demonstrated a steeper decay in consistency as the edges were removed beyond the critical threshold. We also note that there were no significant differences between the consistency patterns of probabilistic and deterministic connectomes across thresholds.

Figure 4.

Effect of density thresholding on consistency for deterministic and probabilistic structural connectomes. Top and bottom rows show consistency of connections (correlation) and network topology (matching accuracy), while the left and right columns contrast intra- (MyConnectome) and inter-subject(PNC) consistency, respectively.

Table 1.

Ratio of non-zero edges to all possible number of edges in deterministic and probabilistic structural connectomes across the two datasets at three parcellations before thresholding. Densities are shown in percentages.

	MyConnectome		PNC
	Probabilistic	Deterministic	Probabilistic	Deterministic
Schaefer 118	85.37	35.18	91.38	45.11
Schaefer 218	69.93	20.81	78.07	28.77
Schaefer 318	59.02	13.96	66.97	20.38

3.4. Consistency in functional connectomes

Finally, we detail our analysis on the consistency of functional connectomes (figure 5). In all test cases, we observed that full and positive connectomes achieved higher consistency relative to negative connectomes, with positive connectomes having a slightly higher accuracy than the full functional connectome. As in previous experiments, we observed that connectomes with coarser parcellations had higher consistency. We also observed that removing edges with weights smaller than a magnitude of 0.2 did not have a significant effect on the consistency of full and positive connectomes. However, it reduced the consistency of negative connectomes even for magnitudes as small as 0.05. We noted a steady decrease in consistency over full and positive connectomes, as well. Between the two datasets, we observed a lower consistency in all functional connectivity types in PNC. We also noted that when compared with correlation based consistency, matching accuracy based consistency reported a larger difference between the negative connectivity and the full and positive connectivities.

Figure 5.

Effect of thresholding on consistency for full, positive, and negative functional connectomes. Top and bottom rows show consistency of connections (correlation) and network topology (matching accuracy), while the left and right columns contrast intra- (MyConnectome) and inter-subject (PNC) consistency, respectively.

4. Discussion

4.1. Structural connectivity is more consistent than functional connectivity

Throughout our analyses, the one common result that clearly stands out is the higher consistency of structural connectomes relative to functional connectomes (figures 1–5), whether it is across subjects, or across scans of the same subject. Despite the fact that both dMRI and rs-fMRI modalities have their own inherent limitations [9, 53], structural connectivity is expected to demonstrate higher consistency than functional connectivity, perhaps due to the more dynamic nature of the latter [11]. Our comparative results indicate that the consistent structural network topology of the brain does not directly translate to a comparably consistent functional network structure.

Although consistency of both structural [44] and resting state functional [24, 27] connectivities in the human brain were previously reported within the context of test-retest reliability or connectomic fingerprinting, to the best of our knowledge, ours is the first study that contrasts the consistency of dMRI and rs-fMRI over the same experimental setup. This result is important in providing a reference point for connectomic studies, especially those that use only one of the modalities (for example, dMRI) and interpret their results based on the reported literature on the other modality (such as rs-fMRI). Thus, we recommend taking the difference of consistency between the dMRI and rs-fMRI connectomes into account, when making inferences across the modalities.

This result could further be interpreted as structural connectomes being more suitable for population based analysis, as it captures group differences and less so the individual variability. On the other hand, lesser consistency observed in function might be interpreted as its suitability for subject specific analysis, as it could be considered to capture individual variability better. This agrees with the conclusions of [1] on functional connectomes being a fingerprint of a subject.

4.2. Consistency differences have consequences in joint structure-function studies

The effect of the disparity in connectomic consistency across structural and resting state functional connectivity is especially acute in the structure-function coupling, where the goal is to find a mapping between the functional activations in the brain and the underlying structural pathways connecting regions [26]. In network neuroscience, the brain is modelled as an information processing network [6] such that the structural pathways connecting brain regions act as conduits through which the information flows to generate functional interactions among brain regions. In such a setup, indirect functional interactions between regions are thought to happen by information exchange through intermediate regions, which is assumed to follow a certain communication scheme. Over the last decade, several communication schemes were suggested as the traffic pattern of the brain including shortest path [66], path transitivity, search information [21], and communicability [17].

Despite several attempts at finding the communication scheme of the brain, structure-function coupling studies have reported imperfect correlations between the structural and functional connectomes (r≲0.5), indicating a misalignment between the structural connectivity and functional activations [21, 26]. One common explanation for this is the inability of the devised communication schemes in capturing the complex multisynaptic interactions in the brain [42, 61]. Results that we presented in this study might provide further insights into this problem. We showed that, although the structural connectomes have relatively small variation across the subjects, functional connectivities that are deemed to emerge from underlying structural pathways demonstrate a large discrepancy (figure 1). This might be indicative of limitations of the functional data captured during the rs-fMRI scan in describing the overall functional connectivity between brain regions.

We note that the connectomes that are used in majority of connectomic research literature based on direct connections, are relatively sparse connected graphs, such as the ones we used in our experiments. These had 15% to 90% edge densities, varying according to the tracking method and fineness of parcellation. On the other hand, communicability and shortest path based connectomes constitute complete (100% density) graphs, since they represent strength of connectivity between any two node pairs which might be established through direct or indirect connections. Our results demonstrated that communicability based connectomes achieve highest consistency, which is followed by direct connection and shortest path based connectomes (figures 2–3). This result indicates that communicability based connectomes, which foresees communication occurring through multiple suboptimal paths in addition to the optimal shortest path, provides a connectivity map that has more commonalities across subjects, relative to those that are based on the shortest path, which restricts communication to occur exclusively through the optimal path. Although the shortest path has been the most commonly adopted scheme to model the communication in the brain in connectomics literature, we recently showed that weighted communicability describes the functional connectivity of brain regions better than other known communication patterns [42]. Our previous study further demonstrated that direct connections are better than shortest path in explaining the functional connectivity of the brain. Thus, our results in this study demonstrating the same ordering among these three connectivity schemes, add another dimension to the choice of communication pattern in structure-function coupling.

4.3. Consistency within a person and across people are different

Connectomes have been shown to have aspects unique to the person leading to the emergence of the concept of connectome fingerprinting [35], which suggests the identifiability of a human by their brain connectivity pattern. This has been evaluated on structural [39] and functional [18, 45] connectomes, separately. These results have demonstrated that two connectomes of a subject obtained from scans at different time points resemble each other more than they resemble connectomes of other subjects. Our results demonstrate a near perfect consistency for the structural connectivity and a very high consistency for the functional connectivity on a subset of the MyConnectome dataset, which consists of repeated scans of the same subject within a 5 month period. This highlights the identity preserving aspect of the connectomic fingerprinting for a single subject (figure 1). Comparing our results on the MyConnectome dataset with that on the PNC dataset, we observed that consistency across different subjects is lower both in structure and function, as compared to within the same person across scans. This indicates the presence of unique connectivity patterns of subjects which highlights the identity differentiation ability of connectomes, leading to a connectomic fingerprinting across subjects. However, this latter result also points to a common connectivity backbone across subjects in both structure and function, as we still observe decent consistency levels in both modalities.

We observed a larger difference in functional than structural connectomic consistency between MyConnectome relative to PNC, which might be due to a combination of reasons. Firstly, the longer rs-fMRI acquisition time of the MyConnectome data, which is 1.5 times longer than that of the PNC data, might have provided a more complete picture of the functional connectivity of the subject. Secondly, compared to structure, functional connectivity might contain more aspects that are unique to the individual, that can be used as a connectivity fingerprint. Finally, lower consistency in functional connectivity across subjects might be due to the activation map being parcellated with a common atlas across all subjects when generating the connectomes. It was recently stated that functional parcel boundaries reconfigure with cognitive state [49], which may be an area to explore in the future.

4.4. Processing of the data affects consistency

With the exponential increase in the number of scientific publications [8], reproducibility of research findings has become a major problem [29]. Neuroimaging research and connectomics are rapidly growing fields of science, with the complicated methods to process and evaluate data [5]. Thus, they face unique reproducibility challenges [46, 75]. A major source of concern in connectomics in this regard, stems from the complicated preprocessing pipelines applied over raw imaging data (such as motion correction, denoising, and registration), involved methods to process curated data (such as tractography on structure and independent component analysis on function), and additional post-processing steps employed before obtaining finalized connectomes (such as thresholding or scaling of connectivity values) [32, 57].

In our analysis, we focused on four steps involved in processing and post-processing of neuroimaging data. Our results on comparison of the tracking methods demonstrate that connectomes obtained through probabilistic tracking are more consistent than the ones obtained by applying deterministic tracking, supporting previous findings (figures 2–3) [7, 59].

Although tractography has successfully been applied in noninvasively reconstructing the structural brain network on dMRI data, it is known to generate spurious fibers which introduce direct connections between two regions which would otherwise be connected indirectly [14]. Functional connectomes, on the other hand, are forced to have connections between every region pair by design since connectivity is calculated as the correlation between the timeseries of regions, potentially introducing false edges in the graph that would be interpreted as direct connections between regions. Since false connections are commonly assumed to be among the weak connections, thresholding of such edges has been a common practice to eliminate noise in the connectomic data. Our results, however, indicate that thresholding does not alter the consistency of structural connectomes significantly unless edges are heavily pruned (figure 4), supporting recent findings [13]. This implies that thresholding of weak edges in structural connectomes might not be necessary, especially if the analysis involves weighted edges. Moreover, it is important to note that weak edges might play an important role in performing normal brain function and therefore should not be removed. In thresholding of positive and whole functional connectomes, our results demonstrate that network topology is more consistent than individual connections, in response to removing lower weighted edges. Negative connectomes, in contrast, demonstrate a steady decline in consistency with thresholding (figure 5). This result might be indicative of the presence of a steady network with strong edge weights in the positive functional connectomes, while the negative functional connectome lacks such a consistent organization.

In analyzing the effect of coarseness of the parcellation on connectomic consistency, we observed that connectomes with lesser number of regions are more consistent within and across subjects, in both structure and function, in accordance with the literature (figures 1, 5) [10, 73]. This result highlights the importance of the spatial scale of nodal parcellation when making a comparison across studies. Another common post-processing step in generating structural connectomes is normalization of the edge weights, which might involve scaling the fiber counts between two regions with the logarithm function or with the total volume of the two regions. Our results indicate that edge normalization yields decreased consistency of individual connections (figure 2). This might be due to the sensitivity of correlation to outliers, as edge distribution of structural connectomes are known to follow power law with large amounts of weaker edges in contrast to very few strong edges. Since edge normalization removes such outliers by shrinking the range of edge weights, it consequently decreases correlation-based edge consistency. On the other hand, results shown in figure 3 demonstrates that normalization yields increased consistency of the network topology. This positive effect of scaling on matching accuracy-based consistency could be attributed to the fact that log function shrinks the range of edge weights, reducing the variability of connectivity structure across subjects. Thus, although such an effect might be desirable for population studies, it should be avoided in subject specific research.

4.5. Network topology is more consistent than individual connectivities

Human brain is known to have a structural and functional organization that is consistent across subjects. Although the consistency has been analyzed at the level of individual connections [24, 27] and network topology [44, 48] separately, a comparison of consistency between these two levels of analysis has been lacking. Our analysis addresses this need by evaluating consistency at both levels. In the analysis of structural connectomes, we observed a near perfect consistency of network topology with matching accuracy ranging in [99%, 100%], while individual connectivities were farther away from a perfect consistency score as correlation was in [0.85,0.95] range (figure 1). A similar pattern was also observed in the consistency of functional connectomes. This indicates that despite variations of individual connections, the overall brain organization is stable within and across subjects. This also demonstrates the effect of similarity measure choice in connectomic consistency analysis as results and ensuing interpretations would differ according to the chosen metric.

4.6. Negative functional connectivity has the lowest consistency among all connectivity types

Although it has been available for analysis to the neuroscientific community since the beginning of the rs-fMRI studies, the negative functional connectivity is very little understood and its mechanism in the context of network physiology has been a subject of debate. Some studies suggested that it could be an artifact of regression of global signals [38, 70] while others demonstrated that it mainly contains long-range connections which might suggest a biological basis [54]. Overall, negative functional connectivity is seldom evaluated in neuroscientific research [12].

In our analysis, we observed that negative functional connectivity has the lowest consistency among all connectivity types especially with the consistency of network topology being negligible in the PNC dataset (figures 1, 5). This can be considered as an evidence against the presence of a biological basis for negative functional connectivity. On the other hand, the small consistency that we observed on MyConnectome might imply the presence of an underlying biological mechanism. Although inconclusive, our results indicate that systematic longitudinal analyses on negative functional connectivity might provide insights into a possible mechanism behind this phenomenon.

We note that our analysis has certain data related limitations that are challenging to isolate from the study. Firstly, although both datasets are acquired on 3 T scanners, difference of acquisition parameters and protocols might have contributed to the consistency variation. Secondly, the age range of subjects in the PNC dataset being [8, 22] years might have contributed to the lower consistency that was observed, as the brain is known to have structural and functional changes during development [23, 50]. Similarly, PNC dataset containing subjects from both sexes might be another factor contributing to lower consistency, as structural and functional sex differences were reported in brain connectivity [28, 64]. In future, we will investigate connectomic consistency with regard to development (variation in age) and between sexes.

We also note that our study evaluates connectomic similarity measures that are currently being used by the community, in order to provide a reference point of connectomic consistency in structure as well as function. It may be worth pondering whether these measures are adequate for connectomic analysis in general, and whether these measures focus on the stronger edges, and as such are debatably biased. We leave this partially scientific and partially philosophical debate to the community as it is beyond the scope of our current study, and urge the community to look into measures that are more sensitive to weak edges in connectomic analysis.

5. Conclusion

In this study, we presented a comprehensive analysis of consistency across structural and functional connectomes. We showed that structural connectomes are more consistent relative to functional connectomes. In analysis of structural connectomes, we showed that probabilistic tracking yields more consistent structural connectomes relative to deterministic tracking. In a detailed analysis of functional connectivity types, we showed that negative functional connectomes demonstrate a drastically lower consistency. We also evaluated the effect of some of the connectome processing steps and demonstrated that (i) consistency is higher in coarser parcellations, (ii) aggressive thresholding of weaker edges affects consistency, and (iii) scaling of structural edge weights increases consistency of network topology. Evaluating a longitudinal dataset on a single person and a cross-sectional dataset of across a population, we showed that the connectomic consistency of a single subject across time is higher than the consistency across a set of subjects. These results broaden our understanding on the relationship between structure and function of the brain and set a reference point for researchers on connectomic consistency.