Community detection in weighted brain connectivity networks beyond the resolution limit

Abstract

Graph theory provides a powerful framework to investigate brain functional connectivity networks and their modular organization. However, most graph-based methods suffer from a fundamental resolution limit that may have affected previous studies and prevented detection of modules, or "communities", that are smaller than a specific scale. Surprise, a resolution-limit-free function rooted in discrete probability theory, has been recently introduced and applied to brain networks, revealing a wide size-distribution of functional modules (Nicolini and Bifone, 2016), in contrast with many previous reports. However, the use of Surprise is limited to binary networks, while brain networks are intrinsically weighted, reflecting a continuous distribution of connectivity strengths between different brain regions. Here, we propose Asymptotical Surprise, a continuous version of Surprise, for the study of weighted brain connectivity networks, and validate this approach in synthetic networks endowed with a ground-truth modular structure. We compare Asymptotical Surprise with leading community detection methods currently in use and show its superior sensitivity in the detection of small modules even in the presence of noise and intersubject variability such as those observed in fMRI data. We apply our novel approach to functional connectivity networks from resting state fMRI experiments, and demonstrate a heterogeneous modular organization, with a wide distribution of clusters spanning multiple scales. Finally, we discuss the implications of these findings for the identification of connector hubs, the brain regions responsible for the integration of the different network elements, showing that the improved resolution afforded by Asymptotical Surprise leads to a different classification compared to current methods.

Highlights

• •Methods to study modularity of brain connectivity networks have a resolution limit.
• •Asymptotical Surprise, a nearly resolution-limit-free method for weighted graphs, is proposed.
• •Improved sensitivity and specificity are demonstrated in model networks.
• •Resting state functional connectivity networks consist of heterogeneous modules.
• •Classification of hubs in function connectivity networks should be revised.

1. Introduction

The brain is thought to consist of a network of interconnected, interacting components whose architecture is critical for the emergence of adaptive behaviors and cognition (McIntosh, 2000). Graph theory provides a powerful means to assess topology and organization of brain connectivity networks, like those derived from MRI and other neuroimaging methods (Eguíluz et al., 2005, Bullmore and Sporns, 2009). Within this framework, the brain is represented as a network of n nodes interconnected by m links. Typically, the nodes correspond to anatomically defined brain regions and the links to a measure of interregional interaction or similarity (Bullmore and Sporns, 2009). For resting state functional connectivity networks, edge weights are defined as interregional temporal correlations in the fluctuations of the BOLD signals, and the resulting graph can be represented by a correlation adjacency matrix. The arcs of structural connectivity networks (the “connectome”), conversely, reflect the number of white matter tracts connecting any two regions. Brain networks have also been defined on the basis of intersubject anatomical covariance (Evans, 2013), co-activation of different brain regions across individuals subjected to experimental tasks (Crossley et al., 2013) or pharmacological challenges (Schwarz et al., 2007, Schwarz et al., 2008). All of these networks are “weighted” by definition, i.e. their edges are associated with real numbers representing a measure of the strength of pairwise interactions between nodes.

Graph-theoretical analysis of these networks has contributed substantially to our understanding of the topological organization of brain connectivity, revealing a small-world, rich-club structure (Eguíluz et al., 2005, van den Heuvel and Sporns, 2011) and the presence of hub regions characterized by high connectivity and network centrality. Additionally, a number of studies (reviewed in Bullmore and Sporns, 2009, van den Heuvel and Hulshoff Pol, 2010)) have investigated the modular structure of brain connectivity networks, highlighting cohesive clusters of nodes that are more densely connected among themselves than with the rest of the network. In the graph-theory jargon, these disjoint clusters are sometimes dubbed “communities”, remnant of early investigations in the field of social sciences (Girvan and Newman, 2002).

Topological modularity is thought to reflect functional and anatomical segregation, a feature that may confer robustness and adaptivity to brain networks. Moreover, the degree of clustering within functional connectivity graphs may provide a measure of the balance between segregation and integration underlying brain function (Bullmore and Sporns, 2009). Finally, the identification of modules and their boundaries is important to understand the topological function of hub regions within the network (Meunier et al., 2010). Indeed, hubs sharing a large number of within-module edges may be critical to determine segregation of sub-structures within the network, while hubs connecting different modules are responsible for network integration (van den Heuvel and Sporns, 2013). It has been suggested that hub regions may have increased susceptibility to the effects of brain disorders (Crossley et al., 2014, Buckner et al., 2009), and their correct identification and classification are important to understand their putative role in the spread and manifestation of brain disease. Finally, alterations in the community structure of the brain have been observed in several neuropsychiatric conditions, including Alzheimer disease (Tijms et al., 2013), schizophrenia (Stam, 2014) and chronic pain (Balenzuela et al., 2010), and assessment of the brain modular organization may provide a key to understanding the relation between aberrant connectivity and brain disease.

Following initial work by Hilgetag et al. (2000), several graph theoretical methods have been deployed to investigate the modular structure of brain networks (Meunier et al., 2010, Meunier et al., 2010, Power et al., 2011). Typically, these methods rely on the optimization of a fitness function that measures the quality of a network partition against that of an ensemble of randomized networks with similar statistical properties (the “null model”). Optimization of the fitness function of choice is often computationally demanding and scales steeply with increasing network size. Hence, heuristics are needed to calculate nearly optimal partitions of large networks, like those derived from neuroimaging data, within reasonable computation time (Blondel et al., 2008, Rosvall and Bergstrom, 2008).

A seminal finding in graph theory is that clustering methods based on optimization of a global function suffer from a resolution limit (Fortunato and Barthélemy, 2007), as they are unable to resolve modules that are smaller than a scale determined by the size of the entire network. This problem was first demonstrated for Newman's Modularity (Newman, 2006), a method included in the Brain Connectivity Toolbox (Rubinov and Sporns, 2010) and most frequently applied to the analysis of neuroimaging data.

Subsequent work by various groups has demonstrated that the resolution limit is quite pervasive and affects, to a different extent, many other methods based on optimization of a global fitness functions (Squartini et al., 2015, Traag et al., 2011, Lancichinetti and Fortunato, 2009), including Reichardt and Bornholdt, 2006, Arenas et al., 2008, Ronhovde and Nussinov, 2009, Rosvall and Bergstrom's (Infomap) (Rosvall and Bergstrom, 2008, Kawamoto and Rosvall, 2015) and others.

The introduction of a resolution parameter has been proposed as a means to mitigate the problem by adjusting the resolving power of the function to a specific scale (Reichardt and Bornholdt, 2006, Ronhovde and Nussinov, 2010, Thomas Yeo et al., 2011). However, this approach enables resolution of smaller clusters at the expense of larger ones, which may be unduly subdivided, thus resulting in partitions with relatively uniform cluster size distributions that do not capture the complex modular structure of real-world networks (Lancichinetti and Fortunato, 2011).

Recently, we have assessed the effects of the resolution limit on the analysis of brain connectivity networks (Nicolini and Bifone, 2016). Specifically, we have shown that this limitation severely curtails the ability to detect small, but functionally and anatomically meaningful clusters of nodes even when they present high densities of intra-cluster edges. Moreover, we showed that resolution-limited methods, like Newman's Modularity, do not reflect the multiple scales of the organization of brain connectivity networks, where small and large modules can coexist. We have also demonstrated that Surprise, a conceptually different fitness function grounded in probability theory, behaves like a resolution-limit-free function (Nicolini and Bifone, 2016). Maximization of Surprise, based on an algorithm dubbed FAGSO, revealed a heterogeneous distribution of modules within brain resting state and coactivation networks. If confirmed, these findings would suggest that a substantial revision of current models of brain modular structure may be in order.

A fundamental limitation of Surprise lies in its definition in terms of discrete probability and binomial coefficients that make it applicable only to binary networks, i.e. graphs with edge values 1 or 0. This may represent a substantial drawback, for it requires binarization of brain connectivity networks, thus discarding potentially important information contained in the edge weight distribution. Moreover, different binarization procedures may lead to different network representations for the same connectivity dataset. Therefore, an extension of Surprise to weighted networks would be highly desirable, and would provide a new and important tool to study the modular organization of brain connectivity beyond the resolution limit.

Capitalizing on recent development in the field of statistical physics of complex networks (Traag et al., 2015), here we describe and demonstrate the use of Asymptotical Surprise, a weighted counterpart to Surprise, in the study of the modular structure of weighted networks. Moreover, we propose a new algorithm, dubbed PACO (PArtitioning Cost Optimization) for the maximization of Asymptotical Surprise.

Since there is no ground-truth structure for brain functional connectivity networks, we have assessed the performance of this novel approach on synthetic networks with a planted modular structures, and compared it to some of the leading graph partitioning methods. Importantly, we demonstrate our approach in networks derived from synthetic data that mimic different structures, levels of noise and variability, such as those observed in functional connectivity experimental data. Indeed, improved resolution afforded by Asymptotical Surprise may imply increased vulnerability to spurious modules resulting from noisy correlations. It is therefore important to assess the benefits of increased resolution against the limitations arising from intrinsic data variability.

Finally, we apply Asymptotical Surprise to weighted functional connectivity networks from resting state fMRI data, revealing a heterogeneous, multiscale community structure. We show that the finer modular subdivision of resting state functional connectivity networks obtained by Asymptotical Surprise leads to substantial differences in the identification of connector hubs compared to other community detection methods.

2. Materials and methods

2.1. Notation

Here we briefly summarize the terminology and the notation that will be used throughout the paper. A binary graph $G=(V,E)$ is a representation of a set V of n nodes, also called vertices, connected by m links (or edges), in a set E. The adjacency matrix $A=\{a_{\mathit{ij}}\}$ of a binary graph is a square $n\times n$ symmetric matrix with elements $A_{\mathit{ij}}=1$ when an edge exists between vertex i and j and 0 otherwise. We denote the total number of possible links in the graph as $p=\left(\genfrac{}{}{0ex}{}{n}{2}\right)$.

A weighted graph $G=(V,E,W)$ assigns as a set of edge weights W to the links. For weighted graphs, the adjacency matrix is square, symmetrical and has real elements.

A clustering $\zeta =\{{\zeta }_c\}$ of G is a partitioning of V into disjoint sets of nodes, ${\zeta }_c\subseteq V$, which we call modules or communities. Each module consists of n_c nodes, m_c edges and $p_c=\left(\genfrac{}{}{0em}{}{n_c}{2}\right)$ pairs of nodes. On weighted graphs we define m_c as the sum of edge weights inside a community. The sum of edges internal to all communities, m_ζ and the intra-cluster pairs p_ζ are $m_{\zeta }={\sum }_c{m}_c$ and $p_{\zeta }={\sum }_c{p}_c$ respectively.

2.2. Surprise and asymptotical surprise

Surprise (Aldecoa and Marín, 2011, Aldecoa and Marín, 2013) is a quality measure of the partition of a binary network that has its roots in probability theory. For a given partition ζ, Surprise represents the probability that a graph drawn uniformly at random from the set of all graphs with n nodes, $p=\left(\genfrac{}{}{0em}{}{n}{2}\right)$ pairs and m edges has at least as many intra-cluster edges as G. Intuitively the lower the probability the better the partition.

For binary networks, Surprise can be computed within the discrete probability theory of urn models as

$$S=\sum _{i=m_{\zeta }}^m\frac{\left(\genfrac{}{}{0em}{}{p_{\zeta }}{i}\right)\left(\genfrac{}{}{0em}{}{p-p_{\zeta }}{m-i}\right)}{\left(\genfrac{}{}{0em}{}{p}{m}\right)}.$$ (1)

Due to numerical precision problems in the evaluation of large binomial coefficients, $\widehat{S}(\zeta )=-\log S(\zeta )$ is often taken as measure of quality of the partition, with higher values corresponding to better clustering.

Surprise quantifies the extent of the departure of the distribution of intra-cluster nodes and edges from that of a randomly drawn partition with the same internal density as in the original graph (Alba, 1973). In the limit of large networks, Surprise $\widehat{S}$ can be approximated by a binomial distribution: this observation led to weighted definition of Surprise, dubbed Asymptotical Surprise ${\mathcal{S}}_a$ (Traag et al., 2015):

$${\mathcal{S}}_a={\mathit{mD}}_{\mathrm{KL}}\left(q\parallel \langle q\rangle \right)$$ (2)

where, for brevity of notation, $q=m_{\zeta }/m$ and $\langle q\rangle =p_{\zeta }/p$ are the observed and expected fraction of intra-cluster links relatively and the binary Kullback-Leibler divergence (Kullback and Leibler, 1951) is

$$D_{\mathrm{KL}}(x\parallel |y)=x\log \left(\frac{x}{y}\right)+(1-x)\log \left(\frac{1-x}{1-y}\right).$$

In the framework of information theory (Cover and Thomas, 2006), Asymptotical Surprise represents the Kullback-Leibler divergence between the observed and expected fraction of intra-cluster edges; it encodes the information lost when the prior distribution $\langle q\rangle$ is used to approximate the posterior distribution q. Kullback-Leibler divergence is a quasi-distance on probability distributions as it is always non-negative, non-symmetric and zero only when $q=\langle q\rangle$, like binary Surprise.

Asymptotical Surprise has a simpler formulation than binary Surprise as there are no binomial coefficients to evaluate and it has been shown to be resolution-limit-free in the limit of large networks (Traag et al., 2015).

2.3. Maximization of asymptotical surprise

Finding the optimal partition of a graph is an NP-hard problem (Fortunato, 2010) and practical implementations of community detection rely on heuristic approaches that enable finding nearly-optimal solutions in a reasonable computation time.

Here we introduce a powerful and general method for the optimization of Asymptotical Surprise dubbed PACO (PArtitioning Cost Optimization). PACO is a non-deterministic agglomerative algorithm based on FAGSO and, like the Louvain method, has an element of randomness that enables a more efficient exploration of the partition landscape.

The operating principle of PACO is based on the triadic closure property, i.e. the fact that in real-world networks nodes with many common neighbors are more likely to be neighbors. This transitive neighborhood property underlies the formation of communities of nodes (Bianconi et al., 2014, Wang and Cui, 2015). In principle, any measure of structural similarity between nodes could guide a community detection heuristic toward the optimal partition. Specifically, PACO uses the Jaccard index (Jaccard, 1901), a measure of the fraction of overlap between the neighbors in common between nodes, as the guiding principle for the agglomeration of similar nodes in the same community.

In the first phase of PACO, the Jaccard metric is evaluated for every edge. More formally, for an edge $e=(u,v)$ the Jaccard index is computed as $J(e)=\frac{|\Gamma (u)\cap \Gamma (v)|}{|\Gamma (u)\cup \Gamma (v)|}$ where $\Gamma (u)$ and $\Gamma (v)$ are the neighboring nodes of u and v respectively.

The agglomerative process starts with an initial partition where every vertex represents a community on its own. This partition has n communities and no intra-cluster edges. The edges of the graph are then ranked in decreasing order by their Jaccard index and iteratively, for every edge in the sorted list, endpoint nodes are merged only if they belong to different communities. In this case one of the two endpoints, selected by chance, is assigned to the other's endpoint community and the increment of Surprise is computed: if it is positive, the partition is updated together with the new value of Surprise (or Asymptotical Surprise), otherwise the algorithm proceeds to the next edge. The running time of PACO is linear in the number of nodes and quadratic in the average degree of the network. Detailed pseudocode for PACO is shown in the Supplementary materials, Fig. S1.

The main difference between PACO and its predecessor FAGSO is the data structure used to store the community structure. FAGSO maintains the community structure in a disjoint-set data structure and when one vertex is moved into another's community, the two modules are merged into one (Supplementary materials, Fig. S2, S3). Conversely, PACO moves single nodes between different communities, and never merges modules (Supplementary materials, Fig. S3, boxes C, D). This results in a more finely-grained optimization that allows a better exploration of the quality function landscape.

2.4. Synthetic benchmark networks

Here we introduce a theoretically sound method for the generation of synthetic FC networks that mimic properties of resting state fMRI networks, including noise and intersubject variability, while presenting a pre-determined ground-truth modular structure against which the performance of community detection algorithms can be tested.

The general idea is that, starting from an adjacency matrix with a given modular structure, we can generate time-courses for each of the nodes whose pairwise correlations reproduce the edge structure of the original matrix. Noise can be added to the time-courses, and the resulting correlation matrix will provide a noisy representation of the original one. This procedure can be repeated multiple times to produce different datasets that represent different “subjects” in the study.

In practical terms, given an undirected weighted graph $\mathbf{C}\in {\mathbb{R}}^{n\times n}$ whose community structure is known a-priori, we have calculated its nearest positive definite matrix (Higham, 1988) and its Cholesky decomposition, i.e. an upper triangular matrix $\mathbf{L}\in {\mathbb{R}}^{n\times n}$ such that ${\mathbf{LL}}^T=\mathbf{C}$. Starting from uncorrelated variables $\mathbf{X}\in {\mathbb{R}}^{n\times l}$, we have generated correlated random variables $\mathbf{Y}=\mathbf{LX}$ such that $\mathbb{E}({\mathbf{YY}}^T)=\mathbf{C}$. Additionally, we have injected different levels of noise into $\mathbf{Y}$ prior to the computation of the correlation matrix. Schematic of this procedure is shown in Fig. 1.

Fig. 1.

Flowchart of the generation and analysis of the synthetic datasets. In A the network with a pre-defined community structure is generated. The adjacency matrix is then processed in block B to obtain the nearest positive definite matrix for the Cholesky decomposition. This enables the generation of node-wise time-courses into which different levels of noise can be injected. The procedure is repeated multiple times to generate different instances (mimicking different subjects in the sample). Finally, correlation matrices are calculated for each instance (block C), and Fisher transformed to calculate the average adjacency matrix for analysis by community detection algorithms (block D). Lastly, resulting partitions are compared with the original, planted one in terms of NMI.

We tested this idea on two different models of planted partition: a variant of the ring of cliques (Fortunato and Barthélemy, 2007) and the Lancichinetti-Fortunato-Radicchi (LFR) network (Lancichinetti et al., 2008) (Fig. 2, whose degree distribution and modular structure can be tuned to replicate topological features of real-world networks, including scale freeness (Hagmann et al., 2008) and the presence of densely interconnected cores (van den Heuvel and Sporns, 2011).

Fig. 2.

The two benchmark networks used in this study, laid out. (A) is a power-law ring of cliques, where cliques present different sizes sampled from a power-law distribution; (B) is the layout of an LFR network with parameters N=600, $\langle k\rangle =12$, ${\max }_k=50$, ${\mu }_t=0.1$, ${\mu }_w=0.1$, ${\min }_c=5$, ${\max }_c=50$. The layout of (B) was generated with the graph-tool library (Peixoto, 2014).

One important finding in Nicolini and Bifone (2016) is that brain networks are organized in modules with heterogeneous size distributions. We implemented this property in our two types of benchmark networks. For the first test, we generated a ring of cliques with 300 nodes, and sizes of the cliques sampled from a power-law with exponent ${\tau }_c=2$, minimum and maximum clique size respectively ${\min }_c=5$, ${\max }_c=50$ (see also Supplementary materials S4). For each subject of the sample, we synthesized 150 time-points for each node using the neuRosim R package (Welvaert et al., 2011). We set the baseline value of all the time series to 100 (Welvaert and Rosseel, 2013).

Finally, we correlated the original synthetic time series $\mathbf{X}$ by multiplication with the matrix $\mathbf{L}$, obtained the correlated time series $\mathbf{Y}$ and added Rician noise (Gudbjartsson and Patz, 1995) to $\mathbf{Y}$ independently for each area. The simulated data $\mathbf{Y}$ did not include slow drift components, simulated physiological noise, nor spatial noise. The average SNR was defined as $\mathrm{SNR}=\overline{S}/{\sigma }_N$ where $\overline{S}$ is the average magnitude of the signal and σ_N is the standard deviation of the noise (Krüger and Glover, 2001).

In order to be more exhaustive and extend the validity of results, we repeated the same procedure on weighted LFR networks with N=600 nodes, sampling nodes degree from a power-law with exponent ${\tau }_d=2$, average degree $\langle k\rangle =12$ and maximum degree ${\max }_k=50$. We set the topological and weights mixing coefficients, i.e. the average fraction of intra-cluster and inter-cluster degree and strengths, to ${\mu }_t=0.1$ and ${\mu }_w=0.1$, respectively. Planted community sizes ranged from 5 to 50 nodes and were sampled from a power law with exponent ${\tau }_c=1$. In the Supplementary information we have extended this analysis to a wider range of network parameters.

Group-level correlation matrices were computed by Fisher-transforming and averaging individual instances of the above matrices. Sparsification was obtained by removing edges with weights below the most stringent threshold that maintained the network connectedness, a procedure known as percolation analysis (Gallos et al., 2012, Bardella et al., 2016, Alexander-Bloch et al., 2010). This approach measures the size of the largest connected component of the network upon iterative removal of the weakest edges and enables data-driven determination of the optimal sparsification threshold that preserves network structure and connectedness while removing potentially spurious correlations.

2.5. Comparative community detection methods

The community structure of the resulting weighted sparsified matrices was detected by Asymptotical Surprise optimized with PACO and compared against two widely used methods, Infomap (Rosvall and Bergstrom, 2008) and Newman's Modularity (Blondel et al., 2008, Newman, 2006), that are affected by the resolution limit, albeit to different extents. In Newman's Modularity, the size of the smallest detectable cluster is of the order of the square root of the number of edges in the entire network (Fortunato and Barthélemy, 2007). Infomap has a limit that depends on the overall number of inter-cluster edges (Kawamoto and Rosvall, 2015).

These two methods are based on different principles to detect the community structure of a graph. Newman's Modularity finds the optimal partition by maximizing intra-cluster edge-density against that of a configuration model (Newman, 2006). Optimization of this fitness function is typically performed using the Louvain method, a greedy agglomerative clustering algorithm that works on hierarchical refinements of the network's partitions. Here we used the Louvain implementation available in the Brain Connectivity toolbox (Rubinov and Sporns, 2010). The idea behind Infomap is the minimization of the description length (Rissanen, 1978) of a random walker defined on the network through a set of heuristics. For this study we used the Infomap implementation available in the igraph-0.7.1 package (Csardi and Nepusz, 2006).

For all methods, including PACO, we launched 10,000 independent runs, and picked the membership corresponding to the partition with the best value of the fitness function, the maximum for Modularity and Asymptotical Surprise, the minimum for Infomap. Graphs showing the dependence of the best value of the fitness function on the number of runs are reported in the Supplementary information section.

Degeneracy of nearly-optimal solutions, whereby similar values of the fitness function around its maximum correspond to substantially different partitions, has been observed for Newman's Modularity (Good et al., 2010). A consensus approach has been suggested in Lancichinetti and Fortunato (2012) as a means to mitigate the degeneracy problem, yielding a stable “average” solution over a large set of partitions. In order to ascertain whether Surprise and Asymptotical Surprise suffer from a similar shortcoming we have performed degeneracy analysis for these fitness functions following (Good et al., 2010). In short, we sampled partitions from a benchmark network consisting in 24 cliques of five nodes, connected by a single link to form a ring-like structure. We sampled the configuration space of partitions through a Montecarlo procedure and annotated its corresponding values of quality function for each partition. We then built a similarity matrix between all sampled partitions and embedded it into a three-dimensional space maintaining similarity relations between partition following a Curvilinear Components Analysis (CCA). In the embedded manifold, two partitions are close if they are similar and the z-axis encodes the quality function. Whereas a large plateau of solutions with similar values of maximum Modularity is observed (3A, consistent with (Good et al., 2010)), Asymptotical Surprise and Surprise display a much sharper peak corresponding to the optimal solution, as shown in Fig. 3B and C. Hence, degeneracy of nearly-optimal solutions does not appear to severely affect Surprise or Asymptotical Surprise, and a consensus approach is not deemed necessary for these functions. This analysis supports our choice to select the solution with the highest value of the fitness function.

Fig. 3.

Degeneracy landscape after dimension reduction of optimal partitions for Newman's Modularity (A), Asymptotical Surprise (B) and Surprise (C) on a benchmark ring of cliques structure consisting of 30 cliques of five nodes. Modularity shows a wide plateau, an indication of the degeneracy of this function. The landscape of Asymptotical Surprise (B) presents a sharper peak that enable identification of the optimal partition.

Our implementation of PACO as well as the code to generate benchmark LFR networks was written in C++ with bindings in Matlab, Octave, Python. PACO is available at goo.gl/vpaggl. The LFR software is available at github.com/carlonicolini/lfrwmx.

2.6. Measures of partition quality

For each method, we analyzed the level of agreement of the detected community structure against the planted one in terms of Normalized Mutual Information (NMI) (Danon et al., 2005). Additionally, we used two different coefficients of similarity between partitions: Sensitivity and Specificity.

To this end, we quantified the confusion matrix $\mathbf{C}$ between the detected and planted modules. Each element C_ij is the number of nodes in the planted community-i that appear in the detected community-j. For each planted community we scored as true positives (TP) the nodes correctly identified as belonging to the ground-truth community, and as false positives (FP) the nodes wrongly assigned to a community; similarly false negatives (FN) were nodes wrongly classified in different communities and true negatives (TN) the nodes correctly classified as out of the community. Sensitivity, defined as $\mathit{TP}/(\mathit{TP}+\mathit{FN})$, decreases with increasing number of False Negatives. Specificity instead is defined as $\mathit{TN}/(\mathit{TN}+\mathit{FP})$ and decreases when many nodes are wrongly assigned in the same community. Additionally, we computed Accuracy and Matthew Correlation Coefficient (see Supplementary materials for definitions).

2.7. Human resting state network

We applied Asymptotical Surprise maximization by PACO to a reference resting state fMRI functional connectivity dataset from healthy subjects (Crossley et al., 2013) made available to the scientific community through the public Brain Connectivity Toolbox (Rubinov and Sporns, 2010). Detailed experimental and image processing procedures are described in the original paper (Crossley et al., 2013), alongside with the ethical statements.

In short, fMRI data were acquired from 27 healthy volunteers at 3 T. Gradient echo-planar imaging data were collected for 5 min with 2 s TR and 13 and 31 ms echo-times. Thirty six interleaved 3 mm slices with in-plane resolution of $3.5\times 3.5\mathrm{mm}$ were acquired. Time series were extracted from 638 brain regions defined by a template (Crossley et al., 2013), corrected for motion and band-passed (0.01–0.1 Hz). Functional connectivity was defined in terms of pairwise Pearson correlations at a subject's level. A group-level functional connectivity matrix was calculated by averaging individuals' matrices after Fisher-transform, and thresholded to retain 18625 edges, as described in Crossley et al. (2013). We used BrainNetViewer as a tool for the visualization of the communities on brain templates (Xia et al., 2013).

2.8. Hub classification

Hub classification is critical for the interpretation of the roles played by the highly connected nodes within the network structure. Resolution limited methods may result in misclassification of hubs, and the resulting partitions may not accurately reflect the modular organization of the connectivity network. To investigate differences in hubs' roles as determined by different community detection methods we have adopted the Guimera et al. (2005) classification scheme, specifying for every node its participation coefficient and within module degree z-score. The participation coefficient reflects the extent to which a node is connected to nodes in other modules, and is defined in Eq. (3)

$$P_i=1-\sum _c{\left(\frac{k_{\mathit{ic}}}{k_i}\right)}^2$$ (3)

where k_ic is the number of links of node i to nodes in module c and k_i is the total degree of node i.

Conversely, the within-module degree reflects the degree of connectivity of a node, and is defined as:

$$z_i=\frac{k_i-\langle k_i\rangle }{{\sigma }_{k_i}}$$ (4)

In the Guimera and Amaral model, connector hubs are those with high values of participation coefficient and high values of within-module degree z-score. These hubs are responsible for integration of the different modules into a cohesive structure, and are thought to be particularly vulnerable in the presence of brain insult or disease, as damage to these nodes is likely to disrupt network connectedness.

We mapped participation coefficients P_i and within module degrees Z_i in our benchmark resting state functional connectivity network for Newman's Modularity, Infomap and Asymptotical Surprise. Following Guimera and Amaral scheme, we identified connector hubs for each of the three methods as those with simultaneously large values of participation coefficient and within module degree (larger than 0.62 and 1.5, respectively).

3. Results

3.1. Synthetic networks

We compared the quality of the partitions of the synthetic benchmark networks obtained by Asymptotical Surprise with those of Infomap (Rosvall and Bergstrom, 2008) and Newman's Modularity (Newman, 2006, Blondel et al., 2008). Fig. 4 shows Normalized Mutual Information, Sensitivity and Specificity of the three methods applied to the ring of cliques for different sample sizes and SNRs; no-noise condition is represented as “Inf”. This model network was constructed to test the ability of the three methods to retrieve heterogeneous community structures under various noise conditions.

Fig. 4.

NMI, Sensitivity and Specificity of the three community detection algorithms applied to a power-law ring of clique network. SNR indicates Signal to Noise Ratio, and Inf the situation with a network structure unperturbed by noise. Number of Subjects indicates the different number of instances used to generate the group level network.

As expected, all methods showed better performance with increasing SNR and number of subjects, as noise and intersubject variability introduce spurious edges that hinder the ability to retrieve the planted structure. Partitions obtained with Newman's modularity showed the lowest NMI with respect to the planted partition under all conditions. Sensitivity of Newman's modularity did not exceed 0.75 even for high SNRs and a large number of subjects, a consequence of its stronger resolution limit. For this network, Infomap performed substantially better in terms of NMI against the planted partition, with a Sensitivity that was superior to that of Modularity across the spectrum of conditions.

Asymptotical Surprise showed highest NMI and Sensitivity across conditions, consistent with its resolution-limit-free behavior. Asymptotical Surprise proved superior in terms of NMI and Sensitivity in the low SNR regimes, and in the presence of relatively large intersubject variability as mimicked by the generation of different instances of the ring of cliques (see Section 2). Specificity of Asymptotical Surprise was not inferior to the other methods under all conditions, thus ruling out increased vulnerability to False Positives, at least in this particular model network.

Comparable results were obtained for the LFR network (Fig. 5), a model graph that replicates the distribution of nodal degree observed in many real-world networks, including those representing brain functional connectivity. All three methods showed similar values of NMI for high SNR and a large number of subjects, with a plateau reaching maximum Sensitivity with a group sample bigger than 20 and SNR above 30. Sensitivity was only slightly worse for Modularity, but it should be noted that for the LFR network the size distribution of the planted modules was narrower than for the ring of cliques (Fig. 2), thus making the resolution limit less evident.

Fig. 5.

NMI, Sensitivity and Specificity of the three community detection algorithms applied to Lancichinetti-Fortunato-Radicchi (LFR) networks. SNR indicates Signal to Noise Ratio, and Inf the situation with a network structure unperturbed by noise. Number of Subjects indicates the different number of instances used to generate the group level network.

In the lower SNR regime, Asymptotical Surprise presented the best performance in terms of NMI and Sensitivity, with a slower decay for decreasing SNR. Specificity was almost equivalent across the three methods, with a quick convergence to the maximum value of 1 for high SNR and good performance (around 0.97) for low SNR. Asymptotical Surprise presented a faster decay with decreasing SNR. However, it should be noticed that the scale of Specificity has a very narrow range (0.97–1.00), and the differences between the three methods were relatively small.

Consensus analysis applied to Newman's Modularity to assess the potential effects of the degeneracy of nearly optimal solutions did not show substantial differences in the comparison with the other methods (Supplementary materials, Figs. S6, S7).

For the sake of completeness, we also computed Accuracy and Matthew Correlation Coefficient for the same model networks, shown in the Supplementary materials. Notably, Infomap showed a large variability in Accuracy for lower SNRs and number of subjects. Under closer examination, however, it appeared that the increased variance for Infomap was due to occasional runs in which the algorithm only retrieved one or two large modules. This is a known problem with Infomap and other algorithms based on random walks that depends on the need to parametrize the teleportation step in order to make the dynamics ergodic (Lambiotte and Rosvall, 2012).

Altogether, the picture that emerges from the analysis of Accuracy and MCC is entirely consistent with the results shown in this section.

3.2. Resting state functional connectivity dataset

Fig. 6 shows a comparison between the modular structure of the resting state fMRI dataset obtained with Newman's Modularity, Infomap and Asymptotical Surprise. For each method, we had 10,000 independent runs and picked the partition with the best value of the respective fitness functions (Q=0.4967, $\mathcal{L}=8.5173$, S_a=5925.3, for Modularity, Infomap and Asymptotical Surprise, respectively). The three methods showed significantly different partitions (relative NMIs in Table 1), with a number of detected communities of 10, 19 and 47 for Modularity, Infomap and Asymptotical Surprise, respectively. Interestingly, Modularity detected a relatively uniform size distribution, consistent with the intrinsic scale built into the fitness function. Infomap showed a wider distribution of module sizes, with number of nodes ranging between 156 and 3, while Surprise showed the largest spread, and included communities as small as single nodes (singletons).

Fig. 6.

Adjacency matrix of the resting state functional connectivity network. The node indices have been reordered by module membership in each graph, and the red lines highlight the community structures obtained by (A) Louvain-Newman's Modularity (Q=0.4967); (B) Infomap (L=8.5173); C) Asymptotical Surprise (S_a=5925.28).

Table 1.

NMI values of partitions obtained from the three different methods on the resting state network.

NMI	Newman	Infomap	Asymptotical Surprise
Newman	1.00	0.75	0.62
Infomap	–	1.00	0.76
Asymptotical Surprise	–	–	1.00

Fig. 7 shows the 16 largest modules detected by Asymptotical Surprise, ranked by number of nodes comprised in each community. The first and largest module (Fig. 7A) includes the pre- and post-central gyri, part of the supramarginal gyrus and supplementary motor area. The second community (Fig. 7B) consists largely of nodes belonging to the occipital lobe: the visual areas and the surrounding calcarine sulcus, the lingual and fusiform gyrus. The third module (Fig. 7C) reflects the Default Mode Network, spanning the temporo-parietal cortex, the medial prefrontal cortex and the posterior cingulate/precuneus. The nodes involved in the executive frontal functions form the fourth largest community. Interestingly, nodes in the communities D, E, G are the major players that take part in the so-called fronto parietal attentional network (Markett et al., 2014). The auditory network, comprising temporal areas, was detected as a distinct community (Fig. 7F). Deeper structures emerge as separate modules in Fig. 7H, with subcortical areas including the basal ganglia, i.e. putamen, globum pallidum, caudate nucleus and the whole thalamus. The hippocampus and the parahippocampal gyrus were identified as separate communities (O and P). Additional, smaller substructures are shown in the third and fourth row of Fig. 7, including the Supplementary Motor Area (Fig. 7J) and the orbital (Fig. 7M) and orbitofrontal (Fig. 7I) modules, containing nodes from Brodmann area 47 (the smaller communities are displayed Fig. S8).

Fig. 7.

Sixteen largest modules found by Asymptotical Surprise Maximization in the resting state network overlaid on an MRI brain template. The modules are ranked by decreasing size, and named after corresponding functional networks previously identified by multivariate analysis of resting state fMRI data, or by the comprised anatomical districts.

Partitions of the functional connectivity network obtained by Newman's Modularity and Infomap are reported in the Supplementary materials Section (Fig. S9 and S10). Newman's Modularity retrieved four large, relatively uniform communities, corresponding to the Default Mode Network, the central network, occipital and frontoparietal networks. This is in keeping with previous studies using Modularity optimization by spectral decomposition (Crossley et al., 2013), and consistent with the strong resolution limit that affects this method. Additionally, a few smaller modules were found by Louvain optimization of Newman's Modularity, corresponding to the basal ganglia, the hippocampal/parahippocampal formation and two asymmetrically distributed subcortical clusters.

Infomap identified 19 communities of various sizes, also shown in the Supplementary materials Section, Fig. S10. The largest modules showed a close correspondence with those identified by Asymptotical Surprise, albeit with some notable differences. By way of example, the largest component includes the motor-sensory and auditory modules, identified as separate communities by Asymptotical Surprise. The Default Mode Network retrieved by Infomap includes parts of the temporal cortices that are not normally associated with the DMN. Similarly, hippocampus and the parahippocampal modules were merged by Infomap, and resolved as individual modules by Asymptotical Surprise. Other modules, including the visual, associative and executive networks (C, E and F in Fig. S10, respectively) were qualitatively very similar to those identified by Asymptotical Surprise.

Altogether, the picture that emerges is consistent with the idea that the resolution limit is more severe in Newman's Modularity than in Infomap, and that Asymptotical Surprise presents the best resolving power among the three methods in a real-world network with finite SNR and variability as the resting state functional connectivity network used for this study.

3.3. Hub classification

Maps of the anatomical distribution of the participation coefficient and within module degree show substantial differences between the three community detection methods (Fig. 8), resulting in discrepancies in the identification of the connector hubs for the same functional connectivity network.

Fig. 8.

Anatomical distributions of the participation coefficient and within-module degree z-score for the resting state functional connectivity network partitioned by the three community detection methods.

Fig. 9 shows the nodes with simultaneously high values of participation coefficient and within module degree (connector hubs, according to the Guimera and Amaral's classification). All three methods pinpoint connector hubs in the superior, superior medial and middle frontal areas, as well as in the supplementary motor area. However, substantial differences are observed for other hub regions. The partition of Asymptotical Surprise localizes connector hubs in the Temporal Middle and Frontal Middle gyri, as well as in the Rectus, Middle Cingulate Cortex, Lingual gyrus and in the Precuneus.

Fig. 9.

Nodes presenting simultaneously large values of the participation coefficient and within-module degree (larger than 0.6 and 1.5, respectively) for the three community detection methods. These nodes are thought to represent the connector hubs responsible for the integration of the networks's modules.

Community detection by InfoMap results in the identification of hubs that are partially consistent with either of the two other methods, in keeping with the idea that its resolution limit is less severe than for Newman's Modularity. Altogether, these findings indicate that node role classification is method-dependent, and may be affected by the resolution limit.

4. Discussions

4.1. Validation of asymptotical surprise in model networks

The performance of Asymptotical Surprise optimization by PACO was assessed in model graphs with a built-in community structure, and compared with two established community detection methods. We have chosen two synthetic benchmark networks, the ring of cliques and the LFR network.

The ring of cliques presents a clear-cut modular structure by construct, with modules corresponding to complete subgraphs of variable sizes sampled from a power-law distribution. This toy network proved useful to assess the effects of the resolution limit in the presence of a wide distribution of cluster sizes. The effects of this limit were particularly apparent for Newman's Modularity (Fig. 4), that showed poor Sensitivity even for noiseless rings of cliques, plateauing at a value of 0.75. This is consistent with the findings of Fortunato and Barthélemy (2007), that showed that for Modularity the resolution limit is set by the square root of the total number of edges in the graph. For Infomap, this limit is less severe and is determined by the number of inter-cluster edges (Kawamoto and Rosvall, 2015). Accordingly, the effects of the resolution limit were not apparent in this model network, where modules are sparingly connected by single edges. Asymptotical Surprise presented the best performance, consistent with the idea that this cost function is quasi-resolution limit free (Traag et al., 2015).

However, real brain networks are characterized by heterogeneous distributions of node degree, with fat tails and power-law decays (Bullmore and Sporns, 2009). Such heterogeneity is critical, as it determines some of the remarkable features of brain connectivity networks, including resilience to random failure and rich-clubness (van den Heuvel and Sporns, 2011, van den Heuvel and Sporns, 2013). To provide a more realistic benchmark, we used the Lancichinetti-Fortunato-Radicchi algorithm (Lancichinetti et al., 2008), that makes it possible to generate networks with realistic and tunable power law degree distribution and community sizes.

For LFR networks, the difference in performance in the low-noise regime was more nuanced for the three methods compared in this study, possibly a result of a fuzzier community structure of the LFR network compared to the ring of cliques, and of the narrower distribution of cluster sizes. However, the picture appears different when noise and intersubject variability were injected into the network structure.

Noise and other sources of variability in the data can significantly affect the structure of the resulting network representation. Noisy fMRI time-courses, for example, may introduce spurious correlations in brain functional connectivity networks. This problem may be particularly relevant for methods endowed with high resolution, like Asymptotical Surprise, that may be more vulnerable to False Positives generated by the mis-assignment of peripheral nodes, particularly in small clusters. Hence, the resolving power of community detection methods should be gauged against Specificity, which may be affected by noise in the distribution of edges that define the network's structure. However, to the best of our knowledge, this aspect has never been considered in the existing literature assessing the performance of community detection algorithms as applied to the study of brain connectivity.

To this end, we have devised methods to inject noise, with amplitude and spectral distribution that mimic those of experimental noise, into networks with a well defined planted structure. Moreover, we have generated different instances for each network, corresponding to different subjects in a group, to account for intersubject variability that occurs in typical neuroimaging studies.

Unsurprisingly, for all methods and networks, detection of the planted structure improved with decreasing levels of noise, and with increasing number of subjects in the study. However, Asymptotical Surprise appeared to provide a superior performance in terms of NMI and Sensitivity to the planted structure for lower SNRs in both types of networks, while its Specificity was in line with that of resolution-limited methods like Newman's and Infomap (Fig. 4, Fig. 5). This rules out the idea that the higher sensitivity to small clusters of Asymptotical Surprise may be detrimental in noisy networks, making it more vulnerable to small, spurious modules.

4.2. Community detection in functional connectivity networks by Asymptotical Surprise

Application of Asymptotical Surprise maximization to a group-level, resting state functional connectivity network from the brains of 27 healthy subjects revealed a heterogeneous distribution of modules, with large and small modules coexisting in the optimal partition. This is in keeping with previous findings with binary Surprise (Nicolini and Bifone, 2016). These modules closely reflect functional networks reported in many studies using Independent Component Analysis or other multivariate methods, including the sensorimotor, visual, default mode, executive, and attentional networks. Moreover, anatomically defined subcortical structures, like the hippocampus and parahippocampal formations emerged as independent moduli.

While this is entirely consistent with our understanding of the neurofunctional and anatomical organization of the human brain, the accuracy of Asymptotical Surprise in identifying these networks is notable. Indeed, Surprise, like other graph-based community detection methods, divides networks into disjoint clusters of nodes on the basis of topological criteria. While a correspondence between topological modularity and functional networks identified by, e.g. Independent Component Analysis, may be expected, it is not a given, for they are defined on different principles. Indeed, multivariate methods like ICA separate components on the basis of the statistical independence of the time-courses, and do not convey information regarding the mutual relationship between modules nor about their topological organization.

Previous studies applying resolution-limited methods like Newman's Modularity to the same dataset hereby analyzed (Crossley et al., 2013) found a few, large modules encompassing large-scale networks, but failed to identify finer, neurofunctionally plausible substructures like those shown in the present study. Infomap, on the other hand, proved sensitive to heterogeneously distributed clusters, thus implying that this method does not have an intrinsic scale, like Modularity and variations thereof based on the introduction of a resolution parameter. However, Asymptotical Surprise appears to provide superior performance in identifying small subnetworks, particularly in the presence of noise, thus suggesting that this method may represent a new standard for community detection in brain networks. It should also be noted that no symmetry constraint was imposed, and the symmetrical bilateral distribution of nodes in the retrieved modules arises entirely from Asymptotical Surprise optimization.

Hierarchical clustering methods have been extensively applied to investigate the structure of brain connectivity networks, showing smaller and smaller clusters as the modules are iteratively subdivided (Meunier et al., 2010). Maximization of Asymptotical Surprise reflects the optimal cut through the dendrogram representing connectivity at these different levels of subdivision, and provides information on the optimal partition of the network. Hence, the heterogeneous distribution of cluster sizes retrieved by Asymptotical Surprise suggests that multiple scales of structure exist at the same level of the dendrogram.

Finally, abnormal functional connectivity has been observed in a number of neurological and psychiatric diseases, but the coarse resolution of methods like Newman's Modularity (Fornito et al., 2015) may have not detected differences in the modular organization of networks in patients compared to healthy controls. The improved resolution and sensitivity to multiscale structure afforded by Asymptotical Surprise may provide a powerful means to assess the brain functional architecture in disease states, thus contributing a potential imaging-based marker and a key to interpret the functional effects of aberrant connectivity.

4.3. Hub classification

The presence of heterogeneously distributed modules in functional connectivity networks may have important consequences for our understanding of the brain functional organization. By way of example, it has been shown that highly connected nodes, or hubs, are critically important in brain connectivity networks, and may play different roles depending on their position and connectivity distribution within and between modules (Bullmore and Sporns, 2009). Hubs that primarily connect to nodes within the same community are dubbed "provincial hubs", and are thought to be responsible for the definition and stability of the modules. Conversely, hubs that connect different modules are referred to as "connector hubs" and ensure integration of the activity of the network. The classification of hubs strongly depends on the modular structure that is considered, and inaccurate partitioning due to the resolution limit can lead to the wrong interpretation of their role in the interplay between segregation and integration of brain function (Bullmore and Sporns, 2009). The present study suggests that this may have been the case in previous studies, in which resolution limited methods characterized by an intrinsic scale have been used, and provides a solution that may enable more accurate classification of hubs and nodes.

The connector hubs identified by our three methods (Asymptotical Surprise, InfoMap and Newman) present some substantial differences, consistent with the idea that hub classification depends on community structure. These differences are particularly interesting in the light of the important role that connector hubs are thought to play in integrating information flow through the brain, and their putative role in brain disease (Crossley et al., 2014, Stam, 2014). By way of example, the Precuneus and the Cingulate Cortex are highlighted by Asymptotical Surprise, but not by Newman's Modularity, as connector hubs. These are two key elements of the Default Mode Network that have been consistently identified as vulnerable regions in neurological diseases (van den Heuvel and Sporns, 2013, Buckner et al., 2009). Community detection by resolution limited free methods should enable more accurate classification of hub nodes, and improve our understanding of their role in brain disease.

4.4. Limitations

Some caution should be taken in the interpretation of the graphs in Fig. 4, Fig. 5. Indeed, the SNRs of the synthetic networks we have generated reflect noise with features, like a Rician distribution, that mimic some, but not all aspects of the variability of experimental data. By way of example, the brain parcellation scheme applied to define the nodes, and the heterogeneity of voxels within these parcels, may play a role that is difficult to model in toy networks (Fornito et al., 2010). Hence, the simulated Sensitivity and Specificity as a function of SNR and number of subjects should not be taken as absolute values to be used in the power and sample size estimation in real experimental designs. Nevertheless, these simulations provide useful information on the dependence of these parameters on noise levels, and a rigorous means to assess the relative merits of different community detection methods.

Finally, we should note that the maximum value of Asymptotical Surprise calculated with PACO is an index of quality of the entire partition, and not of individual modules. Hence, individual modules may not all have the same strength of internal cohesiveness relative to their connection with other modules. We have found hints of this phenomenon in the comparison of nearly-optimal partitions obtained in the 10,000 runs of PACO that we have performed to find the optimal community structure for this network. The overall community structure appeared to be robust, with most modules persistently emerging in every nearly-optimal partition, but in some cases we observed pairs of modules splitting or merging in otherwise similar solutions. Most notably, this was observed for the thalamus that in some instances was merged with the basal cluster and in others, featured as a separate module. This phenomenon may be less critical for methods like Newman's Modularity that have an intrinsic scale and retrieve uniformly distributed modules.

5. Conclusion

We have extended the use of Surprise, a resolution-limit-free fitness function for the study of the modular structure of complex networks, to weighted brain functional connectivity networks. Specifically, we have developed a novel method, dubbed PACO, for the optimization of Asymptotical Surprise, a weighted counterpart of Surprise in the limit of large networks. We have applied PACO optimization of Asymptotical Surprise in synthetic networks to evaluate the relative merits of this novel approach against Newman's Modularity and Infomap, two of the leading methods used for community detection in brain connectivity networks. Specifically, we have implemented a process to inject noise into networks endowed with a ground-truth modular structure to assess the trade-off between improved resolution afforded by Asymptotical Surprise and potential sensitivity to spurious correlations introduced by variability in the data. Asymptotical Surprise optimization proved superior to existing methods in terms of Sensitivity and accuracy in detection of the planted structure as measured by Normalized Mutual Information, while showing comparable Specificity. We have also applied our approach to the partitioning of functional connectivity networks from resting state fMRI experiments. Direct comparison with other methods clearly demonstrated improved capability to identify neurofunctionally plausible and anatomically well-defined substructures otherwise concealed by the resolution limit. Asymptotical Surprise revealed a complex modular structure of resting state connectivity, with communities of widely different sizes reflecting distributed functional networks alongside with small, anatomically or functionally defined modules. This evidence corroborates the idea that the resolution limit may have negatively affected current models of the brain modular organization and the identification of the hubs responsible for integration of functional modules. The application of methods like Asymptotical Surprise provides a novel, powerful approach to study the modular structure of brain connectivity beyond this limit.

Author contributions

C.B, C.N, A.B. conceived the experiments, C.N., C.B. conducted the experiments, C.N. C.B, and A.B. wrote the paper and reviewed the analysis. All authors reviewed the manuscript. Our implementation of PACO as well as the code to generate benchmark LFR networks was written in C++ with bindings in Matlab, Octave, Python. This software is available upon request.

Competing financial interests

The authors declare no competing financial interests.

Appendix A. Supplementary materials

.