Connectivity Subnetwork Learning for Pathology and Developmental Variations

Abstract

Network representation of brain connectivity has provided a novel means of investigating brain changes arising from pathology, development or aging. The high dimensionality of these networks demands methods that are not only able to extract the patterns that highlight these sources of variation, but describe them individually. In this paper, we present a unified framework for learning subnetwork patterns of connectivity by their projective non-negative decomposition into a reconstructive basis set, as well as, additional basis sets representing development and group discrimination. In order to obtain these components, we exploit the geometrical distribution of the population in the connectivity space by using a graph-theoretical scheme that imposes locality-preserving properties. In addition, the projection of the subject networks into the basis set provides a low dimensional representation of it, that teases apart the different sources of variation in the sample, facilitating variation-specific statistical analysis. The proposed framework is applied to a study of diffusion-based connectivity in subjects with autism.

1 Introduction

Analysis of brain connectivity networks created from DTI, MEG/EEG, or fMRI data has provided a novel insight into brain changes arising from pathology, development or aging [1, 2]. The high dimensionality of these networks has necessitated the development of methods that can extract the connectivity patterns of the population and separate components pertaining to each source of variation. In this paper, we provide such a method of extracting the basis of patterns using a locality-preserving basis learning, which helps provide a low dimensional representation of the subject networks that can then be used to study group differences, or train pathology specific or developmental biomarkers. Traditional analysis techniques, such as PCA and ICA, provide dimensionality reduction in brain network investigations [3] but may lack the physiological interpretability of non-negative connectivity matrices. Recently, non-negative matrix factorization (NMF) methods [4] have provided interpretable sparse bases characterizing multivariate data [5–8]. The non-negativity constraints, facilitate interpretability of the components as sparse connectivity matrices [6], but are unable to separate patterns that represent the different sources of variability in the population.

This paper presents a framework for learning sparse subnetwork patterns of non-negative connectivity matrices by their projective non-negative decomposition into sets of i) discriminative or pathology-specific, ii) developmental (age related), and iii) reconstructive components. The decomposition maintains the interpretation of each component as a connectivity matrix and their associated coefficients as the weight of the subnetwork, while providing a succinct low dimensional representation of the population amenable to statistical analysis.

While the method is generalizable to any type of non-negative connectivity matrix, we have demonstrated the applicability of the framework to DTI-based structural connectivity networks for a population of subjects with autism spectrum disorder (ASD). Our method is able to extract components that describe the underlying patterns of pathology related variability, as well as patterns of developmental (age) variation. In addition, the corresponding low dimensional representation of the subjects in these basis components is able to identify group differences in pathology, and quantify the age related variations.

2 Methods

The connectivity network of each subject can be modeled as a linear combination of several components that act as the building blocks of the brain network system [6]. Due to their symmetry, the upper triangular elements of a connectivity matrix are represented by x_i for the subject i. To compute the r connectivity components whose mixture constructs the original non-negative connectivity matrices, a matrix factorization is used as X ≈ WΦ, where columns of X = [x₁, x₂, …, x_n] ∈ ℝ^m×n, i.e. x_i, represent the connectivity matrices, and columns of W = [w₁, w₂, …, w_r] ∈ ℝ^m×r, i.e. w_j, are representative of the normalized basis connectivity components. These components w_j are then mixed by each column of the loading matrix Φ = [φ₁, φ₂, …, φ_n] ∈ ℝ^r×n to approximate the corresponding column of X [8].

2.1 Projective Non-negative Basis Learning

Inspired by [6], we assume that Φ is the projection of X onto W, i.e. Φ = W^TX. The projective properties of the NMF coefficients (Φ) help imbibe orthogonality properties into components [8]. Hence, the reconstruction of original non-negative connectivity matrices can be obtained by minimizing the cost function $F_1(\mathit{W})={\Vert \mathit{X}-\mathit{W}{\mathit{W}}^T\mathit{X}\Vert }_F^2$ with respect to W ≥ 0. This can be denoted by

$$\underset{\mathit{W}\ge 0}{\text{min}}{F}_1(\mathit{W})=\underset{\mathit{W}\ge 0}{\text{min}}\mathit{\text{trace}}\{(\mathit{X}-\mathit{W}{\mathit{W}}^T\mathit{X}){(\mathit{X}-\mathit{W}{\mathit{W}}^T\mathit{X})}^T\}.$$ (1)

2.2 Locality Preserving Bases with Graph Embedding

We split the set of r components into three subsets W = [Ŵ, W̆, W̃] where Ŵ = [w₁, …, w_q] are the discriminative basis components, W̆ = [w_q+1, …, w_q+p] are the developmental basis components, and W̃ = [w_q+p+1, …, w_r] is the complementary space containing the reconstructive basis components which minimizes the reconstruction error together with Ŵ and W̆. Thus, the coefficient matrices are also split into Φ̂ = Ŵ^TX, Φ̆ = W̆^TX, and Φ̃ = W̃^TX. A proper modeling of such intent would provide at most q of those bases which are likeliest to provide population discrimination to belong to Ŵ, and p of those which are likeliest to account for the developmental variations in W̆.

To clarify our mathematical modeling, suppose that the m-dimensional connectivity points (x_i) of two groups lie on a 3-D manifold, as illustrated in Fig. 1(a), and are to be projected into an r = 3 dimensional subspace with q = 1 discriminative (say y⃗ ), p = 1 developmental (say z⃗), and r − q − p = 1 reconstructive components (say x⃗). In order to achieve this, we construct three separate graphs made up of these m-dimensional points as their vertices. The first graph is an intrinsic k-nearest-neighbor graph [5, 9] which connects point i to j if point j is among the k nearest neighbors of point i (as illustrated in Fig. 1(b)). This graph is used for obtaining the discriminative component Ŵ. The second graph connects point i to point j if point j is among the k farthest points to point i (illustrated in Fig. 1(d)), which is used for obtaining the reconstructive component W̃. The third graph, used for obtaining the developmental component W̆, connects point i to j if subject j’s age is among the k nearest ages to subject i, irrespective of the two points’ distance (illustrated in Fig. 1(c)).

Fig. 1.

Illustration of a two-group m-dimensional points lying on a 3-D manifold, shown as a cube in the m-dimensional space. (a) Point distributions when projected into x⃗, y⃗ or z⃗. (b) The 3-nearest-neighbor graph Ĝ of two selected magnified points. (c) The graph Ğ with edges connecting points whose subjects are of similar ages. (d) The 3-farthest-point graph G̃ of the same two points.

There are a variety of approaches that can characterize separability of multivariate data-points. Most of such techniques can be unified in the framework of graph embedding [5, 9]. Let G = {X, S} be an undirected weighted graph of n vertices, i.e. data points x_i, with a symmetric matrix S ∈ ℝ^n×n with non-negative elements, within the range of 0 to 1, corresponding to the edge weight of the graph. The Laplacian matrix L of the graph is then defined by L = D − S where D is a diagonal matrix with $D_{ii}={\sum }_{j=1}^n{S}_{ij}$ [9].

In order for the bases in Ŵ to provide discriminatory information, we would like the resulting coefficients (φ̂_i) of nearby x_i points to stay close to each other to group together when projected into Ŵ. This can be obtained by minimizing

$$\underset{\mathit{W}\ge 0}{\text{min}}{F}_2(\mathit{W})=\underset{\mathit{Ŵ}\ge 0}{\text{min}}\sum _{i=1}^n\sum _{j=1}^n{\Vert {\mathit{\phi ̂}}_i-{\mathit{\phi ̂}}_j\Vert }^2{Ŝ}_{ij}=\underset{\mathit{Ŵ}\ge 0}{\text{min}}\mathit{\text{trace}}\left\{\mathit{\Phi ̂}\mathit{L̂}{\mathit{\Phi ̂}}^T\right\},$$ (2)

where Ŝ = [Ŝ_ij] is the similarity matrix composed of the edge weights in the k-nearest-neighbor graph Ĝ = {X, Ŝ} of the m-dimensional points x_i, as illustrated in Fig. 1(b). According to the equation (2), if data-points x_i and x_j are close, their edge weight S_ij will be large, and therefore, the cost function F₂(Ŵ) gets minimized only if the corresponding coefficients φ̂_i and φ̂_j remain close.

Similarly, in order to capture the connectivity space of developmental variations, we form a developmental graph Ğ = {X, S̆}, as in Fig. 1(c), in which x_i is connected with an edge to x_j if subject i is within the k nearest age to subject j, and vice versa. Therefore, the space of developmental variations, W̆, grouping the coefficients (φ̆_i) of subjects with similar ages, is computed by minimizing $F_3(\mathit{W})={\sum }_{i=1}^n{\sum }_{j=1}^n{\Vert {\mathit{\phi ̆}}_i-{\mathit{\phi ̆}}_j\Vert }^2{\mathrm{S̆}}_{ij}=\mathit{\text{trace}}\left\{\mathit{\Phi ̆}\mathit{L̆}{\mathit{\Phi ̆}}^T\right\}$ when W̆ ≥ 0.

As explained earlier, we exploit the graph of k-farthest points G̃ = {X, S̃} (as illustrated in Fig. 1(d)), to impose the representative coefficients (φ̃_i) of the farthest points to remain as close as possible in the lower dimensional space when projected into the reconstructive set W̃. This is performed by minimizing $F_4(\mathit{W})={\sum }_{i=1}^n{\sum }_{j=1}^n{\Vert {\mathit{\phi ̃}}_i-{\mathit{\phi ̃}}_j\Vert }^2{\mathrm{S̃}}_{ij}=\mathit{\text{trace}}\left\{\mathit{\Phi ̃}\mathit{L̃}{\mathit{\Phi ̃}}^T\right\}$ subject to W̃ ≥ 0.

2.3 Objective Function and Optimization Solution

To achieve the above objectives, and according to the projective properties of the model, i.e. Φ = W^TX, the final objective function is modeled to minimize

$$F(\mathit{W})=\mathit{\text{trace}}\left\{\right(\mathit{X}-\mathit{W}{\mathit{W}}^T\mathit{X}\left){(\mathit{X}-\mathit{W}{\mathit{W}}^T\mathit{X})}^T\right\}+\lambda \left(\mathit{\text{trace}}\right\{{\mathit{Ŵ}}^T\mathit{X}\mathit{L̂}{\mathit{X}}^T\mathit{Ŵ}\}+\mathit{\text{trace}}\{{\mathit{W̆}}^T\mathit{X}\mathit{L̆}{\mathit{X}}^T\mathit{W̆}\}+\mathit{\text{trace}}\{{\mathit{W̃}}^T\mathit{X}\mathit{L̃}{\mathit{X}}^T\mathit{W̃}\left\}\right),$$ (3)

where λ is a tunable parameter to balance the two terms of reconstruction error norm and graph embedding. To minimize (3) with W ≥ 0, we use a gradient descent approach, updating $W_{ij}=W_{ij}-{\eta }_{ij}\frac{\partial F}{\partial W_{ij}}$ with step-sizes η_ij ≥ 0, where

$$\frac{\partial F}{\partial \mathit{W}}=-4(\mathit{X}{\mathit{X}}^T\mathit{W})+2(\mathit{W}{\mathit{W}}^T\mathit{X}{\mathit{X}}^T\mathit{W})+2(\mathit{X}{\mathit{X}}^T\mathit{W}{\mathit{W}}^T\mathit{W})+\lambda [2\mathit{X}\mathit{L̂}{\mathit{X}}^T\mathit{Ŵ},2\mathit{X}\mathit{L̆}{\mathit{X}}^T\mathit{W̆},2\mathit{X}\mathit{L̃}{\mathit{X}}^T\mathit{W̃}].$$ (4)

Regarding that L̂ = D̂ − Ŝ, L̆ = D̆ − S̆ and L̃ = D̃ − S̃, and the fact that both D and S have non-negative elements, our non-negativity constraint is guaranteed by positive initialization of W and applying the step-size ${\eta }_{ij}=\frac{\frac{1}{2}{W}_{ij}}{{(\mathit{W}{\mathit{W}}^T\mathit{X}{\mathit{X}}^T\mathit{W}+\mathit{X}{\mathit{X}}^T\mathit{W}{\mathit{W}}^T\mathit{W}+\lambda [\mathit{X}\mathit{D̂}{\mathit{X}}^T\mathit{Ŵ},\mathit{X}\mathit{D̆}{\mathit{X}}^T\mathit{W̆},\mathit{X}\mathit{D̃}{\mathit{X}}^T\mathit{W̃}])}_{ij}}$. This results in the following multiplicative updating solution

$$W_{ij}=W_{ij}\frac{{(2\mathit{X}{\mathit{X}}^T\mathit{W}+\lambda [\mathit{X}\mathit{Ŝ}{\mathit{X}}^T\mathit{Ŵ},\mathit{X}\mathit{S̆}{\mathit{X}}^T\mathit{W̆},\mathit{X}\mathit{S̃}{\mathit{X}}^T\mathit{W̃}\left]\right)}_{ij}}{{(\mathit{W}{\mathit{W}}^T\mathit{X}{\mathit{X}}^T\mathit{W}+\mathit{X}{\mathit{X}}^T\mathit{W}{\mathit{W}}^T\mathit{W}+\lambda [\mathit{X}\mathit{D̂}{\mathit{X}}^T\mathit{Ŵ},\mathit{X}\mathit{D̆}{\mathit{X}}^T\mathit{W̆},\mathit{X}\mathit{D̃}{\mathit{X}}^T\mathit{W̃}\left]\right)}_{ij}}.$$ (5)

For the stability of convergence, at each iteration, each column of W is normalized by ${\mathit{w}}_i=\frac{{\mathit{w}}_i}{{\Vert {\mathit{w}}_i\Vert }_2}$. Starting with initial random positive elements on W, the iterative procedure will converge to the desired W = [Ŵ, W̆, W̃] ≥ 0.

3 Results

The proposed method provides a framework for extracting three sets of network components from the population. The discriminatory and developmental set of components are expected to show localized sparse sub-networks which mostly capture the changes related to pathology and developmental variations. The reconstructive basis set consists of global networks of dominant connectivity patterns. The number of bases is population dependent; however, we show that even with relatively small numbers, we can obtain stable group differences.

Dataset demographics and connectivity measures

We study the applicability of our method on a dataset of DTI connectivity matrices computed for a population of ASD subjects and typically developing controls (TDCs). Our dataset consisted of 83 male children, 24 ASD and 59 TDCs, aged 6–18 years (mean=12.9yrs, SD=3.0 in ASD, and mean=11.6yrs, SD=3.2 in TDC, no significant group difference in age). DTI was acquired for each subject (Siemens 3T Verio, 32 channel head coil, single shot spin echo sequence, TR/TE = 11000/76 ms, b = 1000 s/mm², 30 gradient directions). 79 ROIs from the Desikan atlas were extracted to represent the nodes of the structural network. Probabilistic tractography [10] was performed from each of these regions with 5000 streamline fibers sampled per voxel, resulting in a 79 × 79 matrix of weighted connectivity values, where each element represents the conditional probability of a pathway between regions, normalized by the active surface area of the seed ROI.

Connectivity component analysis

The 79 × 79 connectivity matrix of each subject was vectorized to its m = 3081 upper triangular elements. To 6 compute the components, we set λ = 1 and used k = 3 to construct the three graphs, and correspondingly calculated their graph edge weights (S) using a Gaussian kernel [9]. We used q = 4 discriminative, p = 2 developmental, and 6 reconstructive components (we suggest the number of reconstructive components be equal to the total of other components). The iterative procedure of equation (5) yielded components shown in Fig. 2. The connectivity components shown were sparse and thresholded for binary visualization to show the dominant edges. Also, in regards to the orthogonality of the discriminative and developmental components, the non-orthogonality between any two of these components was measured by computing their inner products (or angle) which were all less than 0.04 (between 88 and 90 degrees), in a scale of 0 (90-degree) to 1 (0-degree).

Fig. 2.

The r=12 connectivity basis components learned by the proposed method. (a)–(d) are the q=4 discriminative bases, (e)–(f) are the p=2 developmental components, and (g)–(l) compose the set of the 6 reconstructive bases.

The vectorized connectivity of each subject was projected into the discriminative and developmental components to obtain the coefficients for subsequent statistical t-test to validate the discriminative bases and compare between the ASD and TDC groups. We divided the 83 subjects into three closely-balanced age groups of 6–10 (age group I, 25 subjects), 10–13.5 (age group II, 28 subjects), and 13.5–18 years old (age group III, 30 subjects). A t-test was performed between the coefficients of the subjects in these age groups to validate the ability of the developmental components in capturing the effect of age. We then correlated the coefficients of all subjects with their age. Results are given in Table 1.

Table 1.

Statistical group analysis of the coefficients of the ASD-TDC bases

Component label	ASD-TDC group t(p)–values	Age group II vs. group I t(p)–values	Age group III vs. group II t(p)–values	Basis coeffs’ correlation with age
a	−3.2 (0.002)	−1.6 (0.1)	−0.7 (0.5)	−0.24
b	−1.2 (0.2)	−0.6 (0.6)	−0.3 (0.8)	−0.14
c	+0.5 (0.6)	−1.5 (0.1)	−1.3 (0.2)	−0.32
d	−2.3 (0.02)	−2.1 (0.04)	−2.3 (0.02)	−0.48
e	+0.7 (0.5)	+1.9 (0.06)	+3.0 (0.004)	+0.55
f	−3.1 (0.004)	−4.0 (0.0002)	−1.7 (0.09)	−0.54

It is observed that the discriminatory and developmental bases obtained are quite sparse with localized patterns as expected, while the reconstructive network components are globally connected. Table 1 shows the results of group-wise and age-based statistics on the coefficients of discriminatory (a–d) and developmental (e–f) components. Group-based analysis shows that the discriminative basis (a) is able to differentiate ASD and TDC groups with a high statistical significance (p = 0.002). Inspection of this component in Fig. 2 shows distinct connectivity deficiencies in the thalamic network, and inter-hemespheric subcortical connections in children with ASD. The low correlation with age of this component (column 5, table 1) is indicative of the fact that this component concentrates on the pathology-related patterns in the data. The developmental basis (e) does not show any group related differences, but has a high positive correlation with age. It demonstrates significant age increase (p = 0.004) between the second and third age ranges (age>10). Its positive correlation with age suggests that the connections between (mainly) left frontal and its nearby frontal, temporal, and subcortical regions significantly develop with age (correlation=+0.55), likely capturing ongoing maturation of language and executive functioning. The second developmental basis (f) shows an independent sub-network that diminishes (based on its negative correlation=−0.54 with age) significantly with development, especially during the younger ages (age<13.5, p = 0.0002). The behavior of the two components (e) and (f) are opposite with respect to age, as indicated by the sign of the correlation. Of interest is the discriminative component (d), that shows the second highest group difference, with several frontal regions compromised. However, this component also shows a relatively high correlation with age. The analysis of the components shows that our method is able to extract components that capture the changes due to pathology (a) and age (e); there are components such as (d) and (f) that are representative of changes in both. This may be an indication of those aspects of pathology that are linked with age and cannot be completely separated, yet help in providing a comprehensive picture of the pattern of changes in the population.

Our experiments have shown that the reconstructive components do not show any significant age-group differences or age correlation, but two mild significances (components h and j, p > 0.01) in ASD-TDC group difference; while interesting, for the purposes of the paper we have concentrated on the discriminative and developmental components. Also, we have observed that the average of the reconstructive coefficients are an order of magnitude larger than the discriminative and developmental basis coefficients. Thus, due to their relatively small coefficients, the discriminatory and developmental bases do not play a significant role in the reconstruction, and therefore, would not have been captured by solving only for the reconstruction components as has been done in the literature. However, the graph-embedded modeling proposed in this work has been able to extract them from the connectivity matrices of the two populations and comprehensively capture the pathology and age specific changes, which were the two major sources of variation in this population.

4 Conclusion

We have presented a novel technique for simultaneously extracting the discriminatory and developmental sub-networks of a population via graph embedding. Our method consists of an NMF basis learning scheme with locality preserving properties, and provides group-discriminatory as well as developmental network components. Application to a dataset of ASD subjects provided a discriminatory basis which revealed significant inter-hemisphere subcortical connectivity deficiencies. The developmental bases captured subnetworks which changed with age. The framework is generalizable to non-negative functional networks, as well as to modeling and identifying other forms of variation in the population.