Joint Bayesian-incorporating estimation of multiple Gaussian graphical models to study brain connectivity development in adolescence

Abstract

Adolescence is a transitional period between childhood and adulthood with physical changes, as well as increasing emotional development. Studies have shown that emotional sensitivity is related to a second period of rapid brain growth. However, there is little focus on the trend of brain development during this period. In this paper, we aim to track functional brain connectivity development from late childhood to young adulthood. Mathematically, this problem can be modeled via the estimation of multiple Gaussian graphical models (GGMs). However, most existing methods either require the graph sequence to be fairly long or are only applicable to small graphs. In this work, we adapted a Bayesian approach incorporating joint estimation of multiple GGMs to overcome the short sequence difficulty, which is also computationally efficient. The data used are the fMRI images obtained from the publicly available Philadelphia Neurodevelopmental Cohort (PNC). They include 855 individuals aged 8–22 years who were divided into five different adolescent stages. We summarized the networks with global measurements and applied a hypothesis test across age groups to detect developmental patterns. Three patterns were detected and defined as consistent development, late puberty and temporal change. We also discovered several anatomical areas such as the middle frontal gyrus, putamen gyrus, right lingual gyrus and right cerebellum crus 2 that are highly involved in the brain functional development. Functional networks including the salience, subcortical and auditory networks are significantly developing during the adolescent period.

I. Introduction

Adolescence is a transitional period between childhood and adulthood, and has long been tagged with labels like ”impulsive”, ”vulnerable”, ”rebellious”. It is not only a stage of physical changes, but also a time of increased emotional and cognitive development [1]. Several cognitive and neurobiological hypotheses have been postulated to explain the specialty and sensitivity in this period. Studies have shown that the brain’s most dramatic growth occurs in adolescence [2]. With the development of functional magnetic resonance imaging (fMRI), researchers can observe how the brain functions and interacts to gain insight into adolescent brain development. For example, McRae et al. [3] studied the development of emotion regulation using fMRI in children, adolescents and young adults; and Yurgelun-Todd [4] examined emotional and cognitive changes during adolescence using both structural and functional imaging data. Recent research has shown that brain development during adolescence is marked by substantial changes in brain structure and function, followed by a stable network topology in adulthood [5]. Functional brain networks reflect various sub-network communities responsible for different functional systems [6]. Delineating the trajectory of the brain functional connectivity (FC) from late childhood (pre-adolescence) to early adulthood (post-adolescence) can reveal the neural mechanisms of the developing brain.

With regard to mathematical methods, the delineation of the FC trajectory corresponds to estimating a series of association networks under distinct but related conditions. Several methods have been proposed in the literature from different perspectives to solve this issue. For instance, the sliding window approach [7], [8], [9] and the hidden Markov model (HMM) [10] are used for brain dynamics. But these models are more focused on detecting distinct brain states rather than the full brain connectivity trajectory. Another approach is based on Gaussian graphical models (GGMs) for joint estimation of multiple networks. In GGM, the brain is described as a graph, where nodes represent the regions of interest (ROIs) in the brain, linked by edges, i.e. their interconnections. The key idea of GGMs is to use partial correlation coefficients to measure the dependency relationships in a graph. Since the nodes (ROIs) in the brain are all directly or indirectly correlated, the advantage of using partial correlation is that it can distinguish direct dependency relationships. In 1996, Lauritzen [11] showed that for a multivariate Gaussian distributed system, their partial correlation coefficients are equivalent to the precision matrix, i.e., the inverse of the covariance matrix. Various methods have been established [12], [13], [14] and widely used to study functional brain networks due to the mathematical simplicity of the precision matrix [15], [16], [17]. To address the issue of estimating multiple distinct but related graphs, Guo et al. [18] and Danaher et al. [19] enforced regularization terms during estimation to enhance the similarity between graphs. However, these methods require the graph sequence to be fairly long as in the study of brain dynamics [20]. On the other hand, the Bayesian method is effective for a short sequence of small graphs and it can enhance the shared structure of multiple graphical models by employing some specific priors [21], [22]. These existing methods face the challenge of computational burden or are hard to utilize with statistical inference. In this work, we adopt the joint Bayesian-incorporating estimation of multiple GGMs proposed by Jia et. al [23], which can overcome the difficulties mentioned above. The method consists of three steps: ψ-score calculation, Bayesian integration, and joint edge detection. The advantages of the model are summarized as follows. First of all, the joint estimation is based on the ψ-learning method [14], which is known for its computational efficiency and flexibility for statistical testing, as in our recent study [16]. Second, we highlight the similarities of the graph structures by considering proper Bayesian priors and a meta-analysis procedure. Third, the joint detection test at the last step can eliminate the group bias of separate tests.

We applied the Bayesian-incorporating model to understand functional brain development during adolescence. The data were collected by the Philadelphia Neurodevelopmental Cohort (PNC). They include 855 individuals aged 8–22 years who were divided into five different adolescent stages. To further illustrate the advantages of the method, we conducted a series of simulation studies that approximate to brain connecticity patterns and compared it with the fused graphical LASSO (FGL) and group graphical LASSO (GGL) proposed by Danaher [19]. The brain networks over various adolescent stages were obtained through a 10-fold cross-validation process. We summarized the brain functional networks for each stage with several global network statistics [24] and extracted their common framework. We also conducted a hypothesis test among all the groups to capture the development patterns. For each development pattern, the hypergeometric test was applied to detect significant anatomical brain regions and functional modules.

The remainder of this paper is organized as follows. In Section II, we introduce the methodologies used throughout the study including the joint Bayesian-incorporating estimation of multiple GGMs and the hypothesis tests applied to age consecutive groups. The analysis results using PNC data are shown in Section III, followed by discussion of the results and concluding remarks in the last section.

II. Methods

This section describes an innovative way to jointly estimate multiple Gaussian graphical models (GGMs) under distinct but related conditions. In Section II.A, we introduce the detailed procedure of the method, which consists of three steps: 1) distinct graph estimation: ψ-score calculation; 2) common structure construction: Bayesian integration; 3) joint edge detection. Section II.B describes the hypothesis test procedure which can identify connectivity differences among multiple groups as well as the stage it gets to change.

A. Joint Bayesian-incorporating estimation of multiple GGMs

1). Distinct Graph Estimation

In the first step, we build an association network for each distinct condition, separately. Here, we adopt one of the GGM methods – the ψ-learning method [14], which we have applied to brain connectivity study [16]. To be more specific, let X = (X₁,X₂, …,X_p) be a p-dimensional random vector following a multivariate Gaussian distribution N(μ,Σ), where μ and Σ denote the unknown mean and covariance matrix, respectively. The definition of the partial correlation between X_i and X_j is given by ρ_{ij|V \ij} where V = {1, 2, ‥, p} denotes the whole set of variable indices. It has been shown [11] that under a Gaussian distribution, ρ_{ij|V \ij} can be expressed as

$${\rho }_{ij|V\backslash ij}=-\frac{{\Omega }_{i,j}}{\sqrt{{\Omega }_{i,i}{\Omega }_{j,j}}},i,j=1,2,\dots ,p$$ (1)

where Ω_i,j denotes the (i,j) entry of Ω, and Ω is the inverse matrix of Σ called the precision matrix. For high dimensional cases, i.e., sample size n < variable size p, the precision matrix cannot be directly estimated. To address this issue, Liang et al., one of the co-authors, proposed the ψ-learning method [14]. They define the ψ-correlation between X_i and X_j as Ψ_ij = ρ_ij|Sij, where S_ij is the reduced set that only includes the most correlated variables with X_i, X_j. Thus we can invert a smaller dimensional matrix for each pair. Liang et al. [14] showed that the ψ correlation coefficient is equivalent to the true partial correlation coefficient in the sense that

$${\mathrm{\Psi }}_{ij}=0\iff {\rho }_{ij|V\backslash ij}=0$$ (2)

where V = 1, 2, ‥, p denotes the whole set of variable indices.

To facilitate further analysis, we convert the ψ-correlations to ψ-scores via Fisher’s transformation

$$\begin{gathered} {\psi }_{ij}=\frac{\sqrt{n-\left|S_{ij}\right|-3}}{2}log\left[\frac{1+{\widehat{\mathrm{\Psi }}}_{ij}}{1-{\widehat{\mathrm{\Psi }}}_{ij}}\right], \\ i,j=1,2,\dots ,p \end{gathered}$$ (3)

where under null hypothesis H₀ : ρ_ij|V\ij = 0, ψ_ij follows the standard normal distribution. Thus the ψ-score can be used as a test statistic for the identification of non-zero partial correlation and thereby the structure of GGMs.

2). Common Structure Construction

In this step, we take into consideration the similarities among multiple consecutive conditions and estimate the common structure through a Bayesian clustering and meta-analysis method. Firstly, we use ${\psi }_{ij}^k$ to denote the ψ-score from Step 1), where (i, j, k) corresponds to the ψ-score between X_i and X_j under condition k with i, j = 1, 2, …, p and k = 1, 2, …,K. Let $e_{ij}^k$ indicate the status of the edge between X_i and X_j in the underlying graph k: $e_{ij}^k=1$ when there is an edge; $e_{ij}^k=0$ otherwise.

We assume that conditioned on $e_{ij}^k$, ${\psi }_{ij}^k$ are mutually independent following a mixture Gaussian distribution. We consider that the structure of GGM changes slightly over adjacent conditions, thus the mixture Gaussian distribution for each ${\psi }_{ij}^k|e_{ij}^k$ is independent of k, i.e.,

$$p({\psi }_{ij}^k|e_{ij}^k)=\{\begin{array}{ll}N({\mu }_{ij,0},{\sigma }_{ij,0}^2),\hfill & e_{ij}^k=0\hfill \\ N({\mu }_{ij,1},{\sigma }_{ij,1}^2),\hfill & e_{ij}^k=1\hfill \end{array}$$ (4)

Let ${\psi }_{ij}=({\psi }_{ij}^1,{\psi }_{ij}^2,\dots ,{\psi }_{ij}^K)$, $e_{ij}=(e_{ij}^1,e_{ij}^2,\dots ,e_{ij}^K)$. Then conditioned on e_ij, we can acquire the joint likelihood function of ψ_ij:

$$\begin{gathered} p({\psi }_{ij}|e_{ij},{\mu }_{ij,0},{\sigma }_{ij,0}^2,{\mu }_{ij,1},{\sigma }_{ij,1}^2)= \\ \prod _{\{k:e_{ij}^k=0\}}\varphi ({\psi }_{ij}^k|{\mu }_{ij,0},{\sigma }_{ij,0}^2)\times \\ \prod _{\{k:e_{ij}^k=1\}}\varphi ({\psi }_{ij}^k|{\mu }_{ij,1},{\sigma }_{ij,1}^2) \end{gathered}$$ (5)

where ϕ(·|μ, σ²) represents the normal density function with mean μ and variance σ². Taking a product of Eq. (5) over 1 ≤ i < j ≤ p, we can have the joint distribution of all ψ-scores {ψ_ij}i,j conditioned on {e_ij}i,j and other parameters. Based on Bayesian theorem, $e_{ij}^k’\text{s}$ can be inferred from proper priors. We assume that the e_ij’s are independent from each other, since the neighboring dependence has been accounted for in Step 1. Thus we have the following prior distribution:

$$\begin{gathered} p\left(e_{ij}|q\right)=q^{{\sum }_{k=1}^{K-1}{c}_{ij}^k}{\left(1-q\right)}^{{\sum }_{k=1}^{K-1}(1-c_{ij}^k)} \\ q\sim Beta\left(a_1,b_1\right) \end{gathered}$$ (6)

where $c_{ij}^k=\left|e_{ij}^{k+1}-e_{ij}^k\right|$, indicating the change of the edge status from condition k to condition k + 1, and q represents the prior probability of the status change that follows a Beta distribution, a₁ and b₁ are predetermined hyperparameters. Let ${\mu }_{l_0}$, ${\mu }_{l_1}$ have an improper uniform distribution,

$$\pi \left({\mu }_{l_0}\right)\propto 1,\pi \left({\mu }_{l_1}\right)\propto 1,$$

and ${\sigma }_{l_0}^2,{\sigma }_{l_1}^2$ follows an inverse gamma distribution,

$${\sigma }_{l_0}^2,{\sigma }_{l_1}^2\sim IG\left(a_2,b_2\right),$$

where a₂, b₂ are predetermined hyperparameters. We can get the posterior distribution of e_ij by integrating out all the other parameters

$$\begin{gathered} \pi \left(e_{ij}|{\psi }_{ij}\right)\propto \frac{\mathrm{\Gamma }\left(a_1+k_1\right)(\mathrm{\Gamma }\left(b_1+k_2\right)}{\mathrm{\Gamma }\left(a_1+k_1+b_1+k_2\right)} \\ \times \frac{1}{\sqrt{n_0}}{\left(\frac{1}{\sqrt{2\pi }}\right)}^{n_0}\mathrm{\Gamma }(\frac{n_0-1}{2}+a_2) \\ {\left[\frac{1}{2}\sum _{\{k:e_{ij}^k=0\}}{({\psi }_{ij}^k)}^2-\frac{{\left(\sum _{\{k:e_{ij}^k=0\}}{\psi }_{ij}^k\right)}^2}{2n_0}+b_2\right]}^{-\frac{n_0-1}{2}-a_2} \\ \times \frac{1}{\sqrt{n_1}}{\left(\frac{1}{\sqrt{2\pi }}\right)}^{n_1}\mathrm{\Gamma }(\frac{n_1-1}{2}+a_2) \\ {\left[\frac{1}{2}\sum _{\{k:e_{ij}^k=1\}}{({\psi }_{ij}^k)}^2-\frac{{\left(\sum _{\{k:e_{ij}^k=1\}}{\psi }_{ij}^k\right)}^2}{2n_1}+b_2\right]}^{-\frac{n_1-1}{2}-a_2} \end{gathered}$$ (7)

where $n_0=\#\{k:e_{ij}^k=0\}$, $n_1=\#\{k:e_{ij}^k=1\}$, $k_1={\sum }_{i=1}^{K-1}{c}_{ij}^k$, and $k_2=K-1-k_1$. Based on the marginal posterior distribution of e_ij, we can calculate its posterior possibility for each configuration, which in total has 2^K configurations, and we denote it as π_ij,d, for d = 1, 2, …, 2^K.

Our goal in this step is to get the integrated ψ-scores taking account of all conditions. According to our assumptions that ${\psi }_{ij}^k|e_{ij}^k$ is independent of k, we can estimate the expected ψ-value for each configuration through a simple Stouffer’s meta-analysis [25]. The updated ψ-value ${\tilde{\psi }}_{ld}=\left({\tilde{\psi }}_{ij,d}^1,{\tilde{\psi }}_{ij,d}^2,\dots ,{\tilde{\psi }}_{ij,d}^K\right)$ corresponding to the edge status is given below:

$${\tilde{\psi }}_{ij,d}^k=\{\begin{array}{l}{\sum }_{\left\{l:e_{ij,d}^l=0\right\}}{\omega }_l{\psi }_{ij}^l/\sqrt{{\sum }_{\left\{l:e_{ij,d}^l=0\right\}}{\omega }_l^2},\\ e_{ij,d}^k=0\\ {\sum }_{\left\{l:e_{ij,d}^l=0\right\}}{\omega }_l{\psi }_{ij}^l\sqrt{{\sum }_{\left\{i:e_{ij,d}^l=0\right\}}{\omega }_l^2},\\ e_{ij,d}^k=1\end{array}$$ (8)

for k = 1, 2, …,K, where l = 1, 2, …,K, $e_{ij,d}^l$ indicates the edge status between X_i and X_j in the lth condition under configuration d and the weight ω_l accounts for the size of the samples under each condition.

Finally, we combine the Bayesian clustering and meta-analysis results and give the integrated ψ-score:

$${\widehat{\psi }}_{ij}^k=\sum _{d=1}^{2^K}{\pi }_{ij,d}{\tilde{\psi }}_{ij,d}^k,k=1,2,\dots ,K$$ (9)

3). Joint Edge Detection

In the last step, we apply a multiple hypothesis test to jointly estimate multiple networks through a multiple hypothesis test on all of the integrated ψ-scores at once. The method we adopt is proposed by Liang and Zhang [26] which uses an empirical Bayesian procedure that allows for the dependence between test statistics.

B. Hypothesis test for age consecutive groups

Our goal is to track brain network development over a series of age consecutive groups. Therefore, we not only need to identify the significant differences but also are concerned about the periods. To achieve these two purposes, we adopt an approach from statistical process control [27] in that we set the performance in the late childhood (pre-adolescence) stage as the baseline and compare with the other groups to detect the aberrant edges. To be more specific, we define

$${\psi }_{d_{ij}}^k=({\widehat{\psi }}_{ij}^k-{\widehat{\psi }}_{ij}^1)/\sqrt{2},k=2,\dots ,K.$$

Under the null hypothesis $H_0:e_{ij}^k=e_{ij}^1,{\psi }_{d_{ij}}^k$ should follow the standard normal distribution N(0, 1).

III. A Study of Adolescent Brain Connectivity Development

A. Materials

The dataset we used is the publicly available Philadelphia Neurodevelopmental Cohort (PNC). It consists of fMRI images from 855 individuals using an emotion identification task. All MRI scans were acquired on a single 3T Siemens TIM Trio whole-body scanner. During the task, each subject was asked to label emotions displayed which include happy, angry, sad, fearful and neutral faces. The total scan duration was 10.5 min. Blood oxygenation level-dependent (BOLD) fMRI was acquired using a whole-brain, single-shot, multislice, gradient-echo (GE) echoplanar (EPI) sequence of 124 volumes (372s) with the following parameters TR/TE=3000/32 ms, ip = 90°, FOV= 192 × 192 mm, matrix = 64 × 64, slice thickness/gap=3 mm/0 mm. The resulting nominal voxel size was 3.0×3.0×3.0mm [28]. Standard preprocessing steps were applied using SPM12, including motion correction, spatial normalization to standard MNI space, and spatial smoothing with a 3mm full width at half max (FWHM) Gaussian kernel. Then multiple regression considering the influence of motion was performed and the stimulus on-off contrast maps for each subject were obtained. Finally, 264 functionally defined regions of interest (ROIs) were extracted based on the power parcellation [6]. The age range of the participating subjects was between 8 and 22 years. Due to physical and cognitive changes, we divided them into five groups, each representing a stage related to adolescence (Table I).

TABLE I:

Group division information.

Category index	Group name	Age period	# of subjects
1	Pre-adolescence	8–12	194
2	Early adolescence	12–14	150
3	Middle adolescence	14–16	158
4	Late adolescence	16–18	166
5	Post-adolescence	18–22	187

B. Simulation Studies

In this section, we illustrate the performance of the proposed model through a series of simulation studies. We considered two types of network structures, scale-free and small-world, to simulate the brain connectivity patterns and change them slightly with the evolving conditions. We fixed K = 5 and p = 264, and varied the sample size n = 100, 200. We denote Ω^k as the precision matrix at condition k, k = 1, 2, …,K. For each setting, at every condition k, we generated 10 datasets independently from the multivariate Gaussian distribution N(0, (Ω^k)⁻¹).

The scale-free network can be generated through the R package huge, and the small-world network can be generated through the R package igraph. Therefore, for each network structure, we first used the corresponding R package to generate Ω¹ and followed the same random edge deleting-adding procedure as in [23] to generate Ω^k, k = 2, 3, 4, 5, in a sequential manner. To be more specific, we first randomly removed 5% edges in Ω^k−1, k = 2, …, 5, by setting the corresponding non-zero elements to be 0, and then added 5% edges at random by giving them values drawn from the uniform distribution U[0.3, 0.5] to obtain ${\tilde{\mathrm{\Omega }}}^k$. To ensure the positive definiteness, we enforced a regularization term on the diagonal elements,

$${\mathrm{\Omega }}^k={\tilde{\mathrm{\Omega }}}^k+\lambda I,$$ (10)

and chose the minimum λ (λ ≥ 0), that made Ω^k positive definite.

We considered three methods for comparison, which were the original ψ-learning method [14], the fused graphical LASSO (FGL) and the group graphical LASSO (GGL) [19]. The FGL employed the fused LASSO penalty

$$P\left(\left\{{\mathrm{\Omega }}^1,{\mathrm{\Omega }}^2,\dots ,{\mathrm{\Omega }}^5\right\}\right)={\lambda }_1\sum _{k=1}^K\sum _{i\ne j}\left|{\omega }_{ij}^k\right|+{\lambda }_2\sum _{k\le k^{\prime }}\sum _{i,j}\left|{\omega }_{ij}^k-{\omega }_{ij}^{k^{\prime }}\right|,$$ (11)

to enhance the similarity of precision matrices across multiple conditions, while the GGL employed the group LASSO penalty

$$P\left(\left\{{\mathrm{\Omega }}^1,{\mathrm{\Omega }}^2,\dots ,{\mathrm{\Omega }}^5\right\}\right)={\lambda }_1\sum _{k=1}^K\sum _{i\ne j}\left|{\omega }_{ij}^k\right|+{\lambda }_2\sum _{i\ne j}{\left(\sum _{k=1}^K{\omega }_{jk}^2\right)}^{1/2}.$$ (12)

In both equations (11) and (12), ${\omega }_{ij}^k$ denotes the (i, j)th element of the precision matrix Ω^k, the first penalty term λ₁ controls the sparsity of each precision matrix and the second penalty term λ₂ controls the similarity across all precision matrices. To determine the optimal values of λ₁ and λ₂, we followed the same procedure as in [19] by searching over a grid of possible values for a combination that minimizes the Akaike information criterion (AIC). The JGL and the GGL models are available in the R package JGL and the ψ-learning model can be implemented through the R package equSA.

We compared the performances of the four methods through the receiver operating characteristic (ROC) curves shown in Figure 1. To draw the ROC curve for the proposed method and the ψ-learning method, we fixed the significance level of correlation screening to α₁ = 0.2 and varied the value of α₂, the significance level of edge detection; as for the JGL and the GGL, we fixed the λ₂ to its optimal value at which the minimum AIC is achieved, and varied the value of λ₁. Table II further compares the corresponding mean areas under the ROC curves (AUCs) of the four methods. As we can see, the joint estimation method (J-estimation) significantly outperforms the existing ones, especially when the sample size is small. When the sample size gets large, the J-estimation, FGL and GGL tend to perform similarly; however, the J-estimation still performs the best.

Fig. 1:

The ROC curves under various variable settings for the discrete (left column) and mixed (right column) scenarios with n = 200. The top row shows the performance for p = 50, while the bottom row shows for p = 500.

TABLE II:

The mean AUCs of different estimators for various (n, p) settings

	Scale-free		Samll-world
	n = 100	n = 200	n = 100	n = 200
J-estimation	0.971 (0.001)	0.996 (0.001)	0.955 (0.002)	0.986 (0.001)
ψ-learning	0.898 (0.005)	0.958 (0.003)	0.912 (0.005)	0.959 (0.003)
JGL	0.930 (0.006)	0.989 (0.002)	0.935 (0.004)	0.972 (0.003)
GGL	0.915 (0.007)	0.980 (0.002)	0.917 (0.005)	0.941 (0.003)

C. Results

1). Network Summary

We set the predetermined hyperparameters a₁ = a₂ = b₂ = 1, b₁ = 10 for the Bayesian integration and significance level α = 0.05 for the joint edge detection as suggested in [23]. We applied 10-fold cross validation and extracted the subset networks that were consistent on every iteration. We summarized the networks with 5 statistics: network density, mean clustering coefficient (MCC), transitivity, global efficiency (GE), and characteristic path length (CPL). The MCC calculates the mean fraction of triangles around each node in the network, while the transitivity measures the ratio between the number of triangles and the number of connected node triples. For example, if every triplet of connected nodes forms a triangle (as opposed to a V shaped wedge), then the transitivity would be equal to 1. The MCC and the transitivity are two measures of the brain functional segregation, with the ability for specialized processing to occur within densely interconnected groups of brain regions [24]. The CPL is the average shortest path length between all pairs of nodes in the network, while the GE is the average inverse shortest path length. They both indicate the ability to combine specialized information from distributed brain regions, also known as the functional integration ability. Table III lists 5 measures for the 5 adolescent groups. The density results indicate that from pre-adolescence (late childhood) to post-adolescence (young adulthood), the brain functional connectivity under emotion identification task increases and it is most active in early adolescence. The conclusions drawn from the MCC and transitivity are opposite, so are the ones from the CPL and GE. We choose the transitivity and GE with reasons discussed in Section IV. Therefore, from preadolescence to post-adolescence, the brain ability to process and combine specialized information is improved.

TABLE III:

Global measures of the brain connectivity networks

Group Index	Measures
	Density	MCC	Transitivity	CPL	GE
1	0.009849	0.412514	0.114641	4.34395	0.11795
2	0.023422	0.42381	0.199434	3.562147	0.270395
3	0.016456	0.318474	0.172414	4.266679	0.182661
4	0.012812	0.306563	0.172973	4.619742	0.132694
5	0.015054	0.386157	0.202899	4.640903	0.161206

Furthermore, we divided the 264 brain regions into 13 functional network (FN) modules to study regional connectivity, including sensory/somatomotor hand network (SSH), sensory/somatomotor mouth network (SSM), cingulo-opercular task control network (CON), auditory network (AUD), default mode network (DMN), memory retrieval network (MRN), visual network (VN), fronto-parietal task control network (FPN), salience network (SN), subcortical network (SCN), ventral attention network (VAN), dorsal attention network (DAN) and cerebellum network (CERE). Fig 2 shows the change of the degree of each functional network module over the 5 adolescent groups, where the degree of a functional network module is defined as the total number of edges identified in this module. Most FN modules reach their peaks in early adolescence, especially the DMN, SN, FPN, VN and SSH. Only the connectivity in the SCN is most active in the middle adolescent group. The brain connectivity networks for the 5 adolescent groups are shown in Fig. 3 and we also extract their common connectivity patterns given in Fig. 4. For each FN module, we further calculated the ratio of the number of edges between the common connectivity network and the pre-adolescence network. The results show that the common pattern mainly stays in the CON and SCN modules (with the ratio 0.61 and 0.53, respectively).

Fig. 2:

The degree distribution of each functional network module over various adolescent age categories.

Fig. 3:

The visualization of the brain networks from preadolescence (top) to post-adolescence (bottom) using the proposed joint estimation method. The left column shows the ψ-score matrix and the right column shows a sagittal view of the brain connectivity.

Fig. 4:

The common connectivity patterns over the 5 adolescent stages in axial, sagittal and coronal views.

2). Developmental Changes in Brain Connectivity

We used the results from the pre-adolescent group as a baseline to compare with the other groups. This is because, on one hand, the pre-adolescent period is developmentally the earliest. However, on the other hand, it is more clear to track the changes using the same standard. Based on the multiple hypothesis test results, we found two connectivity development patterns during adolescence, which can be described as permanent change and temporal change, respectively. As shown in Fig. 5, the permanent change is defined as the change of the brain connection remaining until the post adolescent (young adulthood) period. The temporal change is the change that only exists before the post-adolescent period, which is the unique connectivity pattern in adolescence. The permanent change can further be divided into two sub-patterns: the development that starts from the early or middle adolescence period is called consistent development, and the ones which happened at or after the late adolescence is defined as late puberty.

Fig. 5:

An illustration of the connectivity development patterns. The x-axis shows the adolescent age categories orderly and each bar represents one development trajectory after preadolescence.

Permanent change

As shown in Fig. 6 (a), we detected 277 edges in the consistent development patterns. Among them, we discovered 20 hub ROIs that were significantly involved (see Table IV). Here the hub ROIs are selected through a hypergeometric test at the significance level α = 0.05 with FDR correction [29]. Five of the hub ROIs were anatomically located in the middle frontal gyrus (MFG), two were located in the putamen (PUT), and the rest were located in the right precentral gyrus (PreCG.R), right precuneus (PCUN.R), right middle frontal gyrus orbital (MFGO.R), right fusiform gyrus (FFG.R), right anterior cingulate gyrus (ACG.R), left thalamus (THA.L), right rectus (REC.R), left parahippocampus (PHG.L), right inferior temporal gyrus (ITG.R), left cerebellum 6 (CRBL6.L) and right cerebellum crus 2 (CRBLCrus2.R). Further, we applied the same hypergeometric test with FDR correction [29] to identify the highly involved FN modules. We discovered that compared to other FN modules, the salience network (SN) and the subcortical network (SCN) were more activated. In Fig 6 (b), we display the 45 connections that were only significantly different from the late adolescent period (after 16 years old) and discovered 3 hub ROIs: two were located in the right lingual gyrus (LING.R) and the other were located in the CRBLCrus2.R. For the FN module, we found that the AUD were significantly involved in this pattern.

Fig. 6:

The visualizations of the three connectivity development patterns.

TABLE IV:

The identified hub ROIs for each development pattern

Development pattern	ROI index	MNI space (X, Y, Z)	Anatomical region (AAL)	FN module
consistent development (* Temporal change)	41	(38, −17, 45)	PreCG.R	SSH
	92	(8, −48, 31)	MCG.R	DMN
	93	(15, −63, 26)	PCUN.R	DMN
	100	(−35, 20, 51)	MFG.L	DMN
	109	(−3, 44, −9)	MFGO.L	DMN
	125	(27, −37, −13)	FFG.R	DMN
	196	(40, 18, 40)	MFG.R	FPN
	197*	(−34, 55, 4)	MFG.L	FPN
	214*	(−28, 52, 21)	MFG.L	SN
	217*	(10, 22, 27)	ACG.R	SN
	218	(31, 56, 14)	MFG.R	SN
	223*	(−2, −13, 12)	THA.L	SCN
	227	(−22, 7, −5)	PUT.L	SCN
	229	(31, −14, 2)	PUT.R	SCN
	243	(−16, −65, −20)	CRBL6.L	Cere
	5*	(8, 41, −24)	REC.R	–
	6*	(−21, −22, −20)	PHG.L	–
	184*	(17, −80, −34)	CRBLCrus2.R	–
	254*	(46, −47, −17)	ITG.R	–
Late puberty	2	(27, −97, −13)	LING.R	–
	140	(8, −91, −7)	LING.R	–
	184	(17, −80, −34)	CRBLCrus2.R	–

^*The hub ROIs for both the consistent development and the temporal change patterns.

- Uncertain FN module

Temporal change: As illustrated in Fig. 6 (c), 373 brain connections were found to be different from the pre-adolescent period and returning to baseline at the post-adolescent period. Among them, nine hub ROIs were detected and they were a subset of the hubs that displayed a consistent development pattern. Three hub ROIs were located in the MFG, and the rest were each located in the ACG.R, THA.L, REC.R, PHG.L, ITG.R and CRBLCrus2.R. The two significant FN modules detected, SN and SCN, were the same as for consistent development.

3). Summary

Overall, from late childhood to young adulthood, the number of brain functional connections increased and the ability to process and combine specialized information improved. Considering the whole adolescent period, the early adolescent stage is the most active. Three development patterns have been discovered, which are consistent development, late puberty and temporal change. From the perspective of brain developmental changes, we found some important ROIs. The middle frontal gyrus (MFG) plays a key role in the brain’s consistent development and temporal change phases, the ROI with MNI coordinates (17,−80,−34) at the right cerebellum crus 2 (CRBLCrus2.R) is a hub center for all 3 patterns, the putamen gyrus (PUT) is significantly functioning in the consistent development and the right lingual gyrus (LING.R) is central in late puberty. Functionally, we first noticed that the number of connections in the DMN, SN, FPN, VN, SCN and SSH increased dramatically in the early adolescent stage. SCN was the only module where the number of connections reached the peak in the middle adolescent stage. We extracted the common pattern from the networks of the 5 groups and compared it with the baseline (pre-adolescent group). The CON and SCN maintained a high ratio (over 0.5) of common connections. From the brain development patterns, the hubs in the consistent development pattern mostly belong to the DMN, SN and SCN, and the hubs in the temporal change pattern are in the SN. Further, the hypergeometric tests on significant FN modules show that the SN and SCN are essential in the consistent development and the temporal change, while the AUD is significant in late puberty.

IV. Discussion and Conclusion

In this paper, we applied a Bayesian-incorporating method [23] to jointly estimate the brain connectivity networks of various adolescent periods using the fMRI images collected during an emotion identification task from PNC. This approach consisted of three steps: we first construct a distinct graph for each group using the ψ-learning method [14], then build the common structure through a Bayesian-incorporating procedure, and finally go through a joint edge detection and obtain the graphs for each group. In the Bayesian-incorporating procedure, several assumptions have been made. First, we assume that conditioned on $e_{ij}^k$, ${\psi }_{ij}^k$ are mutually independent following a mixture Gaussian distribution. Second, we assume the structure of the GGM changes only slightly under adjacent conditions. Therefore, the follow up assumption is reasonable that both the distribution components of $p({\psi }_{ij}^k|e_{ij}^k):N({\mu }_{ij,0},{\sigma }_{ij,0}^2)$, $N({\mu }_{ij,1},{\sigma }_{ij,1}^2)$ are independent of k, k = 1, 2, …,K. Third, we assume that e_ij’s are a priori independent of all i, j, i, j = 1, 2, …, p, as we believe that the neighboring dependence of the Gaussian graphical network has already been accounted for in the calculation of ψ-scores [23]. To illustrate the performance of this joint Bayesian-incorporating estimation method (J-estimation), we compared it with the JGL, GGL models [19], and the ψ-learning method [14] through a series of simulation studies. The results show that the J-estimation method outperforms the other three under various conditions. Additionally, the J-estimation method successfully overcomes the difficulty in analyzing short sequence graphs with low computational cost. As discussed in [23], even when the total condition number K gets larger, the calculations of the correlation coefficients and ψ-scores can be done in parallel. Therefore, it can still be executed quickly.

In Section III.C, we used some network statistics to summarize the brain connectivity networks. Among them, we chose transitivity over the mean clustering coefficient (MCC), and the global efficiency (GE) over the characteristic path length (CPL). This is because MCC is normalized individually for each node and may therefore be disproportionately influenced by the nodes with a low degree, while the transitivity is normalized collectively and consequently does not suffer from this problem [30]. For the other pair, the CPL is primarily influenced by long paths (infinitely long paths are an illustrative extreme), while the global efficiency is primarily influenced by short paths. Achard and Bullmore have argued that this may make the global efficiency a superior measure of integration [31]. From Table III, one can discern that although the density of the brain connectivity networks in early adolescent stage is higher than the one in post-adolescent stage, the transitivity values of both groups are similar. This suggests that although there are dramatic developments of brain interconnections in early adolescence, the adolescent brain is not as efficient and organized as it is in young adult.

In our emotion identification task analysis, the middle frontal gyrus (MFG) was a central functional area of development through the whole adolescent period. The MFG, known as a part of a multiple demand system, is strongly involved in the cognitive control process [32]. The MFG.L is a part of the executive attention network [33], while the MFG.R is a site of convergence of the DAN and VAN and may serve as a circuit-breaker to interrupt ongoing endogenous attentional processes in the dorsal network and reorient attention to an exogenous stimulus [34]. Deeley et al. [35] reported that there was a significant negative correlation between increasing age and neural responses to fearful and disgusted expressions in the MFG from adolescence to middle age. They explained that this may be caused by a reduction in attentional demands as perceptual skill increases, or changes in the processing of the self-relevance of facial expressions during social and cognitive development. The development of the putamen was consistent from early adolescence to post adolescence. Studies have shown that the putamen is involved in facial expression recognition [36] and in response to happiness [37]. Badgaiyan [38] showed that dopamine is released in the striatum during human emotional processing and Montague [39] suggested that the density of dopamine D1 receptor in human caudate and putamen is age related. Therefore, the importance of the MFG and putamen in adolescence is supported by the literature. Although it has been shown in emotional recognition, the sad condition results in significant activation in the right lingual gyrus (LING.R) [40], and its role in late puberty still needs to be explored. The role of cerebellum crus 2 (CRBLCrus2) has been seldom mentioned during the emotion identification task. However, the cerebellum is recognized as a prominent contributor to cognitive and emotional functions [41]. Evidence shows that during adolescence, total cerebellum volume follows an inverted U shaped developmental trajectory and subdivisions of the cerebellum have distinct developmental trajectories in late puberty [42]. We found that the cerebellum crus 2 (CRBLCrus2) appeared in all 3 development patterns, and thus may be an essential region to study.

In regard to the FN modules, the development of the subcortical network (SCN) was the most influential module during the whole adolescent period. First, we discovered that the SCN keeps a high percentage of connections from the pre-adolescent stage. Second, unlike the other modules, the degree distribution in the SCN reaches its peak at the middle adolescent stage. Third, several hub ROIs follow a consistent development pattern analogous to the SCN. Finally, the SCN is significantly activated in both consistent development and temporal change periods. Some of our conclusions have been supported in the literature. Early to mid-adolescence is an important developmental period for subcortical brain maturation and its development provides a framework for interpreting normal and abnormal changes in cognition, affect and behavior [43]. The salience network (SN) also plays a crucial part in the adolescent functional brain development for emotion processing. Several hub ROIs identified in the consistent development and temporal change patterns belong to the SN and it is also significant in both patterns through the hypergeometric tests. The SN contributes to various complex brain functions through sensory, emotional, and cognitive information [44]. We have discovered that the MFG and the right anterior cingulate are critical in development. Other than the MFG, plenty of studies have been conducted on adolescent brain development in the anterior cingulate cortex [45]. In [3], they conducted a fMRI study under an emotion regulation task in children, adolescents and young adults, and observed a quadratic effect of age in the anterior and posterior cingulate cortices. Beyond the SN and the SCN, we also found that the AUD module is significant in late puberty. However, this finding still needs further validation.

In this paper, we have used the undirected GGM to study brain functional network development in adolescence. However, the directionality of functional interactions in the brain remains poorly understood. Directed GGMs [46], [47] can be used to model the causal relationships in the brain. In future studies, we will work on the joint estimation of multiple GGMs like the model by Wang et al. [48] to obtain a better understanding of the adolescent brain.

TABLE V:

The detected FN modules through the hypergeometric test for each development pattern.

Type		FN Module	p – val
Permanent Change	Consistent Development	SN	1.45 × 10−13
		SCN	1.26 × 10−4
	Late Puberty	AUD	0.03
Temporal Change		SN	2.43 × 10−12
		SCN	1.60 × 10−3