Estimating dynamic functional brain connectivity with a sparse hidden Markov model

Abstract

Estimating dynamic functional network connectivity (dFNC) of the brain from functional magnetic resonance imaging (fMRI) data can reveal both spatial and temporal organization and can be applied to track the developmental trajectory of brain maturity as well as to study mental illness. Resting state fMRI (rs-fMRI) is regarded as a promising task since it reflects the spontaneous brain activity without an external stimulus. The sliding window method has been successfully used to extract dFNC but typically assumes a fixed window size. The hidden Markov model (HMM) based method is an alternative approach for estimating time-varying connectivity. In this paper, we propose a sparse HMM based on Gaussian HMM and Gaussian graphical model (GGM). In this model, the time-varying neural processes are represented as discrete brain states which are described with functional connectivity networks. By enforcing the sparsity on the precision matrix, we can get interpretable connectivity between different functional regions. The optimization of our model can be realized with the expectation maximization (EM) and graphical least absolute shrinkage and selection operator (glasso) algorithms. The proposed model is validated on both simulated blood oxygenation-level dependent (BOLD) time series and rs-fMRI data. Results indicate that the proposed model can capture both stationary and abrupt brain activity fluctuations. We also compare dFNC patterns between children and young adults from the Philadelphia Neurodevelopmental Cohort (PNC) study. Both spatial and temporal behavior of the dFNC are analyzed and compared. The results provide insight into the developmental trajectory across childhood and motivate further research on brain connectivity.

I. INTRODUCTION

Recent progress in functional network connectivity (FNC) analysis from fMRI time series (i.e., functional connectivity between coherent brain networks) has enabled a better understanding of human brains. The combined power of brain imaging data and statistical models have provided a window into the spatiotemporal organization of both healthy and diseased brains [1], [2]. For example, Zhi et al. [3] explored the FNCs of individuals with major depressive disorder (MDD) and found abnormal time-varying brain activity and the corresponding network disruptions. Lacy et al. [4] showed that the FNCs could capture the motifs associated with group effects from the diagnosis of attention-deficit/hyperactivity disorder (ADHD), involving key differences in the default mode network (DMN). There are also various FNC related differences revealed in other clinical studies [5], [6], [7], [8], [9] (i.e. schizophrenia and bipolar disorder as well as drug and alcohol abuse). In addition, research has shown changes in FNCs as the brain matures for both task and resting scenarios, such as examining the different inter-regional connectivity of occipital and temporal regions during reading [10], including distinct DMN and cognitive control system connectivity in the resting state [11]. These networks have also been shown to be quite reliable across studies [12].

As research showed that the the brain is intrinsically organized into dynamic functional networks [13], [14], [15], the study of dynamic functional network connectivity (dFNC) began to attract increasing interest. For example, some temporal characteristics like the recurring patterns of brain FNCs [16], [17], [18], [19] and hierarchically organized brain states, can only be identified via a dynamic analysis. More importantly, in several clinical and developmental studies, brain images disclose more details on mental illness and brain development through a dynamic approach [6], [20], [21].

The sliding window approach is the most popular method applied in the dFNC analysis [17], [21]. With a preset window width, one can estimate the brain connectivity along the time axis by sliding the window along the time series under a static assumption. The main limitation of this method is the choice of the window size: if the size is too small, a limited number of observations cannot guarantee a reliable connectivity estimation; if the size is too large, abrupt brain activity will be overlooked in the results. The hidden Markov model (HMM) is an appealing alternative approach for modeling dynamic connectivity. It assumes that brain activity has several discrete states characterized by different connectivity patterns. The time-courses come from distributions parameterized by those connectivity patterns. Although HMM is based on the assumption of the Markov property of connectivity patterns, it avoids the choice of the window size and hence offers a flexible way to track either stationary or abrupt brain connectivity changes. For example, D. Vidaurre et al. [22] revealed fast brain dynamics related to the task in which the sliding window method fails. Baker et al. [23] also applied HMM to group analysis, to discover the reoccurring brain connectivity at the population level. However, the direct application of HMM to brain fMRI time series is challenging due to the high dimensionality of brain images. To address this issue, researchers applied different methods. In [24] and [25], authors applied a strong structural prior on the connectivity matrix. In [22] and [26], researchers concatenated subjects data along the time domain and then applied the estimation. Motivated by the work in [27] and [28], in this paper, we modeled the brain dynamics using the HMM model with a sparse regularization. The regularized HMM enabled us to estimate the brain dynamics under a weaker assumption on the connectivity matrix. Moreover, the model can also be used at either the individual or population level. Based on the EM algorithm [29], we showed that the sparse regularization on the precision matrices can be solved with the famous graphical least absolute shrinkage and selection operator (glasso) approach [30]. To validate the sparse HMM, we applied that model on both simulated data and the rs-fMRI data from PNC study and compared its performance with the sliding window approach. Based on the model, we also did a group comparison study between children and young adults.

The rest of the paper is organized as follows. In the next section, we briefly discuss the generative model based on the sparse HMM, its optimization and the model selection. In Section 3 the performance of our proposed model is validated on both simulated data and real rs-fMRI time series from PNC study. The comparison study of two age groups are also included in this section. In Section 4, we give a brief description of our model and other related works. The results and potential problems are discussed in Section 5.

II. METHODS

In this section we will describe how to model the brain image time series using an HMM framework. Assuming we have $R$ subjects in a brain image data set, each of them has a $p$ dimension time series with $N$ samples (The number of samples may vary in real data but we can assume that without loss of generality). In fMRI data, $p$ can be the number of voxels, regions of interest (ROI), or components. Denote the time series data as $\chi =\{X^{(1)},X^{(2)},\dots ,X^{(R)}\}$ with $X^{(r)}=[x_1^{(r)},x_2^{(r)},\dots ,x_N^{(r)}]\in {\mathbb{R}}^{p\times N}$ and the set of hidden state variables (or latent variables) as $\zeta =\{Z^{(1)},Z^{(2)},\dots ,Z^{(R)}\}$. Here the latent variables come from a discrete state set with size $K$. At every time point, the time course is generated according to the same distribution decided by the time-varying hidden state. That is, the hidden states correspond to specific brain activity patterns which reappear in the time series. For the rth subject, the hidden state is denoted as $Z^{(r)}=[z_1^{(r)},z_2^{(r)},\dots ,z_N^{(r)}]\in {\mathbb{R}}^{K\times N}$, in which each vector is from 1 – of – $K$ coding scheme and the position index of 1 in the vector is the hidden state number. As an initialization, each state has an initial probability ${\pi }_k$. During the time series, the hidden states are associated by a transition matrix $A$, in which each $A_{ij}$ refers to the probability of moving from state $i$ to state $j$. Since the hidden variables satisfy the Markov property, this transition matrix is independent of time. For the $r\text{th}$ subject, given the hidden state at time $n{z_n^{(r)}}_k=1$, the appearance of time course obeys a Gaussian distribution with ${\mu }_k$, ${\Sigma }_k$ as mean and covariance matrix:

$$\begin{gathered} p(x_n^{(r)}\mid {z_n^{(r)}}_k=1)=\frac{1}{{(2\pi )}^{\frac{p}{2}}{\mid {\Sigma }_k\mid }^{\frac{1}{2}}}. \\ \text{exp}(\frac{-{(x_n^{(r)}-{\mu }_k)}^T{({\Sigma }_k)}^{-1}(x_n^{(r)}-{\mu }_k)}{2}) \end{gathered}$$ (1)

where $\mid \cdot \mid$ is the determinant. In Figure 1, we show a schematic illustration of the Gaussian HMM. On the right is a sample of transition matrix between 3 states. All the ${\pi }_{ij}\mathrm{s}$ are time-independent parameters in [0, 1]. Combining that with the transition probability, the probability likelihood of the $r\text{th}$ subject is written as

$$p(X^{(r)},\zeta \mid \theta ,\varphi )=p(z_1^{(r)}\mid \pi )\prod _{n=2}^{N}p(z_n^{(r)}\mid z_{n-1}^{(r)},A)\prod _{m=1}^{N}p(x_m^{(r)}\mid z_m^{(r)},\varphi )$$ (2)

Fig. 1.

Schematic illustration of the multi-variant Gaussian HMM. The left shows the states and observations of $r\text{th}$ subject. The right shows the transition between states when $K=3$.

Maximizing this probability gives the maximum likelihood estimation (MLE) of the model parameters $\theta =\{\pi ,A\}$ and $\varphi ={\{{\mu }_k,{\Sigma }_{k=1}^K\}}_{k=1}^K$. The implementation is generally performed via EM algorithm [31], [32], which is described in Section 1 in supplementary material.

Although ${\Sigma }_k$ is defined as the covariance matrix of Gaussian distribution, its estimation often turns out to be a singular matrix due to insufficient observations. For example, the fMRI data we used has been preprocessed to reduce the data dimensionality from millions to hundreds. However, we still need to deal with this problem because the number of samples is only around 100. Since the EM algorithm is an iterative algorithm, the iteration even fails at the beginning with a singular ${\Sigma }_k$. Another problem is the complexity of a dense covariance matrix. In previous research, investigators have used the covariance matrix or its inverse matrix (known as the precision matrix) indiscriminately to represent the functional network connectivity. But in either case, we want to reveal the most significant connectivity instead of a ”whole connectivity” between each of two functional regions. Sparsity has been proven to be an efficient way to overcome these issues. In previous work [17] and [21], they both constrained the brain connectivity with sparsity; authors also assumed the independence of different ROIs in their spatial-temporal modeling work [25]. We borrow the similar idea as in [27] and [28] and introduce the $l_1$ regularization on the precision matrix ${\Sigma }_k^{-1}$. If $\gamma (z_n^{(r)})=p(z_n^{(r)}\mid \chi ,\tilde{\theta })$ is denoted as the marginal posterior distribution of latent variables $z_n^{(r)}$ and $\gamma (z_{nk}^{(r)})$ is the conditional probability of $z_{nk}=1$, with the trade-off parameters ${\{{\lambda }_k\}}_{k=1}^K$, the objective log-sum maximization function in the M step turns out to be

$$\begin{gathered} \underset{{\varphi }_k}{\max }\sum _{r=1}^R\sum _{k=1}^K\sum _{n=1}^N\gamma (z_{nk}^{(r)})\log p(x_n^{(r)}\mid {\varphi }_k^{(r)})+\sum _{k=1}^K{\lambda }_k{\Vert {\Sigma }_k^{-1}\Vert }_1 \\ \stackrel{\forall k}{\sim }\underset{{\tilde{\Sigma }}_k}{\text{min}}-\log \mid {\tilde{\Sigma }}_k\mid +Tr({\tilde{S}}_k{\tilde{\Sigma }}_k)+{\tilde{\lambda }}_k{\Vert {\tilde{\Sigma }}_k\Vert }_1 \end{gathered}$$ (3)

where

$$\begin{gathered} {\tilde{S}}_k=\frac{{\sum }_{r=1}^R{\sum }_{n=1}^N\gamma (z_{nk}^{(r)})(x_n^{(r)}-{\mu }_k){(x_n^{(r)}-{\mu }_k)}^T}{{\sum }_{r=1}^R{\sum }_{n=1}^N\gamma (z_{nk}^{(r)})}, \\ {\tilde{\lambda }}_k=\frac{{\lambda }_k}{{\sum }_{r=1}^R{\sum }_{n=1}^N\gamma (z_{nk}^{(r)})} \end{gathered}$$ (4)

This is the exact formulation of sparse graphical model. Hence the optimization of Equation(3) can be solved by the glasso algorithm as in [30]. Fortunately, the newly introduced $l_1$ term will not affect the Expectation step computation so we can plug it in the original EM iteration. The algorithm was summarized in Algorithm S1 in supplementary material. Once the EM algorithm converges and the estimation of $\theta$, $\varphi$ is obtained, we can further decode the sequence of hidden states $\theta$ via the Viterbi algorithm [33].

There are two parameters to be determined: the state number $K$ and the sparsity trade-off term $\lambda$. Although most of studies used Bayesian information criteria(BIC) or cross validation for the model selection, Figueiredo et al. [34] has demonstrated that in real and synthetic data, the mixture minimum description length (MMDL) was a better choice with a consideration of effective code lengths for each state in a mixture model. MMDL is defined by

$$\begin{gathered} MMDL(\varphi (K),\theta (K)) \\ =-log(p(\varphi (K),\theta (K);\chi ))+\frac{1}{2}\log (NR)K(K-1) \\ +\sum _k\frac{1}{2}\log (NR{\gamma }_k)Df(k) \end{gathered}$$ (5)

where $Df(k)$ is the degrees of freedom of ${\varphi }_k$ and ${\gamma }_k={\sum }_{r=1}^R{\sum }_{n=1}^N\gamma (z_{nk}^{(r)})$. The $Df(k)$ is the number of non-zeroes in the precision matrix ${\Sigma }_k^{-1}$. The idea of MMDL can be explained with the minimum description length principle in [35]. The state number $K$ is determined by minimizing the MMDL on a given list of $K$. For the model selection on ${\lambda }_k$, we employ the universal regularization technique in [27] and use the effective sample size to reweight each ${\lambda }_k$ by multiplying $\sqrt{NR{\gamma }_k}$. Then a universal choice of ${\lambda }_k$ is ${\lambda }_k={\Sigma }_{k(pp)}^{-1}\sqrt{2NR(\log p)}∕2$, where ${\Sigma }_{k(pp)}^{-1}$ is the last diagonal element in ${\Sigma }_k^{-1}$.

III. RESULTS

A. Analysis of simulated dataset

a), Simulation data generation.

We firstly validate the proposed sparse HMM (SPHMM) with two simulation data sets. The SimTB toolbox [36] was employed to generate the simulated fMRI time series.

For the first simulation data set, The BOLD signals were generated with 60 ROIs. For each ROI, the time series contained 120 time points and repetition time (TR) is 2 seconds. The state number is set to be 4 and each state consists of different permutations of 4 components. Each component includes 8 to 16 ROIs and is activated for different periods. A binary value was assigned to denote activation or inactivation of this component. The 4 components are highlighted in Figure 2(a) and their activation and deactivation activity along the time dimension are represented in Figure 2(b). For example, all of the 4 components are activated during the first 30 time points, while only the 4th component is activated during the last 30 time points. Figure 2 is an illustration of the first simulation data for one run.

Fig. 2.

Illustration of the first simulation data set: (a) 4 components activated in the time series(b) The activation and deactivation of 4 components along the time dimension

The experiment was run on 5 independent synthetic data sets (each containing 200 subjects) with the same groundtruth connectivity patterns. In the second simulation data set, we want to verify if the proposed HMM can track the fast transition of dFNCs. We used 3 states obtained from a sparse HMM estimation on the task fMRI data of PNC data. Then, the estimated HMM is regarded as the ground truth to generate simulation data. To embody the inter-subject variances, a small perturbation was added to the covariance matrix by adding a random matrix with scale factor 0.05. Finally, 200 subjects’ time courses are simulated with 200 discrete samples.

b), Simulation Results.

To evaluate the performance of the proposed sparse HMM method, we compare the estimated dynamic connectivity with the ground truth and the estimation from the sliding window approach. In the sliding window approach, we tested two different window lengths to illustrate its subtle effect ($l=30$, 40) and the sparse regularization parameter $\lambda$ was selected according to [17]. In the sparse HMM method, $\lambda$ was determined by the universal choice. The details of the dynamic connectivity result are summarized in Figure S1 in the supplementary material.

To validate the simulation result, we further calculate the relative error as well as the Wasserstain distance(WD)(Eq 6) between the ground truth of connectivity $\text{matrix}(C_g)$ and the results of those approaches ($C_E$).The relative error is defined as

$$RelError≔\frac{{\Vert C_g-C_E\Vert }_F}{60\times 60}$$

Table I shows the relative error and the WD of three aforementioned estimations to the ground truth. The value in bold indicates that the result is the best in that column. From the table, we can see the proposed model fits two out of four connectivity matrices better than the sliding window approach under both measures. For the rest of the two states, our method also got a comparable result.

TABLE I.

Relative Error of 4 Estimated Connectivity States

$(RelError)\times {10}^{-3}$	State1	State2	State3	State4
SPHMM	0.3801(0.0690)	0.3477(0.0521)	0.3819(0.0540)	0.5145(0.0987)
$\text{SW}(l=30)$	0.4549(0.0225)	0.3381(0.0401)	0.4001(0.0851)	0.7630(0.112)
$\text{SW}(l=40)$	0.6661(0.1030)	0.3224(0.0530)	0.6992(0.0923)	0.9391(0.144)
$(WD)\times {10}^{-3}$	State1	State2	State3	State4
SPHMM	0.3107(0.0594)	0.2961(0.0345)	0.3413(0.0327)	0.4167(0.0798)
$\text{SW}(l=30)$	0.3800(0.0198)	0.2717(0.0315)	0.3165(0.0542)	0.5496(0.0940)
$\text{SW}(l=40)$	0.5222(0.0860)	0.3081(0.0374)	0.4980(0.0903)	0.6257(0.0836)

Then we use the second simulation data to test if the SPHMM can reveal the fast transient brain states. To do so, we firstly find the time points whose state is accurately predicted by SPHMM and then calculate the ratio of these time points to the simulated time length. It shows that we can get a 95.32% ± 1.75% accuracy among the population even when we assume there is high variances among subjects. As an illustration, an example of the inferred state sequence and its ground truth is presented in Figure 3(a) and (b). In Figure 3(c), the ground truth mean activation (mean of the state’s Gaussian distribution) and functional connectivity (off-diagonal elements of the state covariance matrices) are plotted against the estimation from SPHMM. In both figures, each color represents a different state and each dot represents an ROI (if showing the mean) or a pair of ROIs (if showing functional connectivity). Both of these results demonstrate that the proposed SPHMM method can recover both the ground truth state sequence and the underneath Gaussian distribution.

Fig. 3.

Results for the second simulation data. (a), (b). Ground truth and inferred state time courses for one subject, using the multi-variant mixture Gaussian observation model with high inter-subject variability. Time intervals with significant difference are marked with red rectangles. For most of the time points, the proposed model is able to decode the ground truth state accurately. (c) Mean activation (mean of the Gaussian distribution) and functional connectivity (off-diagonal elements of the covariance matrix) of the ground truth model against the SPHMM estimations; each color represents a different state, and each dot represents either a region if showing the mean or a pair of regions if showing functional connectivity.

B. Real data analysis

1). Data acquisition and preprocessing:

The Philadelphia Neurodevelopmental Cohort (PNC) is a large-scale collaborative study between the Brain Behaviour Laboratory at the University of Pennsylvania and the Children’s Hospital of Philadelphia [37]. The data is available in the dbGaP database, which contains (among other data modalities) rest fMRI data from nearly 1000 adolescents of age 8 to 21 years. In our study, 840 subjects(451 female and 389 male) were selected with age and gender labels. TR of the resting fMRI data is $3000ms$ and the voxel matrix size is $RL\times AP\times slices=64\times 64\times 46$, with voxel size= $3m{m}^3$. The 6′18” scanning session contains 124 time points. Standard preprocessing steps were applied using SPM12, including motion correction, spatial normalization to standard MNI space (adult template) and spatial smoothing with a $3mm$ FWHM Gaussian kernel. The influence of motion (6 parameters) was further addressed using a regression procedure, and the functional time series were band-pass filtered using a $0.01Hz$ to $0.1Hz$ frequency range.

Then the preprocessed data were decomposed into functional networks by Group Independent Component Analysis (GICA) [38], [39] as implemented in the GIFT toolbox. Following a similar pipeline as in [17], [40] and [41], we first performed a PCA on each subject to reduce the subject-specific dimension to 120. The PCA reduced data of each subject were concatenated in the time dimension and then were passed through another PCA, reducing the group level dimension to 100. After that, ICA was run and repeated 10 times using the infomax algorithm. The multi-run results were then clustered by ICASSO and the cluster centroids were used as the stable estimation. To further estimate the subject-specific spatial maps and time courses, the GIG-ICA approach [42], [43] was used for back-reconstruction. Finally, we made a selection of the 100 components according to several considerations [43], [44]: removing the components whose peak coordinates (in MNI space) were spatially overlapped with white matter, ventricles, brain stem, or cerebellum according to visual observation; removing the components whose power were dominated by high frequency fluctuations according to the low frequency to high frequency ratio. After the screening, 50 out of 100 components were selected and labeled using the peak functional region. Those 50 components were then divided into 5 functional domains based on the literature [17] and [44]. These functional domains included the sensorimotor network (SM) with 10 components, default mode network (DMN) with 14 components, cognitive network (CCN) with 11 components, visual network (VIS) with 11 components, and auditory network (AUD) with 4 components. Figure 4 illustrates the spatial map of the chosen functional domains. After the GICA, the output time series of size (number of subjects × number of time points × number of ICA components = 840 × 124 × 50), were normalized so that for each subject and component of GICA, the mean is 0 and the standard deviation is 1.

Fig. 4.

Functional domain spatial maps. 50 selected components were visually inspected and grouped into 5 different functional domains. Each component is illustrated with different colors within each domain. For the components spread among several functional domains, we assigned them with the most conspicuous domain. Detailed anatomical information of the selected components can be found in Table S1 and Figure S2 in Supplementary Material.

To investigate the development of functional brain connectivity, the whole data set was divided into two age groups as suggested in [45]. Specifically, subjects whose age was over 200 months were grouped as young adults (age 18.53 ± 1.26 years, 177 females of 305 total subjects) while subjects whose age was under 160 months were grouped as children (11.13 ± 1.37 years, 142 females of 280 total subjects).

2). Postprocessing:

The sparse HMM was then applied to the preprocessed data to estimate the whole brain dynamics according to Algorithm 1. Since the proposed solution converges to a local optimum, we ran the algorithm 15 times to address this issue with different initialization. The setting of initialization follows a similar process as that in the literature [25]. In each run, we randomly sampled 25% of the subjects without replacement and applied mixture Gaussian clustering to the time courses with $k$ clusters centroids. The mean vectors ${\{{\mu }_k\}}_{k=1}^K$ and covariance matrices ${\{{\Sigma }_k\}}_{k=1}^K$ were used as the initial value in EM algorithm. The initial probability ${\pi }_k$ was initialized with equally appearing probability for each state. The transition probability matrix $A$ is initialized as a symmetric diagonal-dominated matrix with $A_{ii}=1∕2$ and $A_{ij}=1∕2(K-1)$, ($i\ne j$).

The MMDL was then employed to select the best state number $K$. The smaller the MMDL value is, the better the model will be for the state number setting. In the previous literatures [17], [22], [24] and [25], the reported number of brain states varies from 4 to 12. Hence we safely set the possible number of states ranging from 2 to 14. We repeated 25 runs for every $K$ between 2 and 14 and calculated the respective MMDL value. Then we averaged those values in 25 runs given $K$ and determined the best $K$ corresponding to the smallest average MMDL value. From the result shown in Figure 5, we can see the smallest MMDL value appears when $K$ is 5, which is admitted as the best choice of $K$. In the subsequent analysis, we will set the number of states to 5 without further explanation.

Fig. 5.

(a) The selection of the parameter $K$. The best $K$ is chosen with respect to the minimal MMDL. The y-axis has been log-scaled to illustrate the difference. (b) The WD of matched five states and the average WD of randomly matched states.

Another issue with the proposed model is that we must decide if the brain dynamics we get from multiple runs can be matched with each other given the state number $K$. In previous studies, researchers used the correlation between the activation map or the connectivity matrix to match states [25], [26]. However, recall that the states are defined with different multi-variant Gaussians, so a more proper way is to find a similarity measure between different Gaussian distributions. Hence we use the Wasserstein distance to measure the similarity between two distributions instead of comparing parameters. For example, Mueller et al. [46] have discussed this issue and its usefulness in a gene expression study. In the literature [47], the authors have shown that for two multi-variant Gaussian $m_1=𝒩({\mu }_1,{\Sigma }_1)$, $m_2=𝒩({\mu }_2,{\Sigma }_2)$ the 2nd order WD is defined with a closed form

$$W_2{(m_1,m_2)}^2={\Vert {\mu }_1-{\mu }_2\Vert }^2+Tr({\Sigma }_1+{\Sigma }_2-2{({\Sigma }_2^{1∕2}{\Sigma }_1{\Sigma }_2^{1∕2})}^{1∕2})$$ (6)

Given the state number $K$, we calculated the WD between any two states from different runs and applied an iterative approach similar to the work in [25], [48] to match the brain states.

To further validate the reproducibility of our model, we randomly divided the data set into two halves without overlap, ran the proposed model separately on two data sets and compared the results. In our experiment, we repeated this procedure 25 times to show that similar results can be reproduced on two halves of data.Additionally, we investigated how the number of subjects influences the estimation results. We found that with enough (over 20% of 840) subjects, we can get consistent results among the population. The details of this result is illustrated in Figure S3 in the Supplementary Material.

Once we estimated the parameters of the sparse HMM, we ran the Viterbi algorithm [33] to decode the sequences of brain states for each subject. The sequence of brain states, as a temporal feature for each subject, was then used to investigate the temporal property in the whole population as well as in different age groups.

In the two age groups studied, the comparison on the temporal feature such as transition matrix and the fractional occupancy of each state can be compared statistically. For the comparison of dFNCs, the permutation test was employed. More specifically, we keep the group difference as the observed difference and shuffle the labels of age for the chosen two groups as a permutation. For each permutation, we get a random group partition and the proposed model is run separately on these two groups. This permutation procedure was repeated 25000 times and Bonferroni adjustment was then employed to correct for false positives. Finally, for each element of every FNC, the distribution of the group difference from random partition is compared to the observed difference. The adjusted $p$ value is recorded and explained as to what extent the group difference from random partition is more extreme than the observed group difference. During this test, the average running time is 432.4s for each permutation using a standard work-station (endowed with four Intel Xeon CPU E5-2630 v3 3.30 GHz processors.)

3). Results:

We first tested the reproducibility of our method. Figure 5(b) shows the matching results of the five states in 25 random half-half divisions and the states were sorted with a descending order of similarity. From the figure, we can tell that the matching of the first four states is consistent among four different splits-and-runs. This consistency was further validated with Student’s t-test which showed a significant difference (p < 10⁻⁴) between the WD of well-matched states and that of the randomly matched states.

Having validated the reproducibility of the proposed model, we investigated both the spatial and temporal property of the 5 brain FNCs. Figure 6 demonstrates one of the FNCs visualized with the BrainNet Viewer [49]. The complete FNCs of five states can be found in Figure S2 of supplementary material. There are several interesting observations from the five discovered FNC states. State 1 and state 2 are similar to each other except that state 1 contains stronger connections in AUD-AUD and AUD-SM network. Comparatively, states 3-5 have more distinct features. State 3 stands out with the least connection between the left and right hemispheres while most of the connectivity in state 5 are inter-hemispheric. In state 4, we can clearly observe a unique SM-SM network connectivity pattern.

Fig. 6.

One of the FNCs estimated by the sparse HMM. The nodes represent the peak coordinates of the Group ICA components in MNI space and are distinguished with different colors according to their functional domain. Concrete information about the nodes is omitted here due to the space limit, but readers can find more details in Table S1 in the supplementary material. The size of the node represents the node connectivity strength (NCS). For the $j\text{th}$ node, its NCS is defined with the sum of the absolute value of the $j\text{th}$ row in matrix ${\Sigma }_k^{-1}$. A greater NCS implies a stronger connection of this node to the other regions. Colors of the edge, in red or blue, indicate the correlation or anti-correlation of its two end nodes. The width of the edge indicates the connectivity strength between two end nodes. The connectivity is displayed from left to right as sagittal(left), axial and sagittal(right) view.

In addition, we considered several temporal properties of the brain dynamics in detail. Figure 7 shows the mean fractional occupancy(FO) for each state among the population along with the variances exhibited for different subjects. From the figure, we see that there was a significant variation of the FO between states, which implied that some states (like 1 or 2) were visited more often than the others. Also, the FO of subjects also varied from one to another. This motivated us to explore more about this temporal feature in different subgroups of the population. We will leave this discussion to the next experiment on age-related subgroups. The state-related variation as well as the subject-related variation were further confirmed based on similar results in the literature [25] and [26].

Fig. 7.

(a) The fractional occupancy (FO) of five states. (b) The transition matrix of five brain states. (c) The correlation matrix of FOs

Figure 7(b) displays the transition probability matrix of brain dynamics. We can check how the brain states switch between each other based on this matrix. For example, state 1 and state 4 rarely switch to each other while some transitions (like switching between states 2, 3 and 4) happen more frequently than others. We further checked the correlation of the FOs in the whole data set and the result is shown in Figure 7(c). Interestingly, the correlation matrix and the transition matrix are pretty similar to each other: states 2, 3, 4 and states 1, 5 formed into two clusters within which the states are strongly correlated with each other and states switch more often between intra-cluster states. In other words, the organization of the brain states showed some hierarchical structure. This discovery can also be considered as the confirmation of the result in previous research [22]. With the great variation shown in the FOs among the population, we further validated our model on the group data, comparing the developmental brain network connectivity in two age groups: children and young adults. The two subgroups are extracted from the PNC data as we mentioned in the preprocessing section.

The differences between young adults and children were first examined based on the temporal feature extracted from the sparse HMM. As we mentioned in the postprocessing section, we plot the means and standard variations of fractional occupancy at the group level in Figure 8. In the child group, the fractional occupancy, the state transition matrix as well as the correlation matrix are pretty similar to the result at the whole data set level. State 1 and state 5 are correlated to each other while anti-correlated with the rest of the states. In the young adult group, the case was different. In young adults, state 1 became more dominant and stable (with a smaller standard variation) while state 2 and 4 appear less frequently. The state 5 frequency increases greatly in young adults. The t-test result indicates a significant difference in the mean of FOs between two age groups with p < 0.001.

Fig. 8.

The temporal features in two age groups: (a)(d)the fractional occupancy, (b)(e) the transition probability matrix, (c)(f) the correlation matrix of FOs.

The switching between states is disparate in terms of the transition matrix, which is plotted in Figure 8(b)(e). Compared to the child group, the transition matrix is less symmetric and turns out to be less random as well. The rare appearance of state 3 and 4 makes themselves isolated from other states in young adults. Figure 8(c) and (f) show the correlation of the FO within each group. It should be noted that the states occurrence frequency of young adults shows a distinct pattern with the children and the whole population. Though state 3 occupies coordinate appearance in both groups, its interaction with the other states behaves distinctly. State 3 is correlated with state 1 and anti-correlated with state 2 in young adults while the correlation is reversed in children. In addition, the correlation between state 2, 3, 4 disappears as age increases. We can divided the 5 states into two meta-states according to this correlation. Meta-state 1 includes state 2, 3, 4 while meta-state 2 comprises state 1 and 5. This finding is consistent with [22], in which researchers found that the brain dynamics are hierarchically organized in time. The result also confirmed that even in a task-free scenario, the transition and the appearance of brain states are not random. Previous state constrains the future states. Since we found this organization disappears in young adults, it is well worth exploring the temporal organization of brain activity from a developmental view. Moreover, we also explored if we can discriminate age groups based on the distribution of FO. The two-dimensional representation of the FO was generated by the t-distributed Stochastic Neighbor Embedding(t-SNE) [50]. Since visualization results could be sensitive to the predefined distance, we chose four different distances to illustrate the separation. The other parameters in t-SNE are set as default values in MATLAB. From Figure S4 in supplementary material, we can see the well-clustered structure of the dimension-reduced FO in two age groups no matter how we chose the distance, which confirmed that the FO of subjects is informative.

To illustrate how the dFNC will change as age advances, we compared the discovered dFNCs between children and young adults. The comparison results of the dFNCs between two groups are demonstrated in Figure 9 (Details of two groups’ FNCs can be found in Figure S5 in supplementary material). We also use the results of the sliding window method as a benchmark to compare with our approach. We observe that two groups barely show connectivity differences in state 1. This finding is confirmed by both approaches. The group difference can be moderately observed in state 2 to 4. For example, the sparse HMM method has revealed significant difference in SM-VIS (state 2, 3), DMN-SM (state 3), which is comparable with the results got from the sliding window approach. State 5 shows distinct patterns between the two groups: young adults show stronger CN-CN, CN-SM, DMN-DMN connectivity relative to children. Several studies have shown that intersubnetwork connectivity or intra-subnetwork connectivity play a key role during higher cognitive functioning and emotional processing. Our finding also confirmed those results.

Fig. 9.

The comparison of five FNCs between children and young adults (young adults - children). Upper row: the permutation test result of the SPHMM. The Bonferroni adjustment was employed to further correct the false positives. Only the entry whose p value is less than(or equal to) $\alpha ∕n=0.05∕1225=1∕24500$ is displayed in $-{\log }_{10}(p)$ scale. Lower row: the two-sample t-test of the sliding window method(SLW). A two-sample t test was performed across subject median dynamic FCs by state, with 5% significance level and under the false discovery rate(FDR) correction(q < 0.001). The results are visualized by plotting the log of p value with the sign of t statistics, $-sign(t){\log }_{10}(p)s$

From the group comparison, it should be noted that state 1 and state 2 share functional connectivity in both groups without much distinction. These two states occupy around 70% of the scanning time (75.3% in children and 68.4% in adults), which is consistent with the fractional occupancy in the whole data set. This implies that the FC networks do not change drastically but rather gradually with brain maturity. This result is also in line with the discovery of the previous research [21], [51]. However, for state 5, despite the spatial differences, there is also a significant difference in its fractional occupancies between the two groups. Young adults have more state 5 appearances during scanning. This age-related differences can also provide a possible explanation of the different development trends in FNC [11] [52]. We believe that the temporal and spatial feature of this specific state carry subtle developmental changes that underlies the emergence of the brain’s cognitive and emotional functions.

IV. Discussion

In this paper, we applied the proposed sparse HMM on the PNC data and estimated the dFNCs at the population level. Moreover, a two-group comparison between young adults and children also demonstrated the approach. The spatial and temporal feature of the brain dynamics were extracted and analyzed. In both age groups, we found that state 1 and state 2 have limited difference in the connectivity matrix with each other and were occupied most of the time during scanning. The similar two states were also confirmed in the whole data set. This finding agrees with the results from [21], [51], in which the researchers concluded that FNC changes gradually rather than drastically as the age increases. The increased intra-DMN connectivity was observed in state 5 of young adults. This is consistent with the finding in [53], [54]. In Fair et al. [53], they found the 13 published ROIs formed an integrated DMN in young adults compared to less connected networks in children. The decreased inter-network connectivity between DMN and CN was observed in state 5. This finding was consistent with previous work showing that decreased connectivity between DMN and the other ICNs appeared in two states of the dFNCs which constituted a core feature of the adults’ state repertoire [11]. The functional development of the brain systems includes the combination of the increasing long-range connections with the decreasing short-range connections [55], [56]. In other words, the development of brain transitions from segregated to integrated. As the brain matures, the whole brain FNC will change into a more interconnected network from a relatively local distributed network [57]. We also used the FO as a feature to illustrate the difference between children and young adults. There is a significant difference between the distribution of the FO in the two age groups. The consistent appearance of hierarchical organization in whole data set and children is in line with the previous study that the human brain dynamics are not randomly organized [22]. Moreover, the disappearance of this organization in young adults implies that this organization is also subject-specific.

Compared to the sliding window based method, sparse HMM can capture brain connectivity even in a small time scale. As we showed in the simulation section, the preset window length in the sliding window method is critical to the success of the sliding window method. Generally, a choice between 25 – 50 TR is necessary for the success of the sliding window method. In the sparse HMM approach, one can estimate the connectivity with only one time-course theoretically. Compared to other HMM-based models, our model has several advantages. As a regularized form of a generative model, the sparse HMM has a simple mathematical form with little assumption on the structure of the covariance matrix. To solve the optimization, one only needs to combine the traditional EM algorithm with the glasso with multiple initializations. The fewer assumptions on the covariance matrix gave a larger feasible region and the sparsity regularizer can lead to a solution that can be easily interpreted.

As a traditional model, many HMM-based methods have been applied and proven to be useful in revealing the dynamic nature of brain functional connectivity. In resting state, the dFNCs have been estimated based on fMRI and other brain imaging modalities and the transitions between dFNCs are shown not to be random [22], [24], [25]. In task fMRI data, the HMM can identify fast transient and short time fluctuations [23], [26]. However, it should be noted that these methods come with different motivations and hence have key differences in the mathematical models. More precisely, these methods have different assumptions on the covariance matrices. In Evani et al.’s work [24], they assumed that the covariance matrix can be approximated by the weighted summation of a family of rank-1 sparse basis matrices, i.e. ${\Sigma }_i\approx {\sum }_{k=1}^K{c}_i(k){\mathbf{b}}_k{\mathbf{b}}_k^T$, where $\overrightarrow{c_i}$ is the state-related weight vector and ${\mathbf{b}}_k{\mathbf{b}}_k^T$ is the rank-1 sparse connectivity pattern. This basis decomposition idea made the result to be more interpretable than the other dFNC estimation approaches. However, the approximation can be problematic since the approximated $\sigma$ can be singular without further regularization, which might lead to an early stop in an EM iteration. The optimization is also complicated in that case. In Chen et al. ’s paper [25], the covariance matrices were assumed to be diagonal and only the mean vectors of the normal distribution were used for analysis as activation patterns. Moreover, in [22], [23], [26], the HMM model was employed to discover the fast transient networks from magnetoencephalography (MEG) data as well as dynamic brain networks from fMRI data. In their approach, no regularization except variational inference was added to improve the computational efficiency of HMM on large data sets. With a dense connectivity matrix setting, it is hard for one to interpret the result.

There are mainly two drawbacks of the sparse HMM approach: the model comparison and the local minimum. As we did in the postprocessing step, we need to find a method to match the states from independent realizations of the algorithm. Yang et al. proposed a greedy algorithm for matching the results from several realizations [48] and the technique was also applied in [25] by matching the correlation between the mean activation maps. Similarly, Vidaurre et al. also used the Hungarian algorithm based on the correlation [26]. However, the HMM cannot guarantee the correlation matrix of mean vectors and correlation matrix between covariance matrices are similar to each other. We used the combination of the Hungarian and the greedy method, to avoid mismatching. Both a finer matching strategy and a measure of similarity are necessary for perfect matching. Recently, the research on the Wasserstein distance [58] has the potential to offer better matching of complicated distributions or stochastic processes. The second problem is that the EM algorithm returns local minima and multi-initialization in optimization. Comparatively, the sliding window method just requests repeating the k-means clustering several times, which will be more efficient than the HMM-based methods. In [27], the authors proposed a backward greedy tuning algorithm to address this problem as well as the model selection. But the deleting and combining of the states should be more cautious because one may discard or merge a less frequently occurring but meaningful brain state in the process.

V. Conclusion

In this paper, we applied the proposed sparse HMM on resting fMRI data to investigate brain dynamics. The underlying FNCs were discretized to be discrete states, and the whole brain FNCs were modeled with precision matrices of a set of Gaussian distributions. The model was validated with both simulation data and the real rs-fMRI time series (e.g., PNC data). The performance of the sparse HMM was compared with the popular sliding window approach on simulation data. The results indicated that our model could estimate the brain FNC even with limited observations or abrupt changes which could be overlooked by the sliding window approach. As a real application, we analyzed the PNC data with our model and compared the differences of brain dFNCs between young adults and children. By comparing the FNC of two age groups, we found that young adults’ brains have well-developed intranetwork connectivity while the children’s brains are characterized with stronger inter-network connectivity. The hierarchical temporal organization of brain states in children and at the whole data set level was also confirmed through the transition matrix and correlation analysis of the FO of states. In brief, the proposed sparse HMM method is an alternative approach to the sliding window method and the other HMM-based models but with better performance. It can efficiently estimate both static and abrupt FNCs with sparse precision matrices. As a simple and powerful tool, it can be used for various brain network analysis tasks.