Multi-subject Independent Component Analysis of fMRI: A Decade of Intrinsic Networks, Default Mode, and Neurodiagnostic Discovery

Abstract

Since the discovery of functional connectivity in fMRI data (i.e., temporal correlations between spatially distinct regions of the brain) there has been a considerable amount of work in this field. One important focus has been on the analysis of brain connectivity using the concept of networks instead of regions. Approximately ten years ago two important research areas grew out of this concept. First, a network proposed to be “a default mode of brain function” since dubbed the default mode network was proposed by Raichle. Secondly, multi-subject or group independent component analysis (ICA) provided a data-driven approach to study properties of brain networks, including the default mode network. In this paper we will provide a focused review of how ICA has contributed to the study of intrinsic networks. We will discuss some methodological considerations for group ICA, and highlight multiple analytic approaches for studying brain networks. We will also show examples of some of the differences observed in the default mode and resting networks in the diseased brain. In summary, we are in exciting times and still just beginning to reap the benefits of the richness of functional brain networks as well as available analytic approaches.

1. Introduction and Background

ICA is a statistical method used to discover hidden factors (sources or features) from a set of measurements or observed data such that the sources are maximally independent. Typically, it assumes a generative model where observations are assumed to be linear mixtures of independent sources, and unlike principal component analysis (PCA) which only uncorrelates the data, ICA works with higher-order statistics to achieve independence. ICA was developed to solve problems similar to the “cocktail party” scenario in which individual voices must be resolved from microphone recordings of many people speaking at once ¹. The algorithm, as applied to fMRI, assumes a set of maximally spatially independent brain networks, each with associated time courses. The model identifies latent sources whose elements (voxels) have the same time course and thus each component can be considered a measure of the degree to which each voxel is functional connected (correlated) to the component timecourse. Note that network is a somewhat ambiguous term. Indeed, in this paper we describe a component as a network (a temporally correlated set of regions) and also consider other types of networks including those built up from the temporal correlations among components or from the spatial mutual information among component maps. A good description of the use of the work network can be found in Ehardt et al².

ICA was first applied to single-subject fMRI data in 1998³. However the application of ICA to multiple subjects is not straightforward due to the presence of a different mixing matrix for each subject. The first approaches for applying ICA to multi-subject data were presented in 2001^4,5 and were quickly followed by a series of publications describing various methodological issues as well as applications to a number of challenging problems such as the analysis of naturalistic paradigms^6,7, resting-state data⁸, complex-valued fMRI data⁹ and various clinical data sets^10,11. Since then, ICA has become widely-used and a standard strategy for evaluating hidden spatiotemporal structure contained in brain imaging data for groups of subjects (see Figure 1). In this paper, we provide a summary of the just over one decade of development and application of ICA multi-subject ICA methods for fMRI.

Figure 1. Number of fMRI papers using ICA by year with a few highlighted landmarks relating group ICA, default mode, and brain disease.

In parallel with the development and application of ICA came the description of what were later described as intrinsic connectivity networks (ICNs) which grew out of work which showed linear correlations to selected seed-voxels resemble interesting functional connections¹². Shortly thereafter, one of the first and most-widely characterized ICNs, described as the default mode network, was identified¹³. This network was also identified in the early ICA results, exhibiting the characteristic pattern of decreasing in response to focused task performance⁶, though it was not described as the default mode network in the early ICA papers since that expression had not yet been coined at the time it was described. Since then, ICA has contributed greatly to the further understanding of the default mode network as one of many ICNs and one that consists of multiple inter-linked networks^14-17 and also provided a more comprehensive functional parcellation of the brain data, which does not rely on the selection of a specific seed-region¹⁵. Though it is nice that once the network locations are approximately identified, multiple methods (including ICA, general linear model (GLM), and seed-based connectivity) show largely convergent results². The identification of interacting ICNs in the prediction of future errors is one good example showing the importance of identifying and characterizing ICNs¹⁸. A brief timeline of some of these events is also highlighted in Figure 1. In this figure we focus on only a few key events including the introduction of ICA and group ICA, the discovery of the default mode network, application of ICA to multiple clinical groups, and more recently to very large resting fMRI studies.

2. ICA (Ways to Estimate Independence/ICA Algorithms)

Data-driven methods have proven effective in many problems such as data analysis and fusion as they minimize the modeling assumptions on the underlying structure of the problem. They typically assume a simple generative model, start with a multiplicative form such as X=AS. Then the task is to achieve the given decomposition through an appropriate interpretation of the underlying components, rows/columns of A and/or of S given a certain metric. For the identifiability of the given decomposition, usually additional constraints such as nonnegativity, uncorrelatedness, and sparsity are imposed. ICA achieves such a decomposition by assuming that the rows of S are samples from statistically independent random variables (or processes). By maximizing independence, one can then achieve the decomposition subject to a permutation and scaling ambiguity.

ICA decomposes a given set of observations by making use of different properties of the data. Most of the ICA algorithms introduced to date have made use of one of the two types of properties, non-Gaussianity and/or sample correlation. More recently, it has been noted that we can have important gains in performance by making use of non-Gaussianity, i.e., higher-order-statistics along with correlation. Also, most types of underlying components satisfy the property that indeed they are both non-Gaussian (hence use of higher-order statistics is appropriate) but they also exhibit sample correlation, which is certainly the case for the fMRI data as well.

In this section, we first give a brief review of different approaches to the ICA problem under the maximum likelihood umbrella, which provides a unifying framework for approaches using different types of data properties, and then in the next section introduce how ICA is applied to fMRI analysis.

2.1. ICA: Cost Function Choice

We start the discussion with the basic ICA problem where x, sR^N and write

$$\mathbf{x}(v)=\mathbf{A}\mathbf{s}(v),1\le v\le V.$$ (1)

where is A full rank square mixing matrix, and hence we assume instantaneous mixing and as many observations x_n as sources/components s_n—which also includes the overdetermined case since one can easily reduce the problem to (1) using e.g., principal component analysis (PCA) for this case. We assume that the index v can be time, or a spatial or volume index, a voxel as in the case of fMRI analysis. In the sequel, we letm x_n(v) and s_n(v) denote the nth random process in and x(v), and s(v) when we consider a given set of observations, i.e., X ∈^N×V, we assume that each row, ${\mathbf{x}}_n^T$ refers to a realization from a V-dimensional random vector.

Given that the sources are mutually independent, one can achieve ICA and form the source estimates u(v)=Wx(v) by estimating the demixing matrix W making use of different properties of the data. The most popular approach has been the use of non-Gaussianity, i.e., using higher-order-statistics (HOS) to achieve the decomposition. Under this umbrella, one can either start with mutual information as the cost function and arrive at the two most popular approaches, based on either maximum likelihood (ML) or maximization of negentropy to achieve the ICA decomposition, or can explicitly calculate HOS as in joint approximation diagonalization of eigenmatrices (JADE)¹⁹. In this overview, we concentrate on the former approach as it allows the study of large sample properties in an ML framework, and can be also used to study properties of another important class of ICA algorithms under the same umbrella, those that make use of sample correlation within a component/source. We start our discussion with the general case that takes advantage of sample correlation together with non-Gaussianity, i.e., HOS, of the sources. The natural cost function in this case is the mutual information rate, which can be written as

$$I_r(\mathbf{W})={\sum }_{n=1}^N{H}_r({\mathbf{u}}_n)-log|det(\mathbf{W})|-H_r(\mathbf{x})$$ (2)

where H_r(u_n)=lim_k→∞H[u_n(1), …, u_n(k)]/k is the entropy rate of the nth source estimate u_n. The entropy rate H_r(X)=lim_k→∞H[x(1), …, x(k)]/k of the observations is a constant with respect to W and thus the statistical dependence among the separated sources is naturally minimized by minimizing the total entropy rate of all source estimates. The regularization term log|det(W)| penalizes small matrices, and reduces (2) to maximization of negentropy as the cost function, i.e., minimization of sum of entropy rates under a variance constraint when W is constrained to be orthogonal (WW^T =I)so that the term is 0.

For a given observation matrix X ∈^N×V, we form the source estimate using U=WX and write the likelihood as

$$L(\mathbf{W})={\sum }_{n=1}^{N}log{p}_{s_n}({\mathbf{u}}_n)+log|det\mathbf{W}|$$ (3)

where we used ${\mathbf{u}}_n^T$ to denote the n^th row of U, a realization of a V -dimensional random vector. When we assume independent and identically distributed (i.i.d.) samples, i.e., ignore sample correlation, (3) takes the more commonly encountered form in ICA formulations

$$L(\mathbf{W})={\sum }_{v=1}^V{\sum }_{n=1}^{N}log{p}_{s_n}(u_n)+Vlog|det\mathbf{W}|$$ (4)

since non-Gaussianity is the more frequently used assumption in ICA. Here, $u_n(v)={\mathbf{w}}_n^T\mathbf{x}(v)$ and ${\mathbf{w}}_n^T$ is the nth row of the demixing matrix. It is the form in (4) that leads to the popular Infomax¹, along with all the ML variations using different density models, e.g., using adaptive scores²⁰ or entropy bound minimization (EBM) as in ²¹—and when the demixing matrix is constrained to be orthogonal to FastICA²², and its variants such as efficient FastICA (EFICA)²³.

An important result in terms of the identifiability of the ICA model is that one can achieve source separation unless there are two sources that are both Gaussian and have covariance matrices that satisfy R_n = αR_m for α ≠ 0, where ${\mathbf{R}}_n=E\left\{{\mathbf{u}}_n{\mathbf{u}}_n^T\right\}$. This is also the condition that guarantees identification of sources when only second-order statistics are used as in WASOBI²⁴. When we ignore sample correlation and use the form in (4), and can only make use of non-Gaussianity, i.e., HOS, in this case we can separate sources as long as there is only one Gaussian in the mixture²⁵—and obviously for both cases, the separation is possible only upto a scaling and permutation ambiguity that is inherent to the problem.

2.2. Algorithm Design

Given a selected cost function, there are a number of important considerations when designing the ICA algorithm. Since the ML cost function has been the most frequently used one in designing ICA algorithms, we primarily focus on ML in this discussion. The most important ones among those are density estimation, optimization, and incorporation of prior information through constrained ICA.

When designing algorithms that take higher-order statistics into account, as it is obvious from the form of the cost functions given in (3) and (4), besides estimating the demixing matrix W, one has to also approximate the source probability density function logp_sn(u_n). This is important for achieving a true ML estimation such that one can take advantage of desirable large sample optimality properties of ML. The success of the Infomax algorithm, which uses a fixed density model for the analysis of fMRI data can be explained by the fact that the signoid nonlinearity used in the algorithm as the score function provides a good match to the underlying fMRI source distribution, which are typically sparse and hence super-Gaussian. However, by adapting the nonlinearity to multiple sources, including artifacts, we improve the overall separation performance. Among the parametric models, generalized Gaussian distribution has been a popular choice as it can model a range of symmetric distributions and can be summarized with a shape parameter besides mean and variance, and is has led to ICA algorithms such as EFICA²³ and adaptive complex maximization of non-Gaussianity²⁶ ICA by entropy bound minimization (ICA-EBM)²¹ uses an efficient entropy estimator based on the bounding of the entropy estimates, and by using a few measuring functions, can approximate the pdf of a wide range of densities including sub- or super-Gaussian, unimodal or multimodal, symmetric or skewed distributions. A more powerful class takes both sample correlation and non-Gaussianity into account. Among those, AR-MoG is an expectation-maximization (EM) algorithm and assumes that the sources are generated by AR models driving by i.i.d. noise come from mixture of Gaussian (MoG) distributions²⁷. Robust, accurate, direct ICA algorithm (RADICAL)²⁸ is a nonparametric ICA algorithm using estimates of entropy based on spacings. Full blind source separation (FBSS) that combines EBM with a flexible correlation model provides a good tradeoff between the parametric and non-parametric approaches and provides reliable performance for ICA of fMRI as we introduce in Section 3.

3. ICA of fMRI Data

The first application of ICA to fMRI demonstrated its ability to separate components for individual subjects that are task-related, transiently task-related, quasiperiodic, slowly varying, and those related to head movement or other phenomena²⁹ and was followed by a more detailed comparison showing superiority over PCA and correlation-based methods³⁰ and an examination of the ICA model assumptions³¹. ICA, like the widely used general linear model (GLM) can be written in a similar matrix form with the difference being the mixing matrix is estimated in ICA whereas the design matrix is specified in the GLM (Figure 2). ICA has also been applied successfully to both EEG and MEG^32,33 and has been extensively used for multimodal data fusion^34,35. fMRI most frequently uses spatial ICA (sICA), which assumes that each image over time is composed of a linear combination of T spatial IC images with associated time courses. EEG and MEG most frequently use temporal ICA (tICA), which assumes that a time course is a composed of a linear combination of T time courses with associated spatial maps³⁶. Choosing sICA or tICA is often a practical issue related to the dimension of the data, where the larger of the time and space dimensions is typically the deciding factor. However combining the two sequentially has become an interesting option as well³⁷.

Figure 2.

Comparison of ICA for fMRI and the general linear model: ICA is a linear space/time decomposition similar to the GLM. The difference is the GLM fixed the design matrix and estimates univariate parameter fits whereas ICA estimates the equivalent mixing matrix by maximizing spatial independence among the rows of the component matrix.

Preprocessing

The number of observations in an ICA decomposition are determined by the number of time points, which usually are in the 100s. Hence, typically a dimension reduction step is applied along with whitening of the data, a common ICA preprocessing step that helps with the convergence of the algorithms. Among different approaches for model order selection, the information-theoretic criteria (ITC) have proven particularly attractive as they do not require the specification of an empirical threshold for order selection, and hence fit naturally into the framework of data-driven analysis methods such as ICA. Among the most commonly used ITC are Akaike's information criterion (AIC), Kullback-Leibler information criterion (KIC)³⁸ and the minimum description length (MDL) criterion (or the Bayesian information criterion (BIC)) ^39,40. For the application of these criteria to order selection in ICA of fMRI, the formulation given by Wax and Kailath⁴¹ provides an attractive framework. They pose the problem in the context of detecting the number of signals in noise where both the signals and the noise are modeled by multi-dimensional complex stationary Gaussian random processes. The only limitation of the approach when directly applied to the problem in fMRI data is that the formulation is based on the assumption of independent and identically distributed (i.i.d.) samples, which is not a property satisfied by fMRI samples as there is the inherent spatial smoothness due to the point spread function of the scanner. Furthermore, smoothing is a common preprocessing step used to suppress the high frequency noise in the fMRI data and to minimize the impact of spatial variability among subjects.

To address this issue, in ⁴², a subsampling scheme is introduced to obtain a set of effectively i.i.d. samples from the given observations, i.e., the dependent original data.

4. Multi-Subject (Group) ICA

Group ICA consists of several stages including data-reduction, forward-estimation, back-reconstruction, and statistical analysis of output features (see Figure 3). We now briefly describe these various approaches.

Figure 3.

Stages of group ICA: The analysis starts with spatially normalized fMRI data and proceeds through a data reduction step using PCA, followed by ICA of the reduced data, then back-reconstruction is used to compute single-subject maps and timecourses for each component which are then analyzed statistically depending on the question of interest.

4.1. Data Reduction

Data-reduction is typically performed using PCA, but other approaches are also possible, such as clustering⁴³. The most common approach used is a two-stage PCA, one at the single subject level and a second one at the group level. There have been various strategies proposed including performing a single-subject PCA, then concatenating the data and performing a group-level PCA. The single-subject PCA can be performed in a common space or can be individual to each subject. These various approaches yield largely similar results, though memory requirements vary considerably⁴⁴.

4.2. Multi-subject ICA of fMRI data

Following data reduction, the next stage includes a forward estimation process (Figure 3). A summary of several multi-subject ICA forward estimation strategies is given in Figure 4. Briefly, there are at least five ways to perform the forward estimation process. Pre-averaging the subject data and performing ICA on the group mean dataset is the least computational method but makes the stringent assumption that all subjects have common time courses (TCs) and spatial maps (SMs)⁴⁵. A number of approaches first perform single-subject ICA on each subject and then attempt to combine the output into a group post hoc by spatial correlation^4,5, self-organized clustering⁴⁶, or retrospective matching⁴⁷ of the components ^4,46. This has the advantage of allowing for unique spatial and temporal features, but has the disadvantage that since the data are noisy, the components might not be necessarily unmixed in the same way for each subject.

Figure 4.

Forward estimation approaches: ICA of groups of subjects can be approached in different ways. The most flexible (and most challenging) is to use single-subject ICA and attempt to group common components post-hoc. Group ICA with temporal concatenation is the most widely used approach (and arguable has assumptions which are the most compatible with the data such as spatial stationarity). Tensor ICA stacks the data into a cube. And one can also spatially concatenate the data.

The other four approaches involve an ICA step that is computed on the group data directly. Temporal concatenation and spatial concatenation have both been examined^5,48. The advantage of these approaches is that they perform one ICA, which can then be divided into subject specific parts, hence the comparison of subject differences within a component is straightforward. The temporal concatenation approach allows for unique TCs for each subject but assumes common group SMs, whereas the spatial concatenation approach allows for unique SMs but assumes common TCs. Temporal concatenation has been widely used for multi-subject ICA of fMRI data because it appears to work better for fMRI data⁴⁵, most likely because the temporal variations in the fMRI signal are much larger than the spatial variations.

The temporal concatenation approach has had a number of methods developed for its implementation. The first group ICA strategy uses subject-level PCA in the time dimensions, temporal concatenation of these reduced data, a group-level PCA, followed by ICA to give an aggregate SM and loading matrix. Finally, subject-specific TCs and SMs are estimated by back projection using inverse PCA projections⁵. Group ICA with temporal concatenation was initially implemented in the GIFT Matlab software (http://icatb.sourceforge.net/) and subsequently in the MELODIC software (http://www.fmrib.ox.ac.uk/fsl/). The GIFT software additionally implements a back-reconstruction step that produces subject specific images⁵. This enables a comparison of both the time courses and the images for one group or multiple groups (see simulations in ^5,49 which shows ICA with temporal concatenation plus back-reconstruction can capture variations in subject specific images). A different back-reconstruction approach was also subsequently added to MELODIC (for details on the differences see brief summary later in this manuscript and also Erhard et al.⁴⁴). Such approaches trade off the use of a common model for the spatial maps against the difficulties of combining single subject ICA. An in-between approach would be to utilize temporal concatenation separately for each group⁵⁰, although in this case, matching the components post hoc becomes again necessary and there is also the possibility of inflating group differences in such a case since different projections are used for each group.

Finally, the tensorial approach in Figure 4 (implemented in MELODIC) involves estimating a common time course and a common image for each component but allows for a subject specific parameter to be estimated. Three-dimensional tensor decomposition for group and multi-group fMRI data is still being explored, estimating a single spatial, temporal, and subject-specific mode (amplitude parameter) for each component to attempt to capture the multidimensional structure of the data in the estimation stage⁵¹. Since this approach assumes common TCs among subjects, it is inappropriate for when they are different, such as in a resting state study or when events are randomized between subjects and/or self paced.

Because of its widespread use and implementation in both the GIFT and MELODIC packages, in the remainder of this paper, we focus on the temporal concatenation approach followed by back-reconstruction. Let Y_i be the T -by- V magnitude data matrix for subject i containing the preprocessed and spatially normalized data with rows of zero mean, where T time points over V voxels are collected on M subjects. Let $\mathbf{Y}\equiv {[{\mathbf{Y}}_1^{\mathrm{T}},\dots ,{\mathbf{Y}}_M^{\mathrm{T}}]}^{\mathrm{T}}$ be the temporally concatenated subject data. Following square noise-free spatial ICA estimation⁵², we can write

$$\mathbf{X}=\widehat{\mathbf{A}}\widehat{\mathbf{S}}$$ (5)

where the generative linear latent variables  and Ŝ are the T₂ -by- T₂ mixing matrix related to subject TCs and the T₂ -by- V aggregate SM, respectively. In this form of spatial ICA, there is no aggregate or group-level TC.

4.3. Subject back-reconstruction methods

Back-reconstruction methods are important as they provide estimates of the signal subject maps and timecourses. The two main categories include PCA based⁵ and regression based^7,53. A third category involves using the group maps as initial constraints using constrained ICA⁵⁴ or other approaches. Under certain conditions, PCA-based and regression-based approaches provide identical solutions. We now briefly review these two approaches (a full description can be found in Erhardt et al.⁴⁴).

PCA-based back-reconstruction (e.g., GICA3)

Let ${\mathbf{Y}}_j^{\ast }={\mathbf{F}}_i^{-}{\mathbf{Y}}_i$ be the T₁ -by- V PCA- reduced data for subject i, where ${\mathbf{F}}_i^{-}={\mathbf{F}}_i^{\mathrm{T}}$ is the T₁ -by- T standardized reducing matrix and T₁ is the number of principal components retained for each subject. For each subject, i=1, …, M, let the subject data Y_i be the product of the subject-specific TC and SM, R_i and S_i, plus error, E_i,

$${\mathbf{Y}}_i={\mathbf{R}}_i{\mathbf{S}}_i+{\mathbf{E}}_i.$$ (6)

In GICA3, assume that subject-specific TCs are the subject-specific PCA back-projected mixing matrix,

$${\mathbf{R}}_i\equiv {\mathbf{F}}_i{({\mathbf{G}}_i^{\mathrm{T}})}^{-}\mathbf{A}={\mathbf{F}}_i{\mathbf{G}}_i{({\mathbf{G}}_i^{\mathrm{T}}{\mathbf{G}}_i)}^{-1}\mathbf{A}.$$ (7)

Let the T₂ -by- V PCA-reduced aggregate data be

$$\mathbf{X}\equiv {\mathbf{G}}^{-}{\mathbf{Y}}^{\ast }=[{\mathbf{G}}_1^{\mathrm{T}},\dots ,{\mathbf{G}}_M^{\mathrm{T}}]\left[\begin{array}{c}{\mathbf{F}}_1^{-}{\mathbf{Y}}_1\\ ⋮\\ {\mathbf{F}}_M^{-}{\mathbf{Y}}_M\end{array}\right]={\sum }_{i=1}^M{\mathbf{G}}_i^{\mathrm{T}}{\mathbf{F}}_i^{\mathrm{T}}{\mathbf{Y}}_i={\sum }_{i=1}^M{\mathbf{X}}_i,$$ (8)

where G⁻ is the T₂ -by- MT₁ standardized reducing matrix.

Then for subject i, PCA compression of the data gives ${\mathbf{Y}}_i^{\ast }={\mathbf{F}}_i^{\mathrm{T}}{\mathbf{Y}}_i$, leading to the compressed data for each subject relating to the common mixing matrix with a subject-specific SM,

$${\mathbf{X}}_i={\mathbf{G}}_i^{\mathrm{T}}{\mathbf{Y}}_i^{\ast }={\mathbf{G}}_i^{\mathrm{T}}{\mathbf{F}}_i^{\mathrm{T}}({\mathbf{R}}_i{\mathbf{S}}_i+{\mathbf{E}}_i)=\mathbf{A}{\mathbf{S}}_i+{\mathbf{E}}_i^{\ast },$$ (9)

where $\mathbf{A}={\mathbf{G}}_i^{\mathrm{T}}{\mathbf{F}}_i^{\mathrm{T}}{\mathbf{R}}_i$ is the common mixing matrix and S_i is the subject-specific SM. Aggregating over subjects gives the group-reduced data related to the aggregate SM,

$$\mathbf{X}={\mathbf{G}}^{\mathrm{T}}{\mathbf{Y}}^{\ast }={\sum }_{i=1}^M{\mathbf{G}}_i^{\mathrm{T}}{\mathbf{F}}_i^{\mathrm{T}}{\mathbf{R}}_i{\mathbf{S}}_i+{\sum }_{i=1}^M{\mathbf{E}}_i^{\ast }=\mathbf{A}{\sum }_{i=1}^M{\mathbf{S}}_i+{\mathbf{E}}^{\ast }\equiv \mathbf{A}\mathbf{S}+{\mathbf{E}}^{\ast },$$ (10)

where the aggregate SM is the sum of the subject-specific SMs, $\mathbf{S}\equiv {\sum }_{i=1}^M{\mathbf{S}}_i$. In noise-free ICA we estimate the mixing matrix and aggregate SM in (10),

$$\mathbf{X}={\sum }_{i=1}^M{\mathbf{X}}_i={\sum }_{i=1}^M{\mathbf{G}}_i^{\mathrm{T}}{\mathbf{F}}_i^{\mathrm{T}}{\mathbf{Y}}_i=\widehat{\mathbf{A}}{\sum }_{i=1}^M{\mathbf{S}}_i=\widehat{\mathbf{A}}\widehat{\mathbf{S}}.$$ (11)

From (11) the natural estimate of the subject-specific SM substitutes our estimate of A into either defining relationship (6) or by solving ${\mathbf{X}}_i={\mathbf{G}}_i^{\mathrm{T}}{\mathbf{F}}_i^{\mathrm{T}}{\mathbf{Y}}_i=\widehat{\mathbf{A}}{\mathbf{S}}_i$ for S_i,

$${\stackrel{\sim }{\mathbf{S}}}_i={\widehat{\mathbf{A}}}^{-}{\mathbf{G}}_i^{\mathrm{T}}{\mathbf{F}}_i^{\mathrm{T}}{\mathbf{Y}}_i.$$ (12)

Note that we have exactly that the aggregate SM is the sum of the subject-specific SMs,

$$\widehat{\mathbf{S}}\equiv {\sum }_{i=1}^M{\stackrel{\sim }{\mathbf{S}}}_i.$$ (13)

Further, from (7) the natural estimator of subject-specific TC R_i substitutes the estimate of A to give

$${\stackrel{\sim }{\mathbf{R}}}_i\equiv {\mathbf{F}}_i{({\mathbf{G}}_i^{\mathrm{T}})}^{-}\widehat{\mathbf{A}}={\mathbf{F}}_i{\mathbf{G}}_i{({\mathbf{G}}_i^{\mathrm{T}}{\mathbf{G}}_i)}^{-1}\widehat{\mathbf{A}}.$$ (14)

Regression-based back-reconstruction (STR/dual regression)

Spatio-temporal regression (STR), or dual regression, is an indirect back-reconstruction approach using least squares to estimate the subject-specific TCs and SMs ^55,56. Given ICA results, as in (11) or PICA ⁵⁷, STR first estimates subject-specific TCs, Ṙ_i , then SMs, Ṡ_i, via multiple regression.

The first assumption is that all subjects share a common SM, SM, S_i ≡ S, i = 1, …, M. Then, from the ICA in (5), substitute Ŝ for S and take the transpose of the equation for

$${\mathbf{Y}}_i^{\mathrm{T}}={\widehat{\mathbf{S}}}^{\mathrm{T}}{\mathbf{R}}_i^{\mathrm{T}}+{\mathbf{E}}_{1i}^{\mathrm{T}}.$$ (15)

Least squares estimation for TC ${\mathbf{R}}_i^{\mathrm{T}}$ gives ${\dot{\mathbf{R}}}_i^{\mathrm{T}}={\left(\widehat{\mathbf{S}}{\widehat{\mathbf{S}}}^{\mathrm{T}}\right)}^{-1}\widehat{\mathbf{S}}{\mathbf{Y}}_i^{\mathrm{T}}$, or the transpose ⁵⁶,

$${\dot{\mathbf{R}}}_i={\mathbf{Y}}_i{\widehat{\mathbf{S}}}^{-}={\mathbf{Y}}_i{\widehat{\mathbf{S}}}^{\mathrm{T}}{\left(\widehat{\mathbf{S}}{\widehat{\mathbf{S}}}^{\mathrm{T}}\right)}^{-1}.$$ (16)

To estimate each subject-specific SM, S_i , the original assumption of common SMs is relaxed. Conditional on the estimated subject-specific TC, Ṙ_i, let

$${\mathbf{Y}}_i={\dot{\mathbf{R}}}_i{\mathbf{S}}_i+{\mathbf{E}}_{2i}$$ (17)

with E[E_2i] = 0 Least squares estimation for S_i gives

$${\dot{\mathbf{S}}}_i={\left({\dot{\mathbf{R}}}_i^{\mathrm{T}}{\dot{\mathbf{R}}}_i\right)}^{-1}{\dot{\mathbf{R}}}_i^{\mathrm{T}}{\mathbf{Y}}_i={\dot{\mathbf{R}}}_i^{-}{\mathbf{Y}}_i={\left({\mathbf{Y}}_i{\widehat{\mathbf{S}}}^{-}\right)}^{-}{\mathbf{Y}}_i.$$ (18)

The product of each subject-specific TC and SM gives

$${\dot{\mathbf{R}}}_i{\dot{\mathbf{S}}}_i={\dot{\mathbf{R}}}_i{\left({\dot{\mathbf{R}}}_i^{\mathrm{T}}{\dot{\mathbf{R}}}_i\right)}^{-1}{\dot{\mathbf{R}}}_i^{\mathrm{T}}{\mathbf{Y}}_i=P_{C\left({\dot{\mathbf{R}}}_i\right)}{\mathbf{Y}}_i=P_{C\left({\mathbf{Y}}_i{\widehat{\mathbf{S}}}^{-}\right)}{\mathbf{Y}}_i,$$ (19)

where P _C(Ṙi) = P _C(YiŜ⁻) is a perpendicular projection operator (PPO) onto C(Y_iŜ⁻). These estimates of Ṙi and Ṡ_i require contradictory assumptions regarding the SMs, where the SMs are assumed to be the common aggregate map to estimate subject-specific TCs, then distinct subject-specific SMs are estimated. The variability among subject can be considerable and the method works quite well in practice^5,44,49.

Rare but common components

The evidence from Schmithorst and Holland⁴⁵ suggests that subject-wise (temporal) concatenation (versus across-subject averaging and row-wise concatenation) performs best in terms of estimating subject-specific rare but common component SMs and TCs. Their Fig 3 suggests (and the text immediately below Fig 3) that with as few as 5 subjects out of 100 (large group size) that the estimated SMs and TCs are quite good, improving slightly with more than 5 subjects. Fig 2 shows same result with a minimum of about 3 out of 20 (small group size) subjects needing the component. Extrapolating to 1000+ subjects, it is not clear whether this is an absolute number of subjects needed with the component (maybe 5-10 subjects) or a proportion of the total number of subjects (5-10%), but in their discussion they claim at least 10 subjects should have the rare but common components to be able to estimate the subject-specific SMs and TCs well for that component.

4.4. Quantifying Intrinsic Networks through ICA Features

The advantage of group ICA is that it provides a summarized set of single-subject features that can then be tested (see Figure 3). This include voxelwise tests of the spatial maps, task-relatedness of a given component, spectra power of the component timecourses, and dependencies among components either temporally (through cross-correlation of the component timecourses⁵⁸) or spatially (through mutual information among spatial component maps⁵⁹).

Intrinsic functional brain networks (INs) are sets of brain regions showing temporal coherence with one another; they provide a key way of evaluating the human (macro) functional connectome^15,60,61. The INs are quite robust and similar (though not identical) with or without a task being performed. Indeed, one can consider the use of a task as a controlled way to study how these networks are modulated both spatially and temporally by a directed task⁶². Numerous INs have been identified consistently by many groups, such as the default mode network, the attentional fronto-parietal networks, the executive control network (or salience network) and bilateral temporal lobe and motor cortex. The INs are likely critical components of healthy and aberrant brain functions; many studies show important cognitive processes appear to be localized to these networks such as prediction of errors¹⁸ and show dysfunction in INs in various mental illnesses^11,63-65. It is also important to note that INs comprise most of the variance of the fMRI data⁶².

Evaluating characteristics of intrinsic networks in health and disease has gained considerable momentum in recent years. However, most previous studies have evaluated only a small subset of the intrinsic networks (e.g. default mode). While this approach has revealed significant differences in, e.g. schizophrenia and bipolar disorder⁶³, it does not enable us to evaluate the underlying functional brain changes in a comprehensive manner. One approach to address this is to use a multivariate testing framework for testing multiple intrinsic networks and multiple aspects of each network while also controlling the false positive rate associated with the multiple testing¹⁵. This approach has been applied to data collected from schizophrenia, bipolar disorder, and healthy controls⁶⁶. Three key measures include the spatial maps for each IN, the pair-wise correlation among these networks (called functional network connectivity or FNC), and the power spectra for each IN (see Figure 5).

Figure 5.

Three output measures from group ICA: ICA enable investigation of multiple output measures including 1) spatial maps (left panel) for each component which can be grouped based on the regions involved, 2) functional network connectivity (correlation among ICA timecourses) provides a measure of how temporally correlated the different components are, note the block structure is consistent with the grouping on the left, and 3) spectra of the ICA timecourses (which can help identify artifacts which tend to have much more high frequency power).

In addition to FNC, which captures temporal dependencies among networks, one can also capture spatial dependencies. Since ICA decomposition naturally takes temporal dependency into account through its underlying model, one can first establish an ICA decomposition—then, group the ICA components using spatial dependence among them, that is, mutual information (MI), to take full-order statistical information into account. In ⁵⁹, component grouping is achieved automatically by incorporating MI-based hierarchical clustering and hypothesis testing, without requiring prior knowledge for the probability distribution of each tuple, number or sizes of tuples⁵⁹ (see Figure 6). Using graph-theoretical analysis⁶⁷ in the spatial dependence structures for ICA components between two groups, physiologically meaningful differences are identified in their networks. As a following analysis step, one can also compute graph theoretic metrics resulting from these spatial and temporal dependencies in which we use components as spatial or temporal nodes, hence providing a more natural, data-driven approach to defining hubs^68-70.

Figure 6.

Spatial dependencies naturally group functionally related networks⁵⁹: It is intuitive that there are still temporal dependencies in the data, but this figure shows a grouping of 6 components containing spatial dependencies measured via mutual information. These spatial dependencies can be quite informative and tend to group artifacts and sensibly group functional regions together as well.

In addition, graph-theoretical analysis can also be applied to either the spatial dependencies or the temporal dependencies. Figure 7 shows an example in which there were a number of physiologically meaningful differences in networks constructed from the spatial dependencies in healthy controls and schizophrenia patients⁶⁷.

Figure 7.

Hubs of spatial dependence among 35 components for schizophrenia and controls: Using a mutual information-based measure, we can compute graph theoretic measures including the shown graph structure of spatial dependencies, which is complementary to the standard approach of using the temporal information to compute the graph structure and appears to be informative about patient versus control differences.

5. Variations on ICA

There are numerous extensions of ICA which have been applied to fMRI data and continue to be developed. We summarize a few of them in this section including the incorporation of prior information in the spatial or temporal domain and flexible ICA algorithms.

5.1. Constrained ICA of fMRI data

There have been multiple ICA algorithms proposed which attempt to incorporate prior knowledge into the algorithm in order to improve performance in certain cases. Constraints can be incorporated both in terms of time courses (columns of the mixing matrix) or spatial maps (estimated components/sources). For example, since in the case of a task we can provide information about the hemodynamic model and constrain the ICA mixing matrix⁷¹. Alternatively, if one is interested in a particular network and can use images from a previous analysis or regional locations in an atlas space to derive spatial template a spatially constrained algorithm would be a useful option⁵⁴.

Constraining the time courses

Using a semi-blind ICA (sbICA)⁷¹ we analyzed data from individuals who had performed an auditory oddball task and constrained the time courses (mixing matrix) using the paradigm timing information. This is done by first computing a GLM model (using the default hemodynamic response function in the SPM software) and then using this model to constrain the component time courses. For the blind ICA analysis, the component-of-interest was selected by performing a multiple regression of the target/novel regressor upon the ICA time courses post hoc. The component which was most highly correlated with this regressor was selected. The blind ICA approach tends to capture temporal lobe regions into a separate component, but is not strongly correlated with the task. The sbICA approach also includes motor and parietal regions and the correlation with the target/novel regressor (e.g. the task-relatedness) is significantly higher (0.51 vs. 0.33), as expected.

Constraining the spatial maps

One can also incorporate spatial information into the ICA algorithm. One example of this is a semi-blind spatial ICA algorithm that uses spatial information within the framework of constrained ICA with fixed-point learning⁵⁴. We performed spatial ICA on data collected during a visuomotor experiment⁷² using three spatial priors. The prior information about the brain function areas related to the three signals of interest, i.e., the right and the left visuo-motor task-related signals and the default mode signal, was obtained from published experimental results on subjects performing similar visuo-motor tasks. The prior information is used to create masks using WFU_PickAtlas⁷³, a tool that allows the user to create masks by selecting different areas of the brain, both appropriate Brodmann Areas (BAs) and functional areas. Performance for the spatially constrained algorithm was higher than blind ICA using either Infomax or FastICA⁵⁴.

5.2. Adaptive ICA Approaches

As discussed in Section 2.2, a major portion of the ICA algorithms exploit either non-Gaussianity (and hence higher-order statistics) or sample correlation, while a few recent algorithms have shown that it is possible to take advantage of both types of information as in most cases, the underlying sources are likely to be both non-Gaussian and have sample correlation. As noted in Section 2.1, fMRI data definitely falls into this class where the underlying sources are expected to be both non-Gaussian and exhibit sample correlation. Unfortunately, a limited view of ICA decompositions might miss this important point and even the importance of density modeling---which can be adapted to prior information such as sparsity of the sources,---and can make claims implying that ICA only favors sparsity⁷⁴.

In this section, we present a comparison of the performance of three ICA algorithms and show the importance of taking sample correlation information along with higher-order statistics into account. The three ICA algorithms are Infomax, the most widely used algorithm for fMRI analysis¹, entropy bound minimization (EBM)²¹ that adapts to a wide range of source distributions, and full blind source separation (FBSS) ⁷⁵ which has the ability to incorporate a flexible density model along with sample correlation information. We apply these three ICA algorithms to fMRI data from multiple subjects performing an auditory oddball task (AOD). In this example, fMRI data from 20 subjects (all healthy controls) performing an AOD task is used for the comparison.

Since all three algorithms are of iterative type, ICASSO⁷⁶ implemented in GIFT is used to check the consistency of the three ICA algorithms to improve robustness of the estimation results. ICASSO runs the ICA algorithm several times and produces different estimated components for each run and then collects the components by clustering them based on the absolute value of the correlation between squared source estimates⁷⁶. Reliable estimates correspond to tight clusters. Eight components of interest were manually selected for the comparison and a map was formed and thresholded at t = 8.94 (p < 0.001 corrected for multiple comparisons using the family wise error (FWE) approach implemented in the SPM5 software). Only voxels beyond this threshold were included in subsequent analysis and the maps of DMN, left parietal, right parietal and temporal lobe are shown in Figure 10. Four components are shown including: 1. DMN; 2. Left parietal (LP); 3. Right parietal (RP); and 4. Temporal lobe.

Figure 10.

Infomax, EBM, and FBSS results. are shown from left to right: T maps of three algorithms are generated for the AOD task. Each component of interest is entered into a one-sample test and is thresholded at P < 0.001 (FWE corrected). Six slices from each component are shown. The more flexible FBSS and EBM appear to have more included positive regions and less anticorrelated white matter signal.

Six masks were generated using WFU Pickatlas⁷³. For each algorithm, t maps generated were compared to these masks for investigating the overlaps with the masks. For evaluation, first, the number of voxels that survive in one-sample t test and are overlapped with the corresponding masks is calculated for algorithms. Second, the value of sensitivity is calculated as the ratio of the number of voxels overlapped with the corresponding mask to the number of voxels that survive in one-sample t test. The results are given in Table 1.

Table 1.

Estimated components overlapping with masks (thresholded at P < 0.001; FWE corrected).

2*Algorithm	Components
(r)2-7	DMN	FL	PL	TL	OL	Ce
Number of voxels overlapped with the corresponding mask
Infomax	2386	1944	1261	2345	3025	2868
EBM	3291	1790	1521	2247	3346	3616
FBSS	3328	1801	1499	2249	3316	3417
Sensitivity of map with corresponding mask
Infomax	0.73	0.79	0.75	0.66	0.86	0.99
EBM	0.82	0.72	0.80	0.67	0.83	0.97
FBSS	0.82	0.73	0.80	0.67	0.83	0.99

The results show that all three algorithms provide competitive performance, however, even though Infomax estimates more voxels than the other two algorithms after thresholding, the positive voxels (after multiple comparison correction for all three algorithms) are significantly higher in all components except the temporal lobe. EBM and FBSS estimate very similar numbers of voxels that survive in one-sample t test. However, for the DMN component, FBSS estimates slightly more voxels than EBM. After thresholding at the same level, EBM and FBSS have similar numbers of estimated voxels overlapped with the corresponding mask and their sensitivity values are close. On the other hand, Infomax leads to smaller number of voxels above the threshold and smaller sensitivity values than the other two algorithms for DMN. For the component of frontal lobe, Infomax has a slightly larger sensitivity value than EBM and FBSS. As observed from the table, FBSS provides consistent estimation of DMN and the estimated DMN spatial map has more voxels overlapping with the mask and shows higher sensitivity. Estimated time course for the DMN component also has the largest negative correlation with the task paradigm as expected. For left parietal and right parietal, the estimated time courses give opposite signs with stimuli and this phenomenon is not observed in the results of Infomax, which is an important characteristic of these networks. In addition, a study of the timecourses (not shown) shows that for the left parietal and right parietal network, the estimated time courses of FBSS and EBM, the two algorithms with adaptive density yield opposite signs as expected⁷⁷, which is not the case for Infomax.

6. Application to Study Brain Disease: Neurodiagnostic Discovery

ICA has been applied quite extensively to study brain disease, most commonly mental illness. Early work focused on schizophrenia^10,64, substance use⁷, and Alzheimer's disease¹¹, and mild cognitive impairment^50,78. More recently it has been used to study bipolar disorder^63,66, epilepsy⁷⁹, psychopathy⁸⁰, antipsychotic drug effects⁸⁰, and intrinsic networks in animal studies^81,82. The evaluation of how disease impacts graph theoretic properties computed from ICA decompositions is also promising^69,70. Indeed, ICA has shown itself to be a key tool in the study of aberrant functional connectivity in many other diseases as well, and its ultimate impact will only be clear at a later point. Recent work has suggested that the use of functional connectivity may be useful as an accurate diagnostic tool⁸³. One promising area is the use of imaging to suggest risk or predict outcome measures. However much more work is needed in order to evaluate the clinical utility of such a tool (e.g. more studies of unmedicated patients⁸⁴ as well as studies of the specificity of the prediction).

7. Recent Work: Dynamics

More recently, there has been interest in studying the dynamics of the intrinsic networks of the brain^85,86. Variations in ongoing activity have been shown to predict changes in task performance and alertness, highlighting their importance for understanding the connection between brain activity and behavior^18,87. Dynamics are potentially even more prominent in the resting-state, during which mental activity is unconstrained. As with recent studies exploring resting-state dynamics^85,86,88, ICA can be used to capture and exploit the dynamic changes and variability over time in brain activity. Addressing dynamics also has the potential to be a better tool for differentiating clinical groups.

One straightforward approach for evaluating dynamics in fMRI data is by evaluating the FNC correlations in either task or rest fMRI over time via a sliding windowed approach^86,89 (called dynamic FNC). Dynamic FNC, based on spatial ICA, provides additional results that are different than, but complementary to, those of static FNC. For example, we have shown dynamic changes in default mode network connectivity with other regions is significantly different in schizophrenia patients in terms of task-modulation. Figure 11 shows two results showing group differences between patients with schizophrenia and healthy controls (top) time frequency analysis reveals persistent low frequency power differences and (bottom) pairwise correlations are showing group differences in the latter portion of the experiment (shaded portion of curve). It is important to note that these two pairs of components did not show a group difference for a static FNC analysis, hence motivating the importance of evaluating the dynamics.

Figure 11.

Dynamic FNC results: (top) low frequency differences in lateral frontal components and (bottom) differences (mainly later in experiment) between temporal lobe and anterior DMN.

Dynamic FNC

A dynamic approach to evaluating the FNC was implemented as pilot data by estimating 100 ICA components, low-pass filtering the FNC timecourses, multiplying the timecourse by a Gaussian with a FWHM of 30 seconds and computing the covariance matrix⁸⁹. This was performed for each point of a 5 minute resting fMRI data set in 400 individuals. Resulting covariances were then clustered to identify stable states^90-92. A null distribution was created by permuting the phase of the timecourses. One interesting feature one can compute is the standard deviation of these state networks over time. Figure 12 shows a matrix of the standard deviation of dynamic states and importantly shows a striking difference between schizophrenia patients (showing less variability) and healthy controls. Data was collected from 100 subjects (50 in each group), site differences were regressed out, stringent motion criteria were applied (though we have found ICA to be much more robust to motion that seed based methods)⁹³. Only data passing the fBIRN QA procedures was used⁹⁴. Results are consistent with a generalized cognitive deficit in schizophrenia and clearly motivate the important of these measures for studying clinical groups.

Figure 12.

Variability of dynamic FNC states (components on x/y axes) varies dramatically in schizophrenia patients (left) and healthy controls (right): This suggests there is important information about the patients in the dynamic changes which is not detectable in the static FNC results.

8. Summary and Future Directions

In summary, group ICA is a powerful tool for analyzing fMRI data which has been developed over the past 10+ years. There are still numerous directions in which the algorithms continue to be developed including incorporation of additional information, characterization of features via graph-theoretic approaches, new approaches for performing multi-subject ICA such as using independent vector analysis⁹⁵, and assessment of the dynamics of the temporal and spatial relationships. The application of ICA to multimodal data fusion³⁵ and genetics^96,97 is another very important area which has been applied to study brain disease which should be considered though it is beyond the scope of this particular review article.

Figure 8.

Comparison of ICA and sbICA in one participant: Results for task-related component for blind ICA (left) and sbICA (right). ICA tends to capture primarily temporal lobe regions and is not highly task-related. The correlation with the novel/target regressor (e.g. the task-relatedness) is significantly increased (0.51 vs. 0.33) for the sbICA analysis and includes expected motor regions (expected since the target stimulus required a button press).

Figure 9.

Results from a spatially constrained ICA analysis: (top) spatial templates used in the ICA-R algorithm for the visuomotor (VM) task for right and left stimulation in addition to a default mode template. Results from an unconstrained Infomax algorithm are shown in the middle row. The ICA-R algorithm using all three templates is shown. Left and Right visuomotor results are similar but slightly improved for ICA-R whereas the default mode result (right column) is markedly improved for the ICA-R.