Sparse Principal Component based High-Dimensional Mediation Analysis

Abstract

Causal mediation analysis aims to quantify the intermediate effect of a mediator on the causal pathway from treatment to outcome. When dealing with multiple mediators, which are potentially causally dependent, the possible decomposition of pathway effects grows exponentially with the number of mediators. An existing approach incorporated the principal component analysis (PCA) to address this challenge based on the fact that the transformed mediators are conditionally independent given the orthogonality of the principal components (PCs). However, the transformed mediator PCs, which are linear combinations of original mediators, can be difficult to interpret. A sparse high-dimensional mediation analysis approach is proposed which adopts the sparse PCA method to the mediation setting. The proposed approach is applied to a task-based functional magnetic resonance imaging study, illustrating its ability to detect biologically meaningful results related to an identified mediator.

1. Introduction

Causal mediation analysis has been widely applied in social, psychological, and biological studies to evaluate the intermediate effect of a variable (called a mediator) on the causal pathway from an exposure/treatment to a target outcome for the purpose of exploring hypothesized causal mechanisms. The single mediator setting has been extensively studied [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]. During the past decade, methods for dealing with multiple mediators have attracted increasing attention [11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]. Most of these methods are designed for dealing with relatively low-dimensional data. With the emergence of modern technologies (e.g., high-throughput technologies in omics studies and neuroimaging technologies), data sets with a large number of variables are being collected. However, methods for conducting mediation analysis with high-dimensional mediators are limited. Motivated by a genetics study, Huang and Pan (2016) [27] proposed a principal component analysis (PCA) based approach for reducing the high-dimensional gene expression mediators to a series of marginal mediation problems. Incorporating a regularized regression for the outcome model, Zhang et al. (2016) [28] introduced an independent screening approach for high-dimensional mediation analysis.

In the field of neuroimaging, studies on the impact of brain mediators on cognitive behavior are becoming increasingly popular [29, 30, 31, 32, 33, 34, 35]. Caffo et al. (2007) [29] presented an early attempt at addressing neurological images as mediators, though the analysis was conducted on univariate summaries extracted from the multivariate images. Recently, Chén et al. (2017) [36] proposed a mediation analysis approach that transforms the high-dimensional mediator candidates into independent directions of mediation (DMs). Geuter et al. [37] extended this approach by transforming components based on the amount of the indirect (mediating) effect they explained. Under a linear structural equation modeling (LSEM) framework, the directions are ranked based on the proportion of the likelihood that they account for. Zhao and Luo (2016) [38] recently proposed a general mediation model under the LSEM framework to account for the causal dependencies between the mediators and introduced a lasso-type penalty to regularize the mediation pathway effects to achieve simultaneous mediator selection and mediation effect estimation.

In both the PCA and the directions of mediation analysis, the acquired PCs and DMs are linear combinations of the original mediating variables. With nonzero loadings, their interpretation is not always straightforward. An informal way of reducing the number of variables is to set a hard threshold, and force loadings with absolute value below the threshold to be zero. However, this can be potentially misleading, as it is not only the loadings but also the variance of the variable that governs its importance [39]. Jolliffe et al. (2003) [40] introduced a modified PCA approach based on the lasso [41]. Built on the fact that sparsifying the PC loadings can be expressed as a regression-type optimization problem, Zou et al. (2006) [42] introduced the sparse principal component analysis (SPCA) approach. The same idea was later implemented in canonical correlation analysis (CCA) [43, 44, 45].

In this study, we propose a sparse principal component of mediation (SPCM) approach to perform high-dimensional mediation analysis. This approach has two stages: the first performs a PCA for high-dimensional mediation using the method proposed in Huang and Pan (2016) [27]; and the second sparsifies the loading vector using a (structured) regularization. In neuroimaging studies, spatial smoothing regularization is generally imposed to enforce spatial smoothness and yield meaningful biological interpretations [46, 47]. In this study, the fused lasso [48], as a special case of the generalized lasso [49], will be employed to impose local smoothness and constancy. We apply the proposed approach to a task-based functional magnetic resonance imaging (fMRI) study, where participants were instructed to preform a probabilistic classification learning task. Our method enables the identification of biologically interpretable brain pathways that are responsive to the task stimulus and also have an impact on the reaction time to the task.

This paper is organized as follows. Section 2 introduces the PCA based mediation approach for multiple mediators proposed in Huang and Pan (2016) [27]. In Section 3, we present the sparse principal component of mediation approach. Section 4 summarizes the simulation results. We apply our proposed method to a task-based fMRI study in Section 5. Section 6 summarizes this paper with discussions.

2. Causal Mediation Analysis with Multiple Mediators

Mediation analysis aims to quantify the causal effect of a treatment/exposure (X) on the outcome (Y) mediated by a third variable, called the mediator (M). This causal relationship can be represented using a causal diagram as in Figure 1a. Linear structural equation modeling (LSEM) is a popular approach for performing mediation analysis. Let M(x) and Y(x, M(x)) denote the potential outcome of the mediator and the outcome under treatment assignment x [50, 51], the mediation models are written as

$$M(x)={\alpha }_0+\alpha x+ϵ,\text{and}$$ (1)

$$Y(x,M(x))={\beta }_0+\gamma x+\beta M(x)+\eta ,$$ (2)

where ϵ and η are independent model errors with mean zero. The average total treatment effect is decomposed as

$$\begin{gathered} \mathrm{ATE}(x,x^{\ast }) \\ =\mathbb{E}[Y(x,M(x))]-\mathbb{E}[Y(x^{\ast },M(x^{\ast }))] \\ =\mathbb{E}[Y(x,M(x))-Y(x,M(x^{\ast }))]+\mathbb{E}[Y(x,M(x^{\ast }))-Y(x^{\ast },M(x^{\ast }))] \\ =\mathrm{AIE}(x,x^{\ast })+\mathrm{ADE}(x,x^{\ast }), \end{gathered}$$ (3)

where x and x* are two distinct treatment assignments. Under models (1) and (2), $\text{ADE}(x,x^{\ast })=\mathbb{E}[Y(x,M(x^{\ast }))-Y(x^{\ast },M(x^{\ast }))]=\gamma (x-x^{\ast })$ is the average (controlled) direct effect of the treatment on the outcome, and $\text{AIE}(x,x^{\ast })=\mathbb{E}[Y(x,M(x))-Y(x,M(x^{\ast }))]=\alpha \beta (x-x^{\ast })$ is the average indirect effect. Under assumptions (see Section A of the supplementary material), these causal estimands can be identified from the observed data [9, 10].

Figure 1:

Causal diagram of (a) single mediator, (b) p dependent ordered mediators and (c) q causally independent transformed mediators.

With multiple mediators, a challenge is to delineate the causal structure among the mediators. When the mediators maintain an ordered structure, one can directly penalize each causal connection in a directed acyclic graph [52]. In certain applications, for example the process of inhibitory motor control [53], causality between brain activation has been established via transcranial magnetic stimulation. This type of causal relationship between brain regions is referred to as the effective connectivity, which captures the influence that one neural system exerts over another [54, 55]. This presumes an ordering of neural systems. Therefore, when considering brain activity from different brain regions as mediators, the assumption that there are p-ordered mediators is consistent with a working model of connectivity. However, functional magnetic resonance imaging (fMRI) is not sufficiently informative to determine the causal ordering of the brain regions, given its low temporal resolution and high noise. To avoid sequentially modeling multiple mediators, Huang and Pan (2016) [27] introduced the concept of applying principal components to the mediators, which can be used to linearly combine the candidate mediators. Imposing orthogonality, the mediation principal components are conditionally independent given the treatment. The complex causal structure in Figure 1b can be transformed into a problem with parallel causal mediation pathways, as shown in Figure 1c. As discussed in Imai et al. (2013) [13] and VanderWeele (2015) [10], with causally independent multiple mediators, it is equivalent to performing a series of marginal mediation analyses.

Let X denote the treatment assignment, $\mathbf{M}(x)=(M_1(x),\dots ,M_p(x){)}^{⊺}\in {\mathbb{R}}^p$ the p-dimensional potential outcome of mediator given the treatment assignment at level x. Let ${\tilde{M}}^{(j)}(x)=\mathbf{M}(x{)}^{⊺}{\varphi }_j(j=1,\dots ,p)$ be a linear projection of the potential outcome M(x), such that

$${\tilde{M}}^{(j)}(x)⫫{\tilde{M}}^{(k)}(x)\mid X=x,\mathrm{for}j\ne k;$$ (4)

that is, the mediators in the projection space are causally independent under the definition in Imai and Yamamoto (2013) [13]. In this setting, the problem is equivalent to conducting a series of marginal mediation analyses. For subject i (i = 1, … , n), under the LSEM framework, for each j = 1, … , p,

$${\tilde{M}}_i^{(j)}={\alpha }_{0j}+{\alpha }_j{X}_i+{\xi }_{ij},\mathrm{and}$$ (5)

$$Y_i={\beta }_{0j}+{\gamma }_j{X}_i+{\beta }_j{\tilde{M}}_i^{(j)}+{\eta }_{ij},$$ (6)

where {α_0j, α_j, β_0j, γ_j, β_j} is the model parameter, and ξ_ij and η_ij are independent model errors normally distributed with mean zero. Here α_jβ_j(x – x*) is the average indirect effect of the projected mediator ${\tilde{M}}^{(j)}$ comparing treatment x and x*, and ${\sum }_{j=1}^p{\alpha }_j{\beta }_j(x-x^{\ast })$ is the total average indirect effect. Under this marginal LSEM, γ_j(x – x*) is interpreted as the treatment effect not mediated through the mediator ${\tilde{M}}^{(j)}$.

As proposed in Huang and Pan (2016) [27], obtaining these causally independent mediators is achieved through principal component analysis. Consider

$$M_{ij}={\tau }_{0j}+{\tau }_j{X}_i+{ϵ}_{ij},\mathrm{for}i=1,\dots ,n\mathrm{and}j=1,\dots ,p,$$ (7)

where {τ_0j, τ_j} are model coefficients, and ϵ_ij is the normally distributed random error with mean zero. Let ϵ_i = (ϵ_i1, … , ϵ_ip)^⊤, assume

$$\mathbb{E}({ϵ}_i{ϵ}_i^{⊺})=\Sigma =\Phi \Lambda {\Phi }^{⊺},$$ (8)

where $\Phi =({\varphi }_1,\dots ,{\varphi }_p)\in {\mathbb{R}}^{p\times p}$ is an orthonormal matrix such that Φ^⊤Φ = I, and Λ = diag{λ₁, … , λ_p} is a p-dimensional diagonal matrix, and thus ΦΛΦ^⊤ is the spectral decomposition of the positive-definite matrix Σ. The columns of $\tilde{\mathbf{M}}=\mathbf{M}\Phi$ are conditionally independent given X, where M = (M₁, … , M_n)^⊤ and M_i = (M_i1, … , M_ip)^⊤ for i = 1, … , n. The method proposed in Huang and Pan (2016) [27] is summarized as follows:

Step 1. For j = 1, … , p, fit model (7) and denote the residuals as {e_i1, … , e_ip} for i = 1, … , n.

Step 2. Conduct PCA on the residuals $\{e_{i1},\dots ,e_{ip}{\}}_{i=1}^n$ to obtain $\widehat{\Phi }=({\widehat{\varphi }}_1,\dots ,{\widehat{\varphi }}_p)$ and $\widehat{\Lambda }$.

Step 3. Let ${\tilde{M}}_i^{(j)}={\mathbf{M}}_i{\widehat{\varphi }}_j$. Using the transformed mediators, perform marginal mediation analysis under models (5) and (6), for j = 1, … , q, where analogous to PCA, q is determined by the designated proportion (for example 80%) of variance explained.

Though the focus of Huang and Pan (2016) [27] is on hypothesis testing for the direct and total indirect effects, one can adapt their approach to make inference about individual pathway effects.

It is well-known that PC loadings are sign nonidentifiable. As a following step, parameters in the mediation models (5) and (6) are sign nonidentifiable as well. Here, we show that though the estimate of α and β are not sign identifiable, the estimate of the indirect and direct effects are sign consistent. Since in mediation analysis, the indirect and direct effects are the effects of interests, with Proposition 1, one can directly interpret the findings.

Proposition 1. Let ${\tilde{\mathbf{M}}}^{(1j)}=\mathbf{M}{\varphi }_j$ and ${\tilde{\mathbf{M}}}^{(2j)}=\mathbf{M}(-{\varphi }_j)=-\mathbf{M}{\varphi }_j$. Let (${\widehat{\alpha }}_j^{(s)}$, ${\widehat{\beta }}_j^{(s)}$, ${\widehat{\gamma }}_j^{(s)}$) denote the estimate from the transformed mediator ${\tilde{\mathbf{M}}}^{(sj)}$ using models (5) and (6), for s = 1, 2. Then

$${\widehat{\alpha }}_j^{(1)}=-{\widehat{\alpha }}_j^{(2)},{\widehat{\beta }}_j^{(1)}=-{\widehat{\beta }}_j^{(2)},and{\widehat{\gamma }}_j^{(1)}={\widehat{\gamma }}_j^{(2)}.$$

Thus the estimated direct and indirect (estimated by the product αβ) effects are sign invariant.

3. Sparse High-Dimensional Mediation Analysis

3.1. Motivation

As discussed in Huang and Pan (2016) [27], the estimated causal effects “do not necessarily have an intuitive interpretation”, since the transformed mediators are linear combinations of the original mediators. This drawback commonly occurs in PCA-based studies. An informal way to reduce the number of variables is to set a hard threshold and force the loadings with absolute value below the threshold to be zero. However, this can be potentially misleading; for example, see [39]. Jolliffe et al. (2003) [40] introduced the modified PCA based on the lasso [41] to yield possible zero loadings. This sparse PCA framework was then further studied by Zou et al. (2006) [42] based on the fact that sparsifying the PC loadings is equivalent to a regression-type optimization problem. In this study, we propose a sparse PCA based mediation analysis approach to estimate the mediator PCs with sparse loadings.

3.2. The lasso, the generalized lasso and the elastic net

The least absolute shrinkage and selection operator (lasso) was introduced by Tibshirani (1996) [41] to perform simultaneous variable selection and estimation in linear regression. Let Y = (Y₁, … , Y_n)^⊤ denote the dependent variable, and X = (X₁, … , X_n)^⊤ where X_i = (X_i1, … , X_ip)^⊤ (i = 1, … , n) the design matrix with p predictors. The lasso solution minimizes the squared-error loss under ℓ₁ regularization. That is,

$${\widehat{\beta }}_{\mathrm{lasso}}=\underset{\beta \in {\mathbb{R}}^p}{\arg \min }\Vert Y-\mathbf{X}\beta {\Vert }_2^2+\lambda \Vert \beta {\Vert }_1,$$ (9)

where $\beta \in {\mathbb{R}}^p$ is the model coefficient, λ ≥ 0 is a tuning parameter, and $\Vert \mathbf{x}{\Vert }_1={\sum }_{j=1}^p\mid x_j\mid$ is the ℓ₁-norm of a p-dimensional vector $\mathbf{x}\in {\mathbb{R}}^p$. When the tuning parameter λ is large enough, some coefficients will be shrunk to exactly zero. Under regularity conditions, the lasso estimator has been shown to be both consistent and sparsistent [see 56, 57, 58].

Tibshirani and Taylor (2011) [49] considered the problem of the generalized lasso to enforce structured constraints instead of pure sparsity. The problem can be formalized as

$${\widehat{\beta }}_{\mathrm{glasso}}=\underset{\beta \in {\mathbb{R}}^p}{\arg \min }\Vert Y-\mathbf{X}\beta {\Vert }_2^2+\lambda \Vert \mathbf{D}\beta {\Vert }_1,$$ (10)

where $\mathbf{D}\in {\mathbb{R}}^{m\times p}$ is a prespecified penalty matrix. The asymptotic properties of the solution was studied in She (2010) [59].

The lasso has several limitations. As discussed in Zou and Hastie (2005) [60], one limitation is that for predictors with high collinearity, the lasso tends to randomly select one of them; and second, when p > n, the lasso selects at most n variables. In the mediation analysis setting, the mediators are potentially causally dependent, which violates the incoherence assumption for the lasso when taking the mediators as predictors in the outcome model. Considering brain voxels as mediators, where p ~ 100, 000, with limited number of trials n < 100, it is not desirable to be constrained to select at most n voxels. Zou and Hastie (2005) [60] introduced the elastic net to address these drawbacks by introducing a convex combination of ℓ₁ and ℓ₂ penalties. The elastic net solution is written as

$${\widehat{\beta }}_{\mathrm{en}}=(1+{\lambda }_2)\left\{\underset{\beta \in {\mathbb{R}}^p}{\arg \min }\Vert Y-\mathbf{X}\beta {\Vert }_2^2+{\lambda }_1\Vert \beta {\Vert }_1+{\lambda }_2\Vert \beta {\Vert }_2^2\right\},$$ (11)

where λ₁, λ₂ ≥ 0. When λ₂ is positive, the elastic net approach can potentially choose all the variables and overcomes the drawbacks with the ℓ₁ penalty only. In this study, we consider the following generalized elastic net solution, i.e.,

$${\widehat{\beta }}_{\mathrm{gen}}=(1+{\lambda }_2)\left\{\underset{\beta \in {\mathbb{R}}^p}{\arg \min }\Vert Y-\mathbf{X}\beta {\Vert }_2^2+{\lambda }_1\Vert \mathbf{D}\beta {\Vert }_1+{\lambda }_2\Vert \beta {\Vert }_2^2\right\},$$ (12)

to impose a structured regularization, for example the fused lasso [48]. Motivated by a protein mass spectroscopy study, the fused lasso was introduced to encourage sparsity in the coefficients as well as in their successive differences. This local constancy property is also desirable in fMRI data, as a small-world topology supports both segregated/specialized and distributed/integrated information processing in the human brain [61].

3.3. Sparse approximation

For PCA, Zou et al. (2006) [42] studied a simple regression approach to recover the PC loadings and showed that with a ridge penalty, the normalized solution to the regression problem, by regressing the loadings on the variables, is independent of λ. With this property, the inclusion of ridge penalty is not meant to penalize the regression coefficients but to ensure the reconstruction of principal components. As described in Section 2, in mediation analysis, PCA is conducted on the residuals of the mediation models. Therefore, we propose to sparsify the loadings using these model residuals, i.e., considering the following optimization problem, for k = 1, … , q,

$${\widehat{\mathbf{v}}}_k=(1+{\lambda }_2)\left\{\underset{\mathbf{v}\in {\mathbb{R}}^p}{\arg \min }\Vert {\tilde{\mathbf{e}}}^{(k)}-\mathbf{Ev}{\Vert }_2^2+{\lambda }_1\Vert \mathbf{Dv}{\Vert }_1+{\lambda }_2\Vert \mathbf{v}{\Vert }_2^2\right\},$$ (13)

where $\mathbf{v}=(v_1,\dots ,v_p{)}^{⊺}\in {\mathbb{R}}^p$; E = (e₁, … , e_n)^⊤ with e_i = (e_i1, … , e_ip)^⊤ is the residual matrix, and ${\tilde{\mathbf{e}}}^{(k)}=\mathbf{E}{\widehat{\varphi }}_k$ is the kth principal component. Then ${\widehat{\mathbf{w}}}_k={\widehat{\mathbf{v}}}_k∕\Vert {\widehat{\mathbf{v}}}_k{\Vert }_2$ is a sparse approximation of ${\widehat{\varphi }}_k$.

In neuroimaging studies, a spatial smoothness constraint is commonly applied (for example, see [46, 47]). In this study, we consider a fused lasso penalty [48] to impose a local constancy of the PC profile, i.e., neighbor brain voxels/regions are assumed to have the same loading weights:

$${\widehat{\mathbf{v}}}_k=(1+{\lambda }_2)\left\{\underset{\mathbf{v}\in {\mathbb{R}}^p}{\arg \min }\Vert {\tilde{\mathbf{e}}}^{(k)}-\mathbf{Ev}{\Vert }_2^2+{\lambda }_{11}\Vert \mathbf{v}{\Vert }_1+{\lambda }_{12}\sum _{(j,j^{\prime })\in \mathcal{E}}\mid v_j-v_{j^{\prime }}\mid +{\lambda }_2\Vert \mathbf{v}{\Vert }_2^2\right\},$$ (14)

where $\mathcal{E}$ is an edge set such that $(j,j^{\prime })\in \mathcal{E}$ if M_j and M_j′ are neighboring brain voxels/regions as defined by their spatial locations. For other areas of application, the edge set can be defined based on domain knowledge. For example in genetic studies, it can be defined based on locus information. Formulation (14) is a special case of the generalized elastic net (12) with D the fused lasso matrix corresponding to the underlying graph with edge set $\mathcal{E}$.

3.3.1. Tuning parameter selection

Zou et al. (2006) [42] showed that the inclusion of a ridge penalty does not penalize the regression coefficients. Thus, when n ⪢ p, we can set the tuning parameter for ridge penalty λ₂ to zero; and when n ⪡ p, we can in principle use any positive λ₂. The objective of sparse PCA is to sparsify the loadings while preserving the proportion of variance explained. Zou et al. (2006) [42] proposed a data-driven approach to choose λ₁ (the ℓ₁-penalty tuning parameter) by examining the trace plot of the percentage of total variance explained calculated from the adjusted total variance. We apply the same strategy to choose tuning parameters λ₁₁ and λ₁₂ in (14).

3.4. Mediation analysis with sparse principal components

In this section, we discuss analysis with the sparse PC of the mediators. Let ${\check{M}}_i^{(k)}={\mathbf{M}}_i^{⊺}{\mathbf{w}}_k={\sum }_{j=1}^p{M}_{ij}{w}_{kj}$, and for k = 1, … , q, fit models

$$\begin{gathered} {\check{M}}_i^{(k)}=a_0+a_k{X}_i+{\check{ϵ}}_i^{(k)},\mathrm{and} \\ {Y}_i=b_0+c_k{X}_i+b_k{\check{M}}_i^{(k)}+{\check{\eta }}_i^{(k)}, \end{gathered}$$ (15)

where ${\check{ϵ}}_i^{(k)}$ and ${\check{\eta }}_i^{(k)}$ are independent random errors normally distributed with mean zero. One advantage of the PCA mediation analysis is that the transformed mediator PCs are conditionally independent, and fitting the LSEM with multiple mediators is equivalent to using marginal LSEMs for each individual mediator. By sparsifying the loading vector, the orthogonality constraint is not explicitly imposed. To achieve the conditional independence, we include a regression projection step to remove the conditional linear dependence between the transformed mediators, analogous to the procedure proposed in Zou et al. (2006) [42].

Let ${\check{M}}_i^{(k\cdot 1,\dots ,k-1)}$ denote the residual after adjusting for ${\check{M}}_i^{(1)},\dots ,{\check{M}}_i^{(k-1)}$ when controlling X_i (for i = 1, … , n). That is,

$${\check{M}}_i^{(k\cdot 1,\dots ,k-1)}={\check{M}}_i^{(k)}-{\check{\mathbf{M}}}_i^{(1,\dots ,k-1)⊺}{\widehat{\Pi }}_{1,\dots ,k-1},$$ (16)

where ${\check{\mathbf{M}}}_i^{(1,\dots ,k-1)}=({\check{M}}_i^{(1)},\dots ,{\check{M}}_i^{(k-1)}{)}^{⊺}$, and ${\widehat{\Pi }}_{1,\dots ,k-1}$ is the estimated coefficient in model

$${\check{M}}_i^{(k)}={\pi }_0+{\pi }_1{X}_i+{\check{\mathbf{M}}}_i^{(1,\dots ,k-1)⊺}{\Pi }_{1,\dots ,k-1}+{\tau }_i^{(k)},$$ (17)

where ${\tau }_i^{(k)}$ is normally distributed model error with mean zero. The new mediators ${\check{M}}^{(k\cdot 1,\dots ,k-1)}$ are uncorrelated given the treatment X. Thus, we can use model (15) to estimate the indirect effect of each individual mediation pathway.

We summarize the steps of mediation analysis with sparse principal components in Algorithm 1. To perform inference on model parameters, we propose a bootstrap procedure.

1. Generate a bootstrap sample ($X_i^{\ast }$, ${\check{M}}_i^{(k\cdot 1,\dots ,k-1)\ast }$, $Y_i^{\ast }$) of size n by resampling the data with replacement, where ${\check{M}}_i^{(k\cdot 1,\dots ,k-1)}$ is the modified mediator PC obtained in Step 4 of Algorithm 1.

2. Estimate the model parameters in (15) using the bootstrap sample.

3. Repeat steps (i)-(ii) B times.

Bootstrap confidence intervals can be then calculated using either the percentile or bias-corrected approach [62] under a pre-specified significance level.

The motivation of sparsifying the loading vectors is to facilitate intuitive interpretations of the findings. The estimated causal effects under the new mediators are desired to be as close as possible to the ones estimated from the original mediator PCs (the ${\tilde{M}}^{(j)}$’s). In Section B.2 of the supplementary material, we provide a sketch of the asymptotic convergence of the estimators from Algorithm 1 and those from ${\tilde{M}}^{(j)}$’s in models (5) and (6).

Algorithm 1 Mediation analysis with sparse principal components.
Step 1. For j = 1, … , p, fit model (7) and denote the residuals as {ei1, … , eip} for i = 1, … , n.
Step 2. Conduct PCA on the residuals $\{e_{i1},\dots ,e_{ip}{\}}_{i=1}^n$ and estimate the loading matrix as $\widehat{\Phi }=({\widehat{\varphi }}_1,\dots ,{\widehat{\varphi }}_p)$.
Step 3. For k = 1, … , q, let ${\tilde{\mathbf{e}}}^{(k)}=\mathbf{E}{\widehat{\varphi }}_k$, where E = (e1, … , en)⊤ and ei = (ei1, … , eip)⊤ (i = 1, … , n). Perform regularized regression using the generalized elastic net penalty (12) and attain the estimator ${\widehat{\mathbf{v}}}_k({\widehat{\lambda }}_k)$ and its normalization ${\widehat{\mathbf{w}}}_k({\widehat{\lambda }}_k)={\widehat{\mathbf{v}}}_k({\widehat{\lambda }}_k)∕\Vert {\widehat{\mathbf{v}}}_k({\widehat{\lambda }}_k){\Vert }_2$, for k = 1, … , q, where ${\widehat{\lambda }}_k$ is chosen based on the method discussed in Section 3.3.1.
Step 4. Let ${\check{M}}_i^{(k)}={\mathbf{M}}_i^{⊺}{\widehat{\mathbf{w}}}_k$ for i = 1, … , n. For k = 2, … , q obtain the modified ${\check{M}}_i^{(k\cdot 1,\dots ,k-1)}$.
Step 5. Fit model (15) using the causally independent $\{{\check{M}}_i^{(1)},{\check{M}}_i^{(2\cdot 1)},\dots ,{\check{M}}_i^{(q\cdot 1,\dots ,q-1)}\}$ to yield estimates of the causal effects.

4. Simulation Study

A simulation study was conducted to examine the performance of the proposed sparse PC based mediation analysis approach. A description of the study and the results are presented in Section D of the supplementary material available online. To summarize, the proposed sparse PC based method achieves comparable estimates of the indirect effect to the PCA based approach introduced in Huang and Pan (2016) [27] under various values of p and n. Using bootstrap confidence intervals following the procedure described in Section 3.4, we find that for those PCs with a nonzero indirect effect, the coverage probability can be slightly lower than the nominal level, but the power is reasonably high and converges to one as n increases. In addition, for those PCs with a null indirect effect, the coverage probability and Type I error are well controlled at the designated level.

5. A Task-Based Functional MRI Study

We analyze a task-based fMRI study using the proposed sparse principal component of mediation approach. The data set is downloaded from the OpenfMRI database (accession number ds000002). In the experiment, participants were instructed to perform a probabilistic classification learning (PCL) task for weather prediction [63]. To avoid inter-subject heterogeneity, we use the data from a single healthy right-handed English-speaking subject aged between 21 to 26. The experiment consisted of n = 80 trials with ten cycles, and within each there are five PCL trials intermixed with three baseline trials. Under the weather prediction trial, a visual stimulus was presented at a randomized location. The participant would respond by pressing either the left button for a “sun” prediction or the right button for “rain”. The experiment included baseline trials to control for visual stimulation, button press and computer response to button press. A fundamental question that PCL task-fMRI study aims to address is whether and how the memory systems interact. In this study, we take a step further considering reaction time of button pressing (Y) as the outcome of interest, and the goal is to discover the brain networks that have an intermediate effect on the reaction time when comparing PCL (X = 1) and baseline (X = 0) trials.

One hundred and eighty functional T2*-weighted echoplanar images (EPI) (4 mm slice thickness, 33 slices, TR = 2 s, TE = 30 ms, flip angle = 90°, matrix 64 × 64, field of view 200) were acquired from a 3 T Siemens Allegra MRI scanner. For registration purposes, a matched-bandwidth High-Resolution scan (same slice prescription as EPI) and MPRAGE (TR = 2.3, TE = 2.1, FOV = 256, matrix = 192 × 192, saggital plane, slice thickness = 1 mm, 160 slices) were acquired for each participant. Preprocessing for both anatomical and functional images was conducted using Statistical Parametric Mapping version 5 (SPM5) (Wellcome Department of Imaging Neuroscience, University College London, London, UK), including slice timing correction, realignment, coregistration, normalization, and smoothing. The blood-oxygen-level dependent (BOLD) time courses are extracted from p = 264 putative brain functional regions defined in Power et al. (2011) [64]. These brain regions are grouped into eleven (ten functional and one uncertain) modules. We employed the general linear model approach to acquire a summary measure of brain activity for each trial at each region [32, 65, 66]. These single-trial brain activities were used as the mediators (M).

We applied the PCA based high-dimensional mediation analysis proposed in Huang and Pan (2016) [27] and our proposed sparse PCA based method. When conducting sparse approximation, we consider a fused lasso penalty (14), where edge set $\mathcal{E}$ is defined based on the spatial location of the brain region, as well as module information given the modular processing nature of human brains [61]. If brain regions j and k are from the same functional module, then $(j,k)\in \mathcal{E}$; if region , k is the spatially nearest neighbor of region j, then $(j,k)\in \mathcal{E}$. In the PCA based analysis, the first 18 PCs, which account for 76.2% of the total variation, were tested for mediation effect (see Figure C.1 in the supplementary material). PC3 shows significant positive indirect effect, where the linear combination of all brain regions shows deactivation in PCL compared to baseline (α estimate −0.376 with 95% confidence interval (−0.592, −0.160)), and this deactivation further increases the reaction time (β estimate −0.235 with 95% confidence interval (−0.365, −0.105)). The loadings of PC3 are shown in Figure C.2. In order to achieve the goal of region selection and make the interpretation feasible, a generally applied approach is to choose a threshold and only regions with magnitude above the threshold are considered to contribute to the PC. However, the threshold is usually chosen arbitrarily. Furthermore, including a thresholding step may lead to deceptive interpretations (see the discussion in Section C of the supplementary materials). Here, we utilize a principled procedure through lasso regularization. Figure 2 shows the sparse approximation under the fused lasso penalty. The estimated model coefficients, as well as the indirect effect are very close to those in the PCA based analysis (Table 1 and Figure C.1 in the supplementary material). From the figure, the whole cerebellum module is regularized to zero. All the visual modules, including lateral, medial and occipital pole visual, yield negative loadings; the positive loadings are mainly from the auditory, default mode network, executive control and frontoparietal modules (see Figure C.3 in the supplementary material). Figure 3 shows the regions with positive and negative loadings in a brain map. The medial frontal and parietal cortex and the auditory cortex are often identified to be deactivated (positive loadings with negative α estimate) when the task involves visual stimuli [67, 68]. The positive loading map (Figure 3a) also includes the medial temporal lobe (MTL) regions, which is in line with the findings in the existing literature [67]. The negative loading map (Figure 3b) consists of the visual cortex and the basal ganglia (caudate nucleus). This classification learning task is a nondeclarative memory procedure. The opposite sign of the loadings in MTL and striatum verifies the competing role of these two memory system regions during learning [67]. Positive correlations between reaction time and activation in visual cortex, fusiform gyrus and thalamus have been discovered, which are presumably involved in processing sensory feedback related to the motor response [69]. Through our mediation analysis, our finding connects the brain pathways that are responsive to the classification learning task and contribute to the longer reaction time.

Figure 2:

The sparse approximation of the loadings of PC3. The x-axis shows the index of the 264 brain regions which are colored by functional modules.

Table 1:

The estimate (Est.) of model parameters α, β, and the indirect effect (IE), as well as the 95% bootstrap confidence interval (CI) of PC3, which yields significant IE.

	PC3
	PCA		SPCA
	Est.	95% CI	Est.	95% CI
α	−0.376	(−0.592, −0.160)	−0.343	(−0.534, −0.151)
β	−0.235	(−0.365, −0.105)	−0.262	(−0.410, −0.115)
IE (αβ)	0.089	(0.020, 0.176)	0.090	(0.025, 0.178)

Figure 3:

Brain regions with (a) positive and (b) negative loadings in sparsified PC3. The module color code is the same as in Figure 2. The size of the node is proportion to the absolute value of the loading.

6. Discussion

In this study, we introduced a sparse principal component analysis (PCA) based approach for performing mediation analysis with high-dimensional mediators. As an extension of the method introduced in Huang and Pan (2016) [27], the proposed approach increases the interpretability of the transformed mediators by employing the sparse PCA method proposed in Zou et al. (2006) [42]. Though our proposed sparse PCA based method introduces a slight estimation bias in model coefficients compared to the PCA based method, the estimate of the mediation effect converges and the findings are more interpretable. Thus, this can be seen as a trade-off between estimation bias and interpretability. In the task-based fMRI application, we consider the fused lasso penalty based on spatial information to enforce local smoothness and constancy. Using sparse loadings, the activation patterns of the brain regions in the PC with significant mediation effect are consistent with those found in the existing literature. The proposed framework is based on the assumption that there are p underlying ordered mediators, where the knowledge of the specific ordering is not necessarily known. When considering brain activation as the mediators of cognitive behaviors, the causal relationship between brain regions is referred to as effective connectivity [54]. Thus, the assumption of p-ordered mediators is consistent with a working model of connectivity; though we acknowledge potential limitations should this prove deficient.

Though our proposed high-dimensional mediation analysis approach is motivated by neuroimaging studies, the fundamental principle can be generalized to other areas, for example genetics studies. Given the spatial information of the brain, we consider a special type of the generalized lasso, namely the fused lasso. Other lasso-type structured regularization can be adopted based on data characteristics, including group lasso [70] and its variations [71].

Highlights.

• We propose an approach that performs mediation analysis on a large number of causally dependent mediators.

• Our proposed method facilitates intuitive interpretations through sparse principal component analysis.

• Software, in the form of R code, together with a sample data set and complete documentation is available on Github at https://github.com/zhaoyi1026/spcma.