A Bayesian regularized mediation analysis with multiple exposures

Abstract

Mediation analysis assesses the effect of study exposures on an outcome both through and around specific mediators. While mediation analysis involving multiple mediators has been addressed in recent literature, the case of multiple exposures has received little attention. With the presence of multiple exposures, we consider regularizations that allow simultaneous effect selection and estimation while stabilizing model fit and accounting for model selection uncertainty. In the framework of linear structural-equation models, we analytically show that a two-stage approach regularizing regression coefficients does not guarantee a unimodal posterior distribution and that a product-of-coefficient approach regularizing direct and indirect effects tends to penalize excessively. We propose a regularized difference-of-coefficient approach that bypasses these limitations. Using the connection between regularizations and Bayesian hierarchical models with Laplace prior, we develop an efficient Markov chain Monte Carlo algorithm for posterior estimation and inference. Through simulations, we show that the proposed approach has better empirical performances compared to some alternatives. The methodology is illustrated using data from two epidemiological studies in human reproduction.

1 |. INTRODUCTION

In biomedical and social sciences research, there is often an interest in investigating the effect of an exposure on an outcome in the presence of a mediator. Mediation analysis aims at decomposing an exposure-outcome relationship into its indirect and direct effects. Panel (A) of Figure 1 provides an illustration of this decomposition, where X, M, and Y are the exposure, mediator, and outcome variables, respectively. A common framework to formulate and implement mediation analysis is the linear structural-equation model such as the Baron and Kenny model¹

$$\begin{gathered} y_i={\mu }_1+{\mathit{x}}_i\alpha +\beta m_i+{\epsilon }_{1i}, \\ {m}_i={\mu }_2+{\mathit{x}}_i\gamma +{\epsilon }_{2i}. \end{gathered}$$ (1)

where x_i, m_i, and y_i are realizations of X, M, and Y; μ₁ and μ₂ are intercepts; α, β, and γ are regression coefficients; and ε_1i and ε_2i are independent error terms. In this framework, it can be shown that the indirect effect of exposure X on outcome Y transmitted through mediator M is δ ≡ βγ, whereas the direct effect is α. This representation of the indirect effect is called the product-of-coefficient approach. Alternatively, the indirect effect can be constructed as δ = α* − α through the difference-of-coefficient approach under a slightly different modeling framework²

$$\begin{gathered} y_i={\mu }_1+{\mathit{x}}_i\alpha +\beta m_i+{\epsilon }_{1i}, \\ {y}_i={\mu }_3+{\mathit{x}}_i{\alpha }^{*}+{\epsilon }_{3i}, \end{gathered}$$ (2)

where μ₃, α*, and ε_3i are intercept, slope, and error term, respectively.

FIGURE 1.

Mediation analysis frameworks with outcome Y and (A) a single exposure X and a single mediator M, (B) multiple exposures X₁,X₂, … ,X_p and a single mediator M, and (C) multiple exposures X₁,X₂, … ,X_p and multiple mediators M₁,M₂, … ,M_q. In all cases, α’s, γ’s, and β’s are regression coefficients, the solid lines represent the direct effects of X’s on Y, and the dashed lines are effects of X’s on Y transmitted through mediators M’s. In panel (C), the box around M’s indicates a block indirect effect is of interest

Recent literature has greatly expanded mediation analysis in various ways. For example, Robins and Greenland,³ Pearl,⁴ Imai et al,⁵ and VanderWeele and Vansteelandt⁶ extended the framework to more generalized situations including nonlinear regressions and noncontinuous outcomes while formalizing assumptions needed for causal interpretations; Sohn and Li⁷ proposed a compositional mediation analysis; Daniels et al⁸ formulated a nonparametric Bayesian approach to estimate natural direct and indirect effects; Preacher and Hayes,⁹ VanderWeele and Vansteelandt,¹⁰ Chén et al,¹¹ Huang and Pan,¹² and Zhao and Luo¹³ considered multiple mediators. In particular, Zhao and Luo¹³ considered regularizations (that is, placing penalty terms on the targets of interest) through a pathway least absolute shrinkage and selection operator,¹⁴ with the assumption that all except a small number of mediators do not transmit the effect of exposure on outcome. This sparse estimation procedure facilitates stable parameter estimations in mediation analysis while allowing variable selection.

We consider the situation where multiple exposures are available, a phenomenon increasingly common in the era of big data and when investigating mixtures, whether they are environmental chemicals or other biomarkers. For example, in the Longitudinal Investigation of Fertility and the Environment (LIFE) study,¹⁵ a mixture of endocrine-disrupting chemicals were assessed in relation to a spectrum of reproductive outcomes, including semen quality and couples’ time-to-pregnancy. Another example is in assessing parental demographic and socioeconomic factors and neonatal outcomes, both through and around maternal lifestyle and immune activity.¹⁶ Similar examples abound in the social sciences, public health, and clinical medicine.

When multiple exposures are available, one can conduct mediation analysis for each individual exposure, eg, in the framework of (1) or (2) with a single mediator. When multiple mediators are present, similar one-exposure-at-a-time procedure can be carried out through aforementioned extensions. Although straightforward to implement, this procedure of separate analysis ignores the interactions and correlations between the multiple exposures and requires multiple tests to assess the effects of interest. Alternatively, one can consider all exposures in one single framework, essentially using a vector of exposures x in place of the univariate x in Equations (1) and (2). However, multicollinearity may result in unstable model fit.

We propose a Bayesian regularized mediation analysis method when multiple exposures exist, regardless of the dimension of mediators. Briefly, regularized mediation analysis aims at simultaneous estimation and selection of both the direct and indirect effects through the use of penalized likelihood function. There are several advantages of using regularizations in mediation analysis. First, regularizations achieve model parsimony, which leads to an enhanced interpretation of the model. Moreover, the use of penalized likelihood can potentially stabilize model fit. Finally, simultaneous effect estimation and selection adequately account for modeling uncertainty, thereby, minimize type I errors. We analytically show that a two-stage approach that directly penalizes the regression coefficients does not guarantee an unimodal posterior distribution. Furthermore, we demonstrate that the product-of-coefficient framework has a serious overpenalization problem when the penalty is imposed on direct and indirect effects. As a result, we propose to regularize in the difference-of-coefficient framework and show that it provides an adequate platform where the posterior distribution is unimodal and no overpenalization occurs. We are not aware of any prior works in mediation analysis that regularizes with respect to the exposures.

The remainder of the paper is organized as follows. Section 2 describes various methods for regularizations in mediation analysis with multiple exposures while delineating the advantages and limitations of each. We also consider a block regularization approach in the case of multiple mediators when the interest is on the total indirect effects. Section 3 provides computational procedures and inference details, Section 4 illustrates simulation setup and presents results, and Section 5 presents the results when applied to the two epidemiological studies. Finally, Section 6 summarizes and discusses future research directions.

2 |. THE METHOD

2.1 |. Notations and assumptions

We first consider the case of p > 1 exposures and a single mediator, respectively denoted by ${\mathit{x}}_i^T=\left(x_{i1},x_{i2},\dots ,x_{ip}\right)$ and $m_i\left({\tilde{\mathit{x}}}^T\right)$, for the ith subject, where i = 1, 2, … , n, ${\mathit{x}}_i^T$ is the transpose of x_i, and $m_i\left({\tilde{\mathit{x}}}^T\right)$ is a potential outcome¹⁷ of the mediator when exposures are at the level of ${\tilde{\mathit{x}}}^T=\left({\tilde{x}}_1,{\tilde{x}}_2,\dots ,{\tilde{x}}_p\right)$. Here, we introduce ${\tilde{\mathit{x}}}^T$ to distinguish from x^T for later usage. To formally define the direct and indirect effects, we further introduce a potential outcome $y_i\left({\mathit{x}}_i^T,m_i\left({\tilde{\mathit{x}}}^T\right)\right)$ for the outcome of interest when exposures and the mediator are ${\mathit{x}}_i^T$ and $m_i\left({\tilde{\mathit{x}}}^T\right)$. Suppose ${\tilde{\mathit{x}}}^T=x^T+(0,\dots ,0,1,0,\dots ,0)=\left(x_1,\dots ,x_{j-1},x_j+1,x_{j+1},\dots ,x_p\right)$, the total effect of the jth exposure is defined as

$$\text{E}\left\{y\left({\tilde{\mathit{x}}}^T,m\left({\tilde{\mathit{x}}}^T\right)\right)\right\}-\text{E}\left\{y\left({\mathit{x}}^T,m\left({\mathit{x}}^T\right)\right)\right\},$$

which measures the expected change of y when the jth exposure increases by one unit, whereas other exposures remain the same. This total effect can be decomposed as the direct effect $\text{E}\left\{y\left({\tilde{\mathit{x}}}^T,m\left({\tilde{\mathit{x}}}^T\right)\right)\right\}-\text{E}\left\{y\left({\mathit{x}}^T,m\left({\tilde{\mathit{x}}}^T\right)\right)\right\}$ and indirect effect $\text{E}\left\{y\left({\mathit{x}}^T,m\left({\tilde{\mathit{x}}}^T\right)\right)\right\}-\text{E}\left\{y\left({\mathit{x}}^T,m\left({\mathit{x}}^T\right)\right)\right\}$. Note that, as in a multiple regression model, these effects are adjusted, given the level of other exposures. We make the following standard assumptions to facilitate estimation of these effects:

(A1) stable unit treatment value assumption (SUTVA);

(A2) no unmeasured confunding;

(A3) sequential ignorability with the mediator;

(A4) no interaction between the exposures and the mediator.

For simplicity, we use y_i and m_i to denote $y_i\left({\mathit{x}}_i^T,m_i\left({\mathit{x}}_i^T\right)\right)$ and $m_i\left({\mathit{x}}_i^T\right)$ in the sequel.

2.2 |. The general Baron-Kenny model

Let y = (y₁, y₂, … ,y_n)^T denote a continuous outcome vector of n subjects, and X = (x₁,x₂, … ,x_n)^T the corresponding n × p exposure matrix. The mediation analysis framework of Baron-Kenny (B-K) model¹ has these structural-equation specifications

$$\begin{gathered} y_i={\mu }_1+{\mathit{x}}_i^T\mathit{\alpha }+\beta m_i+{\epsilon }_{1i}, \\ {m}_i={\mu }_2+{\mathit{x}}_i^T\mathit{\gamma }+{\epsilon }_{2i}, \end{gathered}$$ (3)

where μ₁ and μ₂ are intercepts, α = (α₁,α₂, … ,α_p)^T, β, and γ = (γ₁,γ₂, … ,γ_p)^T are regression coefficients, and ${\epsilon }_{1i}\stackrel{iid}{~}N\left(0,{\sigma }_1^2\right)$ and ${\epsilon }_{2i}\stackrel{iid}{~}N\left(0,{\sigma }_2^2\right)$ are independent error terms. In this framework, α is the vector of direct effects, and the product terms ${\delta }_j\equiv \beta {\gamma }_j$, j = 1, 2, … , p represent the indirect effects mediated through M (see Section S1 in the Supplementary Materials for derivation). The likelihood function of (3) is given by

$$\prod _{i=1}^n\frac{1}{\sqrt{2\pi {\sigma }_1^2}}\exp \left[-\frac{{\left(y_i-{\mu }_1-{\mathit{x}}_i^T\mathit{\alpha }-\beta m_i\right)}^2}{2{\sigma }_1^2}\right]\frac{1}{\sqrt{2\pi {\sigma }_2^2}}\exp \left[-\frac{{\left(m_i-{\mu }_2-{\mathit{x}}_i^T\mathit{\gamma }\right)}^2}{2{\sigma }_2^2}\right].$$

Given priors on the model parameters, we can obtain point estimates, eg, posterior means, and the variability measures, eg, 95% highest posterior density (HPD) intervals, of both the direct effect α and indirect effect $\mathit{\delta }={\left({\delta }_1,{\delta }_2,\dots ,{\delta }_p\right)}^T$. Panel (B) of Figure 1 presents an illustration of the case with multiple exposures and a single mediator.

2.3 |. A regularized two-stage approach

When the number of exposures p is high, it is desirable to conduct effect selection simultaneously with estimation. This is especially necessary when it is believed that most but few of the direct and indirect effects are zeros, and hence should be excluded from the model. To achieve this goal, the Bayesian Lasso approach¹⁸ can be applied by proposing conditional Laplace priors on α and γ

$${\alpha }_j\mid {\sigma }_1^2\stackrel{iid}{~}\text{Laplace}\left(0,\frac{\sqrt{{\sigma }_1^2}}{{\lambda }_1}\right)\text{and}{\gamma }_j\mid {\sigma }_2^2\stackrel{iid}{~}\text{Laplace}\left(0,\frac{\sqrt{{\sigma }_2^2}}{{\lambda }_2}\right),\text{for}j=1,2,\dots ,p,$$ (4)

with corresponding densities

$$\pi \left(\mathit{\alpha }\mid {\sigma }_1^2\right)={\left[\frac{{\lambda }_1}{2\sqrt{{\sigma }_1^2}}\right]}^p\exp \left(-\frac{{\lambda }_1{\sum }_{j=1}^p\left|{\alpha }_j\right|}{\sqrt{{\sigma }_1^2}}\right)$$

$$\pi \left(\mathit{\gamma }\mid {\sigma }_2^2\right)={\left[\frac{{\lambda }_2}{2\sqrt{{\sigma }_2^2}}\right]}^p\exp \left(-\frac{{\lambda }_2{\sum }_{j=1}^p\left|{\gamma }_j\right|}{\sqrt{{\sigma }_2^2}}\right),$$

respectively, where the λ’s are shrinkage parameters. This regularized two-stage (regTS) approach in mediation analysis is straightforward in specification and estimation as it places L₁ penalties on regression coefficients directly. However, it might be more natural to work with the parameters of interest, ie, the direct and indirect effects. Moreover, under regTS, the effects of interest do not necessarily have a unimodal posterior distribution as shown in the following.

Result 1: In the product-of-coefficient framework of mediation analysis in (3), suppose prior specification of (4) for α and γ, a normal prior for β given ${\sigma }_1^2$, and noninformative priors for ${\sigma }_1^2$ and ${\sigma }_2^2$ are used. The posterior distribution $\pi \left(\mathit{\alpha },\beta ,\mathit{\gamma },{\sigma }_1^2,{\sigma }_2^2\mid D\right)$ is unimodal, but $\pi \left(\mathit{\alpha },\mathit{\delta },{\sigma }_1^2,{\sigma }_2^2\mid D\right)$ may not, where D denotes the observed data (y, X)

Proof. The unimodality of $\pi \left(\mathit{\alpha },\beta ,\mathit{\gamma },{\sigma }_1^2,{\sigma }_2^2\mid D\right)$ can be proved following a similar argument as in the work of Park and Casella.¹⁸ To prove the nonunimodality of $\pi \left(\mathit{\alpha },\mathit{\delta },{\sigma }_1^2,{\sigma }_2^2\mid D\right)$, consider a hypothetical example in Tables 1, which shows the joint distribution of γ and β. Clearly, (γ = 2, β = 2) is the mode of the joint distribution of γ and β because Pr(γ = 2, β = 2) = 0.36. However, when focusing on δ, we see that Pr(δ = 4) = Pr(δ = 2) = 0.36. As a result, the distribution of δ does not have unimodality. □

TABLE 1.

Joint and marginal distributions of γ and β

		β values			Total
		1.0	2.0	5.0
	1.0	0.09	0.18	0.03	0.30
γ values	2.0	0.18	0.36	0.06	0.60
	3.0	0.03	0.06	0.01	0.10
Total		0.30	0.60	0.10

2.4 |. A regularized product-of-coefficient approach

To obtain unimodality of the indirect effects, one can place a conditional Laplace prior on δ_js in the product-of-coefficient framework. This way of penalizing both the direct and indirect effects can be desirable as it works directly with the parameters of primary interest in mediation analysis. This approach can be viewed as a Bayesian version of the work of Zhao and Luo¹³ although the latter focused on the case of multiple mediators and a single exposure. However, the use of product-of-coefficient framework can cause overpenalization in β, in the sense that the posterior distribution of β accumulates an increasing amount of mass around zero from the prior when p > 1. To illustrate this phenomenon, we can consider a case where a conditional normal prior N(0, σ²I_p) is assigned to δ, where 0 is a vector of zero with an appropriate length and I_p is the usual identity matrix of size p. This prior can be treated as a product of p conditional normal distributions, $N\left(0,{\sigma }^2/{\gamma }_j^2\right),j=1,2,\dots ,p$, for β, and consequently gives rise to a precision of ${\mathit{\gamma }}^T\mathit{\gamma }/{\sigma }^2$ for the prior of β that centers at 0:$\beta \mid \mathit{\gamma }~N\left(0,{\sigma }^2/{\mathit{\gamma }}^T\mathit{\gamma }\right)$. As p or |γ_j| increases, this precision increases, forcing β to be near zero regardless of the information from data.

2.5 |. A regularized difference-of-coefficient approach

To circumvent this difficulty, we use an alternative representation of mediation analysis proposed in the work of Mackinnon and Dwyer² that utilizes the following two equations:

$$\begin{gathered} y_i={\mu }_1+{\mathit{x}}_i^T\mathit{\alpha }+\beta m_i+{\epsilon }_{1i}, \\ {y}_i={\mu }_3+{\mathit{x}}_i^T{\mathit{\alpha }}^{*}+{\epsilon }_{3i}, \end{gathered}$$ (5)

where μ₃ is the intercept, α* are regression coefficients, and ${\epsilon }_{3i}\stackrel{iid}{~}N\left(0,{\sigma }_3^2\right)$. It can be shown that δ = α* − α. Under this framework, we can consider L₁ penalty on the direct (α_j) and indirect effects (δ_j) and propose the joint prior distribution as follows:

$$\begin{gathered} {\alpha }_j\mid {\sigma }_1^2\stackrel{iid}{~}\mathrm{Laplace}\left(0,\frac{\sqrt{{\sigma }_1^2}}{{\lambda }_1}\right),j=1,2,\dots ,p, \\ {\delta }_j\mid {\sigma }_1^2,\eta \stackrel{iid}{~}\text{Laplace}\left(0,\frac{\sqrt{\eta {\sigma }_1^2}}{{\lambda }_2}\right),j=1,2,\dots ,p,, \\ {\sigma }_1^2,\eta ~\frac{1}{{\sigma }_1^2}I\{\eta \ge 1\}, \\ {\lambda }_1^2~\mathrm{Gamma}\left({\theta }_1,v_1\right), \\ {\lambda }_2^2~\mathrm{Gamma}\left({\theta }_2,v_2\right), \\ {\mu }_1~N\left(0,{\sigma }_0^2\right), \\ {\mu }_3~N\left(0,{\sigma }_0^2\right), \\ \beta \mid {\sigma }_1^2~N\left(0,{\sigma }_0^2{\sigma }_1^2\right), \end{gathered}$$ (6)

where ${\sigma }_0^2$, θ₁, θ₂, v₁, v₂ are prespecified hyperparameters, λ₁ and λ₂ are shrinkage parameters, I {η ≥ 1} is the usual indicator function with value 1 when η ≥ 1 and 0 otherwise, and $\pi \left({\lambda }_s^2\right)=v_s^{{\theta }_s}/\Gamma \left({\theta }_s\right){\left({\lambda }_s^2\right)}^{{\theta }_s-1}\exp \left(-v_s{\lambda }_s^2\right)$ for s = 1, 2. Here, we introduce $\eta \equiv {\sigma }_3^2/{\sigma }_1^2>1$ to reflect the fact that there is more noise in the second equation of (5) than the first. We use regDOC to denote this regularization through the framework of difference-of-coefficient. It can be shown that, under regDOC, the posterior distribution of the direct and indirect effects is unimodal.

Result 2: In the difference-of-coefficient framework of mediation analysis in (5), suppose prior specification of (6) is used. As such, the posterior distributions $\pi \left(\mathit{\alpha },\beta ,\mathit{\delta },{\sigma }_1^2,\eta \mid D\right)$ and $\pi \left(\mathit{\alpha },\beta ,\mathit{\gamma },{\sigma }_1^2,\eta \mid D\right)$ are both unimodal.

Proof. To demonstrate unimodality of $\pi \left(\mathit{\alpha },\beta ,\mathit{\delta },{\sigma }_1^2,\eta \mid D\right)$, we let ${\mathit{\alpha }}^{‡}=\mathit{\alpha }/\sqrt{{\sigma }_1^2}$, ${\mathit{\beta }}^{‡}=\mathit{\beta }/\sqrt{{\sigma }_1^2}$, ${\mathit{\delta }}^{‡}=\mathit{\delta }/\sqrt{{\sigma }_1^2}$, $\rho =1/\sqrt{{\sigma }_1^2}$ and $\varphi =1/\sqrt{\eta }$. After dropping all unrelated terms, the posterior distribution of $\left(\alpha ,\beta ,\delta ,{\sigma }_1^2,\eta \right)$ is

$$\begin{gathered} {\rho }^{2n+2p+3}{\varphi }^{n+p}\exp [-\frac{1}{2}{\left(\rho {\tilde{\mathit{y}}}_1-\mathit{X}{\mathit{\alpha }}^{‡}-{\mathit{\beta }}^{‡}\mathbf{m}\right)}^T\left(\rho {\tilde{\mathit{y}}}_1-\mathit{X}{\mathit{\alpha }}^{‡}-{\mathit{\beta }}^{‡}\mathbf{m}\right) \\ -\frac{{\varphi }^2}{2}{\left(\rho {\tilde{\mathit{y}}}_2-\mathit{X}\left({\mathit{\alpha }}^{‡}+{\mathit{\delta }}^{‡}\right)\right)}^T\left(\rho {\tilde{\mathit{y}}}_2-\mathit{X}\left({\mathit{\alpha }}^{‡}+{\mathit{\delta }}^{‡}\right)\right) \\ -{\lambda }_1\sum _{j=1}^p\left|{\alpha }_j^{‡}\right|-\frac{1}{2{\sigma }_0^2}{\left({\beta }^{‡}\right)}^2-{\lambda }_2{\varphi }^2\sum _{j=1}^p\left|{\delta }_j^{‡}\right|], \end{gathered}$$ (7)

where ${\tilde{\mathit{y}}}_1=\mathit{y}-{\mu }_1{\mathit{J}}_n$, ${\tilde{\mathit{y}}}_2=\mathit{y}-{\mu }_3{\mathit{J}}_n$, and J_n is a n × 1 vector with all elements to be 1. The log posterior is

$$\begin{gathered} (2n+2p+3)\ln \rho +(n+p)\ln \varphi -{\lambda }_1\sum _{j=1}^p\left|{\alpha }_j^{‡}\right|-\frac{1}{2{\sigma }_0^2}{\left({\beta }^{‡}\right)}^2 \\ -\frac{1}{2}{\left(\rho {\tilde{\mathit{y}}}_1-\mathit{X}{\mathit{\alpha }}^{‡}-{\beta }^{‡}\mathit{m}\right)}^T\left(\rho {\tilde{\mathit{y}}}_1-\mathit{X}{\mathit{\alpha }}^{‡}-{\beta }^{‡}\mathit{m}\right) \\ -\frac{{\varphi }^2}{2}{\left(\rho {\tilde{\mathit{y}}}_2-\mathit{X}\left({\mathit{\alpha }}^{‡}+{\mathit{\delta }}^{‡}\right)\right)}^T\left(\rho {\tilde{\mathit{y}}}_2-\mathit{X}\left({\mathit{\alpha }}^{‡}+{\mathit{\delta }}^{‡}\right)\right) \\ -{\lambda }_2{\varphi }^2\sum _{j=1}^p\left|{\delta }_j^{‡}\right|, \end{gathered}$$ (8)

where the four terms in the first line are clearly concave separately in ρ, ϕ, α^‡, and β^‡, and the last three terms are concave in (ρ, α^‡, β^‡), (ρ, ϕ, α^‡, δ^‡), and (ϕ, δ^‡), respectively. Thus, (8) is concave and (7) is unimodal. To show unimodality of $\pi \left(\mathit{\alpha },\beta ,\mathit{\gamma },{\sigma }_1^2,\eta \mid D\right)$, simply replace δ^‡ = β^‡γ in (7) and (8). □

Remark 1. In regPOC, the indirect effects δ is defined as the product of β and γ so that penalization on δ produces overpenalization on β as explained in Section 2.4. On the other hand, the regDOC approach reparameterizes δ as the difference of α* and α, which does not involve β. As a result, β does not suffer from overpenalization. This phenomenon is also evident in the full conditional distribution of β. In regDOC,

$$\beta \mid .~N\left({\left({\mathit{m}}^T\mathit{m}+\frac{1}{{\sigma }_0^2}\right)}^{-1}{\mathit{m}}^T{y}_{\text{A}},{\left({\mathit{m}}^T\mathit{m}+\frac{1}{{\sigma }_0^2}\right)}^{-1}{\sigma }_1^2\right),$$

and is free from ${\tau }_{{\text{D}}_j}^{-1}$, a parameter related to regularization. In contrast, in regPOC,

$$\beta \mid .~N\left({\left({\mathit{m}}^T\mathit{m}+\frac{{\mathit{\gamma }}^T{D}_{{\tau }_{\text{D}}}^{-1}\mathit{\gamma }}{\eta }\right)}^{-1}{\mathit{m}}^T{\mathit{y}}_{\text{A}},{\left({\mathit{m}}^T\mathit{m}+\frac{{\mathit{\gamma }}^T{D}_{{\tau }_{\text{D}}}^{-1}\mathit{\gamma }}{\eta }\right)}^{-1}{\sigma }_1^2\right),$$

and is clearly affected by ${\tau }_{{\text{D}}_j}^{-1}$(in $D_{{\tau }_{\text{D}}}^{-1}$)

Remark 2. The L₁-penalized least squares estimates in the difference-of-coefficient model are given by

$${\mathrm{argmin}}_{\alpha ,\delta }\left\{\frac{\sum {\left(y_i-{\mu }_1-{\mathit{x}}_i^T\mathit{\alpha }-\beta m_i\right)}^2}{{\sigma }_1^2}+\frac{\sum {\left(y_i-{\mu }_3-{\mathit{x}}_i^T(\mathit{\alpha }+\mathit{\delta })\right)}^2}{\eta {\sigma }_1^2}+{\lambda }_1\sum \left|{\alpha }_j\right|+{\lambda }_2\sum \left|{\delta }_j\right|\right\}.$$

These estimates ($\widehat{\alpha }$ and $\widehat{\delta }$) clearly coincide with the posterior mode in (7).

2.6 |. A block regDOC approach for multiple mediators

When there are p exposures and multiple, say q, mediators, the B-K framework can be written as follows:

$$\begin{array}{l}{y}_i={\mu }_1+{\mathit{x}}_i^T\mathit{\alpha }+{\mathit{m}}_i^T\mathit{\beta }+{\epsilon }_{1i},\\ m_{i1}={\mu }_{2_1}+{\mathit{x}}_i^T{\mathit{\gamma }}_1+{\epsilon }_{2_{1}i},\\ \hfill ⋮\hfill \\ m_{iq}={\mu }_{2_q}+{\mathit{x}}_i^T{\mathit{\gamma }}_q+{\epsilon }_{2_{q}i},\end{array}$$

where m_i = (m_i1, m_i2, … , m_iq)^T is the vector of mediators for ith subject; μ₁, α, and β are regression coefficients in the outcome model; ${\mu }_{2_k}$ and ${\mathit{\gamma }}_k={\left({\gamma }_{1k},{\gamma }_{2k},\dots ,{\gamma }_{pk}\right)}^T$ are regression coefficients in the mediation model; and ε_1i, ${\epsilon }_{2_{k}i}$are error terms, k = 1, 2, … , q. The direct effects are α_j, j = 1, 2, … , p, and the indirect effects are β_kγ_jk, j = 1, 2, … , p, k = 1, 2, … , q. Although each individual indirect effect is usually the parameter of interest, the jth total indirect effect, defined as the effect of the jth exposure on Y through the block of all mediators, ie, M₁, M₂, … , M_q, can be the focus. Driven by this consideration, we extend (5) to the following difference-of-coefficient specification:

$$\begin{gathered} y_i={\mu }_1+{\mathit{x}}_i^T\mathit{\alpha }+{\mathit{m}}_i^T\mathit{\beta }+{\epsilon }_{1i}, \\ {y}_i={\mu }_3+{\mathit{x}}_i^T{\mathit{\alpha }}^{*}+{\epsilon }_{3i}, \end{gathered}$$ (9)

where α* = α + δ, and δ = (δ₁, δ₂, … ,δ_p)^T denotes the vector of block indirect effect of each exposure through all mediators, with ${\delta }_j={\sum }_{k=1}^q{\beta }_k{\gamma }_{jk}$. Hence, to put L₁ penalty on each direct and block indirect effect, the same prior setting as (6) can be used except now${\beta }_k\mid {\sigma }_1^2\stackrel{idd}{~}N\left(0,{\sigma }_0^2{\sigma }_1^2\right)$, for k = 1, 2, … , q. With the tractability of all full conditional distributions, this block regularization approach only requires a straightforward Gibbs sampling. Furthermore, the unimodality property of the posterior distribution of direct and block indirect effects still holds. Panel (C) of Figure 1 illustrates this framework with multiple exposures and mediators.

3 |. POSTERIOR COMPUTATIONS AND INFERENCE

In this section, we describe the computational algorithm of the regDOC approach. Details on B-K and regTS are provided in the Supplementary Materials. Based on the difference-of-coefficient specification in (9), the likelihood function is written as

$$\prod _{i=1}^n\frac{1}{\sqrt{2\pi {\sigma }_1^2}}\exp \left[-\frac{{\left(y_i-{\mu }_1-{\mathit{x}}_i^T\mathit{\alpha }-{\mathit{m}}_i^T\mathit{\beta }\right)}^2}{2{\sigma }_1^2}\right]\frac{1}{\sqrt{2\pi \eta {\sigma }_1^2}}\exp \left[-\frac{{\left(y_i-{\mu }_3-{\mathit{x}}_i^T(\mathit{\alpha }+\mathit{\delta })\right)}^2}{2\eta {\sigma }_1^2}\right].$$ (10)

Now, the posterior distribution is proportional to the multiplication of (10) and (6). To facilitate computation in the same spirit as in the work of Park and Casella,¹⁸ the conditional Laplace priors in (6) are expanded to scale mixtures of normal distributions¹⁹ so that

$$\pi \left({\alpha }_j\mid {\sigma }_1^2\right)\pi \left({\delta }_j\mid {\sigma }_1^2,\eta \right)=\frac{1}{\sqrt{2\pi {\sigma }_1^2{\tau }_{{\text{A}}_j}}}\exp \left[-\frac{{\alpha }_j^2}{2{\sigma }_1^2{\tau }_{{\text{A}}_j}}\right]\frac{{\lambda }_1^2}{2}\exp \left[-\frac{{\lambda }_1^2{\tau }_{{\text{A}}_j}}{2}\right]\frac{1}{\sqrt{2\pi \eta {\sigma }_1^2{\tau }_{{\text{D}}_j}}}\exp \left[-\frac{{\delta }_j^2}{2\eta {\sigma }_1^2{\tau }_{{\text{D}}_j}}\right]\frac{{\lambda }_2^2}{2}\exp \left[-\frac{{\lambda }_2^2{\tau }_{{\text{D}}_j}}{2}\right],$$

where $\left({\tau }_{{\text{A}}_1},{\tau }_{{\text{A}}_2},\dots ,{\tau }_{{\text{A}}_p}\right)$ and $\left({\tau }_{{\text{D}}_1},{\tau }_{{\text{D}}_2},\dots ,{\tau }_{{\text{D}}_p}\right)$ are nonnegative latent variables. Within this updated hierarchical structure, the priors for α and δ become conjugate, and consequently, the full conditional distributions all have known forms for ease of sampling. In particular, if we let m_(k) = (m_1k, m_2k, … , m_nk)^T, the full conditional distributions are

$${\mu }_1\mid .~N\left({\left({\mathit{J}}_n^T{\mathit{J}}_n+\frac{{\sigma }_1^2}{{\sigma }_0^2}\right)}^{-1}{\mathit{J}}_n^T{\mathit{y}}_{\text{AB}},{\left({\mathit{J}}_n^T{\mathit{J}}_n+\frac{{\sigma }_1^2}{{\sigma }_0^2}\right)}^{-1}{\sigma }_1^2\right),$$

$${\mu }_3\mid \cdot ~N\left({\left({\mathit{J}}_n^T{\mathit{J}}_n+\frac{\eta {\sigma }_1^2}{{\sigma }_0^2}\right)}^{-1}{\mathit{J}}_n^T{\tilde{\mathit{y}}}_{\text{AD}},{\left({\mathit{J}}_n^T{\mathit{J}}_n+\frac{\eta {\sigma }_1^2}{{\sigma }_0^2}\right)}^{-1}\eta {\sigma }_1^2\right),$$

$$\alpha \mid .~N\left({\left({\mathit{X}}^T\mathit{X}+\frac{{\mathit{X}}^T\mathit{X}}{\eta }+D_{{\tau }_A}^{-1}\right)}^{-1}{\mathit{X}}^T\left({\mathit{y}}_{\text{B}}+\frac{{\tilde{\mathit{y}}}_{\text{D}}}{\eta }\right),{\left({\mathit{X}}^T\mathit{X}+\frac{{\mathit{X}}^T\mathit{X}}{\eta }+D_{{\tau }_{\text{A}}}^{-1}\right)}^{-1}{\sigma }_1^2\right),$$

$${\beta }_k\mid \cdot ~N\left({\left({\mathit{m}}_{(k)}^T{\mathit{m}}_{(k)}+\frac{1}{{\sigma }_0^2}\right)}^{-1}{\mathit{m}}_{(k)}^T{\mathit{y}}_{{\text{A}}_k},{\left({\mathit{m}}_{(k)}^T{\mathit{m}}_{(k)}+\frac{1}{{\sigma }_0^2}\right)}^{-1}{\sigma }_1^2\right),$$

$$\delta \mid .~N\left({\left({\mathit{X}}^T\mathit{X}+D_{{\tau }_{\text{D}}}^{-1}\right)}^{-1}{\mathit{X}}^T{\tilde{\mathit{y}}}_{\text{A}},{\left({\mathit{X}}^T\mathit{X}+D_{{\tau }_{\text{D}}}^{-1}\right)}^{-1}\eta {\sigma }_1^2\right),$$

$$v_{\mathit{J}}={\tau }_{{\text{A}}_j}^{-1}\mid .~\mathrm{InvGaussian}\left(\frac{{\lambda }_1{\sigma }_1}{\left|{\alpha }_{\mathit{J}}\right|},{\lambda }_1^2\right),$$

$$u_{\mathit{J}}={\tau }_{{\text{D}}_j}^{-1}\mid .~\mathrm{InvGaussian}\left(\frac{{\lambda }_2\sqrt{\eta }{\sigma }_1}{\left|{\delta }_{\mathit{J}}\right|},{\lambda }_2^2\right),$$

$${\lambda }_1^2\mid .~\mathrm{Gamma}\left({\theta }_1+p,v_1+\frac{{\sum }_{j=1}^p{\tau }_{{\text{A}}_j}}{2}\right),$$

$${\lambda }_2^2\mid .~\mathrm{Gamma}\left({\theta }_2+p,v_2+\frac{{\sum }_{j=1}^p{\tau }_{{\text{D}}_j}}{2}\right),$$

$${\sigma }_1^2\mid .~\mathrm{InvGamma}\left(\frac{2n+2p+q}{2},\frac{1}{2}\left[{\mathit{e}}_1^T{\mathit{e}}_1+\frac{{\mathit{e}}_3^T{\mathit{e}}_3}{\eta }+{\mathit{\alpha }}^T{D}_{{\tau }_{\text{A}}}^{-1}\mathit{\alpha }+\frac{{\mathit{\delta }}^T{D}_{{\tau }_{\text{D}}}^{-1}\mathit{\delta }}{\eta }+\frac{{\sum }_{k=1}^q{\beta }_k^2}{{\sigma }_0^2}\right]\right),$$

$$\eta \mid .~\mathrm{InvGamma}\left(\frac{n+p}{2}-1,\frac{1}{2}\left[\frac{{\mathit{e}}_3^T{\mathit{e}}_3}{{\sigma }_1^2}+\frac{{\mathit{\delta }}^T{D}_{{\tau }_{\text{D}}}^{-1}\mathit{\delta }}{{\sigma }_1^2}\right]\right)I\{\eta \ge 1\},$$

where ${\mathit{y}}_{\text{AB}}=\mathit{y}-\mathit{X}\mathit{\alpha }-{\sum }_{k=1}^q{\beta }_k{\mathit{m}}_{(k)}$, ${\tilde{\mathit{y}}}_{\text{AD}}=\mathit{y}-\mathit{X}(\mathit{\alpha }+\mathit{\delta })$, ${\mathit{y}}_{\text{B}}=\mathit{y}-{\mu }_1{\mathit{J}}_n-{\sum }_{k=1}^q{\beta }_k{\mathit{m}}_{(k)}$, ${\tilde{\mathit{y}}}_{\text{D}}=\mathit{y}-{\mu }_3{\mathit{J}}_n-\mathit{X}\mathit{\delta }$, ${\mathit{y}}_{{\text{A}}_k}=\mathit{y}-{\mu }_1{\mathit{J}}_n-\mathit{X}\mathit{\alpha }-{\sum }_{\forall k^{\prime }\ne k}{\beta }_{k^{\prime }}{\mathit{m}}_{\left(k^{\prime }\right)}$, ${\tilde{\mathit{y}}}_{\text{A}}=\mathit{y}-{\mu }_3{\mathit{J}}_n-\mathit{X}\mathit{\alpha }$, $D_{{\tau }_A}^{-1}$ is a diagonal matrix of $\left({\tau }_{{\text{A}}_1}^{-1},{\tau }_{{\text{A}}_2}^{-1},\dots ,{\tau }_{{\text{A}}_p}^{-1}\right)$, $D_{{\tau }_D}^{-1}$ is a diagonal matrix of $\left({\tau }_{{\text{D}}_1}^{-1},{\tau }_{{\text{D}}_2}^{-1},\dots ,{\tau }_{{\text{D}}_p}^{-1}\right)$, ${\mathit{e}}_1=\mathit{y}-{\mu }_1{\mathit{J}}_n-\mathit{X}\mathit{\alpha }-{\sum }_{k=1}^q{\beta }_k{\mathit{m}}_{(k)}$, and e₃ = y − μ₃J_n − X(α + δ). Here, the density of an InvGaussian(θ₁, θ₂) random variable v is given by

$$\pi (v)\propto v^{-3/2}\exp \left(-\frac{{\left({\theta }_2\right)}^2{\left(v-{\theta }_1\right)}^2}{2{\left({\theta }_1\right)}^{2}v}\right),$$

and that of an InvGamma(θ₁, θ₂) random variable σ² by

$$\pi \left({\sigma }^2\right)\propto {\left({\sigma }^2\right)}^{-{\theta }_1-1}\exp \left(-\frac{{\theta }_2}{{\sigma }^2}\right).$$

4 |. SIMULATION

4.1 |. Data generation

In the simulated data sets, we consider 30 exposures and a single mediator and use a pairwise correlation of ${0.5}^{\left|k-k^{\prime }\right|}$ between x_(k) and ${\mathit{x}}_{\left(k^{\prime }\right)}$, for k, k^′ = 1, 2, … , 30. We choose 30 exposures to roughly match the number of PCB congeners in the LIFE study data analysis in Section 5.1. With half of α and γ zeros while the other half nonzeros, we use (3) to generate data following eight scenarios, ie, (i) β = −2, ${\sigma }_1^2=4$,${\sigma }_2^2=4$; (ii) β = −2, ${\sigma }_1^2=4$,${\sigma }_2^2=9$; (iii) β = −2, ${\sigma }_1^2=9$,${\sigma }_2^2=4$; (iv) β = −2, ${\sigma }_1^2=9$,${\sigma }_2^2=9$; (v) β = 0, ${\sigma }_1^2=4$,${\sigma }_2^2=4$; (vi) β = 0, ${\sigma }_1^2=4$,${\sigma }_2^2=9$; (vii) β = 0, ${\sigma }_1^2=9$,${\sigma }_2^2=4$; and (viii) β = 0, ${\sigma }_1^2=9$,${\sigma }_2^2=9$. With the choices of β = −2, 0, we can assess the performance of the proposed regDOC approach in a case with half-and-half zero and nonzero indirect effects or in an extreme one that has no nonzero indirect effects. We vary ${\sigma }_1^2$ and ${\sigma }_2^2$ to assess the robustness of regDOC when variation comes from different sources of Y and m. In all scenarios, 500 replicates of data, each with sample size n = 100 or 200, are generated.

4.2 |. Model and computational specifications

For each generated data, we apply three models, ie, (a) the B-K model that fits to an outcome or mediator regression separately, without any penalization; (b) the regTS model that penalizes on α and γ using Laplace priors; and (c) the regDOC approach that puts the penalization on the direct and indirect effects within the difference-of-coefficient framework. In (a), we use vague priors for the parameters ${\mu }_1~N\left(0,{\sigma }_0^2\right),\mathit{\alpha }\mid {\sigma }_1^2~N\left(\mathbf{0},{\sigma }_1^2{\sigma }_0^2{I}_p\right)$, $\beta \mid {\sigma }_1^2~N\left(0,{\sigma }_1^2{\sigma }_0^2\right)$, ${\mu }_2~N\left(0,{\sigma }_0^2\right)$, $\mathit{\gamma }\mid {\sigma }_2^2~N\left(\mathbf{0},{\sigma }_2^2{\sigma }_0^2{I}_p\right),\pi \left({\sigma }_1^2\right)=1/{\sigma }_1^2$, and $\pi \left({\sigma }_2^2\right)=1/{\sigma }_2^2$, where ${\sigma }_0^2=100$. In (b), we follow the priors in (a) except ${\alpha }_j\mid {\sigma }_1^2\stackrel{iid}{~}\mathrm{Laplace}\left(0,\sqrt{{\sigma }_1^2}/{\lambda }_1\right)$ and ${\gamma }_j\mid {\sigma }_2^2\stackrel{iid}{~}\mathrm{Laplace}\left(0,\sqrt{{\sigma }_2^2}/{\lambda }_2\right),j=1,2,\dots ,p$, and additionally, ${\lambda }_1^2~\mathrm{Gamma}\left({\theta }_1,v_1\right)$ and ${\lambda }_2^2~\mathrm{Gamma}\left({\theta }_2,v_2\right)$. Here, the hyperparameters for θ₁, θ₂, v₁, and v₂ are specified as 1, 1, 0.1, and 0.1, respectively. In (c), we use the prior setting in (6) and choose the same hyperparameters as (b). In each analysis, we ran a single MCMC chain, 5000 burn-ins, and a total of 10000 iterations.

4.3 |. Results

Figures S1 to S4 in the Supplementary Materials summarize the simulation results of the three approaches in eight scenarios. In each graph, we plotted the mean square errors (MSEs) of estimated effects (either 30 direct or indirect effects) from the proposed regDOC approach against those from either B-K or regTS method. The 45° degree line is also drawn in each plot. Points above the 45° degree line are evidence that the regDOC approach produces estimates with smaller MSE indicative of superior performance relative to alternative methods. We see that, in the first four scenarios, where half of direct or indirect effects are zeros and half are nonzero, regDOC produces overall better results than the B-K and regTS models when n = 100 or 200. For the last four scenarios, where all indirect effects are zeros because of β = 0, regDOC has superior results in both direct and indirect effects. The results with larger sample sizes (n = 500, 1000) show similar patterns and are also provided in Figures S5 to S8 in the Supplementary Materials. However, the advantage of regDOC becomes smaller as sample size increases, possibly due to the weakening impact of the prior distribution.

4.4 |. The case of block regularization

We also consider two scenarios when q > 1 and demonstrate the usefulness of block regularization. In the first scenario, we extend scenario (i) in Section 4.1 to have four mediators and set β = (0, 2, 4, 2)^T. To generate four mediators, we let $\mathit{M}={\mathit{J}}_n{\mathit{\mu }}_2^T+\mathit{X}\mathbf{\Gamma }+{\mathit{E}}_2$, where Γ = (γ₁, γ₂, γ₃, γ₄) are chosen to achieve half zero total indirect effects, and ${\mathit{E}}_2=({\epsilon }_{2_1},{\epsilon }_{2_2},{\epsilon }_{2_3},{\epsilon }_{2_4})$ with ${\epsilon }_{2_k}\stackrel{iid}{~}N\left(0,4I_n\right)$, for k = 1, 2, 3, 4. We use similar model and computational specifications as in Section 4.2. Figures S9 and S10 in the Supplementary Materials separately present MSEs of 30 direct and block indirect effect estimates. We can see that the regDOC approach performs better than the two alternatives.

In the second scenario, we choose 14 exposures and 5 mediators to match the number of exposures and mediators in the CPP study and consider sample sizes of n = 1000 and 1500. Under the same correlation structure as in Section 4.1, the results of 14 direct and block indirect effects based on 500 data sets are reported in Figures S11 and S12 in the Supplementary Materials. We observe that regDOC has better overall performance in terms of MSE, consistent with our previous findings.

5 |. APPLICATIONS

5.1 |. The LIFE study

The LIFE study comprises 501 couples who discontinued contraception for purposes of becoming pregnant and were followed daily for up to a year of trying.¹⁵ Designed to explore the relationship between exposures to environmental chemicals and fecundity, the LIFE study collected a variety of biological specimens from both partners of the couples. It also measured the number of menstrual cycles required for pregnancy, as determined by a positive human chorionic gonadotropin (hCG) pregnancy test. In this manuscript, we investigate the relation between a male partner’s serum concentrations of 36 polychlorinated biphenyl (PCB) congeners and time to pregnancy (ie, number of cycles before pregnancy) and whether this relation is mediated by his total sperm count, accounting for his age in years. Given that the technical development for regularized mediation analysis with a survival outcome is beyond the scope of this manuscript, we use a subset of the sample (n = 283 (56.49%)) who had an observed pregnancy in the LIFE study. Basic descriptive statistics for the exposures, mediator, and outcome are presented in Table S1 in the Supplementary Materials.

We apply the proposed approach to the LIFE study’s data using similar prior distributions and hyperparameters, as in the simulation. More specifically, we use θ₁ = θ₂ = 1, v₁ = v₂ = 0.1, and ${\sigma }_0^2=100$. For comparison, we also fit the B-K and regTS models to this data set. Using an MCMC sample of 15 000 iterations with a 5000 burn-ins, we present the posterior means and 95% HPD intervals in Tables 2 and 3 for direct and indirect effects, respectively. All three approaches produce similar direct and indirect effects estimates. Overall, all direct (except PCB congener #138 in B-K model) and indirect effects are not significantly different from zero, suggesting that male partners’ serum PCB concentrations are likely not related to couples’ time to pregnancy for those who achieved pregnancy either directly or through total sperm count.

TABLE 2.

Posterior means and 95% highest posterior density (HPD) interval estimates of direct effects of PCB congeners on time to pregnancy (months) in the presence of total sperm count as mediator using a data from the LIFE study. Three models are considered: regDOC is our proposed approach, B-K is the Baron and Kenny model, and regTS is the regularized two-stage approach. Significant findings are displayed in boldface font

PCB Congener	Mean	regDOC			B-K			regTS
		95% HPD Int.		Mean	95% HPD Int.		Mean	95% HPD Int.
		Lower	Upper		Lower	Upper		Lower	Upper
#28	−0.005	−0.306	0.287	−0.845	−3.653	1.844	−0.001	−0.311	0.298
#44	0.015	−0.251	0.335	−0.157	−2.001	1.710	0.007	−0.304	0.304
#49	0.028	−0.241	0.331	0.766	−1.257	2.744	0.012	−0.279	0.325
#52	0.004	−0.294	0.291	0.314	−1.595	2.211	−0.004	−0.294	0.320
#66	−0.008	−0.307	0.263	0.262	−2.047	2.472	−0.010	−0.327	0.284
#74	−0.073	−0.365	0.181	−0.359	−1.286	0.541	−0.049	−0.352	0.215
#87	0.016	−0.136	0.179	0.058	−0.245	0.366	0.008	−0.168	0.195
#99	−0.105	−0.364	0.112	−0.350	−0.914	0.268	−0.055	−0.307	0.175
#101	−0.034	−0.233	0.145	−0.058	−0.531	0.427	−0.028	−0.253	0.177
#105	0.039	−0.164	0.257	0.306	−0.448	0.981	0.025	−0.195	0.265
#110	−0.016	−0.152	0.123	−0.117	−0.415	0.193	−0.014	−0.168	0.151
#114	−0.035	−0.197	0.111	−0.041	−0.327	0.255	−0.033	−0.224	0.138
#118	0.024	−0.190	0.270	0.003	−0.760	0.816	0.013	−0.222	0.259
#128	−0.061	−0.235	0.084	−0.221	−0.525	0.066	−0.052	−0.244	0.124
#138	0.205	−0.061	0.538	0.929	0.269	1.627	0.117	−0.141	0.440
#146	−0.028	−0.234	0.168	−0.023	−0.491	0.428	−0.023	−0.267	0.202
#149	0.027	−0.209	0.265	0.272	−0.828	1.327	0.008	−0.242	0.267
#151	0.002	−0.249	0.230	0.210	−0.896	1.330	−0.007	−0.260	0.249
#153	−0.040	−0.336	0.221	−0.638	−1.549	0.266	−0.015	−0.327	0.264
#156	−0.018	−0.214	0.182	−0.110	−0.577	0.383	−0.010	−0.242	0.200
#157	−0.031	−0.182	0.110	0.009	−0.248	0.291	−0.032	−0.201	0.137
#167	−0.006	−0.128	0.107	−0.023	−0.206	0.162	−0.004	−0.143	0.126
#170	0.016	−0.218	0.270	−0.032	−0.888	0.911	0.018	−0.252	0.278
#172	−0.028	−0.207	0.123	−0.051	−0.378	0.276	−0.015	−0.213	0.171
#177	−0.114	−0.437	0.134	−0.733	−1.560	0.083	−0.068	−0.397	0.192
#178	−0.028	−0.215	0.165	−0.059	−0.473	0.332	−0.025	−0.250	0.183
#180	0.071	−0.183	0.378	0.589	−0.582	1.832	0.041	−0.238	0.340
#183	−0.031	−0.271	0.177	−0.171	−0.733	0.389	−0.012	−0.268	0.235
#187	0.083	−0.145	0.377	0.491	−0.348	1.307	0.054	−0.206	0.362
#189	−0.090	−0.216	0.038	−0.121	−0.295	0.043	−0.085	−0.232	0.051
#194	−0.027	−0.221	0.177	−0.222	−0.743	0.294	−0.022	−0.262	0.195
#195	0.046	−0.101	0.207	0.040	−0.249	0.324	0.048	−0.122	0.235
#196	−0.024	−0.280	0.204	−0.195	−0.947	0.631	−0.015	−0.281	0.252
#201	−0.005	−0.223	0.205	−0.006	−0.712	0.753	−0.004	−0.238	0.230
#206	0.005	−0.178	0.182	0.097	−0.452	0.623	0.004	−0.197	0.208
#209	0.077	−0.053	0.227	0.103	−0.129	0.330	0.069	−0.090	0.229

TABLE 3.

Posterior means and 95% highest posterior density (HPD) interval estimates of indirect effects of PCB congeners on time to pregnancy (months) in the presence of total sperm count as mediator using a data from the LIFE study. Three models are considered. regDOC is our proposed approach, B-K is the Baron and Kenny model, and regTS is the regularized two-stage approach. Significant findings are displayed in boldface font

PCB Congener	Mean	regDOC			B-K			regTS
		95% HPD Int.		Mean	95% HPD Int.		Mean	95% HPD Int.
		Lower	Upper		Lower	Upper		Lower	Upper
#28	0.002	−0.308	0.321	−0.021	−0.285	0.228	0.001	−0.032	0.034
#44	0.011	−0.304	0.319	−0.098	−0.362	0.117	−0.002	−0.036	0.029
#49	0.018	−0.276	0.349	0.068	−0.120	0.302	0.002	−0.028	0.038
#52	0.008	−0.306	0.313	0.040	−0.129	0.254	0.001	−0.029	0.037
#66	−0.001	−0.336	0.290	0.027	−0.183	0.263	0.001	−0.031	0.035
#74	−0.034	−0.334	0.252	−0.010	−0.105	0.073	−0.002	−0.036	0.025
#87	0.018	−0.174	0.220	0.015	−0.018	0.058	0.009	−0.010	0.039
#99	−0.054	−0.338	0.180	−0.046	−0.157	0.042	−0.020	−0.069	0.016
#101	−0.009	−0.241	0.230	−0.001	−0.050	0.041	−0.000	−0.024	0.023
#105	0.025	−0.214	0.284	−0.006	−0.076	0.058	−0.001	−0.028	0.025
#110	−0.008	−0.188	0.166	−0.009	−0.045	0.017	−0.001	−0.021	0.016
#114	−0.007	−0.206	0.196	0.006	−0.021	0.038	0.005	−0.013	0.029
#118	0.011	−0.237	0.270	0.019	−0.056	0.103	−0.001	−0.028	0.029
#128	−0.019	−0.218	0.180	0.004	−0.022	0.034	0.003	−0.013	0.025
#138	0.072	−0.207	0.373	0.018	−0.038	0.096	0.002	−0.025	0.030
#146	−0.014	−0.266	0.220	−0.003	−0.049	0.038	−0.002	−0.029	0.020
#149	0.028	−0.230	0.298	0.034	−0.065	0.161	0.003	−0.022	0.033
#151	0.010	−0.266	0.271	−0.039	−0.172	0.069	−0.002	−0.031	0.025
#153	−0.026	−0.348	0.245	−0.002	−0.090	0.084	−0.000	−0.030	0.031
#156	−0.003	−0.243	0.230	0.002	−0.044	0.050	0.002	−0.020	0.028
#157	−0.007	−0.191	0.182	0.000	−0.026	0.025	0.001	−0.016	0.018
#167	−0.000	−0.156	0.160	−0.000	−0.017	0.017	0.000	−0.015	0.015
#170	0.006	−0.270	0.294	−0.008	−0.098	0.072	−0.003	−0.034	0.025
#172	−0.013	−0.235	0.204	−0.003	−0.035	0.027	−0.005	−0.031	0.013
#177	−0.042	−0.361	0.237	0.014	−0.053	0.101	0.005	−0.022	0.040
#178	−0.013	−0.245	0.216	−0.004	−0.047	0.033	−0.003	−0.028	0.021
#180	0.025	−0.275	0.326	−0.004	−0.121	0.105	−0.001	−0.034	0.028
#183	−0.015	−0.285	0.235	−0.015	−0.077	0.035	−0.002	−0.030	0.022
#187	0.038	−0.237	0.343	0.015	−0.055	0.105	0.003	−0.026	0.034
#189	−0.019	−0.197	0.129	0.001	−0.015	0.017	0.001	−0.015	0.015
#194	−0.015	−0.256	0.219	0.018	−0.029	0.081	0.003	−0.020	0.031
#195	0.021	−0.169	0.224	0.001	−0.024	0.030	0.004	−0.013	0.026
#196	−0.018	−0.311	0.250	0.014	−0.059	0.098	0.000	−0.029	0.029
#201	−0.006	−0.258	0.240	−0.045	−0.166	0.044	−0.008	−0.041	0.016
#206	−0.004	−0.215	0.210	0.016	−0.033	0.079	0.001	−0.021	0.024
#209	0.016	−0.161	0.193	−0.009	−0.039	0.013	−0.006	−0.028	0.009

5.2 |. The CPP study

The Collaborative Perinatal Project (CPP) is a socioeconomically diverse pregnancy cohort conducted between 1959 and 1974 by National Institute of Neurological Diseases and Stroke. Over 50 000 pregnancies and resulting offspring were systematically observed, with the offspring followed up until 7 years of age. The CPP study contains a variety of categories of data such as obstetrics, pediatrics, pathology, and socioeconomics etc, hence, provides an important resource for biomedical and behavioral research in many areas.

In this analysis, we investigate the effect of demographic and socioeconomic status of parents on offspring’s birth weight, and whether those effects are mediated by maternal smoking. In particular, we consider maternal race, highest education level of parents, household income, maternal occupation type, family structure, household crowding (ratio of family size over number of rooms), and a prenatal disadvantage score as exposures, and maternal maximum number of cigarettes smoked per day during pregnancy as mediator. Descriptive statistics are presented in Table S2 in the Supplementary Materials. Under the setting of hyperparameters ${\sigma }_0^2=100$, θ₁ = θ₂ = 1, and v₁ = v₂ = 0.1, the posterior means of direct and indirect effects are summarized in Tables 4 and 5 with 95% HPD intervals. The proposed regDOC approach produces fewer significant effects than B-K and regTS, especially in indirect effects. While there are only 4 out of 14 significant indirect effects in the regDOC model, there are 11 in the B-K and regTS models each. These results suggest that our proposed model achieves parsimony as desired. Substantively, we observe that, while all, except prenatal disadvantage score, have some direct effects on birth weight, only the highest education level of parents and maternal race have significant indirect effects through maternal cigarettes smoking during pregnancy.

TABLE 4.

Posterior means and 95% highest posterior density (HPD) interval estimates of direct effects of demographic and socialeconomic variables on birth weight (g) of new born baby in the presence of maternal maximum number of daily cigarettes as mediator. Three models are considered. regDOC is our proposed approach, B-K is the Baron and Kenny model, and regTS is the regularized two-stage approach. Significant findings are displayed in boldface font

Characteristic		Mean	regDOC			B-K			regTS
			95% HPD Int.		Mean	95% HPD Int.		Mean	95% HPD Int.
			Lower	Upper		Lower	Upper		Lower	Upper
Education	> High school	-	-	-	-	-	-	-	-	-
	High school	−0.017	−0.044	0.008	−0.016	−0.043	0.010	−0.015	−0.042	0.010
	< High school	−0.071	−0.102	−0.039	−0.068	−0.101	−0.036	−0.069	−0.101	−0.038
Household income (thousand $)	> 1.5	-	-	-	-	-	-	-	-	-
	1 ∼ 1.5	0.056	0.031	0.081	0.058	0.033	0.084	0.055	0.029	0.080
	0 ∼ 1	0.047	0.017	0.074	0.050	0.019	0.082	0.046	0.016	0.078
Maternal occupation type	Nonmanual	-	-	-	-	-	-	-	-	-
	Manual	−0.024	−0.049	−0.000	−0.022	−0.048	0.004	−0.024	−0.048	0.002
	Unemployment/Student	0.004	−0.030	0.038	0.006	−0.032	0.042	0.004	−0.030	0.040
Family structure	Complete	-	-	-	-	-	-	-	-	-
	Single	−0.143	−0.170	−0.113	−0.141	−0.170	−0.111	−0.142	−0.171	−0.112
	Divorce/Separate/Widow	0.026	−0.006	0.061	0.031	−0.004	0.067	0.028	−0.006	0.063
Household crowding	≤ 1	-	-	-	-	-	-	-	-	-
	1 ∼ 1.5	0.080	0.053	0.106	0.083	0.055	0.110	0.080	0.053	0.107
	≥ 1.5	0.037	0.012	0.063	0.041	0.012	0.068	0.037	0.011	0.065
Prenatal disadvantage score	≤ 1.5	-	-	-	-	-	-	-	-	-
	1.5 ∼ 2.5	−0.024	−0.057	0.008	−0.029	−0.067	0.009	−0.024	−0.061	0.009
	3+	−0.023	−0.077	0.027	−0.032	−0.094	0.030	−0.023	−0.080	0.035
Maternal race	White	-	-	-	-	-	-	-	-	-
	Black	−0.480	−0.501	−0.457	−0.483	−0.505	−0.461	−0.481	−0.503	−0.460
	Hispanic/Other	−0.309	−0.344	−0.275	−0.315	−0.350	−0.280	−0.313	−0.347	−0.277

TABLE 5.

Posterior means and 95% highest posterior density (HPD) interval estimates of indirect effects of demographic and socialeconomic variables on birth weight (g) of new born baby in the presence of maternal maximum number of daily cigarettes as mediator. Three models are considered. regDOC is our proposed approach, B-K is the Baron and Kenny model, and regTS is the regularized two-stage approach. Significant findings are displayed in boldface font

Characteristic		Mean	regDOC			B-K			regTS
			95% HPD Int.		Mean	95% HPD Int.		Mean	95% HPD Int.
			Lower	Upper		Lower	Upper		Lower	Upper
Education	> High school	-	-	-	-	-	-	-	-	-
	High school	−0.044	−0.080	−0.007	−0.048	−0.053	−0.043	−0.047	−0.053	−0.042
	< High school	−0.051	−0.094	−0.008	−0.057	−0.064	−0.051	−0.057	−0.064	−0.051
Household income (thousand $)	> 1.5	-	-	-	-	-	-	-	-	-
	1 ∼ 1.5	0.009	−0.024	0.042	0.008	0.003	0.012	0.007	0.003	0.012
	0 ∼ 1	0.012	−0.026	0.053	0.010	0.004	0.015	0.009	0.004	0.015
Maternal occupation type	Nonmanual	-	-	-	-	-	-	-	-	-
	Manual	−0.001	−0.035	0.032	−0.003	−0.007	0.002	−0.002	−0.007	0.002
	Unemployment/Student	−0.010	−0.055	0.038	−0.012	−0.018	−0.005	−0.012	−0.018	−0.005
Family structure	Complete	-	-	-	-	-	-	-	-	-
	Single	−0.004	−0.043	0.036	−0.006	−0.011	−0.001	−0.005	−0.010	−0.000
	Divorce/Separate/Widow	−0.049	−0.096	0.001	−0.056	−0.062	−0.049	−0.055	−0.062	−0.049
Household crowding	≤ 1	-	-	-	-	-	-	-	-	-
	1 ∼ 1.5	−0.004	−0.042	0.030	−0.008	−0.012	−0.003	−0.008	−0.012	−0.003
	≥ 1.5	0.007	−0.027	0.041	0.004	−0.001	0.009	0.004	−0.001	0.008
Prenatal disadvantage score	≤ 1.5	-	-	-	-	-	-	-	-	-
	1.5 ∼ 2.5	−0.016	−0.059	0.027	−0.012	−0.018	−0.005	−0.011	−0.018	−0.005
	3+	−0.009	−0.077	0.060	−0.000	−0.011	0.010	0.000	−0.010	0.010
Maternal race	White	-	-	-	-	-	-	-	-	-
	Black	0.097	0.066	0.127	0.101	0.094	0.107	0.101	0.094	0.107
	Hispanic/Other	0.107	0.056	0.154	0.116	0.107	0.124	0.115	0.107	0.123

To illustrate the block regularization, we focus on a subset (n = 1, 540) of pregnant women from the CPP study, for whom data on serum inflammatory biomarkers were measured. We consider the same exposures as in the previous example except maternal race (the majority is white) and use five immune markers in the second and third trimester maternal serum (interleukin(IL)–1β, IL–6, IL–8, IL–10, and tumor necrosis factor(TNF)–α) (pg/ml) as mediators. Descriptive statistics are summarized in Table S3 in the Supplementary Materials. With the same hyperparameters used earlier, Tables S4 and S5 in the Supplementary Materials present the posterior estimation of 12 direct and 12 total indirect effects, among which only family structure of single versus complete is identified as a significantly negative direct effect on birth weights by regDOC and B-K. Household income of 0 ~ 1 versus above 1.5 is a significantly positive indirect effect by the B-K and regTS models.

6 |. DISCUSSION

Mediation analytical techniques have been successfully applied across various scientific disciplines. As research practices continue to focus on mixtures of exposures, often high dimensional, in relation to health disease outcomes in presence of mediators, appropriate techniques are needed for such investigation. This paper provides the first step by supporting simultaneous estimation and selection of effects. We take a Bayesian approach following the work of Park and Casella.¹⁸ While Bayesian Lasso compliments its frequentist counterpart in accommodating priors and providing variability measures of effects deemed to be excluded, it does not estimate them at exact zeros. In this sense, the Bayesian Lasso resembles a shrinkage approach but with the goal of simultaneous effect selection and estimation. We have also explored different ways of regularization and compared their strengths and limitations, both analytically and empirically. The proposed regDOC approach offers nice theoretical features while exhibiting superior empirical performance in comparison to alternatives.

The proposed framework works for both single and multiple mediators. In the multimediator case, we have restricted our attention to the case where the interest is in the total indirect effects. While this focused case is useful in many situations, it is desirable to be able to consider the situations where individual indirect effects are of interest. The extension to individual indirect effects is not trivial and a separate regularization approach is currently under development. Similarly, although interactions between exposures can be considered in the proposed model to obtain varying effects at different levels of other exposures, further research is warranted to investigate the optimal regularization strategy in the presence of such interactions.

In this paper, we use SUTVA to achieve identifiable effects. However, whether SUTVA is valid in the context of mediators can be a point of debate. For example, Imai et al²⁰ and Rubin²¹ raised this issue. Given that there is no consensus in the literature regarding the role of SUTVA when mediators are in use, further research is needed.

We have confined ourselves to the case of continuous outcomes in this paper. However, the extensions to dichotomous, polychotomous, or survival time outcomes are straightforward by choosing proper models. In the additive hazard model for time-to-event data, for example, natural direct and indirect effects in terms of hazard rate can be expressed by regression coefficients and their products²² so that our regularized different-in-coefficient approach is applicable. Similar property can also be seen in the Cox regression model when logarithm of hazard rate is the effect measure of interest.²³

In the application of our method to the CPP study, we have allowed our regularization procedure to make effect selection on the base of individual levels of a categorical variable (eg, education), essentially ignoring the group structures. It is worthwhile to take into consideration the group nature, following the spirit of group Lasso.^24,25