Tutorial on Causal Mediation Analysis with Binary Variables: An Application to Health Psychology Research

Abstract

Mediation analysis has been widely applied to explain why and assess the extent to which an exposure or treatment has an impact on the outcome in health psychology studies. Identifying a mediator or assessing the impact of a mediator has been the focus of many scientific investigations. This tutorial aims to introduce causal mediation analysis with binary exposure, mediator and outcome variables, with a focus on the resampling and weighting methods, under the potential outcomes framework for estimating natural direct and indirect effects. We emphasize the importance of the temporal order of the study variables and of the elimination of confounding. We define the causal effects in a hypothesized causal mediation chain in the context of one exposure, one mediator, and one outcome variable, all of which are binary variables. Two commonly used and actively maintained R packages, mediation and medflex, were used to analyze a motivating example. R code examples for implementing these methods are provided.

Mediation analysis has been widely applied to explain why and assess the extent to which an exposure or treatment has an impact on the outcome in social and behavioral studies since the 1980s (Judd & Kenny, 1981; Baron & Kenny, 1986). A mediator is a variable that transmits the effect of an exposure to an outcome variable (MacKinnon, 2008). Identifying a mediator or assessing the impact of a mediator has been the focus of many scientific investigations. For example, health psychologists investigated the mediating role of change in coping methods in the prospective association between self-rated health and post-operative psychological functioning (Oxlad & Wade, 2008). Mediation analysis can also be used to assess how prevention programs or new health policies influence behavioral or health outcomes through changes in mediating variables, including attitudes, social norms, and beliefs about positive consequences (MacKinnon et al, 2020; Thrul et al., 2021). Traditional mediation analysis methods described in Baron & Kenny’s seminal article (1986) have been used extensively to study mediation in the context of linear regression models and been extended to more complex mediation models (e.g., MacKinnon, 2008; Preacher et al., 2011); however, these methods are not directly applicable to mediation models with binary mediator or outcome variables. Recent causal mediation methods, proposed under the potential outcomes framework for causal inference (e.g., Imai et al, 2010; VanderWeele, 2015; Hong, 2015), explicitly state the assumptions needed for inferring causality and allow for assessment of mediation for various types of outcomes, for example, binary, continuous, count, and time-to-event outcomes. In comparison with the traditional mediation approach, the causal mediation approach has gained popularity in recent years due to its consistent definition of causal effects, transparent assumptions, and flexibility for modeling different types of outcomes. A causal mediation analysis is appealing because it provides clarity and flexibility in defining and interpreting indirect effects, even for non-continuous mediator or outcome variables.

Recent tutorials have described the potential outcomes framework for causal inference in general, with a focus on the estimation of the averaged exposure effect (e.g., Schafer & Kang, 2008; Leite et al., 2019). However, there are fewer tutorials specifically focused on causal mediation methods, especially given the recent proliferation of such methods. This tutorial aims to introduce causal mediation analysis under the potential outcomes framework for estimating natural direct and indirect effects. We aim to help health psychology researchers gain deeper knowledge of causal mediation methods by clearly stating the causal questions, defining causal effects, selecting model estimation methods, and interpreting the results. We expect readers to be familiar with regression analysis methods, but prior knowledge of causal inference and mediation analysis is not assumed.

This tutorial focuses on causal mediation models in which the exposure, the mediator, and the outcome variables are all binary. Binary mediators or outcomes are common in clinical or health settings including, for example, results of a medical test (positive or negative), health status (diseased or healthy), smoking status (yes or no), holding a belief (yes or no), or mortality (alive or dead). Methodological articles, however, have typically focused on models with other types of mediators or outcomes (e.g., continuous, count, time-to-event; Imai, 2011; Valente et al., 2020). Very little attention has been paid to models where both the mediator and outcome variables are binary (Vo et al., 2020), partially due to the fact that mediation analysis with binary variables is more technically demanding and interpretation of results more challenging (Rijnhart et al, 2021). Due to the lack of clear guidance on this situation and the empirical needs from the field, we wrote this tutorial.

In the next sections, we first introduce the motivating example that we will use throughout the rest of the tutorial to illustrate the mediation analysis methods. Then we provide an overview of traditional mediation analysis and causal mediation analysis in the context of a simple mediation model, involving one exposure, one mediator, and one outcome variable, all of which are binary variables. Next, we followed the four elements of causal mediation analysis proposed by Imai et al. (2010), a step-by-step approach to define the causal mediation effects, specify the assumptions needed to identify them, estimate causal effects using parametric methods, and perform sensitivity analyses. Using a longitudinal observational data set, we demonstrate the use of two estimation methods, resampling and weighting, to adjust for confounding in the causal pathways. We demonstrate how to use two commonly used R packages, mediation and medflex, to estimate a causal mediation model and how to interpret the results. In contrast, we also estimate a traditional mediation model using the R package lavaan. Lastly, we discuss important issues when implementing a causal mediation analysis with data from empirical studies. R sample code and sample data for implementing these methods are provided in the appendix.

Motivating Example: The PATH Study

The data for the example are from the Population Assessment of Tobacco and Health (PATH) Study, a large-scale longitudinal study of a nationally representative sample of youth and adults in the United States. We analyze the first four waves of the public-use data from the youth sample (data collected annually from 2013/14 to 2016/18). Participants included the youth (12 – 17 yrs. old) and their parents. Study design and data collection have been reported in Hyland et al. (2017). One of the PATH study’s aims is to evaluate tobacco initiation and use patterns, including the use of new products such as electronic cigarettes (e-cigarettes). The analytic dataset includes a subsample of 7511 youth who were tobacco naïve at Wave 1, initiated e-cigarette use or remained tobacco naïve at Wave 2, and completed four waves of data collection. The proportions of missing values at the item level were low (0% - 2.1%) except for one variable (perception of harm of e-cigarette use at Wave 1, 21.4%); thus, 5 complete data sets were created via multiple imputation by chained equations (Buuren & Groothuis-Oudshoorn, 2011). We used one of the imputed data sets for the motivating example analysis.

The mediation question of interest is whether e-cigarette initiation at Wave 2 (denoted by X; 1 = e-cigarette initiation, 0 = never used any tobacco) changes the perception of harm of e-cigarette use at Wave 3 (denoted by M; 1 = e-cigarettes are less harmful relative to cigarettes, 0 = e-cigarettes are equally or more harmful relative to cigarettes), which in turn, influences tobacco product use at Wave 4 (denoted by Y; 1 = used, 0 = not used). The mediator, relative harm perception of e-cigarette use versus cigarette use, was measured by the question, “Is using e-cigarettes less harmful, about the same, or more harmful than smoking cigarettes?” Following common practice in this domain area (Persoskie, 2019), we combined “about the same” and “more harmful than smoking cigarettes” into a single response category indicating positive perception of harm for e-cigarettes compared to cigarettes. Accordingly, these two response categories were labelled “less harmful” (1) and “same or more harmful” (0). The outcome, current tobacco use status, denotes self-report of past-30-day use of any tobacco products at Wave 4. Pre-exposure covariates at Wave 1 (denoted by C) include age, gender, race/ethnicity, non-prescribed drug use, alcohol use, marijuana use, depressive symptoms, history of asthma, and harm perceptions of e-cigarette use relative to cigarette use. We adjusted for these covariates because, based on the literature, they are associated with both e-cigarette use and other tobacco use (Soneji et al., 2017), and also with harm perception of e-cigarette use and tobacco use (Sharma et al., 2021). Table S1 presents a description of study variables and their descriptive statistics for the whole sample and subsamples stratified by e-cigarette initiation status.

Participants were obviously not randomly assigned to e-cigarette initiation at Wave 2 because it would be unethical to do so. Instead, participants self-selected into each exposure status (e-cigarette initiation vs. tobacco naive). Similarly, participants were not randomly assigned to levels of the mediator because the mediator is an outcome of the exposure (Coffman & Zhong, 2012). We hypothesized that, in comparison with the tobacco never users, e-cigarette users (at Wave 2) would be more likely to perceive e-cigarette use as less harmful to their health (at Wave 3), and in turn, would be more likely to use tobacco products (i.e., become current tobacco users at Wave 4). Figure 1 presents a directed acyclic graph (DAG) of the causal relationships among study variables for the proposed causal mediation model. A DAG consists of nodes, which represent variables, and arrows that represent causal relationships. A DAG in the causal inference literature is similar to a path diagram in structural equation modeling, except that a DAG represents causal relationships and is nonparametric (Nguyen et al., 2021).

Figure 1.

Directed acyclic graph depicting the mediating effect of relative harm perception of e-cigarette use in the pathway between e-cigarette initiation and tobacco current use.

We aimed to address the following three questions to improve our understanding of the mechanism of tobacco use transition.

Q1 (total effect): What is the causal effect of e-cigarette initiation versus staying tobacco naïve on current tobacco use after two years?

Q2 (natural indirect effect): How much would current tobacco use change if all participants perceived e-cigarettes to be less harmful than cigarettes, compared to if all participants perceived e-cigarettes as equally or more harmful than cigarettes?

Q3 (exposure-mediator interaction): Does the e-cigarette exposure effect mediated through harm perceptions have a larger effect among those who initiated e-cigarette use as opposed to those who stayed tobacco naïve?

Traditional Mediation Analysis

Baron and Kenny (1986) describe the steps for determining the mechanism through which an exposure influences an outcome. The total effect of the exposure on the outcome is composed of an indirect effect through the mediating variable (i.e., mediator), and the direct effect, which represents the contribution of other unspecified pathways that are not through the mediator. When the mediator and outcome are continuous variables, a series of three ordinary least squares (OLS) regressions are typically used to define these effects. The coefficient in the first OLS regression of the outcome on the exposure (coefficient c) represents the total effect of the exposure on the outcome. The second OLS regression assesses the effect of the exposure on the mediator (coefficient α). The third OLS regression assesses the effect of the mediator on the outcome (coefficient β), adjusting for the exposure variable. In this third model, the coefficient for the exposure (coefficient c’) is the direct effect, which quantifies the remaining effect of the exposure on the outcome that is not through the mediator. The total effect of the exposure on the outcome is the sum of the direct effect and the indirect effect. The indirect effect (i.e., the mediated effect) is computed either as the difference between the total effect and the direct effect (c – c’), or as the product of the coefficients of α and β. The difference method and the product method of estimating the indirect effect coincide only in the case of OLS (c – c’ = αβ).

When the mediator and/or the outcome are binary variables, generalized linear models may be used to accommodate the distribution of these variables. In these cases, the difference of coefficients and the product of coefficients methods give different estimates of the indirect effect (MacKinnon et al, 2007). In addition, the sum of the direct and indirect effects does not equal the total effect (see VanderWeele, 2015). The traditional mediation methods fail to provide a consistent definition and clear interpretation for the indirect effect (Pearl, 2011; Rijnhart et al., 2021). In addition, the product or the difference of coefficients methods for defining indirect effects are no longer appropriate for the analysis of mediation effects because the indirect effect depends on the values of the exposure and mediator in a nonlinear fashion (Imai et al., 2011; VanderWeele, 2015). In contrast, causal mediation analysis under the potential outcomes framework overcomes these limitations by clearly defining the mediating effects without requiring any particular parametric models or estimating methods (e.g., linear regression estimated via OLS).

Moreover, using the traditional methods, the causal effects of interest can be properly assessed only when (1) there is no confounding of the exposure-outcome, exposure-mediator, and mediator-outcome relationships, and (2) there is no interaction between exposure and mediator affecting the outcome (Hong, 2015; VanderWeele, 2015). Confounding refers to the presence of biased effects due to an additional covariate (or a set of covariates) that disguises the true causal effect of the exposure variable on the outcome variable. For example, a common cause of the exposure and the outcome variables may distort the association between these two variables because the common cause (the third variable) is independently associated with both. The third variable is referred to as a confounder.

Causal Mediation Analysis for Natural Effects

We introduce causal mediation analysis, following a four-element general approach proposed by Imai et al. (2010): 1) define causal mediation effects using the potential outcomes framework, 2) nonparametric identification based on the sequential ignorability assumptions, 3) parameter estimation using certain algorithms, and 4) sensitivity analysis to assess the robustness of empirical results. Using potential outcomes to define causal effects was initially proposed by Neyman (1923) in the context of agricultural experimental design and was later applied to observational studies or quasi-experimental studies (Holland, 1986). A potential outcome is the outcome of an individual under a hypothetical exposure.

As before, let X denote the exposure, M the mediator, and Y the outcome variable, and for simplicity, assume that X, M, and Y are all binary variables. Let C denote a set of pre-exposure covariates which could be either continuous or categorical. These covariates have an effect on both the exposure and the outcome (i.e., they are confounders), hence they should be assessed before the exposure to ensure that they have an effect on exposure rather than vice versa (in which case, the covariate would be a mediator). Given that mediation is a causal chain, the exposure should precede the mediator, which should precede the outcome. The pre-exposure covariates, exposure, mediator, and outcome variables must follow a temporal sequence to allow for a causal interpretation of a mediated effect (Kendall et al, 2017). Otherwise, if any of these variables were assessed in a different temporal order, or two or more variables were measured concurrently, then the causal interpretation of a mediated effect could be compromised.

Given the binary exposure X, each individual has two potential values for the outcome Y. For example, Y_i(x = 1) denotes the potential outcome if individual i initiates e-cigarette use, and Y_i(x = 0) denotes the potential outcome if individual i never uses tobacco products. The causal effect of e-cigarette initiation versus not for individual i is the difference between these two potential outcomes: Y_i(x = 1) - Y_i(x = 0). In the following, we use Y_i(1) and Y_i(0) as a shorthand for Y_i(x = 1) and Y_i(x = 0), respectively, and we drop the subscript i. Unfortunately, only one of the potential outcomes, either Y(1) or Y(0), is observed for any given individual: Y(1) is observed for individuals who initiated e-cigarette use, and Y(0) is observed for individuals who did not. Because only one of the two potential outcomes for individual i can be observed and the other potential outcome is missing (referred to as the fundamental problem of causal inference by Holland, 1986), a causal effect for an individual cannot be estimated from the observed data. Instead, the average causal effect for the whole population, which is a contrast of means of potential outcomes defined as E[Y(1) - Y(0)], is estimated under identifiability assumptions (see subsection Assumptions).

Likewise in the mediation context, given a binary exposure X, each individual has two potential values for the mediator, M(1) and M(0), and in turn each of them has two potential values for the outcome. Nested potential outcomes, Y(x, M(x)), are defined to indicate both the level of the exposure X and the mediator M (Robins & Greenland, 1992). Hence, there are four possible nested potential outcomes for current tobacco use status, Y(1, M(1)), Y(1, M(0)), Y(0, M(1)), and Y(0, M(0)). For example, Y(1, M(1)) indicates the potential current tobacco use status had one initiated e-cigarette use and had the harm perceptions at a level corresponding to the exposed condition. As another example, Y(1, M(0)) indicates the potential current tobacco use status had one initiated e-cigarette use (exposed) and had the harm perceptions at a level corresponding to the tobacco naïve (unexposed) condition. Table 1 summarizes each of these potential outcomes.

Table 1.

Glossary of Potential Outcomes and Causal Effects in a Simple Causal Mediation Model

Label	Notation	Definition/Causal Questions	Motivating Example Interpretation
Potential outcome
Potential outcomes for the mediator variable, M	M(0)	Potential mediator value under the control condition	Potential value for mediator that would have been observed if one had stayed tobacco naïve
	M(1)	Potential mediator value under the exposure condition	Potential value for mediator that would have been observed if one initiated e-cigarette use
Potential outcomes for the outcome variable, Y	Y(0)	Potential outcome under the control condition	Potential value for current tobacco use that would have been observed if one stayed tobacco naïve
	Y(1)	Potential outcome under the exposure condition	Potential value for current tobacco use that would have been observed if one initiated e-cigarette use
Nested potential outcomes for the outcome variable, Y	Y(1, M(0))	Nested potential outcomes under various exposure and mediator combinations	Potential value for current tobacco use had one initiated e-cigarette use and had harm perceptions at a level corresponding to when stayed tobacco naive.
	Y(1, M(1))		Potential value for current tobacco use had one initiated e-cigarette use and had harm perceptions at a level corresponding to when initiated e-cigarette use.
	Y(0, M(0))		Potential value for current tobacco use had one stayed tobacco naïve and had harm perceptions at a level corresponding to staying tobacco naive.
	Y(0, M(1))		Potential value for current tobacco use had one stayed tobacco naïve and had harm perceptions at a level corresponding to when initiating e-cigarette use.
Causal effect
Exposure effect on the mediator variable	M(1) – M(0)	Effect of exposure on the mediator	The difference in potential mediator values had one initiated e-cigarette use vs. stayed tobacco naïve.
Total Effect	TE = Y(1) – Y(0) = Y(1, M(1)) – Y(0, M(0)) = NDE0 + NIE1 = NDE1 + NIE0	Total effect of exposure on outcome	The difference in potential outcomes had one initiated e-cigarette use vs. stayed tobacco naïve.
NIE1 (total indirect effecta)	Y(1, M(1)) – Y(1, M(0))	Effect of exposure induced change in mediator level on the outcome under exposure condition	The effect of e-cigarette initiation on tobacco current use under the e-cigarette initiation condition solely attributable to the e-cigarette-induced change in harm perceptions.
NIE0 (pure indirect effect a)	Y(0, M(1)) – Y(0, M(0))	Effect of exposure induced change in mediator level on the outcome under control condition	The effect of e-cigarette initiation on tobacco current use under the tobacco naive condition solely attributable to the e-cigarette-induced change in harm perceptions.
NDE1 (total direct effect a)	Y(1, M(1)) – Y(0, M(1))	Effect of exposure on outcome holding the mediator to the level that would be obtained under exposure condition	The effect of e-cigarette initiation on tobacco current use, had harm perceptions stayed at the level under the e-cigarette initiation conditions.
NDE0 (pure direct effect a)	Y(1, M(0)) – Y(0, M(0))	Effect of exposure on outcome holding the mediator to the level that would be obtained under control condition	The effect of e-cigarette initiation on tobacco current use, had harm perceptions stayed at the level under the tobacco naïve condition.
Natural exposure- by-mediator interaction effect	NIE1 - NIE0 = [Y(1, M(1)) – Y(1, M(0))] - [Y(0, M(1)) – Y(0, M(0))]	Difference in natural indirect effects between the exposure condition and control condition	The difference in the effect of e-cigarette initiation on tobacco current use under the e-cigarette initiation condition solely attributable to the e-cigarette-induced change in harm perceptions and the effect of e-cigarette initiation on tobacco current use under the tobacco naive condition solely attributable to the e-cigarette-induced change in harm perceptions.

Note. TE = Total effect, NIE = Natural indirect effect, NDE = Natural direct effect

^aIn the R medflex output, NDE₀, NDE₁, NIE₀, NIE₁ are labelled as pure direct effect, total direct effect, pure indirect effect, and total indirect effect, respectively.

Definitions.

Causal effects are defined as contrasts between potential outcomes. For example, the total effect (TE), E[Y(1) – Y(0)] is the effect of initiating e-cigarette use versus being tobacco naïve on current tobacco use status. Similarly, E[M(1) – M(0)] is the effect of initiating e-cigarette use versus being tobacco naïve on harm perceptions. The nested potential outcomes can be used to define causal mediation effects. The TE, E[Y(1) – Y(0)], can be decomposed into the natural direct effect (NDE) and the natural indirect effect (NIE). The NDE is the effect of X on Y that is not due to an X induced change in the mediator, E[Y(1, M(0)) – Y(0, M(0))] (i.e., the mediator is held constant to the potential value under non-tobacco using status). In other words, the NDE addresses the question of what is the effect of e-cigarette initiation on subsequent tobacco use, holding harm perceptions to what they would have been under non-tobacco using status. The NIE is the effect of X on Y that is due to X induced changes in M, E[Y(1, M(1)) – Y(1, M(0))]. That is, holding constant exposure status and changing the mediator potential values to what they would have been under e-cigarette initiation versus tobacco naive. Hence, the NIE defines the mediated effect, which is the part of the total effect that can be explained by the change in the mediator. In contrast to the traditional mediation analysis, these definitions do not start by stating a model (e.g., OLS regression) and then using the model coefficients as definitions to answer the scientific question of interest. Rather, in the potential outcomes framework, the researcher begins by defining the causal effects of interest irrespective of a model or any other assumptions. It is critical to recognize that the meaning of coefficients in traditional mediation analysis change, for example, depending on the type of model and other variables being included in the model. These coefficients may or may not be equivalent to causal effects defined under the potential outcomes framework.

The TE is the sum of the NDE and NIE (TE = NDE + NIE). The TE can be decomposed into NDE and NIE in two ways. One way of decomposing the TE, E[Y(1) – Y(0)] = E[Y(1, M(1)) – Y(0, M(0))] = E[Y(1, M(0)) – Y(0, M(0))] + E[Y(1, M(1)) – Y(1, M(0))], is referred to as the NDE₀ + NIE₁ decomposition, where the subscript 0 indicates the level that M is held to for the NDE and the subscript 1 indicates the level that X is held to for the NIE. An alternative way of decomposing the TE, E[Y(1) – Y(0)] = E[Y(1, M(1)) – Y(0, M(0))] = E[Y(1, M(1)) – Y(0, M(1))] + E[Y(0, M(1)) – Y(0, M(0))], is referred to as the NDE₁ + NIE₀ decomposition, where the subscript 1 indicates the level that M is held to for the NDE and the subscript 0 indicates the level that X is held to for the NIE, respectively.

In cases where the causal mediated effect varies by the exposure level, the NIE₁ will not equal the NIE₀, and an exposure-by-mediator interaction is present. This interaction effect is assumed to be null using the traditional product of coefficients definition. Table 1 summarizes the definitions and presents example interpretations of the total effect, natural (in)direct effects, and the natural exposure-by-mediator interaction effect. These definitions do not rely on specifying any particular model or estimation method, but they are also not identified because only one of the four nested potential outcomes is ever observed (i.e., the fundamental problem of causal inference).

Assumptions.

The primary assumption is referred to as sequential ignorability (Imai, 2010), which states that (1) conditional on pre-treatment covariates, C, the relationship between the exposure and the outcome is not confounded, (2) conditional on C, the relationship between the exposure and the mediator is not confounded, (3) conditional on C and X, the relationship between the mediator and the outcome is not confounded, and (4) there are no confounders of the mediator and outcome that have been affected by X (i.e., there are no post-exposure confounders of M and Y). In the case of randomization to X, the first two conditions are satisfied, but the latter two conditions may not be satisfied even when there is random assignment of X (e.g., in an RCT) because the mediator is an “outcome” of the exposure. Thus, participants cannot be randomized to the levels of the mediator, regardless of the presence of random assignment of X.

Using the potential outcomes approach for causal mediation is appealing because, unlike in traditional mediation analysis, the total effect is consistently decomposed into the direct effect and the indirect effect regardless of the type of model used. The definition and assumptions of a causal effect are independent of data distribution, scales of causal effects, or the linear versus nonlinear form of the model. The technical details have been provided in a few textbooks (e.g., Hong, 2015; VanderWeele, 2015). It is only after the mediation effects have been defined and identifying assumptions stated that models for particular variable distributions and/or methods to estimate them are considered.

Estimation.

Natural (in)direct effects can be estimated using several software packages (for a more comprehensive review, see Valente et al., 2020). In this tutorial, we present mediation analyses of the PATH data using both a resampling and a weighting approach, as implemented in two of the most commonly used and actively maintained R packages, specifically the R mediation package (Tingley et al., 2014) and the R medflex package (Steen et al., 2017). These two R packages have been recently compared against other commonly used packages (Valente et al., 2020) and are recommended for use when missing data are present and when researchers have a multi-categorical exposure variable.

The R mediation package fits a mediation model in two steps. In Step 1, researchers need to specify two separate statistical models, one for the mediator and the other one for the outcome. The mediator model should include the exposure and pre-exposure covariates; the outcome model should include the exposure, mediator, and pre-exposure covariates. The potential outcomes, M(0), M(1), Y(0,M(0)), Y(0,M(1)), Y(1,M(0)) and Y(1,M(1)) are predicted for each participant. When both M and Y are binary, the glm function is fitted with a probit regression (family = binomial(“probit”)). In Step 2, the natural (in)direct effects are estimated for each exposure status using the mediate function. The standard errors and confidence intervals for each model parameter are estimated by default using the quasi-Bayesian Monte Carlo method based on a normal approximation (Imai et al., 2010). For exact reproduction of results, a specific random seed should be set before calling the function. Alternatively, nonparametric bootstrap (boot = TRUE, default number of simulations sims = 1000 resamples) confidence intervals, or robust standard errors using White’s heteroscedasticity-consistent estimator can be computed.

In Step 3 (optional), the test.TMint function can be used to test for the exposure-by-mediator interaction, in which case the NIE (labeled ACME for average causal mediation effect in the package output) is estimated separately by exposure status. The difference between NIE₁ and NIE₀ and its 95% confidence interval are calculated, and the statistical significance of the exposure-and-mediator interaction is evaluated.

Results include estimates for marginal population-level (averaged over the joint distribution of the covariates) natural (in)direct effects (i.e., TE, NDE, and NIE, labelled as Total Effect, ADE, and ACME, respectively). When the outcome is binary, the results are presented as risk differences. Results also include the proportion of mediated effect, which is the ratio of the natural indirect effect to the total effect (i.e., NIE/TE).

The R medflex package uses ratio-of-mediator-probability weighting (RMPW; Hong et al., 2010) to estimate natural effects. RMPW is a propensity-score-based weighting method which transforms the conditional distribution of the mediator in the exposed group such that it represents the distribution in the unexposed group given the pre-exposure covariates. Fitting a natural effect model involves four steps. In Step 1, a logistic regression model for the binary mediator is fitted by including the exposure and pre-exposure covariates in the model. In Step 2, using the neWeight function, an expanded dataset is created in which there are two rows for each individual, one for each of the exposure values, x = 1 and x = 0, respectively. It also creates two new exposure variables corresponding to hypothetical exposure values, labeled as the exposure variable name with a suffix “1” or “0” (e.g., EXPVAR1 and EXPVAR0), respectively. The RMPW weights are stored in the expanded data set and can be easily retrieved for the outcome analysis. In Step 3, using the expanded dataset with the RMPW weights, the neModel function fits an outcome model, including the two new exposure variables and covariates in the model. Robust standard errors based on the sandwich estimator can be created by setting se = “robust.” Bootstrap-based confidence intervals (e.g., percentile bootstrap CIs, bias-corrected bootstrap CIs) can also be obtained by setting the type argument to the desired value. By default, the number of bootstrap replications is 1000. In Step 4, results (NDE, NIE, and TE) are obtained using the neEffdecomp function.

To test for the presence of an exposure-by-mediator interaction, in Step 2, in addition to entering the two hypothetical exposure variables in an additive way, their product (e.g., EXPVAR0*EXPVAR1) is also entered into the regression model. Using the neEffdecomp function, results include estimates for NDE₀, NDE₁, NIE₀, NIE₁ (labelled as pure direct effect, total direct effect, pure indirect effect, and total indirect effect, respectively), and total effect.

Results include the estimates for conditional (on values of the pre-exposure covariates) natural (in)direct effects (i.e., TE, NDEs, and NIEs). By default, these conditional estimates are evaluated at levels that correspond to 0 for continuous covariates and the reference level for categorical covariates. By specifying the argument covLev, the conditional estimates can be evaluated at user-specified levels. The marginal population-average natural (in)direct effects can also be estimated using the same steps, except that the confounders will not be entered into the neModel function in Step 3. Note that the confounders are still accounted for through the use of the weights. When the outcome is binary, the estimates are given on the log-odds scale. The proportion of mediated effect is not provided in the medflex package output.

It should be noted that the results are not necessarily comparable across the R packages or even within each R package. Specifically, the estimates from the mediation package are on the risk difference scale while the estimates from the medflex package are on the odds ratio scale. Within the medflex package, conditional odds ratios are not necessarily similar to marginal odds ratios (Austin, 2011) and are not directly comparable (VanderWeele, 2016) due to the fact that the odds ratio is a measure that is “noncollapsible” (Greenland et al., 1999), where the marginal odds ratio is not a weighted average of the conditional odds ratio (Hernán & Robins, 2020). Finally, having a significant interaction on the risk difference scale does not imply that a significant interaction will be obtained on the odds ratio scale (VanderWeele & Knol, 2014), and vice versa. Only when the outcome is continuous and an identity link (e.g., linear regression model) is used, will all the estimates be similar.

Sensitivity test.

Valid causal inference requires untestable assumptions regarding unconfounded relationships among the exposure, the mediator, and the outcome variables. Confounding is always a concern because the presence of confounding distorts the true relationship between the exposure, the mediator, and the outcome variables. Omitting any confounders poses a risk of obtaining biased estimates of the true effects, hence compromising the causal interpretation of the results. Performing a sensitivity analysis has been highly recommended in the guidelines for reporting causal mediation analysis results (e.g., Lee et al., 2021). A sensitivity test assesses the robustness of the empirical findings against violations of the identification conditions/assumptions. Unfortunately, sensitivity tests in R mediation and R medflex are not currently available for the situation when the mediator and outcome variables are both binary variables. Therefore, we include no results of sensitivity test in this tutorial.

In summary, the definition and identification of causal effects are independent of analytical models, which is different from the traditional approach. The product of coefficients definition from the traditional approach applies only for linear regression. Thus, we would argue that researchers who wish to assess mediation, which is inherently a question about causality, in a context other than linear regression, should use causal mediation analysis based on the potential outcomes framework. Researchers should select models and methods for parameter estimation based on factors such as the scale of estimates (risk difference vs. odds ratio) and desired type of estimated effects (marginal vs. conditional effects), and that are appropriate for the distributions of the mediator and outcome.

Results of the Motivating Example

We investigated whether the effect of e-cigarette initiation (vs. tobacco naïve) on current tobacco use two years later was mediated through harm perceptions of e-cigarette use. To assure temporal ordering, we conducted the mediation analysis using the data from 7511 youth who initiated e-cigarette use or remained tobacco naive at Wave 2 and reported harm perceptions of e-cigarette use at Wave 3 and current tobacco use at Wave 4. All potential confounders were reported at Wave 1. We interpret the results for all parameters, including the ones for our research questions. Among the 7.3% (n = 546) of youth who initiated e-cigarette use at Wave 2, 47.4% (n = 259) perceived e-cigarette use as less harmful than cigarette use at Wave 3, and 33.7% (n = 184) became current tobacco users at Wave 4. Among the 92.7% (n = 6965) of youth who stayed tobacco naive at Wave 2, 27.5% (n = 1912) perceived e-cigarette use as less harmful than cigarette use at Wave 3, and 9.4% (n = 654) became current tobacco users at Wave 4.

Using the R mediation package, we obtained marginal natural (in)direct effects on the percentage (i.e., risk difference) scale (Table 2). The total effect of e-cigarette initiation (compared to remaining tobacco naïve) was a 17.5% (95% CI = 14.0% - 21.2%; Q1) increase in the prevalence of current tobacco use. Harm perceptions explained 9.3% of the e-cigarette initiation effect on current tobacco use. Due to the fact that the interaction between exposure and mediator was statistically significant (p = .002; Q3), the natural direct and indirect effects are different (difference = 1.6%, 95% CI = 0.5% – 2.9%) between e-cigarette users and tobacco naïve participants at Wave 2. The NDE₁ indicated that had participants initiated e-cigarette use, compared to remaining tobacco naïve, they would on average have a 16.7% (95% CI = 13.1% - 20.4%) higher prevalence of current tobacco use, had harm perceptions stayed at the level that would have been observed under the e-cigarette initiation condition. The NDE₀ indicated that had participants initiated e-cigarette use, compared to remaining tobacco naïve, they would on average have a 15.1% (95% CI = 11.7% - 18.9%) higher prevalence of current tobacco use, had harm perceptions stayed at the level that would have been observed under the e-cigarette naïve condition. The NIE₁ indicated that had participants initiated e-cigarette use, the effect of e-cigarette initiation on current tobacco use that was solely attributable to e-cigarette initiation-induced changes in harm perceptions was 2.4% (95% CI = 1.2% – 3.9%) (Q2). The NIE₀ indicated that had participants remained tobacco naive, the effect of e-cigarette initiation on current tobacco use that was solely attributable to e-cigarette initiation-induced changes in harm perceptions was 0.8% (95% CI = 0.5% - 1.3%) (Q2).

Table 2.

Estimated Causal Effects for the PATH Data

	R mediation	R medflex		R medflex
	(marginal)	(conditional)		(marginal)
	percentage (95% CI)	Adj. log-odds	AOR (95% CI)	log-odds	OR (95% CI)
Total Effect	17.5 (14.0 – 21.2)	1.30	3.69 (2.97 – 4.57)	1.58	4.86 (4.03 – 5.85)
Average NIE	1.6 (0.9 – 2.4)	0.10	1.11 (1.07 – 1.15)	0.10	1.11 (1.06 – 1.15)
NIE0	0.8 (0.5 – 1.3)
NIE1	2.4 (1.2 – 3.9)
Average NDE	15.9 (12.4 – 19.5)	1.20	3.32 (2.68 – 4.10)	1.48	4.39 (3.64 – 5.29)
NDE0	15.1 (11.7 −18.9)
NDE1	16.7 (13.1 – 20.4)
X-M Interactiona	1.6 (0.5 – 2.9)	0.04	1.05 (0.97 – 1.13)	0.03	1.03 (0.96 – 1.10)
PEb	9.3 (5.4 – 14.1)

Note. CI = Confidence Interval, NIE = Natural indirect effect, NDE = Natural direct effect, Adj. log-odds = adjusted log-odds, AOR = adjusted odds ratio.

^aX-M interaction for the R medflex analyses were based on a model where the X-M interaction were included. Given the X-M interaction was not significant, it was excluded from subsequent analyses based on which we reported the total effect, the average NIE, and the average NDE.

^bPE = Proportion of Mediated Effect; PE can be obtained from R mediation, not from R medflex.

Using the R medflex package (Table 2), we obtained conditional natural (in)direct effects on the odds ratio scale. Covariates in the analysis included the aforementioned baseline variables and the estimates are conditional on the level of these covariates by the package default setting. E-cigarette initiation and harm perceptions did not interact with regard to their effect on current tobacco use (p = .23; Q3), hence, stratified natural (in)direct effects (e.g., NDE₀ and NDE₁) across the exposure groups are not reported. Instead, an averaged NDE and NIE are reported. The NDE indicated that initiation of e-cigarette use (compared to remaining tobacco naïve), resulted in a 3.22 times (95% CI = 2.68 – 4.10) increase in the adjusted odds of current tobacco use, holding harm perceptions at the levels naturally observed. The NIE indicated that the effect of e-cigarette initiation (compared to remaining tobacco naïve) on current tobacco use that was due to e-cigarette initiation induced changes in harm perceptions corresponded to a 1.11 times (95% CI = 1.07 – 1.15) increase in the adjusted odds of tobacco use (Q2). Finally, the TE indicated that e-cigarette initiation (compared to remaining tobacco naïve) resulted in a 3.69 times (95% CI = 2.98 – 4.57) increase in the adjusted odds of current tobacco use (Q1).

Marginal estimates of averaged natural (in)direct effects are also reported. The NDE indicated that initiation of e-cigarette use (compared to remaining tobacco naïve), resulted in a 4.39 times (95% CI = 3.64 – 5.29) increase in the odds of current tobacco use that was not due to harm perceptions. The NIE indicated that the effect of e-cigarette initiation (compared to remaining tobacco naïve) on current tobacco use that was due to e-cigarette initiation induced changes in harm perceptions corresponded to a 1.11 times (95% CI = 1.06 – 1.15) increase in the odds of current tobacco use (Q2). Finally, the TE indicated that e-cigarette initiation (compared to remaining tobacco naïve) resulted in a 4.86 times (95% CI = 4.03 – 5.85; Q1) increase in the odds of current tobacco use.

For completeness, we also obtained the estimates for a traditional mediation model using the R package lavaan. For ease of interpretation, the coefficients from the probit model were converted to odds ratios by exponentiating 1.8 times the values of the coefficients, and we report the adjusted odds ratios (AORs) and corresponding 95% CIs. Overall, e-cigarette initiation was associated with a higher prevalence of current tobacco use (total effect: AOR = 3.84, 95% CI = 3.06 – 4.93; Q1). Controlling for harm perceptions, e-cigarette initiation was associated with a higher prevalence of current tobacco use (direct effect: AOR = 3.35, 95% CI = 2.67 – 4.21). The indirect effect of e-cigarette initiation on current tobacco use mediated through harm perceptions was calculated as the product of two coefficients: one for the association between e-cigarette initiation and harm perceptions (AOR = 2.21, 95% CI = 1.76 – 2.78), and the other for the association between harm perceptions and current tobacco use (AOR = 1.47, 95% CI = 1.33 – 1.61). The bootstrap confidence intervals derived from 1000 samples indicated that the indirect effect was significant (indirect effect: AOR = 1.19, 95% CI = 1.11 – 1.27; Q2).

Discussion

Causal mediation analysis is now routinely used by behavioral and health researchers to determine the mechanisms through which an exposure or treatment affects an outcome. Causal mediation effects can be used to answer important questions from health psychology research and help to provide important empirical evidence for policy decision-making and intervention development. After properly controlling for confounding, the total effect, which compares the outcome if everyone is exposed versus when everyone is not exposed, is of main research interest when evaluating the effects of exposures, including treatments in clinical trials, or interventions in policy studies. Researchers may further determine whether the exposure variable influences the mediator variable, which in turn influences the outcome variable. Researchers may also want to know what happens if participants’ exposure status stays constant and only the level of the mediator is changing (natural indirect effect). Alternatively, they may want to know what would be the average change in the outcome if participants’ status changes from unexposed to exposed, while holding the level of the mediator constant at the value naturally observed (natural direct effect). The answers to these “what-if” questions are key to scientific inquiries, which provide evidence for policy making decisions. In other cases, if the natural indirect effect varies by the exposure status one received (exposure-and-mediator interaction), then policy or intervention programs should target individuals differently.

In this tutorial, we described the potential outcomes framework, defined the causal effects in a hypothesized causal mediation chain, and stated the primary assumptions under which the causal effects could be estimated. Two commonly used and actively maintained R packages, mediation and medflex, were used to analyze a motivating example regarding tobacco use. We emphasized the importance of the temporal order of the study variables and of the elimination of confounding. We also interpreted the model estimates, a topic which has not been satisfactorily addressed in many published articles. In the context of a simple mediation model, where the exposure, mediator, and outcome variables are all binary, the results from these two packages are not directly comparable due to (1) the parametric models used and corresponding estimation method, (2) the scale the estimates are based on, and (3) the type of effects (marginal vs. conditional). Paying attention to these matters would allow researchers to properly apply these methods and to accurately interpret the results. In the motivating example, estimates from R lavaan are conditional effects on the odds ratio scale, those from R mediation are marginal effects on the risk difference scale, and those from R medflex are marginal or conditional effects on the odds ratio scale. These estimates are not directly comparable. Consistent with the literature (Rijnhart et al, 2021), in the absence of an exposure-by-mediator interaction, the conditional total effect of R lavaan and R medflex provide a similar conditional estimate of the total effect, but not the estimate of the indirect effect. Even though the results from R medflex indicated no statistically significant exposure-by-mediation interaction, the results from R mediation indicated the interaction was significant but small in size (1.6%).

Causal mediation analysis offers clear definitions, transparent assumptions, and meaningful interpretations that make it an attractive method for researchers. Using a causal inference approach, we estimated natural indirect effects (NIE), which capture the effect of X on Y that is due to X-induced changes in M. In addition, we estimated the exposure-mediator interaction, which refers to differences in the effects of X on Y that result from X-induced changes in M between the exposure group (NIE₁) and non-exposure group (NIE₀). This type of interaction can be estimated with causal mediation analysis, but it is assumed to be null in a traditional mediation analysis, which could lead to bias (VanderWeele, 2016). In addition, the indirect effect as defined in traditional mediation analysis as the product of the α and β coefficients cannot be interpreted as such when based on a nonlinear regression model.

We dichotomized the responses to the question about relative harm perceptions of e-cigarette use in this study. We recommend always justifying why one needs to dichotomize a variable and be aware of its potential adverse consequences, including loss of effect size and power, underestimating the extent of variation in outcome, and concealing any non-linearity in the relationship between variables (MacCallum et al., 2002). In this case, consistent with previous analyses of harm perceptions in the PATH Study, responses were dichotomized into “less harmful” and “same or more harmful.” Such dichotomization may be important for tobacco product regulatory purposes. For example, if the Food and Drug Administration decides to allow companies to market e-cigarette products with comparative risk claims it is likely they would require such claims to be simple and easy to communicate.

Missing data are the norm in empirical studies. When missing data occurs in either the exposure, mediator, or pre-exposure covariates, one may impute a small number (e.g., 5 – 10) of datasets, analyze each separately, and then use Rubin’s (1987) rule for combining the results (Coffman et al., 2020). Otherwise, analysis in both the mediation and medflex packages will be restricted to participants with complete data. The mediation package includes a pair of functions, mediations and amelidiate, that facilitate the analysis and combination of results from multiple imputed data sets. This procedure can be performed by manually running mediate on the imputed data, stacking the vectors of quantities from bootstrapping, and computing confidence intervals (Imai et al, 2011). Advances in bootstrap inference using multiple imputation has been proposed in Schomaker & Heumann (2018) but has not yet been implemented in any causal mediation analysis packages.

Lastly, the presentation from this tutorial is limited in a few ways. We focused on two commonly used R packages and ignored other software implementations for causal mediation methods. Further, neither the R mediation or medflex packages provide a sensitivity test for a simple mediation analysis when the exposure, the mediator, and the outcome variable are all binary. Alternatively, researchers may apply the methods from Hafeman (2011) to a setting where there are no measured covariates, and there exists only one unmeasured binary covariate that potentially confounds the relationship between the mediator and outcome variables. VanderWeele (2015) suggests a sensitivity test for each stratum of the exposure and mediator variables when the outcome is rare. Lindmark et al. (2018) developed the R sensmediation package to assess sensitivity to exposure-mediator, exposure-outcome, or mediator-outcome confounding. The results are based on probit regression models and the effects are based on the risk difference scale. We echo Vo’s suggestion (Vo et al., 2020) that more development in sensitivity analysis for causal mediation analysis with binary variables is needed. There is no single package that provides researchers with all the features they may need; however, we hope our tutorial provides useful guidelines on when to consider a causal mediation analysis, how to implement the methods, and how to properly interpret the results.

While beyond the scope of the current tutorial for binary variables, a causal mediation model could involve a higher level of complexity by including various types of mediator or outcome variables (e.g., multi-categorical) and/or multiple mediators (Tingley et al., 2011). In addition, intensively measured longitudinal data (e.g., from ecological momentary assessment), results in a time-varying mediator and/or outcome, which requires more complex analysis (e.g., Coffman et al., 2021). Importantly, similar to other analytical methods, in addition to the identification assumptions presented in this tutorial, researchers may also need to evaluate the appropriateness of other assumptions, for example, the assumption that the variables have no measurement error and/or are not misclassified. Hong (2015) and VanderWeele (2015) provide comprehensive reviews of these and additional assumptions underlying the use of causal mediation models.