Multimediation Method With Balanced Repeated Replications For Analysis Of Complex Surveys

Abstract

We propose a method suitable for analysis of cross-sectional studies with complex sampling and continuous variables. The method consists of R+4 steps, where R denotes the number of replications. In the first R+1 steps, the main and R replicate weights are used (one at a time) to estimate the product of coefficients for all mediation effects using a structural equation model. In step R+2, the standard errors of these estimates are computed via balanced repeated replications. In step R+3, the raw p-values corresponding to mediation effects are computed based on the generalized Sobel’s tests. In the final step, R+4, the p-values are adjusted for multiplicity and statistical inferences regarding mediation effects are drawn. To illustrate the approach we examined significance of attitudes toward smoking bans as mediators in the association between smoking restrictions at work and nicotine dependence among male daily smokers.

1. Introduction

The main goal of the causal mediation analysis with multiple mediators is to assess the effect of each mediator as a potential factor intervening between the relationship of an independent variable and a dependent variable (Hayes, 2009; Hayes & Rockwood, 2017; MacKinnon, 2008; Preacher & Hayes, 2008). While the mediation analysis has been proposed and discussed primarily in the context of experimental studies (Danaher, Smolkowski, Seeley, & Severson, 2008; Emsley, Dunn, & White, 2010; Kelly, Hoeppner, Stout, & Pagano, 2012), mediation analysis techniques found wide range of applications in observational studies (Aryee, Budhwar, & Chen, 2002; Bravo, Pearson, & Henson, 2017; Ismail, 2017). For example, two recent cross-sectional studies examined (1) problem-focused thoughts, counterfactual thinking, repetitive thoughts, and anticipatory thoughts as mediators in the association between depressive symptoms and drinking motives among U.S. college student drinkers (Bravo et al., 2017), and (2) posttraumatic stress disorder symptoms, depression symptoms, and alcohol use symptoms as mediators in the association between deployment stressors and post-deployment suicidal ideation among veterans (Gradus, Street, Suvak, & Resick, 2013).

In this article, we propose a generalization of the multi-mediation methodology for analysis of survey data collected using complex sampling, e.g., complex sampling is common in national surveys. One approach to adjust for unique features of complex sampling is to incorporate the survey weights and estimate variance of estimators using the balanced repeated replication (BRR) method (Fay & Train, 1995; Judkins, 1990; Lohr, 2009; Wolter, 2007). For example, this strategy is recommended by the U.S. Census Bureau for analysis of the data from all Current Population Survey (CPS) Supplements (U.S. Department of Commerce & U.S. Census Bureau, 2017). The survey weights help assure that the national- and state-level estimates computed using the CPS data are correct for multiple labor-force characteristics. Specifically, the survey weights are computed to adjust for nonresponse, sampling with unequal probabilities, monthly rotation, over-sampling of some racial/ethnic minority groups and other features of the CPS (U.S. Bureau of Labor Statistics & U.S. Census Bureau, 2006). Prior research illustrated that ignoring the survey weights in the statistical analysis could affect the research findings (Ha & Soulakova, 2018). For example, the estimates of smoking-related characteristics derived based on the TUS may not be valid for the civilian non-institutional adult (18+ years old) population if the survey weights are ignored in the statistical analysis.

The survey weights are commonly supplied together with the original data (possibly in a separate data file), e.g., the file with 160 replicate weights corresponding to the 2014-15 CPS Tobacco Use Supplement (TUS) data are available online for public use (U.S. Department of Commerce & U.S. Census Bureau, 2016). Incorporating the weights and BRR in statistical computing is straight-forward in multiple software packages, e.g., computing using SAS® Survey Package (SAS Institute Inc., 2016) has been discussed (Ha and Soulakova 2017, 2018).

The paper is outlined as follows: in Section 2, we review the traditional multi-mediation method. In Section 3, we propose the generalized approach where the primary goal is to identify all mediators (out of a set of considered mediators) which significantly influence the association between an independent and dependent variable while controlling for multiple covariates and adjusting for the features of the complex design. In Section 4, we illustrate the proposed approach with respect to a study of attitudes toward smoking bans, smoking restrictions at work and nicotine dependence in male daily smokers. Finally, in Section 5, we provide some discussion on limitations and future research.

2. Traditional Multi-Mediation Model

We consider a set of K mediators M₁,…,M_K, the independent variable X and dependent variable Y. Furthermore, we consider L control factors Z₁,…,Z_L. In a case where all variables are continuous, the following approach can be used to assess the goals. Suppose we have a cohort of n subjects and the data set is represented by D = (y,m₁,…,m_K, z₁,…,z_L,x) of the dimension n × (K + L + 2), where y = (y₁,…,y_n)^T, m_k = (m_k1,…,m_kn)^T for k = 1,2,…,K, z_l = (z_l1,…,z_ln)^T for l = 1,2,…,L, and x = (x₁,…,x_n)^T. Then the multi-mediation model can be expressed as the following structural equation model (MacKinnon 2008; Preacher and Hayes 2008):

$$\begin{gathered} Y=v_0+a_{0}X+b_1{M}_1+\cdots +b_K{M}_K+{\mathbf{d}}_0\mathbf{Z}+{\epsilon }_0, \\ {M}_1=v_1+a_{1}X+{\mathbf{d}}_1\mathbf{Z}+{\epsilon }_1, \\ {M}_2=v_2+a_{2}X+{\mathbf{d}}_2\mathbf{Z}+{\epsilon }_2, \\ \cdots \\ {M}_K=v_K+a_{K}X+{\mathbf{d}}_K\mathbf{Z}+{\epsilon }_K, \end{gathered}$$ (1)

where Z = (Z₁,…,Z_L)^T is a 1 × L vector of covariates; ε₀,…,ε_K denote the disturbance terms; v₀,…,v_K denote the intercepts; a₀,…,a_K, b₁,…,b_K and d_m1,…,d_mL (for m = 0,1,…,K) denote the model slopes. We assume that the following conditions are satisfied:

1. (Z₁,…,Z_L,X,ε₀,…,ε_K)^T follows a multivariate normal distribution,

2. (Z₁,…,Z_L,X)^T and (ε₀,…,ε_K)^T are independent, and

3. ε₀ and (ε₁,…,ε_K)^T are independent (Bollen, 1989; Briggs, 2006; Kaplan, 2009; Preacher & Hayes, 2008; Raykov & Marcoulides, 2006).

The model is an extension of the two-mediator model (Briggs, 2006) and allows assessing K mediators simultaneously (treating mediators as dependent). We also note that the above model (1) is different from the causal-inference approach discussed by VanderWeele and Vansteelandt (2014). The latter approach incorporates unmeasured confounders and allows for interactions between the independent variable and a mediator as well as interactions between the mediators (VanderWeele & Vansteelandt, 2014).

Because the product of coefficients a_kb_k (k = 1,2,…,K represents the mediation effect of the mediator M_k (Baron & Kenny, 1986; Briggs, 2006; Preacher & Hayes, 2008; Sobel, 1982), the model-based estimates can be used to assess different aims regarding the significance of the mediation effects (Aryee et al., 2002; Ismail, 2017; Rucker, Preacher, Tormala, & Petty, 2011). For example, one can test for the significance of the effect of M_k via testing the null hypothesis H_0k:a_kb_k = 0 against the alternative hypothesis H_1k:a_kb_k ≠ 0 (Doaei, Rezaei, and Khajei 2011; Ismail 2017; MacKinnon 2008, Page 108).

3. Proposed Multi-Mediation Method for Complex Data with Replicate Weights

Let w₀ = (w₀₁,…,w_0n)^T and {w_r = (w_r1,…,w_rn)^T, r = 1,2,…,R} denote, respectively, the array of main weights and the set of R arrays of replicate weights. For convenience of wording, we will refer to the arrays of main and replicate weights as “main weights” and “replicate weights”, respectively. Then the proposed method includes the following R + 4 steps:

Step 1 is to estimate the product of coefficients in the multi-mediation model using the main weights. First, we fit the multi-mediation model (1) to the dataset D using the main survey weights w₀. Next, we note the estimated coefficients ${\widehat{a}}_{k0}$ and ${\widehat{b}}_{k0}$ (k = 1,2,…,K), and compute their products ${\widehat{a}}_{k0}{\widehat{b}}_{k0}$ (k = 1,2,…,K). The estimate ${\widehat{a}}_{k0}{\widehat{b}}_{k0}$ provides a point estimate for the mediation effect of M_k, but it differs from the one based on the multi-mediation model (1) because it is based on the main survey weights.

Step s (for s = 2,3,…,R + 1) is to estimate the product of coefficients in the multi-mediation model using the (s − 1) – th replicate weights. This step is similar to Step 1 but it incorporates the replicate weights w_s−1 (s = 2,3,…,R + 1) instead of the main weights w₀. This step results in the estimates ${\widehat{a}}_{k,s-1}{\widehat{b}}_{k,s-1}$ (k = 1,2,…,K; s = 2,3,…,R + 1). For example, Step 2 (i.e., s = 2) incorporates the replicate weights w₁ and results in the estimates ${\widehat{a}}_{k,1}{\widehat{b}}_{k,1}$ (k = 1,2,…,K).

Step R + 2 is to compute the standard error of the product of coefficients estimate using the BRR approach. Use the estimates obtained in Steps 1 through R + 1 to calculate the standard error using the BRR with the Fay correction formula (Fay & Train, 1995; Judkins, 1990; Wolter, 2007).

$${\mathrm{SE}}_{\mathrm{BRR}}({\widehat{a}}_{k0}{\widehat{b}}_{k0})=\sqrt{\frac{1}{R{(1-f)}^2}{\sum }_{r=1}^R{({\widehat{a}}_{kr}{\widehat{b}}_{kr}-{\widehat{a}}_{k0}{\widehat{b}}_{k0})}^2},k=1,2,\dots ,K,$$

where f (0 ≤ f < 1) denotes the Fay factor. The special case with f = 0 corresponds to the original BRR method. The default value of Fay factor (when Fay option is specified in the BRR statement) in SAS is 0.5 (SAS Institute Inc., 2016). This is also the recommended value for analysis of TUS (U.S. Bureau of Labor Statistics & U.S. Census Bureau, 2006).

Step R + 3 is to test individual hypotheses. Consider the family of K hypothesis problems, where H_0k:a_kb_k = 0 is tested against the alternative hypothesis H_1k:a_kb_k ≠ 0; k = 1,2,…,K. To test these hypotheses, compute values for K test statistics:

$$T_k=\frac{{\widehat{a}}_{k0}{\widehat{b}}_{k0}}{{\mathrm{SE}}_{\mathrm{BRR}}\left({\widehat{a}}_{k0}{\widehat{b}}_{k0}\right)},k=1,2,\dots ,K.$$

The asymptotic distribution of each statistic (under the corresponding null hypothesis) can be assumed to be standard normal (Sobel, 1982; U.S. Bureau of Labor Statistics & U.S. Census Bureau, 2006). Thus, we can compute the two-sided p-values, denoted by p_k (k = 1,2,…,K), via p_k=2{1 − Φ[|t_k|]} for all k = 1,2,…,K, where Φ(·) denotes the standard normal cumulative distribution function and |t_k| is the absolute value of the computed value of the test statistic T_k.

Step R + 4 is to assess the significance of each mediation effect while controlling the overall error rate. Several multiple comparison strategies can be used to adjust the p-values so that the overall error rate is controlled at level α when testing the hypothesis family (Holm, 1979; Hsu, 1996). The simplest methods include fixed-sequence testing, i.e., testing the completely ordered hypotheses, the Bonferroni adjustments or Holm adjustments (Holm, 1979; Hsu, 1996). Fixed-sequence testing allows using the original p-values; however, the order of hypotheses should be pre-specified in advance. This strategy is highly beneficial when mediators can be pre-ranked according to their importance in the study. For example, suppose we have two mediators: Mediator A – smoker’s attitude toward smoking when children are present, and Mediator B – smoker’s attitude toward smoking when other people are present. We might be more interested in assessing the effect of Mediator A than Mediator B and assess the effect of Mediator B only if the effect of Mediator A is shown to be significant. In this setting, it would be logical to use the fixed-sequence method with the order: (1) hypothesis for Mediator A, and (2) hypothesis for Mediator B. In a fixed-sequence testing, a hypothesis is tested only if all prior hypotheses were rejected; the testing stops as soon as a hypothesis is accepted, e.g., if the null hypothesis for Mediator A is accepted, then the null hypothesis for Mediator B is not tested.

Holm adjustments (Holm, 1979) and Bonferroni adjustments do not require pre-ordering the hypotheses as is in the fixed-sequence testing. Here we outline Holm adjustments because this approach is usually less conservative than Bonferroni adjustments (Soulakova, 2009; P. H. Westfall, Young, & Wright, 1993): first, order the p-values p_k (k = 1,2,…,K) as p₍₁₎ ≤ p₍₂₎ ≤ ⋯ ≤ p_(K) and the corresponding null hypotheses as H₀₍₁₎,H₀₍₂₎,…,H_0(K). Then compute the adjusted p-values, denoted by p_a(k) (k = 1,2,…,K) using the formulas:

$${\mathrm{p}}_{a(1)}=K{\mathrm{p}}_{(1)},{\mathrm{p}}_{a(2)}=\max \{{\mathrm{p}}_{a(1)},(K-1){\mathrm{p}}_{(2)}\},\dots ,{\mathrm{p}}_{a(k)}=\max \{{\mathrm{p}}_{a(K-1)},{\mathrm{p}}_{(K)}\}.$$

If any adjusted p-value exceeds 1 then it should be set to 1 so that it complies with the definition of a p-value. Finally, the adjusted p-values are compared to the level α. All null hypotheses with corresponding p-values satisfying p_a(k) ≤ α where k ∈ {1,2,…,K} are rejected and the corresponding mediation effects are concluded to be significant.

4. Applications of the Proposed Method: Smoking Restrictions at Work and Nicotine Dependence in non-Hispanic White Men Who Smoke Daily

The study goal was to determine whether each of the two mediators, M₁: supporting smoking bans in adult-exclusive areas, and M₂: supporting smoking bans in kid-related areas influence the association between X: smoking restrictions at work and Y: the average number of cigarettes smoked per day among non-Hispanic White male daily smokers. Adult-exclusive areas included bars, cocktail lounges, and clubs; kid-related areas included outdoor children’s playgrounds, sports fields, and cars when children were present inside. Both mediators were represented by a scale from 0 (no support) to 10 (high support). The independent measure X: smoking restrictions at work, was also represented by a scale from 0 (no restrictions, smoking is allowed in all public areas) to 10 (high restrictions, smoking is banned in all public places). Appendix A outlines how all scale measures were defined. The average number of cigarettes smoked per day (Y) is a common nicotine dependence measure. All measures were treated as continuous. We controlled for smokers’ age and number of children under the age of 18 living in the same household. Figure 1 depicts the hypothesized two-mediator model.

Figure 1.

Hypothesized Two-Mediator Model

We used the 2014-15 TUS (U.S. Department of Commerce & U.S. Census Bureau, 2016) reports for employed adults. The surveys were administered via phone (56%) and in-person (44%) interviews. The smokers (n = 2,260; the projected population count N = 3,294,568) were on average 41 years old (SE=0.3) and had 1 child under the age of 18 living in the same household (SE<0.1). Computing was done using developed SAS macros which incorporated SAS PROC CALIS and PROC IML, depicted in Appendix B. We used R=160 replicate weights and Fay’s factor of 0.5 (U.S. Department of Commerce & U.S. Census Bureau, 2016). Therefore, in our case, the method had 164 steps.

The results were as follows: Step 1 resulted in the point estimates ${\widehat{a}}_{10}{\widehat{b}}_{10}=-0.0156$ and ${\widehat{a}}_{20}{\widehat{b}}_{20}=0.0004$. Step 162 resulted in: ${\mathrm{SE}}_{\mathrm{BRR}}\left({\widehat{a}}_{10}{\widehat{b}}_{10}\right)=0.0073$ and ${\mathrm{SE}}_{\mathrm{BRR}}\left({\widehat{a}}_{20}{\widehat{b}}_{20}\right)=0.0045$. Step 163 resulted in the (unadjusted) p-values p₁ = 0.033 and p₂ = 0.937. The final step, 164, resulted in the Holm-adjusted p-values p₍₁₎ = 0.066 and p₍₂₎ = 0.937; thus, none of the mediation effects is significant.

We note that if the survey weights are (incorrectly) ignored then the estimated quantities would be as follows: ${\widehat{a}}_{10}{\widehat{b}}_{10}=-0.0170$ and ${\widehat{a}}_{20}{\widehat{b}}_{20}=0.0018$, $\mathit{SE}\left({\widehat{a}}_{10}{\widehat{b}}_{10}\right)=0.0066$ and $\mathit{SE}\left({\widehat{a}}_{20}{\widehat{b}}_{20}\right)=0.0033$, the corresponding (unadjusted) p-values would be p₍₁₎ = 0.010 and p₂ = 0.598, and the Holm-adjusted p-values would be p₍₁₎ = 0.020 and p₍₂₎ = 0.598. Hence we would conclude that M₁ is significant while M₂ is not. In addition, if the study goal was different and stated that assessing the effect of M₁ is of primary importance, while assessing the effect of M₂ is of secondary importance, then we could use the fixed-sequence testing. In the latter case the conclusion would be “there is a significant effect of M₁”. However, the testing strategy may not be picked after the data have been analyzed. Thus, despite this observation, we conclude that none of the mediators is important in the association between smoking restrictions at work and nicotine dependence among non-Hispanic White men who smoke daily.

5. Limitations and Future Directions

One of the limitations of the proposed method is that we assume that each test statistic (under the corresponding null hypothesis) follows the standard normal distribution. While in studies dealing with large samples this assumption is likely to hold, the tests might not be valid if the assumption does not hold. In the latter cases generalizations of alternative mediation methods to handle survey weights would be highly practical (Carpenter, Goldstein, & Rasbash, 2003; MacKinnon, 2008; Sobel, 1982; Valente, Gonzalez, Miočević, & MacKinnon, 2016). Another limitation is that the proposed approach incorporates BRR for computing standard errors. However, some modifications of the proposed approach, e.g., incorporating Taylor linearization to compute standard errors (Wolter, 2007), are expected to be straight-forward. Generalizations to other types of multi-mediation problems, such as the ones incorporating interactions among mediators, sequential and parallel mediators (Hayes, 2015), as well as categorical variables, require additional methodological developments. In this paper we limited to Holm adjustments for multiplicity, because it is one of the simplest methods to control the family-wise error rate. To handle logically ordered or partially-ordered hypotheses, one could use a gatekeeping method (Dmitrienko, Offen, & Westfall 2003; Westfall & Krishen 2001; Dmitrienko, Soulakova, & Millen 2011). In addition, when the number of tested hypotheses (mediators) is large, it could be more reasonable to apply methods that control the false discovery rate – the proportion of incorrectly rejected null hypotheses (Benjamini & Hochberg, 1995). We used SAS to perform all computing. Nonetheless, one can use R or another software package that fits a structural equation model. For example, one can use SEM package for R (Fox, 2006) to fit the model with multiple mediators for weighted data and then use the output to compute standard errors via BRR and test the hypotheses of significance of mediators.

In addition, interpretation of mediation analysis results obtained using survey data should be done with caution: no cause-effect relationships could be claimed.

APPENDIX A. Definitions of the Mediators and Independent Variable

Mediator measure – supporting smoking bans in adult-exclusive areas – was defined using responses to two survey items depicted in Table A1. Higher scores correspond to greater support for smoking bans.

Table A1.

Supporting Smoking Bans in Adult-exclusive Areas

		Inside casinos, do you think that smoking should be allowed in all areas, allowed in some areas, or not allowed at all?
		Allowed in all areas	Allowed in some areas	Not allowed at all
Inside bars, cocktail lounges, and clubs, do you think that smoking should be allowed in all areas, allowed in some areas, or not allowed at all?	Allowed in all areas	0.0	2.5	5.0
	Allowed in some areas	2.5	5.0	7.5
	Not allowed at all	5.0	7.5	10.0

Mediator measure – supporting smoking bans in kid-related areas – was defined as follows: first, respondents were asked the question, “Inside a car, when there are other people present, do you think that smoking should always be allowed, allowed under some conditions, or never be allowed?” If the respondent answered that smoking should be allowed always or under some conditions, then he/she was asked the follow-up question, “If children are present inside the car, do you think that smoking should always be allowed, allowed under some conditions, or never be allowed?”. For those respondents who answered that smoking should never be allowed when other people are present, we imputed the answer “smoking should never be allowed” to the follow-up question.

Table A2 presents the survey items and the corresponding scores, where higher scores correspond to stronger levels of support for smoking bans.

Table A2.

Supporting Smoking Bans in Kid-related Areas

		If children are present inside the car, do you think that smoking should always be allowed, be allowed under some conditions, or never be allowed?
		Always be allowed	Be allowed under some conditions	Never be allowed
On outdoor children’s playgrounds and outdoor children’s sports fields, do you think that smoking should be allowed in all areas, allowed in some areas, or not allowed at all?	Allowed in all areas	0.0	2.0	3.0
	Allowed in some areas	1.0	2.5	5.0
	Not allowed at all	1.5	3.0	10.0

The independent variable – i.e., smoking restrictions at work – was defined as follows: first respondents were asked “Is smoking restricted in any way at your place of work?” If the respondent answered “Yes” then he/she was asked two follow-up questions, depicted in Table A3. If the respondent answered “No” then he/she was not asked the follow-up questions and we imputed that “smoking was allowed in all public areas” as the answer to each follow-up question.

Table A3 presents the definition of this variable; higher scores correspond to higher levels of smoking restrictions at place of work.

Table A3.

Smoking Restrictions at Work

		Which of these best describes the smoking ban at your place of work for indoor work areas?
		Allowed in all public areas	Allowed in some public areas	Not allowed in any public areas
Which of these best describes the smoking policy at your place of work for indoor public or common areas, such as lobbies, rest rooms, and lunchrooms?	Allowed in all public areas	0.0	2.0	4.0
	Allowed in some public areas	1.0	2.5	5.0
	Not allowed in any public areas	1.5	3.0	10.0