Assessing Mediation in HIV Intervention Studies

SYNOPSIS

Objectives

A mediator is a psychosocial construct that is targeted by an intervention to bring about behavior change. Recent literature suggests that a widely used approach for assessing mediation, namely the causal steps method, can be severely statistically underpowered. This article describes three standard methods for assessing mediation: causal steps, difference in coefficients, and product of coefficients. We also demonstrate the use of asymmetric confidence limits (ACLs) in testing mediation.

Methods

We compared the results obtained by ACL construction with results obtained based on the causal steps and product of coefficients approaches to analyze data from the Seropositive Urban Men's Intervention Trial.

Results

ACL construction uncovered previously unidentified mediating factors. We also identified a marginally significant suppressor, which means that, with regard to this factor, the intervention had effects that were opposite from the desired direction.

Conclusions

ACLs are preferred for this type of analysis because of their statistical power and because they are informative regardless of whether the intervention has a significant effect on the outcome. Furthermore, ACLs present the size of the mediating effect rather than just a binary decision regarding significance.

Since the beginning of the human immunodeficiency virus (HIV)/acquired immunodeficiency syndrome epidemic in the early 1980s, public health practitioners have sought to effectively treat infected people and prevent the transmission of HIV to uninfected people. Whereas clinical trials are performed to improve treatment of the disease, the goal of HIV intervention research is to develop interventions that encourage participants to reduce or eliminate sexual or social behaviors that put themselves or others at increased risk for HIV infection.

THEORIES OF BEHAVIOR CHANGE

In the development of HIV prevention interventions, behavioral scientists have posited various factors associated with HIV transmission risk. Social cognitive theory, for example, asserts that by changing people's expectations regarding the consequences (or outcomes) of their behavior (e.g., “If I don't wear condoms, I might get HIV”), and their beliefs regarding their abilities to carry out the behavior change targeted by the intervention (e.g., “I can convince my partner to wear condoms”), people will begin to engage in safer sexual behavior.¹

In addition to elements specified by formal theories, other factors have been targeted by HIV interventions to influence behavior. For example, reductions in psychological distress or increases in social support, both of which may be consequences of participation in an intervention, could in theory serve to improve intervention effectiveness. In a recent trial testing an antianxiety, antidepressant medication to treat compulsive sexual behavior, the intervention effects were found to be partly explained by the medication's negative sexual side effects (e.g., impotence).²

Definition of mediation

Mediation (also called the “mediated effect,” “indirect effect,” “surrogate endpoint effect,” or “intermediate endpoint effect”) can be described as the mechanism by which an independent (or predictor) variable causes change in a dependent (or response) variable.³ The concept of mediation suggests that the effectiveness of a predictor variable is at least partially transmitted through an intermediate variable (or mediator) to the response variable. In short, the independent variable changes the mediator, which in turn changes the dependent variable.

Figure 1 illustrates the basic mediation model with a single mediator and a single outcome. In the context of an intervention trial, this illustration depicts the presumed direct and indirect causal pathways by which the intervention changes the outcome. Pathways α and β jointly form the indirect causal pathway. The direct pathway (τ') represents all other pathways, both known and unknown, by which the intervention changes the outcome.

Figure 1.

Basic mediation model

Mediation vs. confounding

A mediator is similar to a confounder in that it is a type of third, intermediate variable that, when entered into a regression model, modifies the association between the intervention and outcome. These two effects differ, however, in that a mediator (unlike a confounder) is specifically targeted by an intervention and is a hypothesized component of a causal sequence. Although a mediator is conceptually distinct from a confounder, the two effects can be statistically assessed in the same way.⁴

Mediation analysis in HIV interventions

In intervention trials, mediation analysis is the process by which one tests the significance of a theorized mechanism of behavior change. The objectives of a mediation analysis go beyond merely determining whether an intervention has an effect on an outcome to assessing how this effect is transmitted (i.e., the causal pathway). When an intervention aims to alter a set of beliefs, skills, or other factors, mediation analysis provides a method of determining which factors were responsible for the observed change in outcome (behavior) and which may have been superfluous. This information can then be used to reduce the intervention cost and participant burden by providing guidance on how to streamline the intervention without compromising its effectiveness. The findings of a mediation analysis may also be generalizable to other settings in which a similar intervention is being developed.⁵

Despite the fact that the theories upon which many HIV interventions are based are fairly well-developed, a rigorous assessment of these theoretical factors is rare. In fact, a recent review of a subset of psychological articles published from 1996 to 1999 that reported any mediation analyses revealed that less than a third reported a formal test of significance. Of the articles that did report a formal test of significance, the vast majority used the causal steps method developed by Judd and Kenny in 1981 and Baron and Kenny in 1986.³

This article describes three standard methods for assessing mediation: causal steps, difference in coefficients, and product of coefficients. The latter two methods yield identical results when the dependent variable is continuous and very similar results when the dependent variable is dichotomous and the sample size is large. We also demonstrate the use of asymmetric confidence limits (ACLs) in testing mediation.^6,7 We illustrate these methods using data from the Seropositive Urban Men's Intervention Trial (SUMIT).

METHODS FOR TESTING THE SIGNIFICANCE OF THE INDIRECT EFFECT

Causal steps

Judd and Kenny and Baron and Kenny published a now widely used set of criteria for assessing mediation.^5,8 These criteria are based on a mediation model that describes both the direct and indirect effects of an intervention (Figure 1). This approach stipulates that (1) a change in the level of the intervention be associated with a change in the mediator (α path), (2) a change in the mediator be associated with a change in the outcome of interest (β path), and (3) when the α and β paths are controlled, a previously significant relationship between the independent and dependent variable is neutralized (τ' path).⁸

Controversy: is a significant intervention effect needed for mediation analysis?

In Judd and Kenny and Baron and Kenny, the first step in mediation analysis was testing for a relationship between the independent variable (X) and the outcome (Y).^5,8 This step was described as necessary to establish that there was an effect to be mediated. However, MacKinnon et al. argued that a particular type of mediation, the suppression effect, could result in a nonsignificant direct effect of X on Y.⁴ Suppression, also called inconsistent mediation, is defined as the situation in which the direct and mediated effects of an independent variable on a dependent variable have opposite signs.⁹ Shrout and Bolger also recommended that mediation analyses not require a significant effect of X on Y, because for long-term processes, power may be particularly low for detecting an X-Y effect.¹⁰ They also discussed the suppression effect and advocated for a mediation analysis based on theoretical justification rather than an X-Y association.

Difference in coefficients (τ–τ')

Another method of assessing third variable effects is by regressing the outcome variable on the intervention independently and then in a model that adjusts for the mediator's influence. A statistical comparison of the coefficients obtained from both models reveals the degree to which a mediator explains the indirect causal pathway between an intervention and an outcome.

Consider the model equations:

where

• Y = dependent variable

• X = independent variable

• Z = mediator variable

• β_01,β₀₂ = intercepts

• τ = coefficient relating independent and dependent variables (unadjusted)

• τ' = coefficient relating independent and dependent variables adjusted for mediator effect

• β = coefficient of the mediator variable

• ε_1,ε₂ = unexplained variability (normally distributed)

Referring to Equations 1 and 2, the difference between the unadjusted and adjusted regression coefficients describing the association between the intervention and the outcome (τ and τ', respectively) provides a direct measure of the mediating effect of the third variable (Z).

Product of coefficients method

Referring back to Figure 1, the product of coefficients method involves dividing the product of the two coefficients describing the indirect causal pathway, α and β, by its estimated standard error and comparing this value to the standard normal distribution. MacKinnon, Lockwood, and Williams used simulation studies to show that the product of coefficients method consistently provided greater statistical power than mediation analyses based on the causal steps and the difference in coefficients approaches.⁷ The product of coefficients approach, however, has its limitations, due in part to the fact that the distribution of the product of regression coefficients α and β is not normally distributed, as explained later in this article. Thus, attempting to determine significance using a normal distribution approximation can still have relatively low power.

The distribution of the product of two normally distributed random variables, in this case αβ, is markedly non-normal. As a result, confidence intervals and tests based on the assumption of normality perform poorly. In light of this poor performance, an alternative method was presented in that same article; namely, confidence limits based on the asymmetric distribution of the product of α and β.⁷

ACLs method

Meeker et al. published tables of critical values for the distribution of the product of two normal random variables, and provided a FORTRAN algorithm to compute the cumulative density function of the product distribution.¹¹ These values can be used to estimate statistical power and type I error, and construct confidence limits for the indirect effect based on the product of coefficients. Following the steps outlined in Figure 2, ACLs are computed using the values contained in the published tables.

Figure 2.

Steps for constructing ACLs

^aDerived from: Meeker WQ, Cornwell LW, Aroian LA. The product of two normally distributed random variables. Selected tables in mathematical statistics. Providence (RI): W.J. Kennedy and R.E. Odeh; 1980.

^bSobel ME. Asymptotic confidence intervals for indirect effects in structural equation models. Sociological Methodology 1982;13:290-312.

ACL = asymmetric confidence limit

LCL = lower confidence limit

UCL = upper confidence limit

Also, MacKinnon et al. recently published an article describing a program, PRODCLIN, written for the SAS®, SPSS®, and R programming languages that computes confidence limits for the product of two normal random variables.¹² A significant mediating effect is said to be present when this interval does not contain zero.

METHODS

SUMIT example

SUMIT tested the effectiveness of a peer-led behavioral intervention for gay and bisexual men living with HIV. A full description of the study has been published elsewhere.¹³ For this analysis, the behavioral outcome under investigation was a dichotomous variable for unprotected insertive anal intercourse (UIAI) with a seronegative or serostatus unknown partner in the previous 90 days. The objective of the mediation analysis was to assess the importance of eight potential mediators in the causal pathway by which the SUMIT behavioral intervention effects reported UIAI. Each potential mediator is described in more detail in O'Leary et al.;¹⁴ we briefly describe those used in the mediation analysis for this article in Figure 3.

Figure 3.

Description of potential mediators in the Seropositive Urban Men's Intervention Trial

^aItems for each scale (with the exception of sexual compulsivity) were measured on either a four- or five-point Likert scale ranging from “strongly agree” to “strongly disagree.” For sexual compulsivity, participants indicated how “like me” each item was. The overall score is the mean of the individual items.

^bKalichman SC, Rompa D. Sexual sensation seeking and Sexual Compulsivity Scales: reliability, validity, and predicting HIV risk behavior. J Pers Assess 1995;65:586-601.

^cKalichman SC, Rompa D. The Sexual Compulsivity Scale: further development and use with HIV-positive persons. J Pers Assess 2001;76:379-95.

^dAnxiety, hostility, and depression were measured using BSI subscales. Source: Derogatis LR, Melisaratos N. The Brief Symptom Inventory: an introductory report. Psychol Med 1983;13:595-605.

HIV = human immunodeficiency virus

BSI = Brief Symptom Inventory

Using ACLs, we assessed the mediating effects of eight psychosocial factors on reported UIAI at three-month post-intervention follow-up. We also compared these findings with those obtained using the causal steps and product of coefficients methods.

RESULTS

The direct effect of the intervention on the UIAI outcome was not significant. Therefore, according to the originally published criteria, performing mediation analysis is inappropriate. As a result, the original investigation only examined correlates of behavior change.¹⁴

By comparison, using a less stringent definition of mediation that permits a nonsignificant direct effect and applying the product of coefficients method that was shown in simulation to be the test with the highest power among the three standard methods, we obtained the results in the Table. Using Sobel's standard error estimator and the standard normal distribution, only serostatus assumption appears to significantly mediate the effect of the intervention on UIAI.¹⁵ In other words, the SUMIT intervention appears to decrease the assumptions that participants made with regard to potential partners' serostatus. This decrease, in turn, resulted in a decrease in reported UIAI.

Table.

Analysis of eight potential mediators in the Seropositive Urban Men's Intervention Trial with outcome unprotected insertive anal intercourse in the previous 90 days

^aA comparison of the signs of the coefficients for the direct (τ') and indirect (αβ) effects. Similar signs indicate mediation (M). Opposite signs indicate effect suppression (S). (Tzelgov J, Henik A. Suppression situations in psychological research: definitions, implications, and applications. Psychol Bull 1991;109:524-36.)

^bResults of significance test using a normal distribution approximation (α=0.05; z=1.96)

^cStatistically significant at p<0.05

α = regression coefficient for intervention effect on mediator controlling for baseline mediator value

σ_α = standard error of α

β = regression coefficient for relation between mediator and outcome controlling for intervention and baseline outcome value

σ_β = standard error of β

αβ = product of α and β

τ' = regression coefficient for intervention effect on outcome controlling for baseline and three-month mediator values

SE = standard error

z = standard z-score

LCL = lower confidence limit

UCL = upper confidence limit

M = mediation

S = suppression

Using the same data, we constructed ACLs using the mediating and outcome variables. First, we estimated the parameters, α, σ_α, β, σ_β, and τ'. We determined the type of intervening variable effect (mediator or suppressor) by comparing the signs of αβ and τ'.⁴ This analysis revealed two significant mediators, namely serostatus assumption and hedonistic outcome expectancies. So, the intervention effect can be partially explained by its effect on a participant's tendency to make assumptions about a potential partner's serostatus and the participant's belief that condom use will decrease sexual pleasure.

Also, because their upper confidence limits were very close to zero, we identified sexual compulsivity as a marginally significant suppressor (a factor that was influenced by the intervention and caused an increase in the behavior that it was supposed to decrease) and depression as a marginally significant mediator. Thus, the intervention had an unexpected effect on sexual compulsivity that, rather than decreasing reported UIAI, increased it. Similarly, although only marginally significant, the effect of the intervention on reported UIAI is partially explained by it successfully decreasing depression (Table).

DISCUSSION

Unlike a traditional analysis of intervention effectiveness, which only tests whether a change occurred as the result of an intervention, mediation analysis seeks to determine how this effect was transmitted to an outcome. By including an analysis of suppressive effects, mediation analysis can also identify factors that may affect the outcome in unexpected or counterintuitive ways. In so doing, mediation analyses provide richer information for understanding the results of current studies and for improving future intervention programs.

Our use of ACLs was motivated by a previous analysis of SUMIT data that failed to identify mediating effects through strict adherence to the causal steps method.¹⁴ Using less stringent criteria and constructing ACLs, however, uncovered both mediating and suppressive effects. The moderately significant suppressive effect may help explain the overall null results obtained in the intervention trial.

CONCLUSIONS

The use of ACLs to test the significance of a mediating effect has several practical and statistical benefits. First, ACLs are statistically more precise, because the underlying distribution of the mediating effect is better approximated by an asymmetric distribution. Also, ACLs present the size of the mediating effect rather than just a binary decision of significance. By examining the width of the interval, the investigator has a better sense of the uncertainty in estimating the mediating effect.