Using novel approaches to evaluate behavioral interventions: Overlooked significant HIV prevention effects in HPTN 015 Project EXPLORE

Abstract

Background:

Mediated and moderated processes that lead to intervention efficacy may underlie results of trials ruled as non- efficacious. The overall purpose of this study was to examine such processes to explain the findings of one of the largest, rigorously conducted behavioral intervention randomized controlled trials, EXPLORE.

Methods:

4,295 HIV-negative MSM in the U.S. were randomized in a two-armed trial. Participants completed follow-up and an HIV test every 6 months up to 48-months. We used multiple and causal mediation analysis to test five mediators, including safer sex self-efficacy and condomless receptive anal sex with HIV-positive or status unknown partners on our primary outcome (HIV seroconversion). We also examined whether intervention effects on the mediators would be moderated by robust correlates of HIV among MSM, including stimulant use.

Results:

There were significant effects of the intervention on all hypothesized mediators. Stimulant use moderated the effect on condomless receptive anal sex. In stratified, multiple mediation models we found that among MSM with low stimulant use, the intervention significantly prevented HIV by reducing condomless receptive anal sex with HIV- positive or status unknown partners. Among MSM with higher stimulant use, there were no indirect effects of the intervention on HIV through any of the hypothesized mediators.

Conclusion:

The results suggest that the null effect found in the original EXPLORE trial might have occurred as a function of previously unexplored mediated and moderated processes. This study illustrates the value of testing mediated and moderated pathways in randomized trials, even in trials ruled as non-efficacious.

Introduction

For over 30 years, HIV primary prevention scientists have developed and evaluated behavioral interventions to reduce HIV among populations at elevated risk. These interventions are often time- and resource-intensive, and have produced mixed findings, particularly for biological outcomes (e.g., HIV incidence).^1-3 Upon finding a statistically significant effect on the primary outcome (“efficacious”), time- and financially-intensive interventions may be perceived as difficult to implement, especially where resources may be limited. Upon finding a null effect (“non-efficacious”), interventions may be unnecessarily discarded or modified with additional resources. In both situations, the data from these trials are not typically analyzed further. Moving beyond efficacy, understanding mediators and moderators of existing behavioral HIV prevention interventions – even in null trials – may help explain mixed results, and can inform how to pare down or ramp up interventions to their most effective elements, and which subgroups must be targeted to produce briefer, less expensive, and more impactful interventions.⁴ To this end, in the current paper we examine mediation and moderated mediation effects in one of the largest, rigorously conducted, HIV prevention behavioral intervention trials- HPTN 015 Project EXPLORE, or simply, “EXPLORE” conducted from 1999 to 2004. Although the HIV epidemiological context has changed since conducting EXPLORE, the methods and processes that we evaluate in the current manuscript remain highly important for today’s clinical trials.⁵

In the EXPLORE trial, a total of 4,295 HIV-negative men who have sex with men (MSM) in six cities across the United States were randomized to the theory-based behavioral risk reduction EXPLORE intervention or a control intervention (itself an evidence-based intervention, CDC’s Project RESPECT model). Participants completed a follow-up assessment and an HIV test every 6 months (for 30 months minimum and 48 months maximum). The primary outcome of the study was HIV seroconversion. In addition, three theory-based mediators were measured: safer sex self-efficacy, safer sex communication skills, and perceived norms for safer sex. The intervention also addressed substance (alcohol and drug) use, particularly with respect to substance use in the context of sex (before or during sex). Thus, the intervention may have reduced HIV incidence through several indirect pathways: safer sex self-efficacy, safer sex communication skills, perceived norms for safer sex, alcohol or drug use before/during sex, and condomless receptive anal sex. MSM were recruited who varied in a number of ways including depression, childhood sexual abuse, and substance use. Each of these has been shown to have a robust association with HIV in this population, in addition to being theoretically important joint co-factors with HIV (i.e., Syndemic Theory). However, none of these factors were targeted by the intervention and were therefore not expected to change as a function of the intervention. Therefore, we examined potential subgroup differences across these variables (moderators) in the indirect/mediated pathways (also known as moderated mediation or conditional indirect effects). Such knowledge would inform whether MSM who experience depressive symptoms, have experienced childhood sexual abuse, and/or who or who do not report higher stimulant use may need additional or targeted intervention support to effectively reduce risks for HIV via specific mediators.

The primary outcome analysis of EXPLORE found that the intervention led to an 18·2% lower HIV incidence rate compared to control, which was attenuated to 15·7% after accounting for baseline covariates. However, the difference between groups was non-significant (i.e., the 95% CI ranged from −4·7 to 36·0), and the results did not meet the specific benchmarks for efficacy that were defined at the outset of the trial.⁶ Thus, the EXPLORE behavioral intervention was ruled as non-efficacious. Experts in mediation now understand that even in null trials, significant mediated effects can be present.^7,8 It is often assumed that an overall significant effect must be found to examine mediation, but this is not the case, as there are conditions with significant mediation even in the context of nonsignificant intervention effects (e.g., when the indirect effect equals the total effect with a large sample size and small effects).⁸ In such situations, efficacy through an indirect process is essentially masked – and the intervention is rendered as non-efficacious – despite the fact that it might facilitate behavior change through specific mechanisms and/or under specific conditions. Mediation analysis also help understand why an intervention may not be effective by providing evidence about hypothesized mediators that were not actually changed by an intervention, and also hypothesized mediators that were not actually related to the dependent variable.

Further, potential moderators must also be examined to fully understand efficacy in an intervention trial. Outcome analyses of the EXPLORE trial evaluated three potential moderators (or “effect modifiers”) of the intervention effect on the primary and secondary outcomes, alcohol use, non-injection drug use, and depression, but none of these were statistically significant.⁶ However, in the context of understanding mediation, instead of a moderator changing a direct effect of the intervention on the outcome, a moderator might dampen or exacerbate the role of a mediator variable in explaining efficacy. For example, while a behavioral intervention may show statistically significant improvements in self-efficacy across an entire sample, a significant moderated (interaction) effect may show that this effect is weaker among a particular sub-population (e.g., those with depression). Previous behavioral intervention studies have demonstrated similar moderated indirect effects (moderated mediation), including in both HIV-related and non-HIV-related interventions.^9-14 For example, in one study, an injection risk reduction intervention for women in sex work who inject drugs demonstrated reductions in receptive needle sharing via improvements in safer injecting self-efficacy (i.e., the mediator), particularly among women who reported police syringe confiscation (i.e., the moderator).⁹ In another example, a physical activity counseling intervention increased physical activity via improved motivation (i.e., the mediator), particularly among participants who had higher baseline self-determined physical activity motivation (i.e., the moderator).¹⁰

The overall objective of this paper is to examine mediation and moderated mediation of efficacy in the EXPLORE trial. In the current paper, we use a more comprehensive and novel approach than one previously published in 2013¹⁵. In the 2013 paper by Eaton and colleagues, the investigators conducted a mediation analysis of EXPLORE focusing only on unprotected anal sex as a potential mediator of efficacy; this analysis did not test any other potential mediators or any moderators, and did not examine unprotected anal sex by partner type (e.g., with HIV-positive partners). Our primary aims are twofold: 1) Examine potential indirect pathways driving efficacy of the intervention; and 2) Examine whether effects of the intervention on hypothesized mediators depend on different potential moderators.

Methods

Study Design

EXPLORE was a two-armed trial tested among MSM in Boston, Chicago, Denver, New York, San Francisco, and Seattle from 1999 to 2004 (n=4,295). Participants were followed-up every 6 months, up to a maximum of 48 months, depending on the date of enrollment. Additional details on the design and procedures can be found elsewhere. ⁶

Control Intervention

The control condition consisted of twice-yearly HIV pre-and post-test risk reduction counseling sessions based on CDC’s Project RESPECT (Kamb, et al., 1998) – an efficacious risk reduction counseling session that included assessing personal risk, identifying barriers to risk reduction, and discussions around condom use barriers and facilitators.

EXPLORE Intervention

Study participants in the EXPLORE intervention condition completed the same activities as those in the control arm and received additional HIV prevention counseling in the form of multiple, intensive behavioral counseling sessions based on Social Cognitive Theory,¹⁶ Theory of Reasoned Action,¹⁷ Information-Motivation-Behavioral Skills Model,¹⁸ and other approaches/frameworks including cognitive-behavioral therapy.^19,20 Based on these theories, the intervention targeted: needs and HIV-related risks, skills for negotiating and implementing risk reduction in the context of various partner relationships, self-management specific to the relationship between substance use and sex, and managing emotions leading to elevated sexual behavior. Intervention counseling sessions occurred within the first four to six months of the study, along with quarterly maintenance sessions throughout the entire follow-up period.

Measures

Primary Outcome

The primary outcome in the present study was HIV seroconversion, operationalized as a binary variable indicating that the participant did or did not seroconvert at any follow-up point up for the entirety of the study. At each follow-up visit, blood samples were collected for HIV-antibody testing (ELISA). Serum samples shown to be reactive after a first test were then retested. Repeat reactive samples were confirmed by western-blot assay or immunofluorescence assay.

Hypothesized Mediators

Safer sex self-efficacy.

An example item is, “I am able to avoid behavior that may put me at risk of HIV infection.” (alpha= 0·83).

Safer sex communication skills.

An example item is, “I have trouble letting a sex partner know that I want to have safer sex (reverse coded).” (alpha= 0·72).

Perceived norms for safer sex.

An example item is, “Most gay men I meet only engage in safer sex practices.” (alpha= 0·72).

The three scales above were created following recommendations from the EXPLORE codebook (i.e., items were summed) and used a 6-point Likert-type response scale from 1=strongly disagree to 6=strongly agree. Higher scores on each scale indicate greater safer sex self-efficacy, better safer sex communication skills, and greater perceived norms for safer sex, respectively.

Alcohol or drug use before/during sex.

Participants were asked the question, “Since your last interview, about how often did you get high or have a few drinks immediately before or during sex?” Response choices ranged from 0=never, 1=occasionally, 2=often, and 3=all the time.

Condomless receptive anal sex acts with HIV-positive and/or status unknown partners.

In separate items, participants were asked about how many of their male sex partners (since the last assessment) were HIV positive, and about how many male sex partners “never told you their HIV status, or told you they were negative and you have reason to doubt it?” Then participants were asked about how many times they had receptive anal sex with these partners. The information was used to derive the total number of condomless receptive anal sex acts with HIV-positive and/or status unknown partners.

Hypothesized Moderators of “a” Paths

Depression.

Participants completed a shortened version of the Center for Epidemiologic Studies-Depression (CES-D) scale.²¹ Incorporating the same approach previously used when studying depression in EXPLORE,²² we created a sum score and conducted a median split with higher score representing more depressive symptoms (versus less depressive symptoms).

Childhood Sexual Abuse.

Participants were asked about sexual experiences before the age of 17 in two items: “before you turned 13 years old, did you have any sexual experience with someone who was five years or more older than you?” and “between the time you turned 13 and your 17^th birthday, did you have any sexual experiences with someone who was ten years or more older than you?” Participants who responded “yes” to one or both questions were categorized as reporting childhood sexual abuse (versus not).

Stimulant Use.

Participants were asked how often in the last six months they used different stimulants in separate items: “used poppers or inhaled nitrites, including ampules,” “smoked crack or rock cocaine,” “snorted or sniffed cocaine,” and/or “swallowed, snorted, or smoked amphetamines such as speed, crystal, or crank.” Response options included 0=never, 1=less than once a week, 2=1-2 days a week, 3=3-6 days a week, and 4=each day of the week. The four items were summed, and participants who scored 0-1 were categorized as reporting “low stimulant use” and those who score 2 or higher were categorized as reporting “higher stimulant use.”

Control Variables

Throughout the course of the EXPLORE trial, retention was higher in the control group compared to the intervention group (a CONSORT study flow diagram is published elsewhere).⁶ The original outcome analysis found that lower retention (defined by final retention status) was found among racial minority MSM (vs. white MSM), younger MSM (i.e., less than 25 years old vs. more than 25 years old), MSM who reported at least one female sex partner at baseline (vs. none), and among MSM who reported any unprotected receptive anal intercourse at baseline (vs. none). Mirroring the approach used in the original outcome analysis, we controlled for these variables in final models.

Statistical Analyses

Prior to estimating the full mediation model with multiple mediators, we conducted preliminary analyses in model-building steps (described in “Supplementary Material”). After conducting the preliminary analysis, we evaluated the indirect effects of the intervention on HIV seroconversion through all of the hypothesized mediators that were shown to have a statistically significant “a” path (p < ·05) with a model assuming no interactions between the mediators and with exposure.²³ To strengthen the plausibility of the assumption in causal mediation that there is no confounding (i.e., there are no confounding variables that explain mediated effects), each mediator’s baseline score was included as a predictor, each of the mediators is allowed to be predicted by baseline scores of the other mediators, and each mediator is allowed to correlate with one another. Mediators were treated as continuous variables in the multiple mediation analysis to simplify estimation. Modeling was conducted using the structural equation modeling framework in Mplus (i.e., path analysis), using maximum-likelihood estimation, probit link function (for the HIV binary outcome), and Monte Carlo integration. Indirect effects were tested for each hypothesized mediator that had a statistically significant ‘a’ and ‘b’ path using the potential outcomes model (causally defined-effects) with bootstrapped confidence intervals (k=1,000). Mplus version 8 was used to conduct these analyses.

Results

Sample characteristics

Detailed baseline characteristics of the sample are published elsewhere.^6,24 Most of the participants were White, college-educated, and employed full-time. For the purpose of this paper, we provide descriptive statistics across the entire sample (e.g., means and standard deviations) on key variables that were tested in the models (i.e., baseline scores on the hypothesized mediators; Table 1).

Table 1.

Sample characteristics at baseline on key variables

	Total Sample (n=4,295)		Intervention Arm (n=2,144)		Control Arm (n=(2,151)
	Mean	Standard Deviation	Mean	Standard Deviation	Mean	Standard Deviation
Safer sex self-efficacy	40.84	8.93	40.92	8.88	40.76	8.98
Safer sex communication skills	24.60	7.18	24.53	7.01	24.67	7.35
Perceived norms for safer sex	21.83	4.76	21.95	4.78	21.70	4.73
Alcohol or drug use before/during sex	1.08	0.87	1.08	0.86	1.08	0.88
Condomless receptive anal sex acts with HIV-positive and/or status unknown partners (total number)	2.07	10.16	1.95	10.19	2.19	10.14
Higher Stimulant Use (n, %)	1002	23.3%	496	23.1%	506	23.5%
Higher Depression (n, %)	2028	47.2%	1022	47.7%	1006	46.8%
Childhood Sexual Abuse (n, %)	1723	40.1%	893	41.7%	830	38.6%

Mediation analysis with multiple mediators

Based on the results of the preliminary analyses (see “Supplementary Material”), we examined multiple mediation models (including all five mediators following MacKinnon 2008) across the entire sample, and in stratified analyses among MSM with low stimulant use and among MSM with higher stimulant use. This provided a comprehensive understanding about indirect pathways underlying efficacy for these different subgroups in EXPLORE. Depression and childhood sexual abuse were not considered as moderators in final models because they did not significantly moderate ‘a’ paths.

MSM with low stimulant use.

We tested a multiple mediator model among MSM who reported low stimulant use. Results are summarized in Figure 1. After controlling for site, age, race, having a female partner, reporting any condomless receptive anal sex at baseline, baseline scores on all mediator variables, and with all mediator variables in the model, the following ‘a’ paths (beta coefficients) were statistically significant (p<·05) in the predicted directions: intervention increasing safer sex self-efficacy, intervention increasing safer sex communication skills, intervention decreasing alcohol or drug use before/during sex, intervention reducing condomless receptive anal sex with HIV-positive or status unknown partners. There was no effect of the intervention on perceived norms for safer sex. The following ‘b’ path was statistically significant in the predicted direction: any condomless receptive anal sex with HIV-positive or status unknown partners increasing the odds of HIV seroconversion. There were no effects of safer sex self-efficacy, safer sex communication skills, perceived norms for safer sex, and alcohol or drug use before/during sex on HIV. The results suggested that only a single mediator, condomless receptive anal sex with HIV-positive or status unknown partners, mediated the effect of the intervention on HIV (i.e., both ‘a’ and ‘b’ paths were significant). For this mediator variable, causally-defined indirect, direct, and total effects are summarized in Table 2 (top panel). The total natural indirect effect was statistically significant (p=0·03), as was the total effect (p=0.02), indicating that there was causal mediation such that the intervention significantly reduced condomless receptive anal sex with HIV-positive and/or status unknown partners, and this changed behavior in turn reduced the odds of becoming infected with HIV across the 48-month follow-up period, among men with low stimulant use.

Figure 1. Multiple mediator model results among MSM with low stimulant use (n= 2,749).

**p < .01; ***p < .001

Table 2.

Total, indirect, and direct effects of the intervention to HIV seroconversion through condomless receptive anal sex with HIV-positive or status unknown partners, based on counterfactuals (causally-defined effects)

MSM with Low Stimulant Use
	Odds Ratio	Lower 95% CI	Upper 95% CI	p
Total Natural Indirect Effect	0.931	0.865	0.988	0.027
Pure Natural Direct Effect	0.739	0.502	1.061	0.060
Total Effect	0.688	0.476	0.982	0.016
MSM with High Stimulant Use
	Odds Ratio	Lower 95% CI	Upper 95% CI	p
Total Natural Indirect Effect	0.961	0.879	1.038	0.347
Pure Natural Direct Effect	1.103	0.722	1.733	0.696
Total Effect	1.060	0.683	1.723	0.818

Note: Causally-defined effects were estimated in Mplus using a logit model for the HIV outcome (Mplus is not currently capable of producing causally-defined effects for probit models). Total Natural Indirect Effect (TNIE) is the effect of the intervention on the outcome, conditional on all individuals being in the treatment group, and is computed as the difference between being in the treatment group with a mediator value estimated as the mean for the treatment group, and being in the treatment group with a mediator value estimated as the mean for the control group. The Pure Natural Direct Effect (PNDE) is interpreted as the causal direct effect of X on Y if all individuals were in the control condition. For more information about causal effects in the potential outcomes framework for mediation, please refer to Smyth et al. (2021)³³

MSM with higher stimulant use.

The results of a multiple mediator model among MSM with higher stimulant use are summarized in Figure 2. In this subgroup, the intervention significantly increased safer sex self-efficacy and safer sex communication skills, but did not significantly affect perceived norms for safer sex, alcohol or drug use before/during sex, or condomless receptive anal sex with HIV-positive or status unknown partners. Alcohol or drug use before/during sex and condomless receptive anal sex with HIV-positive or status unknown partners significantly increased the odds of HIV, but there was no effect of the other variables on HIV. In sum, none of the hypothesized mediators showed significant effects for both ‘a’ and ‘b’ paths. For the purpose of comparison, we tested causally-defined indirect, direct, and total effects for condomless receptive anal sex with HIV-positive or status unknown partners among this subgroup of MSM with higher stimulant use, and none were statistically significant (lower panel of Table 2).

Figure 2. Multiple mediator model results among MSM with higher stimulant use (n= 885).

*p < .05; ***p < .001

HIV seroconversion by stimulant use.

For descriptive purposes and to aid interpretation of the data in light of the results, we present the frequencies of HIV seroconversion by stimulant use. Among MSM with low stimulant use (n=3,286), a total of 4.5% (n=148) seroconverted, whereas among MSM with higher stimulant use (n=1,002), a total of 11.1% (n=111) seroconverted.

Discussion

The purpose of this study was to examine mediated and moderated effects to explain null efficacy in one of the most well-known behavioral interventions trials to prevent HIV. A priori, the EXPLORE trial was designed to meet a specific benchmark for efficacy; if the difference between conditions was significantly below 35% then the intervention would be “ruled out.”⁶ The primary outcome analysis of EXPLORE demonstrated that the difference between groups was non-significant (i.e., the 95% CI ranged from −4·7 to 36·0), and the results did not meet the specific benchmarks for efficacy that were defined a priori.⁶ Thus, the EXPLORE behavioral intervention was ruled as non-efficacious. The results from the current paper demonstrate that EXPLORE can be effective in preventing HIV, particularly among MSM with low stimulant use. Another way that the current results can be interpreted is that the intervention failed to change behavior for about a quarter of the sample – MSM with higher stimulant use. Among this specific subgroup, the intervention was not efficacious in preventing HIV in the way that is was efficacious among MSM with low stimulant use, which was by significantly reducing condomless receptive anal sex with HIV-positive or status unknown partners.

It is worth noting that EXPLORE was evaluated under highly conservative conditions. The control arm was itself an efficacious intervention, and counselling sessions for this arm were delivered regularly throughout the follow-up. Further, the 48-month follow-up period was unprecedented, and since then behavioral HIV prevention trials have not mirrored the same length of follow-up. In fact, during the initial 12-18 months following counselling, there was a more than 35% reduction in HIV incidence, which declined to 15·7% at 48-months and after accounting for covariates. Prior research has provided insight around the justification for the 35% a priori benchmark of success, and the largely negative interpretation of the EXPLORE results. Specifically, Kalichman and colleagues²⁵ described that the benchmark was defined based on epidemiologic models to define an effective HIV vaccine, setting a high standard for a behavioral intervention. Their review of the literature found that 80% of articles that have cited EXPLORE have judged the intervention to be ineffective, which is related to discrediting of the benefits of behavioral counseling.

The prevalence of past year stimulant (specifically non-medical amphetamine) use among MSM in the U.S. is estimated to be about 10%.²⁶ Further, modeling research has shown that MSM who use stimulants experience a disproportionate burden of health harms, including HIV.²⁷ The present study builds upon this research to further demonstrate that MSM who use stimulants require targeted prevention and intervention approaches to effectively prevent HIV and reduce other harms. Specifically, this study demonstrated how MSM who use stimulants were essentially “left behind” by the EXPLORE intervention. While the intervention was efficacious among MSM with low stimulant use at reducing condomless receptive anal sex with HIV positive or status unknown partners, which in turn explained reductions in HIV seroconversion, this pattern was not found among MSM with higher stimulant use. Thus, we build upon the literature arguing for more targeted intervention approaches for MSM who use stimulants. As one promising example, early results from a randomized trial of behavioral activation for treatment of concurrent crystal methamphetamine dependence and sexual risk among MSM (Project IMPACT) demonstrate that this approach is feasible and potentially efficacious at reducing both behaviors.²⁸

In addition to providing supporting evidence that MSM with higher stimulant use require targeted intervention approaches, the current paper highlights the important, or imperative, need for researchers to examine mediation and moderation in intervention trials, including those that do not demonstrate an overall effect (are ruled as “non-efficacious”). It is often assumed that an overall significant effect must be found to examine mediation, but this is not the case. Interventions may be shown to have significant indirect impacts on outcomes even when an overall effect is not statistically significant. Although the HIV epidemiological context has changed since the EXPLORE trial was conducted (1999-2004), the types of analytic methods (multiple mediation, causal mediation, moderated mediation) and processes that we evaluate in the current manuscript remain highly important for the field of HIV prevention, and can be applied to fields intervening on other public health outcomes (e.g., substance use, nutrition and exercise, smoking cessation). In this study, only a single mediator was statistically significant (in the low stimulant group), and in this case causal mediation techniques in the potential outcomes framework were used to estimate the direct and indirect effects. Approaches for causal mediation analysis for the multiple mediator model are starting to appear in the methods literature.^23,29,30

The results of this study must be interpreted in light of its limitations. As mentioned, while we argue that the same techniques are important to apply for all past and future intervention trials in public health,⁵ EXPLORE data are ~15 years old. Compared to the current era, when the trial was conducted in 1999-2004 it may have been more challenging for MSM to reliably assess the HIV status of their sex partners, potentially affecting the reliability of these items (e.g., condomless receptive anal with HIV positive or unknown partners). Also, EXPLORE focused solely on MSM in the U.S. The findings may not apply to other populations and regions of the world, including for example MSM in low- and middle-income countries and racial/ethnic minority heterosexual women and men. We focused on stimulant use because of its robust impact on HIV-related outcomes among MSM. Focusing on other substances (e.g. alcohol or heroin) or on injecting versus non-injecting drug use may show similar or different effects in future research. EXPLORE did not measure or examine socio- or structural-factors related to risks for HIV (e.g., social networks, stigma, medical mistrust, access to care). There has been a growing recognition that such factors must be considered to curb and end the HIV epidemic, and that multi-level intervention approaches (i.e., targeting individual-level psychological factors like self-efficacy as well as structural factors like stigma and discrimination) are most likely to produce significant and sustained effects on risk behavior and HIV incidence.^31,32 To determine statistical significance we set alpha at p <.05, and with multiple tests there is the potential that this increased the experimentwise Type I error rates and the significant effects are due to chance. However, the processes we examined are theoretically-grounded, and we did observe statistically significant effects at p<.001 for some individual paths in the multiple mediator model. In any case, future HIV intervention research should test similar moderated mediation hypotheses also using theory-based variables. In particular, the results suggest that stimulant use is an important moderating variable to consider in future research.

Given the time and resources that are needed to develop and evaluate intervention programs, it is essential for researchers to have a full and complete understanding of the data. Complete analysis is important for both efficacious and non-efficacious interventions, but may be even more essential with non-significant trials. When an intervention is ruled as non-efficacious and mediators and moderators are not examined, it may be set aside without its full potential to inform science and public health ever being realized.