More Behavioral Recommendations Produce More Change: A Meta-Analysis of Efficacy of Multi-Behavior Recommendations to Reduce Non-Medical Substance Use

Abstract

Objective:

Death and morbidity associated with substance use have risen continuously over the last few decades, increasing the need for rigorous examination of promising programs. Interventions attempting to change multiple behaviors have been designed to address interconnected problems such as use of both alcohol and drugs. This meta-analysis aimed to examine the efficacy of multi-behavior interventions to curb non-medical substance use in relation to the theoretical relation among different substance use behaviors. Specifically, our synthesis aimed to estimate the optimal number of recommendations for intervention efficacy and evaluate the impact of different combinations of recommendations on intervention efficacy.

Methods:

A synthesis of multi-behavior interventions addressing non-medical substance use was conducted to measure behavioral changes between the pretest and the follow-up. These changes were then compared across different numbers of recommendations.

Results:

Sixty-nine reports and 233 effect sizes (k of conditions = 155, n = 28,295) were included. A positive linear relation was found between the number of targeted behaviors and intervention efficacy, which was stronger for drug use than alcohol use. Furthermore, recommendations on drug use worked better when paired with recommendations targeting other behaviors, whereas recommendations on alcohol use worked more independently. Lastly, multi-behavior interventions were especially efficacious when delivered by experts.

Conclusions:

Overall, our synthesis indicated that targeting multiple substances is beneficial for changing drug use outcomes, but less so for alcohol use outcomes. Therefore, in the current substance use epidemic, innovative multi-behavior programs appear to hold promise, especially to combat non-medical drug use.

The detrimental effects of non-medical substance use are well documented. According to the Centers for Disease Control, between 2006 and 2010, alcohol was responsible for an average of 88,000 deaths and 2.5 million years of potential life lost each year in the United States (Centers for Disease Control and Prevention, 2016). Deaths from illicit drug overdoses (heroin, natural and semisynthetic opioids, methadone, synthetic opioids, cocaine, etc.) in the United States are currently at a record high in both sheer numbers and prevalence per 100,000 people (Hedegaard et al., 2017). In 2015, 16.3 per 100,000 people died of a drug overdose, which represented over 2.5 times the 6.1 per 100,000 rate in 1999 (Hedegaard et al., 2017). Fortunately, many treatments and programs have been created to address non-medical substance use and addiction, including multi-behavior interventions, which are interventions geared towards changing two or more behaviors.

The efficacy of interventions aimed at reducing substance use has often been underwhelming. For example, nine prior meta-analyses estimated efficacy across populations and interventions. The sources, number of studies, critical comparisons, and effect sizes of these nine syntheses appear in Table 1. The syntheses spanned several decades with no apparent changes in efficacy, leading to the question of whether the newest generation of more complex interventions is ready to reduce substance use.

Table 1.

Summary of Prior Meta-Analyses of Substance Use Interventions.

	Title	M year	k of studies	Effect size		Comparison (& type of controls)
Prendergast, Podus, Chang, & Urada, (2002)	The effectiveness of drug abuse treatment: A meta-analysis of comparison group studies	1980	78	g = 0.34		Drug use treatment vs. active control
Prendergast, Podus, Finney, Greenwell, & Roll (2006)	Contingency management for treatment of substance misuse disorders: A meta-analysis	1986	47	g = 0.42		Contingency management vs. control
Magill & Ray (2009)	Cognitive-Behavioral Treatment with Adult Alcohol and Illicit Drug Users: A meta-analysis of randomized controlled trials	1994	53	g = 0.154		CBT vs. active control
				g = 0.115		CBT vs. active control,
Carney & Myers, (2012)	Effectiveness of early interventions for substance using adolescents: Findings from a systematic review and meta-analysis	2006	9	g = 0.24		Drug and alcohol treatment vs. control
Blodgett, Maisel, Fuh, Wilbourne, & Finney, (2014)	How effective is continuing care for substance misuse disorders? A meta-analytic review	1999	19	g = 0.187		Continuing care vs passive control
				g = 0.271	Continuing care vs. passive control comparison
Benishek et al., (2014)	Prize-based contingency management for the treatment of substance abusers: A meta-analysis	2007	19	d = 0.46	Contingency management vs. active control
				d = 0.33	Contingency management vs. active control
Tanner-Smith & Risser (2016)	A meta-analysis of brief alcohol interventions for adolescents and young adults: Variability in effects across alcohol measures	NR	190	g = 0.25	Brief intervention vs. passive control (adolescents)
				g = 0.15	Brief intervention vs. passive control (young adults)
Boumparis, Karyotaki, Schaub, Cuijpers, & Riper (2017)	Internet interventions for adult illicit substance misusers: A meta-analysis	2010	17	g = 0.31	Internet interventions vs. mostly active control
				g = 0.22	Internet interventions vs. mostly active control
Sayegh, Huey, Zara, & Jhaveri (2017)	Follow-up treatment effects of contingency management and motivational interviewing on substance misuse: A meta-analysis	NR	84	d = 0.43	Contingency management vs. control
				d = 0.06	Contingency management vs. control
				d = 0.10	Motivational interviewing vs. control
				d = 0.22	Motivational interviewing vs. control

g = Hedge’s g, corrected for small sample bias. d = Cohen’s d, not corrected for small sample bias.

It is noteworthy that many of the programs included in Table 1 have addressed drug or alcohol use as relatively independent of other related behaviors or contextual factors that are likely to maintain the behavior. But in fact, research has shown that substance use often stems from a plethora of biological, psychological, and social factors, all working in tandem (Borrell-Carrió et al., 2004). Substance use is often interdependent with other disorders and behaviors. For example, a number of underlying factors that affect a person’s propensity to alcohol dependence may also lead to a propensity to tobacco dependency (Dawson, 2000; Friedman et al., 1991). Moreover, different forms of substance use are often interrelated. For example, alcohol and drug use are strongly correlated and using alcohol is often a precursor to drug use (Regier et al., 1990; Agosti, Nunes, & Levin, 2002). Relative to people who do not use alcohol, people who use alcohol are 35 times more likely to use cocaine, 17 times more likely to use sedatives, and 13 times more likely to use opioids (Helzer & Pryzbeck, 1988; Agosti, Nunes, & Levin, 2002; Stinson et al., 2005).

Recognizing the interrelation among different behaviors, cutting-edge intervention approaches often attempt to target multiple behaviors by making multiple recommendations. These interventions that attempt to change two or more behaviors are known as integrated interventions (Prochaska et al., 2008) or multi-behavior programs (Wilson et al., 2015), and can be implemented in several ways. First of all, interventions can promote multiple principal goals regarding substance use or other risks, based on knowledge of comorbid risks in the population, or programs and behaviors that the population needs. For example, an intervention tested by Haller et al. (2014) provided two main recommendations, namely to reduce alcohol use and to reduce cannabis consumption. Besides directly targeting risky behaviors, multi-behavior interventions can also address multiple behaviors that are often disrupted by the use of substances and can improve the outlook and lifestyle of people who use non-medical substances. For example, an intervention tested by Kypri and McAnally (2005) recommended increasing physical activity, increasing fruit intake, and increasing vegetable intake, besides reducing hazardous drinking. Both types of recommendations discussed above are the goals of these programs and are considered to be main recommendations.

Interventions can also promote multiple auxiliary recommendations, which are skills or behaviors that are prerequisite for recipients to implement the main recommendations. For example, the intervention tested by Deady, Mills, Teesson, & Kay-Lambkin (2016) provided three auxiliary recommendations, including to monitor drinking behaviors, to monitor thoughts, and to plan for activities, besides the main recommendation to reduce alcohol use. These recommendations concern behaviors that, if implemented, can help participants to be successful at implementing the main recommendation.

Given the demonstrated existence of clusters of harmful behaviors and the antagonistic effects between drug use and health promoting behaviors, multi-behavior programs are the most logical solution to addressing the complex problem of substance use. Before our meta-analysis, some evidence suggested that multi-behavior interventions are effective in the domain of lifestyle behaviors (Wilson et al., 2015). Although these results are auspicious, it is still unclear whether similar models will improve substance-related interventions over and above prior generations of programs, and whether these programs apply differently to drug and alcohol outcomes. Therefore, the primary purpose of this meta-analysis was to examine the efficacy of these multi-behavior interventions in the domain of substance use.

Commonly Used Recommendations and Combinations of Recommendations in Multi-Behavior Programs for Non-Medical Substance Use

Besides recommending a reduction in alcohol use and drug use, multi-behavior interventions often provide recommendations to change other related behaviors. These behaviors can range from major goals, such as maintaining a healthy lifestyle and preventing transmission of sexually transmitted infections, to ones such as seeking social support and making better use of community services. To better understand the efficacy of multi-behavior interventions in the substance use domain, it is thus crucial to first describe combinations of recommendations that are most commonly used in multi-behavior programs for non-medical substance use. Therefore, another goal of this meta-analysis was to provide this description and further investigate the efficacy of such combinations of recommendations.

Optimal Number of Recommendations to Reduce Drug and Alcohol Use: Facilitating, Independent, and Antagonistic Relations among Behaviors

Besides the type of recommendations to include, the number of recommendations to include in an intervention is also an important question to consider when aiming for the most efficacious intervention program. Whereas too few recommendations may not be stimulating enough to elicit change, an excessive number of recommendations can create burden and undermine the motivation to change (Wilson et al., 2015; Albarracín et al., 2018; Albarracín et al., 2019). Therefore, our meta-analysis also investigated the optimal number of recommendations to produce the greatest behavioral change.

The optimal number of recommendations to include in an intervention should depend on the relations among the target behaviors. Although previous research has shown a high intercorrelation among the use of multiple substances, the use of some substances may still be relatively more independent than the use of other substances. Better knowledge of the relative independence among the network of different behavior can help better predict the optimal number of recommendations targeting the use of each specific substance. To achieve this goal, in this paper, we proposed two models of relations among behaviors (see Figure 1). On the one hand, a Relative Interdependence Model represents behaviors that are causally associated. For example, a person may experience cocaine cravings each time he drinks alcohol, or not seek health care during periods of heavy drug use (Behavior A causally related with Behavior B). Therefore, alcohol use increases the probability of drug use and drug use decreases the probability of seeking health care. On the other hand, a Relative Independence Model represents behaviors that are relatively independent from each other. For example, a person may drink alcohol (Behavior A) without having a craving for cocaine (Behavior A not causally related with Behavior B) (see e.g., Livingston, Oost, Heck, & Cochran, 2015). Thus, the Relative Interdependence model should be associated with stronger correlations among these two behaviors than the relative independence model, although neither model is likely to lead to either a perfect correlation or a 0 correlation and correlations among behaviors may also occur due to other factors.

Figure 1.

Models of Relatively Interdependent and Relatively Independent Behaviors

In the current meta-analysis, we tested our theoretical hypothesis by comparing the efficacy of multi-behavior interventions targeting alcohol use, drug use, or both. In summary, if behaviors follow the Relative Interdependence Model, such as alcohol use facilitating drug use, or drug recovery requiring a cluster of support and health care services, then an intervention should be more efficacious when it targets both alcohol use and drug use, compared to targeting drug use alone. If behaviors follow the Relative Independence Model, such as alcohol use being independent from drug use, then an intervention should be equally efficacious when it targets both alcohol use and drug use, compared to targeting alcohol use alone.

A related important question is whether adding behavioral recommendations increases efficacy in a monotonic fashion. If the behaviors contribute to positive outcomes because they are mutually facilitating, then more recommendations should be better than fewer. However, if behaviors contribute to a positive outcome but compete for resources and cause fatigue, then more recommendations should add only up to a point (Baumeister & Heatherton, 1996; Muraven et al., 1998; Vohs & Heatherton, 2000). In the domain of lifestyle (smoking, exercise, and diet), Wilson et al. (2015) showed that the improvements in efficacy resulting from adding recommendations flattened out or even decreased after 3 recommendations. However, the pattern may be different in the substance use domain. Whereas quitting smoking can make dieting more difficult, quitting alcohol can make quitting drugs easier. Likewise, protecting one’s child, completing drug treatment, maintaining drug abstinence, and complying with court orders act synergistically, so that protecting a child is generally not possible without maintaining drug abstinence, complying with court orders, and completing drug treatment. Therefore, we hypothesized that in the substance use domain, there should be a linear relation between number of recommendations and intervention efficacy, such that more recommendations generally lead to more behavioral changes.

The Current Meta-Analysis

The synthesis included 69 research reports (k of conditions = 155), which looked at the outcomes of interventions related to both alcohol and drug use. The list of the articles and its description can be found in Supplementary Table 1. We determined the change between the baseline and first follow-up and compared change across interventions with different numbers of recommendations, with passive controls coded as providing zero recommendations. The average gap between the baseline and first follow-up was two and a half weeks.

Importantly, the prior meta-analysis of multi-behavior interventions in the lifestyle domain identified several factors such as targeting a specific population and delivery format, as moderators of the effects of these multi-behavior interventions (Wilson et al., 2015). Often, targeting a specific population (e.g., specific ethnic group, vulnerable population) leads to more efficacious programs than not targeting (Ammerman et al., 2002; Ebrahim et al., 2006; Hardcastle et al., 2013), and group delivery format is associated with higher efficacy than individual delivery format (Ayala, 2006; Wright et al., 2011). Besides these features, we also coded a variety of study settings and methodological moderators that have been commonly studied in intervention research (Albarracín et al., 2005), as well as the expertise of the facilitator of the intervention as represented by professional training in health or substance use issues (Durantini et al., 2006; Amaro et al., 2007; Burton et al., 1995). We also coded whether the intervention was designed to be culturally specific or generic (Dushay et al., 2001; Fogel et al., 2015), whether the intervention made use of active (e.g., behavioral skills training) vs. passive (e.g., informational arguments) strategies (Albarracín et al., 2005; Grossbard et al., 2010), whether the intervention relied on attitudinal elements (e.g., attitudinal arguments, threat arguments; Albarracín et al., 2005; Burke et al., 2005), motivational elements (e.g., feedback, encouragement, a written contract; Jungerman et al., 2007; Latkin et al., 2009), and/or skills training (e.g., role-playing, goal setting; Albarracín et al., 2005; Calsyn et al., 2010; Kaner et al., 2013), as well as whether the intervention included biological methods (e.g., use of nicotine patches, methadone; Hanson et al., 2008; Penberthy et al., 2013). A last moderator we included was whether the intervention targeted an alcohol / drug dependent population. This moderator was added to distinguish prevention of alcohol or drug use increase from treatment in dependent populations, which are often different in efficacy and characteristics (Nemoto et al., 2002; Volkow & Li, 2005; Fischer et al., 2015). The present meta-analysis explored the influence of all these factors in the domain of substance use.

Critical to our objectives, we described the synthesized multiple-behavior interventions in terms of whether they targeted drug use, alcohol use, and any additional behaviors, such as using available community services, seeking social support, and developing behavioral skills to combat non-medical substance use. We then analyzed the impact of the number of all recommendations, as well as the number of main recommendations only, and considered this impact in the context of the types of behaviors that were combined. These analyses controlled for other factors identified as having an impact on change.

Method

Literature Search

We conducted an internet search of six research databases (EBSCO, Scopus, Web of Science, PubMed, JSTOR, and Crossref). The search terms used were as follows: intervention, health education, persuasion, recommendation, treatment, educational program, rehabilitation, counseling outcomes, treatment outcomes, treatment effectiveness evaluation, treatment compliance, health promotion, behavior change, and randomized trial. To specifically find articles related to multiple alcohol and/or drug use behaviors, we used the syntax ((alcohol or drug or substance) AND ((intervention or randomized trial)) NOT ((review or meta-analysis or campaign or cost-effectiveness or theoretical)). This search yielded a total of 2,022 articles. In an attempt to include as many pertinent articles as possible, we searched through the reference list of related meta-analyses, conference titles, and emailed the researchers who appeared most frequently in the database search, requesting any unpublished works. Article searches were performed iteratively in 2014, 2015, 2017, and 2019.

Inclusion Criteria

After finding research reports, we screened the articles with our inclusion criteria, which led to a total of 69 eligible articles (see Figure 2). The inclusion criteria were as follows:

1. Existence of at least two groups, including at least one multi-behavior group. To be considered eligible, all reports had to have at least two groups to allow for comparison of change across conditions. They also had to have at least one multi-behavior intervention group that provided at least one substance use recommendation.

2. Existence of at least one behavioral or clinical outcome. To be included, all reports had to include at least one behavioral or clinical outcome variable. For example, if a paper reported behavioral intentions only, it was excluded.

3. Number of recommendations available. To be included in this meta-analysis, all reports had to include information that allowed us to count the number of recommendations. Reports were excluded if the number of recommendations could not be ascertained.

4. Presence of sufficient statistical information. All reports that did not include enough statistical information to calculate effect sizes for change over time were excluded. Therefore, to be included, articles had to report outcome values at both baseline and at least one follow-up, or other statistics (e.g., a paired t-test) to calculate change effect sizes. Reports that only included a follow-up outcome value were excluded, due to the fact that they disallowed us to calculate effect sizes to represent change over time.

Figure 2.

Flow of reports in this systematic review.

Measuring Change in Overall Outcomes, Drug use Outcomes, and Alcohol use Outcomes

Similar to the coding process used in Wilson et al. (2015), we recorded both behavioral and clinical intervention outcomes. The most frequent outcomes were alcohol consumption and drug consumption. The most frequent measures for alcohol consumption were the amount of alcohol consumed in a specific period (e.g., in a day, in a week), the frequency of excessive alcohol use (e.g., number of days of excessive drinking in a month), the proportion of abstinence days, and the number of drunk days. The most frequent measures for drug consumption were the amount of drug use (heroin, opioids, methadone, cocaine, crack, etc.) in a specific period (e.g., in the past week), the frequency of ecstasy use, the frequency of excessive drug use, the proportion of abstinence days, and the number of times having sex while under the influence of drugs.

Behaviors that help to reduce the risk from non-medical substance use were measured as well. These variables included adherence to medications, managing negative consequence of substance use, and counseling attendance. Clinical outcomes were measured infrequently. Clinical outcomes for alcohol use were urine test results, the number of medical conditions related to alcohol use, and diagnoses of alcohol use disorder or dependence. Clinical outcomes for substance use were urine test results, the number of drug use disorders diagnosed, and the number of dependence symptoms.

We obtained each effect size by representing the change from the pretest to the posttest (e.g., d = (M_postest −M_pretest/SD_pretest) for each of the outcome variables in a sample, including drug use outcomes, alcohol use outcomes, and any auxiliary behavioral outcomes, such as medical adherence and counseling attendance. For example, we obtained 5 effect sizes from Reback et al. (2010), including one alcohol-use outcome (i.e. percentage of people free of alcohol in a urine test), three drug-use outcomes (i.e. percentage of people free of cocaine / amphetamine / methamphetamine in a urine test), and one auxiliary outcome (i.e. intervention attendance). Importantly, the decision to analyze change over time was based on the fact that performing post-test comparisons across conditions would typically provide one or two effect sizes of the many possible comparisons across interventions with 0, 1, 2, 3, 4, 5, 6, 7, 8 recommendations, and so on. Such a situation would thus require abnormally high levels of imputation to estimate the impact of the number of recommendation count, much higher than is the norm for network meta-analysis (Lumley, 2002; Caldwell et al., 2005; Glenny et al., 2005). Thus, it seemed more appropriate to impute the correlation between measures than to model tons of differences based on the scarce data provided by the trials we synthesized.

When running analyses on drug use and alcohol use separately, we only included the outcomes that only pertained to drug use or alcohol use from each report, depending on what they reported. For example, in Reback et al. (2010), we obtained one effect size for alcohol use from the only reported alcohol-use outcome, and three effect sizes for drug use from each of the three drug-use outcomes. Outcomes that pertained only to alcohol use included alcohol consumption, sex and drinking, and managing harmful consequences of alcohol use. Outcomes that pertained only to drug use included drug consumption, exchanging drugs for sex, starting drug treatment, and sharing syringes or other equipment. In each case, effects were scored to reflect improvement. For example, a decrease in alcohol use and an increase in the percentage of a sample starting drug treatment were coded positive.

Coding of Number of Overall and Main Recommendations

We coded the number of recommendations by counting the total number of behavioral goals, including both main and auxiliary goals. We also created a separate count of number of main recommendations for the purpose of some analyses. Main recommendations were defined as the main goals defined in an intervention which had a direct impact on clients’ quality of life, whereas auxiliary recommendations are those that can help clients to achieve the main goals of the intervention. For example, the intervention in Santa, LaRowe, Armeson, Lamb, and Hartwell (2016) was coded as providing four recommendations because the intervention encouraged participants to reduce substance use, stay clean and sober, attend treatment, and take medication for the substance use disorder. Among these recommendations, the first one was considered a main recommendation, because it directly targeted the risky behavior of substance use and was the intervention’s focus. The other three were coded as auxiliary because they were introduced to help clients to achieve the main goal of reducing non-medical substance use.

Coding of Type of Recommendations

In addition to understanding the impact of the number of recommendations, we were interested in describing the most commonly used recommendations and combinations of recommendations, and their influence on change. Therefore, for each intervention, we coded whether it included recommendations targeting alcohol use, drug use, and any other commonly targeted behaviors, including sexual behavior, lifestyle behavior, service use, support seeking, and development of behavioral skills. However, due to the small ks for each of these types of recommendations, we combined all these recommendations into the category of “other recommendations” and classified all interventions into seven categories based on whether they included recommendations targeting alcohol use, drug use, and at least one other domain: (a) neither alcohol nor drug use, (b) alcohol use only, (c) drug use only, (d) alcohol and drug use, (e) alcohol use and at least one other domain, (f) drug use and at least one other domain, and (g) drug use, alcohol use, and at least one other domain.

Coding of Exploratory Study Characteristics

Relevant characteristics of the reports, as well as the methods used in the studies, were coded by two independent raters as described in Supplementary Materials. Disagreements between coders were resolved by discussion and further examination of the reports.

Description of the report.

We coded report characteristics, including the (a) publication year, (b) first authors’ institution (e.g., college, hospital, research center), (c) first authors’ institutional area (e.g., psychology, public health, medicine), (d) source type (e.g., journal article, dissertation), (e) location of the intervention, and (f) language of the intervention.

Demographic and other participant characteristics.

We also coded characteristics of the sample, including the (a) sample size, (b) percentage of males in each group, (c) mean age, (d) percentages of participants of European, African, Latin, Asian, and Native American descent, (e) percentage of participants who completed high school and their mean years of education, (f) percentages of participants who identified as heterosexual, gay and bisexual, and (g) percentage of participants with a pre-existing health condition (e.g., HIV, heart disease).

We further coded factors related to the intervention participants. We coded reports for (a) the specific sample targeted in the intervention (e.g., college students, people who inject drugs), and recoded this variable to indicate whether the intervention targeted a population that was alcohol / drug dependent, (b) whether the intervention was targeted to an ethnic minority group, (c) whether the intervention was targeted to a specific gender group, and (d) whether the sample was self-selected, indicated by whether participation in the intervention was voluntary or whether the study conducted with a captive audience (e.g., prison inmates).

Intervention characteristics.

Finally, we coded for characteristics of the intervention programs, including (a) whether participants were randomly assigned and (b) the mean number of days between the intervention and posttest. We also coded for factors related to the implementation of these interventions. Specifically, we classified each intervention group according to (a) where participants were recruited (e.g., drug treatment facility, social service agency), and recoded this variable to describe a hospital/clinic setting vs. a non-hospital/clinic setting, (b) whether the facilitator was a professional (e.g., physician, nurse, social workers, licensed counselors) or lay community member (e.g., community leaders and peers), (c) whether the intervention was delivered in a group setting, to individuals, or a combination of the two, (d) the exposure format (e.g., radio, brochure), which was recoded as face-to-face vs. other formats, (e) the exposure setting (e.g., school, community), and (f) clinic vs. non-clinic setting. We also determined (g) whether the intervention was designed to be culturally specific or generic, (h) whether the intervention made use of active (e.g., behavioral skills training) vs. passive (e.g., informational arguments) strategies, (i) whether the intervention relied on attitudinal elements (e.g., attitudinal arguments, threat arguments), motivational elements (e.g., feedback, encouragement), or skills training elements (e.g., role-playing, goal setting), (j) whether the intervention included biological methods (e.g., use of nicotine patches, methadone), and (k) whether the intervention include a written behavioral contract.

Data Analytic Plan

To obtain effect sizes from the mean scores we subtracted the mean at posttest from the mean at pretest and divided it by the pooled standard deviation. To get effect sizes from proportions, we calculated the odds ratio and divided its natural log by 1.81 to convert it into Cohen’s d as outlined by Chinn (2000). For cases in which the proportion was equal to 0 or 1 at either posttest or pretest, we applied the correction from Sweeting, Sutton, and Lambert (2004), which involved adding 0.005 to (or subtracting 0.005 from) both pretest and posttest scores. Because the correlation between the pre-test and post-test measures is nor reported, we set this correlation constantly as r = .5. We note, however, that supplementary analyses were conducted with r = .8, as s sensitivity analysis, and all our conclusions remained the same. Hedges and Olkin’s (1985) correction factor was applied to all effect sizes to correct for small sample size bias. Reverse factors were applied to all the effect sizes so that a positive effect size always showed improvement, whereas a negative effect size showed worsening in the targeted health outcome.

For reports that included multiple outcomes or multiple measurements for one outcome (e.g., substance use, alcohol use), we kept all the effect sizes in the final analyses and used the robust variance estimate to deal with the dependency among correlated effect sizes obtained from the same study (Tanner-Smith et al., 2016).

Most data analysis was conducted using RVE (i.e. robust variance estimate) meta-regression in the statistical program R (Tanner-Smith & Tipton, 2014). R was also used to generate a funnel plot and run trim and fill analysis to gauge publication bias. Due to high levels of heterogeneity (I² = 0.953), we used only random-effect models for all analyses. That is, the effect sizes were weighted by the inverse of their sampling variance plus a random variance component.

Results & Discussion

Description of Sample

We included 69 reports, resulting in 155 research groups and 233 effect sizes. Among the 155 research groups, 103 groups recommended multiple behaviors, 20 groups recommended a single behavior, and 22 groups did not recommend any behavior. The number of recommendations in the synthesized studies ranged from 0 to 11. When only counting the main recommendations, 87 groups included more than one main recommendation, 44 groups included one main recommendation, and 24 groups did not include any main recommendation. Of the 155 groups, 106 included outcomes on alcohol use and 86 included outcomes on drug use. The total number of participants included in this meta-analysis was 28,295.

A summary of the demographic characteristics, intervention set-up details, and participants information appears in Supplementary Table 2. As can be seen, most of the reports included in this meta-analysis were journal articles, published around 2011, and conducted in the United States. The samples included both males (56%) and females (44%), who, on average, were in their mid-thirties. On average¹, 37% of participants were gay or bisexual (k = 31), 61% had completed high school (k = 80), 57% had a risk factor or health condition at baseline (k = 51), 48% were European-American (k = 121), 28% were African-American (k = 112), 86% were randomly assigned to conditions (k = 155), and 85% voluntarily participated in the interventions (k = 151). Among all interventions, 83% were delivered face-to-face (k = 126), 25% were delivered to groups (k = 136), 53% were delivered to individuals (k = 136), 87% used professional experts as facilitators (k =155), 26% targeted alcohol / drug dependent populations (k = 155), 6% targeted a specific at-risk ethnic group (k = 155), 24% targeted a specific gender (k = 148), and only 8% were described by the authors as culturally appropriate (k = 155).

Average Intervention Effect Size

Overall, there were significant improvements in the studied samples. The grand average for change over time was d = 0.42 (CI = [0.34, 0.50], p < .001, k = 233, I² = 0.953), which was close to a medium effect². Comparing the efficacy of multi-behavior interventions to that of previous intervention programs on substance use (Table 1), multi-behavior interventions performed better than most of the intervention programs that have been examined in the past. When separating alcohol and drug use outcomes, the average change on alcohol use outcomes over time was d = 0.31 (CI = [0.24, 0.38], p < .001, k = 118, I² = 0.892). The average change on drug use outcomes over time was d = 0.53 (CI = [0.39, 0.67], p < .001, k = 91, I² = 0.972). Multi-behavior interventions led to significantly more change on drug use outcomes than on alcohol use outcomes (B = 0.245, SE = 0.079, p = .002).

Optimal number of Recommendations

Our objective was to estimate the association between number of recommendations and change. To test for the linear and quadratic predictions between number of behavioral recommendations and intervention efficacy, we first ran an RVE (i.e. robust variance estimate) meta-regression analysis of overall efficacy on number of recommendations, including mean-centered linear and quadratic terms in the model. As shown in Table 2a, the linear term was significant (B = 0.073, SE = 0.016, p < .001) but the quadratic term was not (B = 0, SE = 0.007, p > .05). This pattern replicated for both alcohol use outcomes alone (Linear: B = 0.046, SE = 0.017, p < .05) and drug use outcomes alone (Linear: B = 0.085, SE = 0.025, p < .01). To better understand the effect of the main intervention recommendations, we counted only the main recommendations and reran the above regression analysis. The patterns were mostly similar to the previous analysis. For overall outcomes, the linear term was significant (B = 0.129, SE = 0.031, p < .001) but the quadratic term was not (B = 0.022, SE = 0.023, p > .05). This pattern replicated for drug-use outcomes alone (Linear: B = 0.182, SE = 0.080, p < .05) but not for alcohol-use outcomes alone (Linear: B = 0.042, SE = 0.024, p > .05).

Table 2a.

Summaries of the Meta-Regression Analysis for Change with Robust Variance Estimate (Correlation between Pre- and Post-Score Set as 0.5)

	Estimated coefficients? (Standard error)
	Overall Outcome Change (k = 233)	Alcohol Use Change (k = 118)	Drug Use Change (k = 91)
All recommendations included
Recommendation number linear	0.073 (0.016) ***	0.046 (0.017) *	0.085 (0.025) **
Recommendation number squared	0.000 (0.007)	−0.002 (0.003)	−0.005 (0.011)
Only main recommendations included
Recommendation number linear	0.129 (0.031) ***	0.042 (0.024)	0.182 (0.080) *
Recommendation number squared	0.022 (0.023)	0.021 (0.026)	−0.003 (0.074)

^*p < .05

^**p < .01

^***p < .001.

k is the total number of effect sizes included.

Knowing that underestimation of the pre-post correlation can lead to potential errors in estimation of the effect (Cuijpers et al., 2017), we changed the correlation between the pre and post measures from 0.5 to 0.8 and rerun the above meta-regression, as a sensitivity analysis. As shown in Table 2b, changing the correlation between the pre and post measures only changed the numerical value of the estimated effect but did not change the overall linear pattern between number of recommendation and the intervention efficacy. This pattern replicated for both alcohol and drug use outcomes and for both main and auxiliary recommendations.

Table 2b.

Summaries of the Meta-Regression Analysis for Change with Robust Variance Estimate (Correlation between Pre- and Post-Score Set as 0.8)

	Estimated coefficients β (Standard error)
	Overall Outcome Change (k = 233)	Alcohol Use Change (k = 118)	Drug Use Change (k = 91)
All recommendations included
Recommendation number linear	0.070 (0.015) ***	0.045 (0.016) **	0.080 (0.025) **
Recommendation number squared	0.000 (0.007)	−0.003 (0.003)	−0.005 (0.010)
Only main recommendations included
Recommendation number linear	0.125 (0.030) ***	0.041 (0.023)	0.177 (0.078) *
Recommendation number squared	0.020 (0.022)	0.020 (0.026)	−0.007 (0.074)

^*p < .05

^**p < .01

^***p < .001.

k is the total number of effect sizes included.

To further describe an optimal number of recommendations, we calculated the mean change for each number of recommendations on all outcomes. Due to the small ks for interventions recommending 6 or more than 6 behaviors, we combined them all into one group as “6 & 6+ recommendations”. As shown in Panel A of Figure 3, the mean change steadily increased as the number of recommendations increased, with interventions recommending 6 or more than 6 recommendations bringing the largest change over time. The overall pattern of means is offered for descriptive purposes and further support the positive linear relation between number of recommendations and intervention efficacy that we formally tested in Table 2.

Figure 3.

Findings for Number and Type of Recommendations. Note. AU = Alcohol Use; DR = Drug Use; O = Other Category

We then calculated the mean change for each number of recommendations for alcohol use outcomes and drug use outcomes separately. As shown in Panel B of Figure 3, the mean change for alcohol use outcomes did not change much as the number of recommendations increased beyond one, suggesting that a single recommendation might be sufficient in this case. In contrast, the mean change for drug use outcomes steadily increased as the number of recommendations increased and flattened out after 4, although the quadratic pattern was not significant (see Table 2). This pattern suggested that multiple recommendations were needed to maximize efficacy in drug interventions.

Overall, the results in Table 3 show a linear association between number of recommendations and intervention efficacy for our three measures of substance use outcomes. First, although the regression analysis shows a linear effect of number of recommendations on efficacy for alcohol use, Panel B of Figure 3 indicates little benefit in adding recommendations beyond one, suggesting that a single recommendation might be enough to change alcohol use behavior. In contrast, for drug use, recommending reducing drug use along with other behaviors is most efficacious. Regarding the number of included recommendations, intervention including four or more than four recommendations appeared to be most efficacious in changing drug use behavior. These patterns provide more support for the Relative Interdependence Model for drug use than for alcohol use.

Table 3.

Exploratory Moderator Analyses.

Moderators	d [CI]	Estimated coefficients B (Standard error)	k
Is the target group alcohol / drug dependent?		0.230 (0.106) *	233
Yes	0.59 [0.37, 0.80]		69
No	0.36 [0.28, 0.43]		164
Is the intervention delivered by experts?		0.385 (0.106) **	110
Yes	0.57 [0.33, 0.80]		99
No	0.18 [0.00, 0.36]		11
Is there behavioral contract?		0.726 (0.238) *	233
Yes	1.02 [0.52, 1.72]		7
No	0.39 [0.32, 0.47]		226

^*p < .05

^**p < .01.

k is the total number of effect sizes included.

Types of Recommendations

To better understand the impact of different types of recommendations included in the interventions and further test our hypothesized models (i.e., Relative Interdependence Modal and Relative Independence Model), we then calculated the mean change for different groups of interventions on overall outcomes based on the types of recommendations included. ³

If drug use is more dependent on alcohol use than alcohol use is on drug use, then the effects of interventions targeting both alcohol and drug use may be larger than those targeting alcohol use only. As shown in Panel D of Figure 3, however, the effects of interventions targeting both alcohol and drug use appeared to be similar to those targeting alcohol or drug use only. Still, we found that intervention efficacy was especially high when it included interventions recommending reducing both drug use and another behavior compared to only drug use. However, the efficacy of interventions recommending reducing alcohol use did not improve when paired with recommendations of other behaviors.

All in all, substance use outcomes improved more when drug recommendations were combined with at least one other recommendation, such as seeking service use and support seeking. This finding provides further support for the Relative Interdependence Model for drug use. Replicating the finding that alcohol-use outcomes changed equally in response to single- and multi-recommendation interventions, a single recommendation of alcohol use achieved roughly the same level of change as did combinations of alcohol use and other recommendations such as seeking service use or reducing drug use. This finding again supported the earlier finding (see Panels B and C at Figure 3) that alcohol use is relatively more independent from other related behaviors, compared to drug use.

Exploratory Moderator Analysis

We then conducted exploratory moderator analyses to determine whether participant or intervention characteristics were associated with efficacy. All analyses were conducted with the overall samples of conditions except for those with intervention characteristics as moderators, which pertained only to intervention groups. These analyses appear in Table 3 and showed three major significant moderators. Specifically, interventions were more efficacious when they targeted an alcohol / drug dependent population (d = 0.59, 95% CI = [0.37, 0.80], k = 69) than when they did not (d = 0.36, 95% CI = [0.28, 0.43], k = 164); when they were delivered by experts (d = 0.57, 95% CI = [0.33, 0.80], k = 99) than when they were not (d = 0.18, 95% CI = 0.00, 0.36], k = 11); and when they included a behavioral contract (d = 1.02, 95% CI = [0.52, 1.72], k = 7) than when they did not (d = 0.39, 95% CI = [0.32, 0.47], k = 226). The rest of the described moderators had no significant effect.

Testing the Linearity between Number of Recommendations and Efficacy after Controlling for Significant Moderators

We lastly tested whether number of recommendations predicted intervention efficacy above and beyond each of these predictors. Specifically, we conducted three multiple regression models, each including the number of recommendations and one of the three moderators above as the predictors. The linear term of number of recommendations was still significant after controlling for targeting drug / alcohol dependent population (B = 0.067, SE = 0.020, p < .01), expert delivery (B = 0.136, SE = 0.034, p < .001), and inclusion of behavioral contract (B = 0.067, SE = 0.020, p < .01). However, only expert delivery remained as a significant predictor after controlling for the number of recommendations included (B = 0.249, SE = 0.083, p < .05). Again, these findings confirmed the positive linear relationship between number of recommendations and intervention efficacy, showing the promising prospect of multi-behavior interventions in combating non-medical substance use.

Summary

In conclusion, our analyses revealed that multi-behavior interventions were efficacious at reducing non-medical substance use behaviors, especially for drug use behaviors. The data also suggested a linear association between the number of recommendations and change, such that the more behaviors recommended, the more efficacious the interventions were, particularly for drug use outcomes. This pattern held even after controlling for alcohol / drug dependent subpopulations, facilitators’ characteristics, and inclusion of behavioral contracts. Finally, we found strong support for the Relative Interdependence Model for drug use in relation to other behaviors, particularly seeking services that support recovery. In contrast, we found less support for the Relative Interdependence Model for alcohol use, which could be targeted more independently of other outcomes and achieve the same level of success as in combination with other outcomes.

Inclusion Bias

Finally, we assessed inclusion bias in multiple ways. First, we visually inspected the funnel plot of the effect size against the standard error (see Figure 4). If the distribution of effect sizes were unbiased, the plot should resemble a funnel, with studies with greater errors (assessed as smaller sample sizes) displaying greater variability (Sterne et al., 2006). Our plot revealed a bias suggesting that some positive effects were missing across different levels of precision. Therefore, we ran Egger’s test of asymmetry (Egger et al., 1997). For our data, the intercept for Egger’s test was 4.08 (p < .001), which suggested that there is asymmetry in the distribution of effect sizes. To address the asymmetry, we ran Duval and Tweedie’s (2000) trim and fill analysis. This method first fills in any missing effect sizes and then adds them to the analysis to recalculate an adjusted effect size corrected for observed biases (Rothstein, 2005). The trim and fill analysis (see Figure 4) added 43 studies to the right side, which moved our estimate from 0.42 (CI = [0.34, 0.50] to a new adjusted d = 0.53 (CI = [0.47, 0.60]) for the intervention groups, which remained significant. All the above analyses of bias indicated that our estimated effects were overly conservative and not affected by the usual publication bias that shows reluctance to publish negative effects. In this case, the synthesis appears to be missing positive effects.

Figure 4.

Funnel plot for inclusion bias analysis (dots added by trim and fill methods are shown in white).

Given the existing criticism of the above methods as lacking a statistical model and proper evaluation (McShane et al., 2016), we then applied the selection methods to further assess and adjust for publication biases relating to the statistical significance of effect sizes. Selection methods assume that the probability of publication depends on the p-value of its effect size. In other words, different p-values tend to have different chances of getting published and therefore being included in a meta-analysis (Hedges & Vevea, 1996; Vevea & Woods, 2005). Given that our dataset showed an untraditional bias against positive effects, we run a two-tailed sensitivity analysis for our dataset. Assuming a moderate two-tailed selection bias, the adjusted effect dropped to 0.39, which is a 0.03 reduction from the original effect size. Assuming a severe two-tailed selection bias, the adjusted effect turned into 0.36, which is a 0.06 reduction from the original effect size. These sensitivity analyses did suggest some potential bias, but the bias should have very little impact on our estimated effect sizes.

Altogether, the distribution of effect sizes in our meta-analysis was asymmetric. However, the asymmetry does not suggest exclusion of negative or non-significant effects. Instead, the asymmetry indicates that positive effects might be missing, which would then lead to overly conservative estimates of the efficacy of multi-behavior interventions. The sensitivity analyses also showed that, even assuming a severe bias, the observed effect remains significant. All in all, the observed bias is most likely due to the heterogeneity in the current dataset created by different methodological variables and unlikely to nullify our conclusions.

Conclusions

Overview of Findings

Given the rising rates of alcohol-related deaths and drug overdoses in the United States, understanding intervention efficacy in this domain is critical. Our study is the first to describe the body of evidence on multiple-behavior interventions for substance use. The majority of multi behavior programs analyzed in the current meta-analysis involved 2 or 3 recommendations targeting more than one behavioral domain, either alcohol use plus drug use, alcohol use plus one other domain (e.g. lifestyle, sexual behavior, service use), or drug use plus one other domain.

Our study showed that multi-behavior interventions were overall effective in reducing non-medical substance use, more so for drug use than for alcohol use. Our results also showed a general positive linear relation between number of recommendations and overall change, which was stronger for drug use outcomes and weaker for alcohol use outcomes. Regarding the type of included recommendations, we found that inclusion of recommendations for other behaviors increases the efficacy of drug use interventions but does not seem to improve the efficacy of alcohol use interventions. Altogether, these findings supported the notion that drug use often accompanies alcohol use (i.e., Relative Interdependence Model; see Figure 1), whereas alcohol use is more independent from drug use and other related behaviors (i.e., Relative Independence Model, see Figure 1). All in all, although the heterogeneity of the data caution against broad generalizations, multi-behavior interventions seemed better for drug use, whereas single-behavior interventions might be sufficient for alcohol use.

It is important to note that this linear pattern is different from the curvilinear pattern found by Wilson et al. (2015) in the lifestyle domain, suggesting that the optimal number of recommendations to be included in intervention programs differs across domains. Although three recommendations might be sufficient to change lifestyle related behaviors (e.g. exercising, smoking) and single recommendation might be sufficient to decrease alcohol use, more recommendations are necessary to reduce drug use. These findings surrounding the domain-specific optimal number of recommendations have important implications for the design of future intervention programs. Based on our findings, alcohol use can be treated as a relatively independent problem and an alcohol intervention program can simply target the alcohol use behavior itself (i.e. aiming to reduce the amount of alcohol consumed in the target population). In contrast, intervention programs to curb non-medical drug use need to consider the interconnections among the drug use behaviors and other related behaviors. For example, these interventions may benefit from determining all the substances that a patient is currently using and all the relevant behaviors that are leading to or relevant to the use of a specific substance (e.g., lack of social support, limited access to drug treatment program). Interventions may also benefit from including recommendations targeting all of these relevant behaviors, instead of just drug use. Accordingly, when reassessing patient behavior at a follow-up point, the interventionists should not limit the assessment to just drug use but instead measure the related behaviors and a more holistic improvement.

Lastly, our exploratory moderator analyses found participant and intervention characteristics that moderate the efficacy of interventions. Specifically, interventions were more efficacious when targeting an alcohol / drug dependent population, being delivered by experts, and including a behavioral contract. Number of recommendations still remained as a significant predictor after controlling for each of these predictors, but only expert delivery remained as a significant predictor after controlling for the number of recommendations. These results, if supported by further evidence from randomized controlled trials, may also provide important guideline for future interventions. For example, future intervention programs may invest more resources in the training of the facilitators to ensure their expertise in delivering the interventions.

Limitations and Future Work

There are a few limitations to be noted for the current meta-analysis. First of all, although we included both behavioral and clinical outcomes, most of the outcome measurements included in the current meta-analyses were behavioral (roughly 90 percent), due to the fact that most studies in the substance use domain rely solely on behavioral outcomes. Since most of the behavioral outcomes are measured through self-report, various problems can rise, such as social desirability bias and participants’ lack of knowledge about health behaviors (Newell et al., 1999). To enhance the generalizability of the current findings, future research should examine the impact of multi-behavior programs on clinical outcomes.

Next, although the intervention programs included in our synthesis covered a wide range of number of recommendations (i.e., 0–11), they did not assess all possible numbers of recommendations. Past research suggests that related behavioral recommendations might compete for resources and lead to a fatigue effect (Baumeister & Heatherton, 1996; Muraven et al., 1998; Vohs & Heatherton, 2000). Although this fatigue effect does not seem to appear in our studies with up to 11 recommendations, a higher number of recommendations could still be problematic. Therefore, future research should replicate the current meta-analytic study findings by examining future multi-behavioral interventions with more than 11 recommendations to determine the effect of higher numbers of recommendations.

Lastly, there are conflicting opinions on use of effect sizes that reflect change over time as posed to treatment-control conditions, although those apply primarily to the use of pre-post designs without controls (Cuijpers et al., 2017). Despite the validity of these arguments, we believe that the problem at hand required the chosen effect size. In addition, our findings were robust to changes in the estimated correlation between the pre- and post- measures from 0.5 to 0.8 as a sensitivity analysis. The additional analyses showed that any uncertainty about this correlation could influence the point estimation of the effect but is very unlikely to alter the primary conclusions from our study. Secondly, although it is true that it is hard to separate the intervention effect from the effect coming from natural processes, we are always comparing across conditions rather than relying on a single effect.

Closing Remarks

Designing more efficacious intervention programs is key to halting the current substance use crisis. This meta-analysis found a general positive linear relation between the number of recommendations and change, which was stronger for drug use programs than for alcohol use programs. This result ensures that multi-behavior interventions are promising methods to combat the current drug use epidemic and may be an effective solution through widespread implementation. Our analyses also showed that drug use interventions appeared to be more efficacious when also targeting other related behaviors, whereas alcohol use interventions can be efficacious enough when only targeting alcohol use itself. Additionally, this meta-analysis identified multiple moderators that can influence the efficacy of this particular type of promising intervention in the substance use domain. To maximize change, interventions should be delivered by experts, which imply the necessity of greater funding in the area of behavioral health and more training of health professionals. Lastly, the efficacy of multi-behavior programs seemed robust to differences in contexts and target populations, which makes the findings generalizable. Altogether, although more work still need to be done to reach confident conclusions about the optimal number and type of recommendations, our meta-analysis contributes to our understanding of multi-behavior programs in the substance use domain and has actionable implications for the development and implementation of future intervention programs, as well as policy surrounding them.