Cognitive-Behavioral Interventions Targeting Alcohol or Other Drug Use and Co-Occurring Mental Health Disorders: A Meta-Analysis

Abstract

Aims

This meta-analysis reviewed 15 clinical trials (18 study sites/arms), examining the efficacy of an integrated cognitive-behavioral intervention (CBI) delivered to individuals with an alcohol or other drug use disorder and a co-occurring mental health disorder (AOD/MHD). Outcomes were alcohol or other drug use and mental health symptoms at post-treatment through follow-up.

Methods

The inverse-variance weighted effect size was calculated for each study and pooled under random effects assumptions.

Results

Integrated CBI showed a small effect size for AOD (g = 0.188, P = 0.061; I² = 86%, τ² = 0.126, k = 18) and MHD (g = 0.169, P = 0.024; I² = 58%, τ² = 0.052, k = 18) outcomes, although only MHD outcomes were statistically significant. Analysis by subgroup suggested that effect magnitude varied by type of contrast condition (integrated CBI + usual care vs. usual care only; integrated CBI vs. a single-disorder intervention), follow-up time point (post-treatment vs. 3–6 months) and primary AOD/MHD diagnosis, although these sub-groups often contained significant residual heterogeneity. In a series of mixed effects, meta-regression models, demographic factors were non-significant predictors of between-study heterogeneity. For AOD outcomes, greater effects were observed in higher quality studies, but study quality was not related to effect size variability for MHD outcomes.

Conclusions

The current meta-analysis shows a small and variable effect for integrated CBI with the most promising effect sizes observed for integrated CBI compared with a single disorder intervention (typically an AOD-only intervention) for follow-up outcomes, and for interventions targeting alcohol use and/or post-traumatic stress disorder. Given the clinical and methodological variability within the sample, results should be considered a preliminary, but important step forward in our understanding of treatment for co-occurring AOD/MHD.

INTRODUCTION

A major challenge in treating populations with alcohol or other drug use disorders (AOD) is that a large number also have co-occurring mental health disorders (MHD), including post-traumatic stress disorder (PTSD), mood disorders and personality disorders (Grant et al., 2015). As of 2018, ~9.3 million adults in the USA had a mental health problem along with an AOD (Substance Abuse and Mental Health Services Administration [SAMHSA], 2018). Often, AOD providers must address both AOD and MHD during counseling sessions, despite having little training in the latter (Flynn and Brown, 2008). As a result, attempts have been made to administer treatments both sequentially and parallelly. ‘Sequential care’ involves the treatment of one condition followed by the other condition; ‘parallel care’ involves the simultaneous treatment of both conditions, but without coordination between systems (Burnam and Watkins, 2006). Sequential care is limited in clear empirical guidance as to which condition should be treated first. For parallel care, acquiring and coordinating two different forms of treatment is a burden often placed on patients.

To address the above noted issues, 74.8% of mental health treatment facilities in the USA have attempted to administer ‘integrated care’ to patients with co-occurring disorders as of 2018 (Spivak et al., 2020). This entails a coordinated healthcare approach, where treatment for the AOD and MHD is delivered by the same team of caregivers and substance use and mental health interventions are combined in one clinical program. Most models of integrated treatment also involve multidisciplinary case-handling and provide mental health skill development along with outreach (Drake et al., 1998). This eases the burden on the patients who no longer have to coordinate between two systems, often acting as their own advocates for quality healthcare. Integrated treatments for AOD and MHD may also help to address treatment delays for MHDs that are experienced disproportionately by individuals with AODs (Patel et al., 2015). Given that integrated approaches are relatively new, but have already been implemented in clinical settings, additional research is needed to understand and develop guidelines for best practices.

Previous reviews have looked at the comparative efficacy of integrated versus non-integrated interventions for co-occurring AOD and MHD (AOD/MHD), but findings are mixed in this emerging literature. A narrative review and meta-analyses of both randomized and non-randomized trials showed similar improvement in trauma and AOD symptoms regardless of whether treatment was integrated (number of studies (k) = 17, 0 overlap of primary studies with the current review; Torchalla et al., 2012). A meta-analysis of 15 studies indicated that outcomes for co-occurring AOD and depression/anxiety could be improved moderately (standardized mean difference [d range]) = 0.15–0.36) by combining AOD treatment with mental health treatments such as cognitive-behavioral therapy or psychopharmacology (one primary study overlaps with the current review; Hobbs et al., 2011). Furthermore, a meta-analysis of 12 studies found that depression outcomes were achieved faster than AOD outcomes in integrated cognitive-behavioral and motivational interventions, leading authors to hypothesize a ‘sleeper effect’, or a delay in the application of AOD coping skills (one primary study overlaps with the current review; Riper et al., 2014). However, it is uncertain whether this pattern can be replicated in the current research with a broader range of MHD conditions, and with a sample of studies that are primarily independent of those in prior reviews.

A common conclusion from the previous studies is that more research needs to be conducted on integrated interventions, their efficacy and underlying mechanisms. For example, it is important not to dismiss the little evidence present in favor of integrated treatments as more research could inform better recommendations for future implementation (Torchalla et al., 2012). Even studies with small effects have demonstrated the need for further investigation since it is unclear whether this is enough to warrant the implementation of integrated interventions, which could be costlier to implement than non-integrated interventions (Hobbs et al., 2011). Integrated treatment is also a relatively new idea and thus one that requires more examination, randomized controlled trials and quantitative review. Moreover, only one of the reviews noted here looked specifically at a single evidence-based modality (Riper et al., 2014), and we expand this work with a more heterogeneous diagnostic sample, as diagnostic variability is highly representative of frontline clinical populations (National Survey of Substance Abuse Treatment Services; SAMHSA, 2018).

Cognitive-behavioral models are evidence-based for adult AOD as well as numerous mental health conditions (Chambless and Hollon, 1998). For AOD, components include teaching patients to change their thought processes and behaviors, to learn a range of coping skills and to use social supports to reduce substance use (Carroll, 2004). Homework is often assigned and completed outside of therapy sessions, and modeling and behavioral practice occurs during therapy sessions to facilitate the uptake of new behavioral skills (Kadden et al., 1992). Integrated cognitive-behavioral interventions (CBIs) for treating co-occurring disorders utilizes these components to target both AOD and MHD symptomology and may add additional cognitive-behavioral components that are MHD specific.

Current study aims

Given co-occurrence rates between AOD and MHD and the use of CBIs as a standard of care for these populations individually (i.e. AOD or MHD), we conducted a meta-analysis on the efficacy of integrated CBIs for AOD/MHD. Aim one explored the overall pooled effect size of an integrated CBI (i.e. AOD/MHD) in contrast to a comparison condition for substance use and mental health symptom outcomes. Here, the substantive question was about the efficacy of integrated CBI over single-disorder or other usual care interventions. Aim two examined these effect sizes in subgroups by follow-up time point, type of contrast condition and targeted disorder. Given the clinical heterogeneity of our sample (i.e. a range of substance use and mental health conditions), potential clinical and methodological moderators of effect size variability were explored. The pooled effect estimates were additionally examined in sensitivity analyses (i.e. publication bias).

MATERIALS AND METHODS

Primary study inclusion

Studies meeting inclusion criteria were English language, peer-reviewed articles published between 1990 and 2019. These were outcome reports of randomized controlled trials that included both substance use and mental health outcomes. The targeted population was adults (age ≥ 18) meeting criteria for an AOD and at least one co-occurring MHD (DSM III-R through V; American Psychiatric Association, 1987, 1994, 2000, 2013). The treatment must have been identified as cognitive-behavioral or based on a cognitive-behavioral approach. Commonly reported intervention components were functional analysis, relapse prevention, affect management, and social and life skills training. These cognitive-behavioral therapies must have been integrated and thus also included components targeting mental health symptoms, such as exposure-based interventions, medication management or an exploration of the relationship between mental health symptoms and substance use (see Supplemental Table 1 for details). While trials of Seeking Safety, an integrated CBI for AOD and PTSD (Najavits, 2002), might have met our inclusion criteria, this evidence-based model was the subject of recent review (Torchalla et al., 2012; Roberts et al., 2015) and is thus not reviewed here.

Literature search

A literature search was conducted through December of 2019 to identify eligible studies for a large-scale, meta-analytic project on cognitive-behavioral therapy in addictions care (R21AA026006). The first step involved a title, abstract and keyword search by treatment (‘cognitive-behavioral therapy’ OR ‘relapse prevention’ OR ‘coping skills training’), AND outcome (‘alcohol’ OR ‘cocaine’ OR ‘methamphetamine’ OR ‘stimulant’ OR ‘opiate’ OR ‘heroin’ OR ‘marijuana’ OR ‘cannabis’ OR ‘illicit drug’ OR ‘substances’ OR ‘polysubstance’ OR ‘dual disorder’ OR ‘dual diagnosis’ OR ‘co-occurring disorder’) AND study terms (‘efficacy’ OR ‘randomized controlled trial’ OR ‘randomized clinical trial’) in the PubMed database. Then, a search of the Cochrane Register and EBSCO database (i.e. Medline, PsycARTICLES) was performed, removing duplicates from the results of the PubMed search. Abstract screening occurred by two raters in Abstrackr (Wallace et al., 2012). A bibliographic search of topically related systematic reviews and meta-analyses was also performed to identify any candidate studies not identified by the original search methods (e.g. Hobbs et al., 2011; Torchalla et al., 2012; Riper et al., 2014). Figure 1 provides a visual representation of study inclusion for the present report, following PRISMA guidelines (Moher et al., 2009).

Fig. 3.

Mental health outcomes. All observed values are plotted by precision where high effect studies would be assumed to have low precision if bias were present. Figure shows some asymmetry and regression intercept is marginally significant (b = 2.26, P = 0.051). High precision, negative effect study is Hunter et al. 2012b, site 2.

Fig. 1.

PRISMA flowchart of study inclusion. k = number of studies. CBT = cognitive-behavioral therapy. An updated search that was conducted in PUBMED for eligible studies between December 2019 and January 2021 was completed and yielded no new eligible studies.

Primary study characteristic variables

Study characteristic variables were used as sample-level descriptors and potential effect size moderators. The following variables were considered: mean age of sample, percent female participants, percent White participants (While a more specific treatment of racial and ethnic distribution in samples would have been desirable, reporting of this information was inconsistent across studies, and resulted in a high proportion of missing data.), primary AOD (i.e. alcohol use disorder vs. other), primary MHD (i.e. PTSD, depression/anxiety, vs. other), type of comparison condition (i.e. integrated CBI + usual care vs. usual care only; integrated CBI vs. a single-disorder intervention; see Supplemental Table 1 for details on contrast conditions in primary studies), outcome time point (i.e. post-treatment vs. follow-up [3–6 months]) and outcome level risk-of-bias (Higgins et al., 2011). Data extraction guidelines were detailed in a study codebook available, upon request, from the contact author. Data were extracted in two independent passes conducted by trained raters and showed a between-rater agreement rate of 93%. Final data entry, where disagreement was observed, required a consensus review with the contact author.

Primary study outcome variables

Hedges’ g was used to calculate effect sizes for efficacy outcomes in this meta-analysis. (Hedges’ g is a variation on Cohen’s d with a slight adjustment for small sample bias. Effect sizes can be interpreted using the following generic benchmarks: 0.2 ‘small’, 0.5 ‘medium’ and 0.8 ‘large’—Cohen, 1988.) Primary studies often provided data on more than one outcome; therefore, data for effect size estimation were selected based on a decisional hierarchy for AOD and mental health symptom outcomes. For AOD outcomes, we prioritized the measures of frequency or quantity in the form of means and standard deviations, followed sequentially by sample proportions or other outcomes (e.g. diagnostic measures [Addiction Severity Index; McLellan et al., 1980]). For mental health symptom outcomes, we prioritized self-report measures of symptoms using psychometrically validated measures (e.g. Beck’s Depression Inventory; Beck et al., 1996), followed by measures of global functioning (e.g. Global Assessment of Functioning; DSM III-R through V; American Psychiatric Association, 1987, 1994, 2000). When multiple months of follow-up data were provided, the latest time point was selected. Effect sizes were reverse scored for negative outcomes (e.g. number of days drank; number of depression symptoms) so that positive effect sizes would indicate a positive treatment outcome. When data from publications were insufficient for effect size calculation, first and second authors were contacted for raw data requests. There were two eligible studies removed due to author non-response.

Data analysis

Prior to pooling, standardized effect sizes were inverse-variance weighted to allow larger studies more influence on the overall estimate (Hedges and Olkin, 1985). Effect sizes were pooled using a random effects model, which assumes a distribution for the population effect size with both systematic and random sources of heterogeneity (Hedges and Vevea, 1998). The I² value provided a percent estimate of systematic heterogeneity with values exceeding 50%, considered ‘substantial’ (Higgins and Green, 2011). For heterogeneous substance use and mental health outcomes, subgroup analysis and a series of meta-regression models explored potential moderators of effect sizes. Additional sensitivity analyses were conducted for heterogeneity and publication bias, including a visual inspection of funnel plots and a test for funnel plot asymmetry using Egger’s regression test of the relationship between study effect size and precision (Egger et al., 1997).

Results

Primary study descriptive characteristics

The sample included 15 randomized trials with 18 study sites/arms, targeting CBIs for adult AOD/MHD. The sample size of included studies ranged from 30 participants (Ball, 2007) to 344 (Kushner et al., 2013) and totaled at N = 1914. The mean participant’s age was 39 years (SD = 5), and an average of 40% of participants were female (SD = 17%). Although reporting of race and/or ethnicity was inconsistent, White participants made up 61% (SD = 31%) of the samples, on average. Studies primarily targeted polydrug use (67%) followed by alcohol use (28%). For MHD, PTSD was the most common diagnosis targeted (44%), followed by mood and/or anxiety disorders (39%) and ‘other’ (i.e. attention-deficit hyperactivity disorder, personality disorder, schizophrenia, suicidality). Study-level risk-of-bias assessment showed that 50% of studies were high risk (Higgins et al., 2011; see Supplemental Figs 1 and 2 for details). Studies designated as high risk were typically due to: (a) no pre-specified outcome assessment plan, (b) presence of differential or high attrition and (c) no report of blinding of study personal. Table 1 describes each study with respect to key design characteristics and AOD and MHD effect estimates.

Table 1.

Study-level descriptive data and effect size estimates

First author (date)	N	Treatment	Comparison condition	AOD   outcome measure	g(SE)  1	MHD   outcome measure	g(SE)  2
Acosta et al. (2017)    Ball (2007)    Brown et al. (2006)  Eack et al. (2015)  Hunter et al. (2012a)— study site 1  Hunter et al. (2012b)— study site 2  Kushner et al. (2013)  McGovern et al. (2011)  McGovern et al. (2015) study site 1  McGovern et al. (2015) study site 2  Mills et al. (2012)  Morley et al. (2014)  Morley et al. (2016)  Randall et al. (2001)  Ruglass et al. (2017)– study site 1  Ruglass et al. (2017))—study site 2  Soder et al. (2019) van Emmerik – van-Oortmerssen (2019)	162 30 66 31 73 299 344 53 119 111 103 185 37 93 82 67 37 119	Thinking Forward + usual VA services Dual Focus Schema Therapy Integrated Cognitive-Behavioral Therapy Cognitive Enhancement Therapy + usual clinic services Group Cognitive-Behavioral Therapy for Depression and Substance Use Group Cognitive-Behavioral Therapy for Depression and Substance Use + Usual care Hybrid Cognitive-Behavioral Therapy Integrated Cognitive-Behavioral Therapy Integrated Cognitive-Behavioral Therapy Integrated Cognitive-Behavioral Therapy + Usual care Concurrent Treatment of PTSD and AOD using prolonged exposure + Usual care Opportunistic CBI Package + Usual care Specialized Integrated Treatment for Alcohol and Comorbid Anxiety and/or Mood Disorder Integrated Cognitive-Behavioral Therapy Concurrent Treatment of PTSD and AOD using Prolonged Exposure Concurrent Treatment of PTSD and AOD using Prolonged Exposure Treatment of Integrated PTSD and AOD Integrated Cognitive-Behavioral Therapy	Usual VA Services Twelve-Step Facilitation Twelve-Step Facilitation Usual Clinic Services Group Addiction Counseling Usual Care Progressive Muscle Relaxation Training Individual Addiction Counseling Individual Addiction Counseling Usual Care Usual Care Usual Care Treatment for Alcohol Only Cognitive-Behavioral Therapy for Alcohol Only Relapse Prevention Therapy for AOD Assessment/monitoring Cognitive-Behavioral Treatment for AOD Cognitive-Behavioral Treatment for AOD	alcohol days drinking polydrug days used polydrug days abstinent polydrug proportion decreased use polydrug proportion abstinent polydrug days abstinent alcohol days used polydrug days used polydrug proportion not relapsed polydrug proportion not relapsed polydrug proportion abstinent alcohol drinks/ drinking day alcohol days abstinent alcohol days abstinent polydrug proportion abstinent polydrug days used polydrug days used polydrug heavy use days	0.055(0.156) −0.058(0.019) 0.432(0.284) 0.770(0.589) 0.023(0.275) 0.370(0.117) 0.203(0.108) 0.592(0.283) 0.538(0.211) 0.563(0.211) −0.281(0.261) −0.888(0.161) 2.764(0.457) −0.360(0.208) −0.222(0.285) 0.501(0.249) 0.120(0.332) 0.078(0.182)	PTSD PTSD checklist for DSM - Military personality disorder Multiple Affect Adjective Check List-Revised depression Hamilton Depression Rating Scale schizophrenia composite measure of symptoms depression Beck Depression Inventory-II (BDI-II) depression BDI-II anxiety State–Trait Anxiety Inventory (STAI) PTSD Clinician-administered PTSD scale (CAPS) PTSD CAPS PTSD CAPS PTSD CAPS suicidality Beck Scale for Suicide Ideation anxiety/depression composite  depression anxiety stress scales social anxiety composite social phobia and anxiety PTSD CAPS PTSD CAPS PTSD CAPS ADHD ADHD Rating Scale	0.142(0.157) 1.304(0.409) 0.125(0.297) 0.205(0.394) 0.541(0.284) −0.451(0.117) 0.122(0.108) 0.130(0.277) 0.225(0.165) 0.111(0.164) 0.446(0.199) 0.329(0.155) 0.181(0.325) 0.022(0.206) 0.022(0.206) 0.175(0.245) 0.046(0.332) 0.298(0.183)

Notes. K = 15 studies with 18 study sites/arms. ¹Effect size for substance use outcome at latest follow-up. ²Effect size for mental health outcomes at latest follow-up. N = Total participants randomized; g = Hedges’ g; SE = standard error. CBT = cognitive-behavioral therapy; AOD = alcohol and other drugs; ADHD = attention deficit hyperactivity disorder. For a listing and citation of all measures used, see Supplemental Table 2.

CBI effect on substance use: overall and by subgroup

Primary study effect sizes were pooled for AOD outcome as well as by contrast condition type (i.e. integrated CBI + usual care vs. usual care only; integrated CBI vs. single-disorder intervention), follow-up time point (i.e. post-treatment; 3–6 months) and by primary MHD group (i.e. PTSD; depression/anxiety; other; see Table 2 for summary). The overall pooled effect size was small, non-significant and heterogenous at g = 0.188 (95% CI = −0.009, 0.386, P = 0.061; I² = 86%, τ = 0.355; k = 18). The pooled effect size for the CBI + usual care versus usual care only contrast was not statistically significant at g = 0.042 (95% CI = −0.450, 0.534, P = 0.867; I² = 90%, τ = 0.559; k = 6) but was significant for integrated CBI versus a single-disorder intervention with g = 0.274 (95% CI = 0.034, 0.513, P = 0.025; I² = 84%, τ = 0.349; k = 12). The pooled effect size at post-treatment was non-significant at g = −0.121(95% CI = −0.313, 0.071, P = 0.218; I² = 78%, τ = 0.276; k = 13) but was significant at later follow-up with g = 0.280 (95% CI = 0.043, 0.518, P = 0.021; I² = 80%, τ = 0.371; k = 13). Pooling by MHD yielded positive and significant results only for PTSD samples with g = 0.245 (95% CI = 0.002, 0.489, P = 0.048; I² = 54%, τ = 0.253; k = 8). Results were non-significant for depression and/or anxiety disorder samples with g = 0.262 (95% CI = −0.244, 0.768, P = 0.310; I² = 93%, τ = 0.639; k = 7) and for other disorders with g = −0.023 (95% CI = −0.178, 0.131, P = 0.769; I² = 19%, τ = 0.084; k = 3). Figure 2 does not suggest bias due to publication status (e.g. small sample studies only published when treatment effects are significant).

Table 2.

Summary of pooled effect sizes—overall and by sub-group

		g	95% CI	P	I  2	τ	k
Overall pooled size effect	AOD	0.188	−0.009, 0.386	0.061	86%	0.355	18
	MHD	0.169	0.022, 0.316	0.024	58%	0.229	18
By contrast condition
CBI + usual care versus usual care	AOD	0.042	−0.450, 0.534	0.867	90%	0.559	6
	MHD	0.110	−0.206, 0.426	0.495	81%	0.334	6
Integrated CBI + single-disorder intervention	AOD	0.274	0.034, 0.513	0.025	84%	0.349	12
	MHD	0.188	0.067, 0.309	0.002	2%	0.035	12
By outcome time point
Post-treatment	AOD	−0.121	−0.313, 0.071	0.218	78%	0.276	13
	MHD	0.100	−0.137, 0.336	0.409	75%	0.363	13
At follow-up	AOD	0.280	0.043, 0.518	0.021	80%	0.371	13
	MHD	0.119	−0.040,0.279	0.143	59%	0.213	13
By primary disorder
PTSD	AOD	0.245	0.002, 0.489	0.005	54%	0.253	8
Anxiety/Depression	AOD	0.262	−0.244, 0.768	0.310	93%	0.639	7
Other MHD	AOD	−0.023	−0.178, 0.131	0.769	19%	0.084	3
Alcohol	MHD	0.160	0.022, 0.298	0.023	0%	0.000	5
Other drugs	MHD	0.193	−0.025, 0.410	0.083	69%	0.316	13

Notes. AOD = alcohol and other drugs; MHD = mental health disorder; CBI = cognitive behavioral intervention.

Fig. 2.

Substance use outcomes. All observed values are plotted by precision where high effect studies would be assumed to have low precision if bias were present. Figure and regression test do not suggest bias due to publication status (b = 1.31, P = 0.079).

CBI effect on mental health symptoms: overall and by subgroup

Primary study effect sizes were pooled for MHD outcome as well as by contrast condition type (i.e. integrated CBI + usual care vs. usual care only; integrated CBI vs. single-disorder intervention), follow-up time point (i.e. post-treatment; 3–6 months) and by primary drug group (alcohol; other; see Table 2 for summary). The overall pooled effect size was positive, significant and had less heterogeneity than the AOD pooled effect, at g = 0.169 (95% CI = 0.022, 0.316, P = 0.024; I² = 58%, τ = 0.229; k = 18). Pooling by contrast type showed that CBI + usual care versus usual care only was non-significant at g = 0.110 (95% CI = −0.206, 0.426, P = 0.495; I² = 81%, τ = 0.334; k = 6) but significant for integrated CBI versus a single-disorder condition at g = 0.188 (95% CI = 0.067, 0.309, P = 0.002; I² = 2%, τ = 0.035; k = 12). The pooled effect size was non-significant when pooled at post-treatment g = 0.100 (95% CI = −0.137, 0.336, P = 0.409; I² = 75%, τ = 0.363; k = 13) and at later follow-up g = 0.119 (95% CI = −0.040, 0.279, P = 0.143; I² = 59%, τ = 0.216; k = 13). When grouped by primary AOD, the effect size was positive, significant and homogeneous for alcohol studies at g = 0.160 (95% CI = 0.022, 0.298, P = 0.023; I² = 0%, τ = 0.000; k = 5), and for other drugs, it was non-significant at g = 0.193 (95% CI = −0.025, 0.410, P = 0.083; I² = 69%, τ = 0.316; k = 13). Figure 3 shows some asymmetry although it is unclear whether this is due to small sample bias or, simply, overall heterogeneity.

Meta-regression by patient and study-level factors

Table 3 summarizes findings for demographic factors (i.e. mean age, gender and race of participants), along risk of bias designation for the studies. For AOD outcomes, low risk of bias was significant with b = 0.575 (SE = 0.263, z = 2.19, P = 0.029), suggesting higher effect sizes when bias was low. The Q value for this model, however, showed the difference in variance explained by the model which was not significant at Q = 4.82 (P = 0.09). For MHD outcomes, no regression covariates were significant.

Table 3.

AOD and MHD meta-regression results

	β	SE	z	P
AOD outcomes
Mean age of participants	0.031	0.024	1.270	0.202
Percent female participants	0.005	0.007	0.790	0.429
Percent White participants	0.001	0.003	0.270	0.784
Risk of bias: Low risk	0.575	0.263	2.190	0.029
Risk of bias: Some concerns	0.252	0.251	1.000	0.316
MHD outcomes
Mean age of participants	−0.009	0.019	−0.480	0.629
Percent female participants	0.001	0.005	0.250	0.801
Percent White participants	0.002	0.003	0.840	0.399
Risk of bias: Low risk	0.075	0.176	0.430	0.670
Risk of bias: Some concerns	−0.205	0.170	−1.210	0.228

Notes. AOD = alcohol and other drugs. SE = standard error. These are a series of simple regressions. Risk of bias reference for low and some concerns is ‘high’. Because only one coefficient was significant, multivariable models were not considered.

Discussion

This meta-analysis examined integrated CBI efficacy for AOD and MHD outcomes at post-treatment and at 3- to 6-month follow-up across 15 randomized controlled trials published in the last 30 years. We considered intervention effects for substance use and mental health symptom outcomes under two key implementation conditions: integrated CBI combined with usual care in contrast to usual care only and integrated CBI in contrast to a single disorder treatment (i.e. most often AOD treatment only). To our knowledge, this is the first meta-analysis of this nature, and we consider our findings in the context of a small, burgeoning meta-analytic and review literature on the treatment of co-occurring substance use and MHDs. In a clinical area that was referred to as ‘mission impossible’ in a previous review (Chow et al., 2013), understanding study-level predictors of efficacy is arguably important for guiding recommendations for future research and frontline care.

When pooling effect sizes, across a clinically heterogeneous sample of CBI trials, overall effect estimates were positive and non-significant for substance use outcomes (g = 0.188, P = 0.061) and positive and significant for mental health symptom improvement (g = 0.169, P = 0.024). In the current review, it also appeared that the significance of effects was driven by the method of implementation. Specifically, for both outcomes, we found a small but significant effect size (g = 0.188 and 0.274, respectively) for integrated CBI versus a single-disorder intervention, but not for the addition of integrated CBI to usual community services. When converted to a more clinically intuitive metric, our results suggest that 57–60% of integrated CBI participants had better outcomes than the mean outcome for single-disorder condition participants. Previous studies have cited insufficient evidence for the superiority of integrated over non-integrated treatments for psychiatric and substance use outcomes in co-occurring disorders (Torchalla et al., 2012). However, our results indicate that integrated CBI treatments specifically may result in improvements for substance use and mental health symptoms in comparison to interventions that target a single disorder. Examples of these interventions were 12 step-based interventions, community drug counseling and even single-disorder focused CBIs. In other words, the bar for demonstrated efficacy was rather high. When the single-disorder CBI contrast studies were removed (Randall et al., 2001; Soder et al., 2019; van Emmerik-van Oortmerssen et al., 2019), effect sizes were substantially larger for substance use (g = 0.485, P = 0.005) and stable for mental health (g = 0.256, P = 0.003) outcomes. Therefore, this study supports the notion of integrated CBI treatment over a range of single disorder treatments, but there may be a ceiling effect when attempting to add integrated services to existing community-based care.

The current study found that effect estimates for substance use outcomes varied with time. While the overall pooled effect size favored the selection of later follow-up data points (i.e. with a goal of being more conservative in treatment effect estimation), analyses by follow-up showed that treatment effects were non-significant at post-treatment. In contrast, we found a significant effect of integrated CBI for AOD outcomes at 3- to 6-month follow-up. The effect size was also larger at later follow-up for AOD outcomes (g = −0.121 and 0.280) but stayed relatively stable for MHD outcomes (from g = 0.100 to 0.119). This suggests a need for targeted interventions to ensure the maintenance and durability of integrated CBI effects on MHD outcomes. The AOD outcomes reflect a pattern of findings, a ‘sleeper effect’, that has been observed in previous reviews (Riper et al., 2014) and could suggest that patients learn skills during treatment that they continue to apply after treatment to promote abstinence and prevent relapse (Carroll, 1996). It is not clear why this effect was not observed for mental health symptoms, but the lack of concordance in outcomes warrants further study, particularly as the definition of optimal outcomes for addictions treatment trials continues to evolve (e.g. Kiluk et al., 2018).

In this clinically heterogeneous sample, we found that effect sizes for specific outcomes varied by the primary disorder targeted. For AOD outcomes, studies with populations with PTSD showed a positive and significant effect size with moderate heterogeneity (g = 0.245, P = 0.048; I² = 54%; k = 8), but not for other diagnostic sub-groups. Depression and anxiety disorder samples were grouped together due to a small number of studies available for analysis, and this categorization may have been too broad. For MHD outcomes, studies with alcohol using populations showed a small, significant and homogeneous effect size (g = 0.160, P = 0.023; I² = 0%; k = 5), while polydrug samples were non-significant. Overall, this suggests that integrated CBI could be more beneficial for some conditions compared with others. However, this speculation should be viewed with caution due to the small number of studies targeting each disorder and/or combination of disorders. For example, previous research has found differences in outcome results for anxiety compared to depression, citing larger effect sizes for anxiety (i.e. generalized anxiety, panic disorder and social phobia; Hobbs et al., 2011).

Limitations

Our findings need to be interpreted with caution due to the clinical and statistical heterogeneity within the sample. Furthermore, meta-regression analyses were unable to determine systematic sources of this variability. Due to the nature of our research aims, which considered integrated CBI for co-occurring disorders, there may be concern about comparing ‘apples to oranges’ and underpowered moderator analysis even in the context of high statistical heterogeneity (Wilson, 2000). As a result, we consider our results as preliminary and adding to the emerging review literature on this important topic. We also believe that the subgroup analyses allowed us to inspect some of the systematic differences between primary studies. For example, a particular benefit was observed for alcohol studies and for studies with those affected by PTSD. We also excluded roughly 10 trials on the PTSD treatment Seeking Safety from our review, and this could be viewed as a limitation. However, this evidence-based CBI treatment has been the subject of two prior meta-analyses (Torchalla et al., 2012; Roberts et al., 2015) and is the subject of an ongoing, large-scale meta-analytic project on integrated treatments for alcohol use disorders and PTSD (R01AA02583). An additional caveat is the absence of empirical benchmarks for interpreting the magnitude of the effect observed and instead relying on Cohen’s (1988) guidelines, which are generic to any form of effect estimation, meta-analytic or otherwise. Finally, 50% of our MHD outcome studies and 44% of our AOD outcomes studies received a ‘High Risk’ designation for risk of bias. This suggests that study quality could have an effect on our findings, although meta-regression suggested this was only marginally the case (and the direction was the opposite of what is typically the concern [i.e. low-quality studies have higher effect sizes]).

Conclusion

The current meta-analysis shows a small and variable effect for integrated CBI, with the most promising effect sizes observed for integrated CBI compared with a single disorder intervention (typically a non-CBI, AOD intervention) at later time points and for interventions targeting alcohol use and/or PTSD. Given clinical and methodological variability within the sample, results provide preliminary, but important data to guide future research and intervention design for this under-served and under-studied population. References marked with an asterisk (*) are included in the meta-analysis.

Data availability statement

The contact author will make the meta-analytic data available upon request.