An Evaluation of Cognitive Behavioral Therapy for Substance Use Disorder: A Systematic Review and Application of the Society of Clinical Psychology Criteria for Empirically Supported Treatments

Abstract

Cognitive behavioral therapy (CBT) is a commonly used treatment for substance use disorders (SUDs) but has not been evaluated using the American Psychological Association’s “Tolin Criteria” for determining the empirical basis of psychological treatments. The current systematic review evaluated five meta-analyses of CBT for SUD. One meta-analysis had sufficient quality to be considered in the evaluation of effect sizes. CBT produced small to moderate effects on substance use when compared to inactive treatment and was most effective at early follow-up (1–6 months post-treatment) compared to late follow-up (8+ months post-treatment). Sensitivity analyses including all five meta-analyses found similar results. A “strong recommendation” was provided for CBT as an empirically supported treatment for SUD, based on effects on substance use, quality of evidence, and consideration of contextual factors (e.g., efficacy in diverse populations).

Introduction

Substance use disorders (SUDs) are heterogeneous conditions characterized by the recurrent use of a substance that results in harm, distress, or impairment. According to the 2019 National Survey on Drug Use and Health, approximately 7.7% of United States individuals aged 18 or older had a substance use disorder in the past year (Substance Abuse and Mental Health Services Administration, 2020). Heavy and prolonged substance use is associated with a range of consequences, including physical (e.g., death, injury, infection), psychological (e.g., exacerbation of other mental health symptoms), social (e.g., family stress), and economic (e.g., unemployment or underemployment) problems (American Psychiatric Association, 2013; Compton et al., 2007; Schulte & Hser, 2013; Substance Abuse and Mental Health Services Administration [SAMHSA], 2013). To reduce and address substance-related risk and impairment, it is imperative to identify empirically supported treatments¹ for SUDs².

Cognitive behavioral therapy (CBT) is a psychosocial treatment that is premised on learning principles. CBT aims to reduce symptoms and improve functioning through targeting behavioral and cognitive processes that underlie or contribute to psychological disorders. Specifically, CBT for SUD frames the use of substances as a behavior that can be both positively and negatively reinforced and is influenced by social and other environmental contexts. CBT for SUD focuses on intervening upon these processes through increasing awareness of antecedents and consequences of use, building skills that address internal or external antecedents, and leveraging behavior change principles to reduce or eliminate substance use (e.g., through introducing rewarding alternatives to substance use) (Carroll & Kiluk, 2017; McCrady, 2000; McHugh et al., 2010). CBT for SUD can be used as a monotherapy or as adjunct (e.g., to treatment as usual, pharmacotherapy, or other behavioral interventions like Contingency Management).

A large and growing body of literature indicates CBT is an efficacious treatment for a variety of SUDs, both as a monotherapy and an adjunct, with effect sizes ranging from small to large depending on the substance (Carroll & Kiluk, 2017; Magill et al., 2019; McHugh et al., 2010). Data from the 2020 National Survey of Substance Abuse Treatment Services, which surveys United States treatment facilities, suggests that a notable number of facilities indicated frequently using relapse prevention (96%) or CBT (94%) in the treatment of SUD (SAMHSA, 2020). The Veteran’s Administration (Veteran’s Administration, n.d.) and National Institute on Drug Abuse (NIDA; National Institute on Drug Abuse, 2018) both recommend CBT for SUD as an evidence-based approach to addiction treatment. The apparent acceptability of CBT (as well as relapse prevention, which includes many of the same principles) and widespread utilization among providers and facilities would suggest real world efficacy and indicates the need for formal recognition as an evidence-based treatment.

The Society of Clinical Psychology (Division 12 of the American Psychological Association) is a leader in evaluating the empirical basis of psychological treatments. Although CBT for SUD has not yet been evaluated, per the Division 12 website, Motivational Enhancement Therapy plus CBT is an empirically supported treatment for “mixed substance abuse/dependence” (strong research support).³ Other researchers have applied the Division 12 criteria to CBT and relapse prevention for SUD and concluded that these treatments met criteria for empirically-supported treatments, but these evaluations were not formally adopted by Division 12, rendering this information invisible to clients, providers, insurance companies, among other key parties (McCrady, 2000; McGovern & Carroll, 2003). Furthermore, these designations are based on outdated criteria for evaluating empirically supported treatments (Chambless & Hollon, 1998; Chambless & Ollendick, 2001).⁴

In 2015, the criteria for empirically supported treatments were revised and updated to account for the increased volume of treatment outcome research as well as improved methodological rigor and evaluation tools (see Tolin, McKay, et al., 2015 for discussion of specific updates made). The criteria require more stringent evidence for the efficacy of treatments, as well as explicit consideration of key contextual factors such as efficacy among people from diverse backgrounds and cost. The Society for Clinical Psychology subsequently adopted these revised criteria, referred to as the “Tolin Criteria,” and established procedures for updating empirically supported treatment designations under the Tolin Criteria. To date, the Tolin Criteria have been applied to Exposure and Response Prevention for Obsessive-Compulsive Disorder (Tolin, Melnyk, et al., 2015), Cognitive Behavioral Therapy for Insomnia (Boness et al., 2020), and Contingency Management for Substance Use Disorders (Pfund et al., 2021). The adoption of the Tolin Criteria by the Society of Clinical Psychology and the subsequent application of the criteria to specific treatments is important because the Society of Clinical Psychology is seen as an authority on evidence-based treatment. Therapists and other key parties, such as insurance companies, use this information to inform treatment selection and reimbursement, for example. Taken together, there is a need to directly evaluate CBT for SUDs alone (i.e., in the absence of Motivational Enhancement Therapy) per the Tolin Criteria, which has the potential to provide more rigorous and contextualized recommendation than have been previously applied to CBT for SUD.

The purpose of the present evaluation was to apply the Tolin Criteria to CBT for SUD and to provide an evaluation of the quality of the evidence for CBT as an empirically supported treatment for SUD. Here, CBT is defined as a multisession intervention that targets cognitive, affective, behavioral, and/or environmental risks for substance use and provides training in skills to help an individual achieve and maintain substance use abstinence, moderation, or reduced substance-related harm. Consistent with prior meta-analyses in the areas of CBT for substance use (e.g., Magill et al., 2019), relapse prevention (RP) and coping skills training cognitive-behavioral interventions were also considered, given such treatments use critical CBT elements, such as functional analysis, avoidance of high-risk situations, and drug refusal skills, among others. Although there are other efficacious cognitive and behavioral treatments for SUD (e.g., mindfulness-based interventions, contingency management, motivational interviewing; see Ray et al., 2019 for a more comprehensive overview of SUD treatments), these were not considered in the present manuscript, given their hypothesized active ingredients and mechanisms of behavior change are sufficiently different from CBT to warrant separate evaluations.

Methods

The present evaluation followed the methods and procedures recommended by the APA Division 12 Committee on Science and Practice (Boness et al., 2021) and are described in further detail below. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Page et al., 2021) were also followed. Although the evaluation was not formally pre-registered, the authors submitted a letter of interest (LOI) to the Committee on Science and Practice to evaluate CBT for substance use disorder on September 23, 2021. The LOI was approved on October 1, 2021.

Search Strategy

Seven databases, including Scopus, MEDLINE, ScienceDirect, PsycINFO, Social Sciences Citation Index, Science Citation Index, and Academic Search Premier were searched to identify meta-analyses of CBT for SUDs on September 24, 2021. Search terms included the following: (TI ((“Cognitive Behavior* Therapy” or “CBT” or “relapse prevention” or “coping skills training” or “skills training”) AND (addiction or alcohol or drug or substance or abuse or dependence or cocaine or opioid or cannabis or marijuana or heroin or cocaine or amphetamine or prescription drug))) AND ((TI (Review or “Systematic Review” or “Quantitative Review” or “meta analysis”) OR (SU(Review or “Systematic Review” or “Quantitative Review” or “meta analysis”))) NOT TI (“qualitative review” or “narrative review”) where TI = title and SU = subject. The search was restricted to the University of Missouri’s peer-reviewed library collections, “reviews” from the “filter by format” option was selected. There were no restrictions on year of publication⁵ or language.

Inclusion criteria included being a quantitative review focused on CBT for substance use disorders, substance use, or substance-related problems among adults. Exclusion criteria included lack of reporting on substance use-related outcomes (e.g., quantity, frequency of use) and a sole focus on CBT + pharmacotherapy. To limit the scope and draw targeted conclusions, quantitative reviews that only focused solely on CBT for nicotine dependence were excluded, given that meta-analyses examining CBT for substance use have generally excluded trials focused on nicotine dependence (e.g., Dutra et al., 2008; Magill et al., 2019). Quantitative reviews focused only on Behavioral Couples Therapy were also excluded. Although Behavioral Couples Therapy is a form of CBT, modules administered and hypothesized mechanisms of change differ between traditional (individual or group) CBT and Behavioral Couples Therapy (McHugh et al., 2010).

Study Selection

Figure 1 displays a flowchart of the study selection process. The initial search yielded 44 records (43 unique articles and one erratum) after removing duplicates. The authors (CB and VV) double-coded each of the 44 meta-analyses as eligible, not eligible, or possibly eligible based on their title and abstracts. For meta-analyses coded as eligible or possibly eligible, full texts were obtained and read to further determine eligibility. All discrepancies were resolved via consensus.

Figure 1.

Flow Chart for Cognitive Behavioral Therapy for Substance Use Search Process. This figure illustrates the search process for locating reviews eligible for inclusion in the treatment evaluation.

Of the 44 possibly eligible records, 12 were excluded because they were not focused on CBT for substance use as a standalone intervention (e.g., they focused on other interventions for substance use or primarily on pharmacotherapy or CBT + pharmacotherapy), 22 were excluded for not being a meta-analysis, and 12 were excluded for not reporting on substance-related outcomes (e.g., they primarily enrolled individuals with behavioral addictions [e.g., gambling] or other psychological disorders [e.g., depression, trauma] and therefore did not report on substance-related outcomes). This resulted in two eligible meta-analyses. Backward searches of these meta-analyses were also conducted, resulting in 7 additional possibly eligible manuscripts. Upon full-text review, 3 of these were eligible. This resulted in 5 eligible meta-analyses for inclusion in the current evaluation. A full list of ineligible records with the primary reason for exclusion can be found in Supplemental Table 1.

Data Extraction and Coding

For each of the 5 eligible meta-analyses, two authors (CB and VV) independently double-coded the PICOTS (population, intervention, comparison, outcomes, timeline, setting) criteria (e.g., Schardt et al., 2007). PICOTS allows for a full consideration of review characteristics (see Table 1) and assists the reader in evaluating the appropriateness of the eligible reviews included for answering the clinical question of interest. Discrepancies among coders were resolved by consensus between CB and VV. Effect sizes and confidence intervals were also extracted by two coders (VV and FS) for all meta-analyses. Specifically, VV and FS independently extracted all effect sizes related to substance use quantity and frequency and psychosocial outcomes (if available) for each comparator group and each follow-up time frame reported. If a given meta-analysis reported separate effect sizes for primary substances of use and method of substance use data collection (i.e., biologically verified, self-reported), these are also extracted; however, many of these effect sizes were reported by only one meta-analysis and/or meta-analyses of “low” or “critically low” quality (see below), and therefore were not included in the evaluation. A third coder (CB) was responsible for comparing effect sizes and resolving discrepancies.

Table 1.

Description of Reviews Included in the CBT for SUD Treatment Evaluation

Study	Intervention(s)	Number of Studies	Population	Setting	Comparison Condition	Outcome	Time Points
Dutra et al., 2008	Psychosocial treatments for SUDs, including CBT and RP	18 (13 CBT; 5 RP)	Adults with illicit SUDs, including cannabis (n = 4), cocaine (n = 4), opioids (n = 4), polysubstance (n = 6); also included participants with co-occurring borderline personality disorder, bipolar disorder, and posttraumatic stress disorder	Non-intensive outpatient treatment	Inactive or active treatment	Self-reported or biologically-verified substance use	Post-treatment
Irvin et al., 1999	RP (individual, group, or couples format; with or without an adjunctive treatment)	26	Alcohol (n = 10), smoking (n = 8), or other substance use, including polysubstance use (n = 5) and cocaine use (n = 3)	Inpatient; outpatient	Pre-post change within person; waitlist or no-additional treatment control; active intervention	Self-reported or biologically-verified substance use; psychosocial adjustment	Post-treatment; 1-month, 3-month, 6-month, and 12-months post-treatment
Magill & Ray, 2009	CBT, RP, or coping-skills training (individual or group format; with or without an adjunctive treatment)	53	Adults with SUDs, including alcohol (n = 23), cocaine/stimulants (n = 11), polydrug (n = 11), cannabis (n = 6), opioids (n = 2); “…64% of studies allowed nonsubstance-related co-occurring diagnoses (exclusive of suicidal or homicidal ideation and active psychosis)” (p. 520)	NS	Active treatment; passive treatment or usual service; no treatment; no CBT adjunct	Self-reported or biologically-verified substance use	Post-treatment to 4 months post-treatment; 6–12 months post-treatment
Magill et al. 2019	CBT or RP (individual or group format)	30 (across 32 study sites)	Adults with SUDs or problematic use, including alcohol (n = 15), cannabis (n = 3), opioids (n = 2), stimulants (n = 6), and polydrug (n = 6)	Community sample; specialty substance use or mental health clinic; Medical setting; College setting; Criminal legal setting; Other setting	Minimal treatment; Non-specific therapy; Other specific therapy	Self-reported or biologically-verified substance use	Early (1–6 months) post-treatment; Late post-treatment (8+ months)
Windsor et al., 2015	CBT (individual, group, or combined)	16	Adults (≥70% White or Black and/or Hispanic) with substance use, including cocaine (n = 3), cannabis (n = 3), alcohol (n = 6), alcohol + other drugs (n = 4)	NS	Comparison treatment; pre-post change within person	Substance use	Post-treatment; Average follow-up from baseline (1–24 months)

Note. CBT = Cognitive Behavioral Therapy; RP = Relapse Prevention; SUD = Substance Use Disorder; NS = not specified. 734

Methodological Quality

For each meta-analysis, authors (DM and CB) double-coded the methodological quality using the Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR2; Shea et al., 2017). AMSTAR2 is a tool for critically evaluating systematic reviews of randomized and nonrandomized studies of health care interventions and includes 16 domains (see Table 2). For each domain, authors coded “Yes” or “No,” and for some items, “Partial Yes.” “Yes” or “Partial Yes” indicates a positive result on the stated domain, and a code of “No” indicates a negative result. Consistent with the procedures of Boness et al. (2020), several domains were deemed critical: (a) including components of PICOTS in the research questions and inclusion criteria, (b) using a comprehensive search strategy, (c) describing the included studies in adequate detail, (d) using appropriate methods for statistical combination of results, (e) accounting for risk of bias in individual studies, and (f) providing explanation for a discussion for any heterogeneity observed in the results. Coding “No” in any critical domain reflected a critical weakness.⁶

Table 2.

AMSTAR2 Results for Eligible Studies

Item	Irvin et al. (1999)	Dutra et al. (2008)	Magill & Ray (2009)	Windsor et al. (2015)	Magill et al. (2019)
1. Did the research questions and inclusion criteria for the review include the components of PICO?	Y	Y	Y	Y	Y
2. Did the report of the review contain an explicit statement that the review methods were established prior to the conduct of the review and did the report justify any significant deviations from the protocol?	N	N	N	N	N
3. Did the review authors explain their selection of the study designs for inclusion in the review?	N	N	Y	Y	N
4. Did the review authors use a comprehensive literature search strategy?	N	N	PY	N	PY
5. Did the review authors perform study selection in duplicate?	N	N	N	Y	Y
6. Did the review authors perform data extraction in duplicate?	N	N	N	Y	Y
7. Did the review authors provide a list of excluded studies and justify the exclusions?	N	N	N	N	N
8. Did the review authors describe the included studies in adequate detail?	N	PY	PY	PY	PY
9. Did the review authors use a satisfactory technique for assessing the risk of bias (RoB) in individual studies that were included in the review?
RCT	N	N	N	N	Y
NRSI	N	NA	NA	NA	NA
10. Did the review authors report on the sources of funding for the studies included in the review?	N	N	N	N	N
11. If meta-analysis was performed did the review authors use appropriate methods for statistical combination of results?
RCT	N	N	Y	Y	Y
NRSI	Y	NA	NA	NA	NA
12. If meta-analysis was performed, did the review authors assess the potential impact of RoB in individual studies on the results of the meta-analysis or other evidence synthesis?	N	N	N	N	Y
13. Did the review authors account for RoB in individual studies when interpreting/ discussing the results of the review?	N	N	N	N	Y
14. Did the review authors provide a satisfactory explanation for, and discussion of, any heterogeneity observed in the results of the review?	N	Y	Y	Y	Y
15. If they performed quantitative synthesis did the review authors carry out an adequate investigation of publication bias (small study bias) and discuss its likely impact on the results of the review?	Y	Y	Y	Y	Y
16. Did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review?	Y	N	Y	Y	Y
Overall Rating	Critically low	Critically low	Low	Critically low	Moderate

Note. Y=yes, PY=partial yes, N=no, NA=not applicable, PICO: (P=population, I=intervention, C=comparator group, O=outcome), RCT=randomized controlled trial, NRSI=non-randomized studies of interventions. Bold = critical weaknesses if coded “no.”

Independent coders indicated their overall confidence (critically low, low, moderate, and high) in the results of each meta-analysis based on the pattern of results of AMSTAR2, including consideration of critical and non-critical domains. A rating of “High” was assigned if the meta-analysis had zero or one non-critical weakness, “Moderate” if no critical weaknesses and more than one non-critical weakness, “Low” if one critical weakness with or without non-critical weaknesses, and “Critically Low” if more than one critical flaw with or without non-critical weaknesses. Discrepancies among coders were resolved between DM, CB, and VV. Those rated as low or critically low (n = 4) were excluded from the main effect size estimates but were considered in supplemental sensitivity analyses, consistent with the Tolin Criteria guidance document (Boness et al., 2021).

Data Analysis

Because only a single meta-analysis (Magill et al., 2019) had sufficient quality for inclusion, per AMSTAR2, effect sizes were not statistically aggregated across separate meta-analyses. However, because Magill et al., 2019 reported four outcomes (early [1–6 months] and late [8+ months] follow-up substance use frequency and early and late follow-up substance use quantity) for three separate comparator groups (minimal treatment [e.g., waitlist, brief psychoeducation], non-specific therapy [e.g., treatment as usual, supportive therapy, drug counseling], and specific therapy [e.g., Motivational Interviewing, Contingency Management]), two separate sets of effect sizes are reported. First, all comparator groups for each outcome are reported separately. Second, averaged effect sizes across the non-specific and specific comparator groups are reported for each of the three outcomes to understand how effect sizes differ for inactive versus active treatments. In this second set of effect sizes, an average across substance use quantity and frequency was taken to indicate overall substance use effect sizes. All estimates used Hedge’s g and Cohen’s (1988) guidelines for classifying small effects as g = 0.2, medium effects as g = 0.5, and large effects as g = 0.8.

Results

Search Results and Meta-Analysis Characteristics

First, effect sizes are reported by comparator group for each of the four outcomes (early and late follow-up substance use frequency and quantity). These estimates are displayed in Figure 2, and all raw effect size estimates are provided in Supplemental Table 2. Of note, for the late follow-up substance use quantity outcome, the only effect size reported was for the specific treatment comparator group, given these data were not reported in trials examining minimal and non-specific comparator groups.

Figure 2.

Effect size estimates (Hedge’s g) with 95% confidence intervals for Magill et al., 2019 for each comparator group type.

Overall Effect Sizes

Regarding the overall pattern of results from Magill and colleagues (2019), CBT was most effective at early follow-up (defined as 1–6 months post-treatment) compared to late follow-up (defined as 8+ months post-treatment) and had the largest effect sizes when compared to minimal (inactive) treatment. Effect sizes for substance use quantity were generally larger than substance use frequency.

CBT Compared to Minimal Treatment

In the minimal comparator group, effect sizes for early follow-up substance use frequency (g=.58, 95% CI=.15–1.01; k = 4) and quantity (g=.67: 95% CI=.41–.98; k = 2) were medium in magnitude (>0.50) and the effect size for late follow-up frequency was still small to medium (g=.44, 95% CI=.02–.86; k = 2) in magnitude.

CBT Compared to Non-Specific Treatments

Unsurprisingly, effect sizes decreased in magnitude when looking at non-specific (inactive; e.g., waitlist, psychoeducation) treatments as comparator groups. For the non-specific treatment comparator group, effect sizes ranged from 0.18 (substance use frequency; 95% CI =.02–.35; k = 9) to 0.42 (substance use quantity; 95% CI =.03–.81; k = 2) for early follow-up. At late follow-up, the effect size for substance use frequency was 0.05 (95% CI = −.09–.19; k = 7). There was no late follow-up substance use quantity reported for the non-specific comparator group. Thus, although smaller in overall magnitude compared to the minimal treatment group, CBT still had small to moderate effects at early-follow up when non-specific treatment was used as the comparator group.

CBT Compared to Specific Treatments

Similarly, for the specific treatment comparator group, effect sizes were all close to zero regardless of the follow-up period. It is worth noting that Magill and colleagues (2019) found heterogeneous effects of CBT on early follow-up use frequency in contrast to minimal treatment (I² = 59%, k = 4) and in contract to a non-specific therapy (I² = 45%; k = 9).

CBT Compared to Inactive vs. Active Treatments

For ease of comparison in the supplemental analyses (described later), effect sizes between the non-specific and specific treatment comparator groups (to indicate active treatment effect sizes) and between substance use quantity and frequency (to indicate overall substance use effect sizes) were averaged. These results are displayed in Figure 3. The general patterns previously reported are also true here. For example, all effect sizes are larger when the minimal (inactive) group is used as the comparator versus the active treatment group. Worth noting, however, is that there is a moderate effect of CBT on early follow-up quantity when compared to active treatments.

Figure 3.

Effect size estimates (Hedge’s g) with 95% confidence intervals for Magill et al., 2019 for minimal versus active treatment comparator groups. Treatment (Active) represents an aggregate of effect sizes for the original non-specific and specific treatment comparator groups. “Use” is the average of quantity and frequency, where available.

Sensitivity Analyses

As a sensitivity analysis, the four reviews of low or critically low quality were considered. Effect sizes were first converted to a consistent metric (Hedge’s g) and then combined by taking the mean of the effect sizes reported for a given outcome. In aggregating effect sizes, the approach taken in the CBT for Insomnia (CBT-I) evaluation (Boness et al., 2020) was maintained, resulting in average (rather than weighted) effect sizes. This may also be useful for minimizing, to the extent possible, bias in the precision of the overall effect sizes due to non-independence. Although it is difficult to know the full extent of non-independence on an estimate’s variance, Supplemental Table 3 is included as an index of the extent to which the meta-analyses considered eligible for the purposes of this evaluation included the same primary study in their effect estimates. Variance estimates are expected to be minimally biased, given most of the estimates reported below were derived from single meta-analyses.

Combined effect sizes are reported for substance use outcomes at four time points: 1) combined post-treatment and follow-up, 2) post-treatment, 3) early follow-up (1–6 months post-treatment), and 4) late follow-up (6+ months post-treatment). As previously mentioned, the comparison groups examined included inactive and active treatments.

Combined Post-Treatment and Follow-Up

For combined post-treatment and follow-up substance use, only effect sizes from Magill and Ray (2009) were used. Although these outcomes were reported for Windsor et al. (2015) and Irvin et al. (1999), Windsor et al. (2015) included the pre-post comparison in their aggregate and Irvin et al. (1999) reported on this outcome in the r metric but did not report on total sample size for each effect so we could not convert r to Hedge’s g for inclusion. Based on Magill and Ray (2009), the combined post-treatment and follow-up effect for CBT on substance use was large in magnitude (g=0.80, 95% CI=0.45–1.14) when compared to an inactive comparison group and very small (g=0.14, 95% CI=0.13–0.15) when compared to an active treatment comparison group.

Post-Treatment

The only study that reported on post-treatment substance use was Dutra et al. (2009). This study suggested a small to moderate effect of CBT on substance use at post-treatment compared to an inactive control comparison group (g=0.30, 95% CI=0.27–0.33).

Early Follow-Up

For early follow-up substance use, Irvin et al. (1999) and Magill et al. (2019) reported on this outcome. However, because Irvin et al. (1999) did not report on total sample size for the effect, it was not possible to convert r to Hedge’s g for inclusion. Thus, the estimate for the effect of CBT on substance use at early follow-up is equivalent to what is reported in the main analyses above such that there is a small effect of CBT on substance use when compared to active treatment (g=0.15, 95% CI=0.06–0.24) and a moderate to large effect of CBT on substance use when compared to inactive, or minimal treatment (g=0.63, 95% CI=0.57–0.69).

Late Follow-Up

For late follow-up substance use, Magill and Ray (2009) and Magill et al. (2019) report on this outcome. Of note, Magill and Ray (2009) reported effect sizes by comparison group types (but across all follow-up periods) and by follow-up periods (but across all comparison group types), and therefore we could not separate late follow-up substance use estimates into inactive versus active treatment comparison. Thus, comparison groups were combined across inactive (minimal) and active comparison groups for Magill et al. (2019). The overall effect of CBT on substance use at late follow-up across all comparison groups was small (g=0.17, 95% CI=0.08–0.25)

Together, these supplemental analyses support the main analyses such that CBT’s effects on substance use are larger in magnitude when inactive (minimal) versus active treatment is the comparison group and these effects tend to diminish with time.

Quality of the Evidence

Based on these effect size results, the quality of the evidence for CBT in reducing substance use among those with SUD is of moderate quality (see Table 3) per the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system (Atkins et al., 2004; Guyatt et al., 2008). The GRADE system aims to guide the quality of the evidence and strengths of treatment recommendations comprehensively by including consideration of: methodological quality of the evidence; the importance of the outcome that the treatment improves; the magnitude of the treatment effect, and the precision of its estimate; the burden, costs, and potential risks associated with the therapy; and other consumer values that might be expected to influence decision processes regarding a treatment.

Table 3.

Judging the Quality of the Evidence for CBT for SUD

Quality	Criteria
□ High quality	All of the following: There is a wide range of studies included in the analyses with no major limitations. There is little variation between studies. The summary estimate has a narrow confidence interval.
✓ Moderate quality	At least one of the following: There are only a few studies, and some have limitations but not major flaws. There is some variation between studies, or the confidence interval of the summary estimate is wide.
□ Low quality	Any of the following: The studies have major flaws. There is important variation between studies. The confidence interval of the summary estimate is very wide.

Note. This table represents one component of the full GRADE criteria.

A designation of moderate quality, per GRADE, was made because only one meta-analysis of adequate quality was included in the overall evaluation of the evidence (Magill et al., 2019), and this study was considered to have “moderate” quality per AMSTAR2, suggesting it had some limitations but not major flaws. However, it’s important to note that even those excluded based on quality produced similar results when included as part of sensitivity analyses. When considering the effect estimates extracted from Magill et al. (2019), confidence intervals were wide in some cases.

Additional Contextual Factors

In addition to the quality of the evidence rating, the GRADE system also allows consideration of additional contextual factors in the overall treatment recommendation. In the case of CBT for SUD, the quality of the evidence is strengthened by several additional contextual factors (see Table 4). First, CBT for SUD generates an effect size and pattern of effects (i.e., decreasing treatment effects over time, within studies) similar to other well-studied treatments, such as Motivational Enhancement Therapy and Contingency Management (Benishek et al., 2014; Pfund et al., 2021; Smedslund et al., 2011), but CBT has strong evidence for flexibility and scalability via technology-delivered CBT (e.g., Computer Based Training for CBT or CBT4CBT; Carroll et al., 2008). Although not included in the meta-analyses reviewed here, there is evidence that technology-delivered CBT for alcohol use as a standalone intervention demonstrates efficacy over minimal treatment (small effect size), and technology-delivered CBT for alcohol use as an adjunct to treatment as usual demonstrates efficacy over treatment as usual alone (small effect size) (Kiluk et al., 2019). Technology-delivered CBT has also demonstrated efficacy for substances beyond alcohol, including cocaine use disorder and cannabis use disorder (Carroll et al., 2008, 2014). The effect sizes observed for technology-delivered CBT on substance use outcomes are similar to effect sizes for all technology-delivered substance use treatments, including CBT and non-CBT interventions (e.g., Motivational Interviewing) (Rooke et al., 2010; Tait et al., 2013).

Table 4.

Additional contextual factors considered in increasing or decreasing the GRADE recommendation for CBT for SUD

Positive	Negative
□ Treatment appears superior to other established and effective treatment(s)	□ There are other psychological treatments that have well-documented and much larger effects
✓ The treatment generates an effect that is similar to other well-studied treatments and has strong evidence for flexibility via technology-delivered CBT	□ The treatment generates an effect that is similar to other well-studied treatments, but requires a very large number of sessions or length of time to generate the same effect at a much higher cost
✓ Evidence supports the purported mechanism or active ingredient(s) of treatment	□ Evidence fails to support the purported mechanism or active ingredient(s) of treatment
□ Treatment has demonstrated good effects with marginalized groups	□ Treatment has demonstrated weak effects with marginalized groups
□ Treatment has been studied by a wide array of researchers without strong allegiance to the treatment	□ Treatment has been studied by a narrow array of researchers with strong allegiance to the treatment
✓ Other: Demonstrated efficacy across several patient populations	□ Other:

Note. This table identifies additional positive contextual factors to consider in the overall GRADE treatment recommendation that are supported by the CBT for SUD literature and was adapted from Tolin et al., 2015. Lack of identification of a positive or negative assessment of a contextual factor indicates that there is not enough data to make a firm conclusion in this category for CBT for SUD.

There is additional evidence that CBT for SUD exerts effects through hypothesized mechanisms of behavior change, including increased coping skills and self-efficacy. However, it is currently unclear the extent to which these mechanisms are unique to CBT versus common mechanisms of psychosocial treatments for substance use (Magill et al., 2020). Irvin et al. (1999) also found a medium effect (r=0.48) for relapse prevention (combining all follow-up timepoints and combining pre- to post-treatment effects and all comparator groups) on psychosocial adjustment outcomes. Notably, the definition of psychosocial adjustment outcomes included measures of purported mechanisms of change, such as self-efficacy, coping skills, and social and problem-solving skills.

Further strengthening the GRADE rating is that CBT for SUD is generally efficacious across several patient populations, including those with varying primary substance used (e.g., cannabis, alcohol, cocaine), those with co-occurring disorders (e.g., borderline personality disorder, posttraumatic stress disorder), those receiving adjunctive pharmacotherapy, and those in varying settings (e.g., community sample, specialty substance use or mental health clinic, medical setting, college setting, criminal legal system, etc.) (Dutra et al., 2008; Magill et al., 2019), demonstrating the flexibility of this intervention. Several meta-analyses in the present evaluation explicitly examined these patient factors as moderators of CBT effects or conducted subgroup analyses. Magill et al. (2019) concluded that primary substance was not associated with early follow-up substance use frequency effect size (they only examined effect size with enough heterogeneity and a large enough sample size to examine subgroup effects). Conversely, Irvin et al. (1999) found that relapse prevention was more efficacious for alcohol and polysubstance use than smoking or cocaine use, while Magill and Ray (2009) found that effect sizes for CBT across substance type were similar (and small), except for cannabis, for which the effect size was moderate. Although the primary quantitative review we examined (Magill et al., 2019) excluded those with adjunctive pharmacotherapy, therefore indicating the efficacy of CBT alone, meta-analyses that included adjunctive interventions found that CBT + pharmacotherapy generally had greater effect sizes than CBT alone (Irvin et al., 1999; Magill & Ray, 2009). Other patient population factors that were examined as moderators or using a subgroup approach, but were not significantly related to effect sizes, included treatment setting (Irvin et al., 1999), co-occurring psychological disorders (Magill & Ray, 2009), and treatment format (i.e., group vs. individual) (Irvin et al., 1999; Magill et al., 2019). However, Magill et al. (2019) did find that for CBT effects on early follow-up use frequency in contrast to a non-specific therapy, older age was associated with smaller effect sizes (b = −.072, p = .044), suggesting that those of older age may benefit less from CBT for SUD.

Overall Treatment Recommendation

Taken together (see Tables 3 and 4), there is moderate quality evidence that CBT produces small to moderate effects on substance use among people with various types of SUDs when compared to inactive treatment. This remains true even when effect sizes from studies considered to have low or critically low quality are considered. There is also some evidence, although from a dated study that is considered to have “critically low” quality, that CBT for SUD may influence psychosocial/functional outcomes. Although there is only moderate quality evidence for the efficacy of CBT in treating SUD, consideration of additional contextual factors such as flexibility in CBT delivery modality and evidence for efficacy across patient populations bolsters the overall recommendation. As such, based on the criteria outlined by Tolin and colleagues (2015), the current status of the literature merits a “strong” recommendation of CBT for SUD (see Table 5).

Table 5.

Overall Treatment Recommendation for CBT for SUD

Recommendation	Criteria
□ Very strong recommendation	All of the following: There is high-quality evidence that the treatment produces a clinically meaningful effect on symptoms of the disorder being treated There is high-quality evidence that the treatment produces a clinically meaningful effect on functional outcomes There is high-quality evidence that the treatment produces a clinically meaningful effect on symptoms and/or functional outcomes at least three months after treatment discontinuation At least one well-conducted study has demonstrated effectiveness in nonresearch settings
✔ Strong recommendation	At least one of the following: There is moderate- to high-quality evidence that the treatment produces a clinically meaningful effect on symptoms of the disorder being treated There is moderate- to high-quality evidence that the treatment produces a clinically meaningful effect on functional outcomes
□ Weak recommendation	Any of the following: There is only low- or very low-quality evidence that the treatment produces a clinically meaningful effect on symptoms of the disorder being treated There is only low- or very low-quality evidence that the treatment produces a clinically meaningful effect on as well as on functional outcomes There is moderate- to high-quality evidence that the effect of the treatment, although statistically significant, may not be of a magnitude that is clinically meaningful

Note. This table was adapted from Tolin et al., 2015.

Discussion

Results of the present evaluation suggest that CBT for SUD has strong research support per the Tolin Criteria. Notably, this evaluation builds upon the prior Society of Clinical Psychology evaluation of Motivational Enhancement Therapy plus CBT, which used the Chambless and Hollon (1998) criteria, by indicating that CBT alone is an effective intervention for SUDs, even when using more rigorous and updated criteria. Importantly, CBT for SUD generates an effect similar to well-studied treatments, can be flexibly delivered via technology-based CBT, exerts effects on substance use via hypothesized mechanisms of change, and demonstrates efficacy across various patient populations. Given the Society of Clinical Psychology has yet to evaluate CBT for SUD, this evaluation provides a much-needed examination of the status of CBT for SUD as an empirically supported treatment. Formal recognition may help encourage insurance reimbursement for CBT and inspire further dissemination of and training in CBT.

Limitations of the Current Literature and Future Directions

Despite finding strong research support for CBT for SUD, several factors limited the quality of the evidence and confidence in results and, accordingly, precluded the ability to recommend “very strong” research support for CBT for SUD. Perhaps most notably, there is a need for increased rigor in future meta-analytic work; this includes ensuring that included studies are described in adequate detail (including specific components of CBT for SUD that were implemented, fully describing the population represented by the effect sizes including any co-occurring diagnoses, details on therapist characteristics across primary studies, etc.), use appropriate methods for statistical combination of results, and account for potential risk of bias when interpreting results. Given the eligible meta-analyses required primary studies to be randomized clinical trials, future meta-analyses may consider the inclusion of studies conducted in more naturalistic settings⁷. Similarly, CBT for SUD has been studied by research groups with mixed allegiance to the treatment (i.e., one or more authors developed the intervention and/or engage in supervision or training the therapists delivering the intervention; Dragioti et al., 2015). It may be especially important for research teams without allegiance to the treatment to contribute to this literature, particularly meta-analytic investigations.

Second, the impact of CBT for SUD on psychosocial outcomes was mixed and limited. One included meta-analysis indicated that CBT for SUDs influences psychosocial/functional outcomes, but this study was dated and considered to have “critically low” quality. Studies examining the impact of CBT for SUD on psychosocial outcomes in the broader literature have also been inconclusive. For example, a recent meta-analysis of substance use interventions on emotional outcomes indicated that CBT was not statistically significantly associated with reductions in emotional distress, and that mindfulness-based and affect-regulation interventions had greater efficacy in reducing emotional distress than CBT (Kang et al., 2019). However, a relatively small number of studies examining CBT (k = 4) were evaluated, as compared to the studies evaluating mindfulness-based (k = 11) and affect-regulation (k = 6) interventions. Thus, future work should aim to broaden the types of outcomes included in trials of CBT (e.g., quality of life, affective symptoms; see Witkiewitz et al., 2021; Witkiewitz & Tucker, 2020), particularly given the importance of biopsychosocial outcomes in defining recovery from SUD (Hagman et al., 2022; Witkiewitz & Tucker, 2020).

Critically, we are unable to conclusively comment on the evidence for the efficacy of CBT for SUD among historically marginalized and excluded populations, including those from racially and ethnically diverse communities. Windsor et al. (2015) compared the effect of CBT on substance use outcomes for studies with predominantly non-Hispanic and White participants versus studies with predominantly Hispanic and/or Black participants. This meta-analysis concluded that effects sizes of CBT versus comparison groups were similar for studies enrolling predominantly non-Hispanic and White, and Hispanic and/or Black samples. However, although the effect sizes of CBT pre- to post-intervention were large and statistically significant in both groups, these effects were larger (indicating greater change) for studies enrolling predominantly non-Hispanic and White versus Hispanic and/or Black samples. Windsor et al. (2015) also identified significant weaknesses in the literature, including a paucity of studies comparing retention and engagement rates by racial and ethnic identity and few studies enrolling primarily Hispanic and/or Black samples. In addition, Magill et al. (2019) found that the percentage of participants who identified as White was not associated with early follow-up substance use frequency effect size (they only examined effect sizes with enough heterogeneity and a large enough sample size to examine subgroup effects). Overall, there is insufficient evidence that CBT for SUD has demonstrated good effects with marginalized populations. Further, there exists more general criticisms of the use of CBT as an approach among marginalized groups because it has been developed and tested in overwhelmingly White samples and overlooks cultural values that are likely to be held by marginalized groups (e.g., interdependence over personal independence; see Hays, 2009). Cultural adaptations of CBT, including modifications of setting, language, and content, to improve access, acceptability and efficacy in marginalized groups have shown initial promise but will require additional research (e.g., Jordan et al., 2021; Paris et al., 2018).

Lastly, additional clarity is needed regarding the conditions under which CBT for SUD might be most effective. There were somewhat mixed findings regarding the efficacy of CBT by specific SUD type. One meta-analysis concluded that relapse prevention was more efficacious for alcohol and polysubstance use than smoking or cocaine use (Irvin et al., 1999). Another found that CBT was most effective for cannabis (with similar effect sizes across all other substances; Magill and Ray, 2009), while another indicated no differences in effect sizes by primary SUD (Magill et al., 2019). Future work might clarify which patient populations benefit most from CBT, based on primary SUD. Such research might help inform which individuals could benefit from targeted adjunctive interventions to CBT, given evidence that CBT plus pharmacotherapy or psychosocial interventions might have greater efficacy than CBT alone (Irvin et al., 1999; Magill & Ray, 2009).

Although we concluded that CBT for SUD can be flexibly delivered via technology-based CBT, the published effect sizes for technology-delivered CBT are smaller than the primary effect sizes reported here (for in-person group and individual CBT) (see Kiluk et al., 2019). A meta-analysis of CBT for alcohol use disorder indicated that technology-delivered CBT effect sizes are substantially larger when used as an adjunct to usual care, as opposed to a standalone intervention (Kiluk et al., 2019). Given that technology-delivered CBT as a standalone intervention has advantages, such as cost-effectiveness and the ability to be widely disseminated, future research should continue to examine patient populations who can achieve success via technology-delivered CBT.

Limitations of the Present Evaluation

Several methodological limitations qualify the results of this evaluation. First, our search strategy was limited to the search terms and databases used, and we may have overlooked additional relevant meta-analyses. To limit the scope of the review, we excluded quantitative reviews that focused solely on CBT for nicotine use disorder. Although most of the meta-analyses we included also excluded studies focused on CBT for nicotine use disorder, Irvin et al. (1999) did include those focusing on tobacco and nicotine use disorder. Future work should continue to examine CBT as an established empirically supported treatment for tobacco and nicotine use disorder.

Although the effects of an intervention on functional outcomes are a consideration when providing an overall treatment recommendation, we chose to only include quantitative reviews that reported substance use outcomes. It is possible that broadening our inclusion criteria to include meta-analyses focusing on psychosocial outcomes only (without requiring substance use outcomes) via CBT for SUD may have provided additional evidence regarding this consideration. Future investigations that include meta-analysis focused on biopsychosocial outcomes are important for evaluating CBT for SUD given recent updates to the definition of recovery from SUD (e.g., Hagman et al., 2022; Witkiewitz & Tucker, 2020). Similarly, we chose to focus only on traditional CBT for SUD, thus excluding integrated CBT approaches for co-occurring psychological disorders, given these approaches typically integrate additional skills (e.g., exposure for anxiety disorders). This may further limit our understanding of the impact of CBT on psychosocial outcomes, given these studies are potentially more likely to evaluate psychosocial outcomes and identify effects on psychosocial outcomes (Mehta et al., 2021; Swan et al., 2020).

In the present evaluation, only a single meta-analysis (Magill et al., 2019) had sufficient quality for inclusion, and therefore future meta-analytic investigations of adequate quality will be a useful test of the robustness of the overall efficacy of CBT for SUD. However, for several reasons, we believe that evaluating this single meta-analysis is sufficient to render a “strong” research support recommendation for CBT for SUD. First, applying the Tolin Criteria to a single meta-analysis is acceptable, with recommendations that investigators conduct an original meta-analysis to evaluate research support in the absence of a pre-existing meta-analysis (Boness et al., 2021). Second, the evaluated meta-analysis (Magill et al., 2019) was conducted relatively recently and included 30 randomized trials published between 1982–2018, capturing a large body of empirical research on CBT for SUD. Lastly, it is unlikely that only evaluating one meta-analysis led to a misestimation of effects, given that the pattern of effects (i.e., larger effects when CBT was compared to inactive vs. active treatment; decreasing treatment effects over time, within studies) was similar across primary and sensitivity analyses and is similar to meta-analyses of other psychosocial treatments for SUD (Benishek et al., 2014; Pfund et al., 2021; Smedslund et al., 2011).

More generally, there may be limitations of the Tolin Criteria as currently written. These limitations have been discussed elsewhere (e.g., Boness et al., 2020; Pfund et al., in press) but include, for example, the appropriateness of AMSTAR2 for evaluating meta-analytic quality and the challenges of aggregating across meta-analyses rather than primary studies which may obscure important information such as patient characteristics or treatment delivery settings. Thus, use of the Tolin Criteria for determining the status of treatments in terms of their evidence base, although an improvement in some ways, may still have important limitations that need to be addressed in future iterations and applications of the criteria.

Conclusions

Despite these limitations, our rigorous evaluation of meta-analyses examining CBT for SUD suggests this intervention has strong research support. This recommendation provides further support for the widespread dissemination of and training in CBT for SUD, in addition to underscoring the necessity of additional evaluation of the cost-effectiveness of CBT and advocacy around addiction treatment policy and funding. However, future research is needed to identify patient characteristics that might moderate response to CBT for SUD, such as primary SUD type, and under which circumstances CBT should be used as a standalone versus an adjunct intervention. Future clinical trial and meta-analytic work would be needed to support a “very strong” research support recommendation, including research conducted by researchers without a strong allegiance to the treatment and meta-analytic work with greater transparency and rigor, as well as research that aims to delineate the effects of CBT for SUD among underserved and understudied populations.

Public Health Statement:

Substance use disorder is a significant public health problem globally. This systematic review and treatment evaluation found that cognitive behavioral therapy’s effects on substance use are larger in magnitude when inactive (minimal) versus active treatment is the comparison group and that these effects tend to diminish with time. Cognitive behavioral therapy is therefore considered an evidence-based treatment for substance use disorder according to current evidence-based treatment guidelines.