Cognitive Behavioral Therapy: A Meta-Analysis of Race and Substance Use Outcomes

Abstract

Cognitive behavioral therapy (CBT) is an effective intervention for reducing substance use. However, because CBT trials have included predominantly White samples caution must be used when generalizing these effects to Blacks and Hispanics. This meta-analysis compared the impact of CBT in reducing substance use between studies with a predominantly non-Hispanic White sample (hereafter NHW studies) and studies with a predominantly Black and/or Hispanic sample (hereafter BH studies). From 322 manuscripts identified in the literature, 17 met criteria for inclusion. Effect sizes between CBT and comparison group at posttest had similar effects on substance abuse across NHW and BH studies. However, when comparing pre-posttest effect sizes from groups receiving CBT between NHW and BH studies, CBT’s impact was significantly stronger in NHW studies. T-test comparisons indicated reduced retention/engagement in BH studies, albeit failing to reach statistical significance. Results highlight the need for further research testing CBT’s impact on substance use among Blacks and Hispanics.

There is consistent evidence from clinical trials that cognitive behavioral therapy (CBT) is an effective treatment for reducing substance use (e.g.: Barrowclough et al., 2009; Drummond et al., 2005; Glasner-Edwards et al., 2007; Maude-Griffin et al., 1998; Morgenstern, Blanchard, Morgan, Labouvie, & Hayaki, 2001). Two meta-analytic studies also indicate that CBT is efficacious for treating alcohol and drug use disorders. In the first meta-analysis (N = 53), Magill and Ray (2009) provide a broad view of CBT efficacy for adults diagnosed with alcohol or drug abuse/dependence and found a small overall pooled CBT effect, g = .14 (95% CI, .09–.19). In the second meta-analysis (N = 26) Irvin, Bowers, Dunn, & Wang (1999) found a small overall effect of relapse prevention interventions for alcohol and substance use disorders as well as smoking, g = .28 (95% CI, .79–1.35).

Despite the evidence that CBT is an efficacious intervention for reducing substance use, the majority of participants in these efficacy trials identified as White (e.g., Anton et al., 2005; Brown et al., 2006; Budney, Moore, Rocha, & Higgins, 2006), leading to questions about whether these findings can be generalized to ethnoracial minorities (Griner & Smith, 2006; Sue, Zane, Hall, & Berger, 2009). Additionally, while higher numbers of ethnoracial minorities have been included in CBT effectiveness trials, these trials typically take place in the community (as opposed to more controlled environments) and use less rigorous methods to evaluate the effect of the intervention when compared to efficacy trials (Singal, Higgins, & Waljee, 2014). Studies examining the effectiveness of CBT also have high dropout rates among non-White participants and/or do not generate subgroup analyses by race/ethnicity (Magill & Ray, 2009; Mak, Law, Alvidrez, & Pérez-Stable, 2007; Miranda et al., 2005; Voss Horrell, 2008), making it difficult to generalize these findings to Black and Hispanic populations (Schmidt, Greenfield, & Mulia, 2006; Voss Horrell, 2008). Finally, due to ethical considerations, randomized clinical efficacy and effectiveness trials testing behavioral interventions rarely compare experimental treatments to no treatment control groups (“true control”). Instead, these trials tend to compare the experimental therapy to other existing treatments (“comparison control”) which complicates meta-analytic procedures for subgroup analysis.

The question about whether CBT is effective for ethnoracial minorities is complicated by another issue. Studies that do indicate CBT is effective for ethnoracial minorities have used culturally-adapted approaches (Foster, 2007; Kohn & Oden, 2003; Miranda et al., 2003, 2006; Organista, Muñoz, & Gonzalez, 1994), such as conducting the intervention using the client’s ethnic language, integrating cultural elements into treatment, or using same-race groups (Sue et al., 2009). Claiming CBT is an effective treatment for reducing substance use among ethnoracial minorities without reporting that this finding was based on studies using predominantly White, middle-class, and English-speaking samples or culturally adapted versions of CBT can be misleading (Bernal & Scharró-del-Río, 2001; Chambless & Hollon, 1998).

With few studies comparing the effect of CBT by race/ethnicity two questions remain: (a) are standard CBT interventions for substance use effective for ethnoracial minorities, and (b) if so, are they equally effective for ethnoracial minority and White populations? The purpose of the current study is to compare the effect of CBT for substance use between White and Black/Hispanic individuals. Given the existing health disparities between ethnoracial minority and White individuals, special attention to race/ethnicity is needed. For instance, despite relatively uniform rates of substance use across racial/ethnic groups (Center for Substance Abuse Treatment, 1999), Black and Hispanic individuals experience more substance use-related consequences than other groups (e.g., higher prevalence of HIV; National Institute on Drug Abuse, 2005, 2012). Thus, it is important to develop, implement, and evaluate substance abuse interventions that meet the unique needs of Black and Hispanic individuals.

CBT, Substance Abuse, and Cultural Relevance

CBT has been shown to help individuals reduce substance use by anticipating problems and developing effective coping strategies, examining the positive and negative consequences of using substances, and identifying situations that may trigger substance use (National Institute on Drug Abuse, 2012b). The theoretical framework underlying CBT proposes “that cognitive factors mediate all interactions between the individual, situational demands, and the person’s attempts to cope effectively” (Center for Substance Abuse Treatment, 1999; page 69). Therefore, to change behavior, individuals need to alter their beliefs, thoughts, and assumptions about the self and their external environment. With the help of the therapist, clients learn to recognize their triggering events, automatic thoughts regarding these events, emotional and behavioral responses, and in certain situations their underlying core beliefs. Once clients are able to identify this process, they can work toward changing their emotions and behavior by altering their automatic cognitions (Beck, 2011).

Despite the benefits of CBT in reducing a number of mental health problems, there is concern that traditional CBT approaches may not account for the unique experiences encountered by marginalized populations (David, 2009; Eamon, 2008). Generally, counseling approaches have focused on the therapeutic needs of upper-and middle class European Americans (Griner & Smith, 2006; Hall, 2001). More specifically, CBT’s approach may “reflect the values of Western culture, most notably a persistent bias toward individualism” (Griner & Smith, 2006, p.532). Given the limitations of CBT, scholars have argued that substance use among Black and Hispanic populations should be understood and addressed as a complex social phenomenon (Alegria et al., 2006; Maher, Dunlap, D. Johnson, & Hamid, 1996; Small, 2001; Zerai & Banks, 2002). By focusing solely on individual psychological factors, the use of a non-culturally adapted form of CBT may ignore or minimize shared values, stigmatized identities, and contextual factors (e.g., socioeconomic status, systemic forms of discrimination) contributing to substance use among Black and Hispanic individuals (Griner & Smith, 2006; McKay et al., 2007; Schmidt et al., 2006; Windsor & Dunlap, 2010). For example, a Black or a Hispanic person who experiences anxiety due to the fear of remaining unemployed may seem to have an irrational belief, since it is common for Black and Hispanic individuals to be in the labor market. The inability to find employment may lower their self-esteem, which in turn may be associated with increased substance use. However, traditional CBT approaches may overlook how experiences of prejudice and discrimination affect one’s beliefs and behavior, even though studies show disparities between White and Black or Hispanic individuals in the labor market (DeNavas-Walt, Proctor, & Smith, 2011; Maume, 2012). Because CBT focuses on the individual as an agent for change, it typically does not address the effect of prejudice and discrimination on individuals, families, and communities (Schmidt et al., 2006; Voss Horrell, 2008; Waldron & Turner, 2008). Traditional CBT methods that minimize or ignore realistic environmental factors by focusing solely on changing cognitions and behaviors may exacerbate ethnoracial minority clients’ psychological distress (Hays & Iwamasa, 2006).

In addition to the potential clashing of cultural and mainstream values inherent in many traditional psychotherapy approaches (Griner & Smith, 2006), the psychological harm associated with prejudice and discrimination may not be adequately addressed by traditional approaches, including CBT (David, 2009; Griner & Smith, 2006; Speight, 2007; Sue et al., 2009). Scholars note that as a consequence of prejudice and discrimination, Black and Hispanic individuals may come to accept the negative societal beliefs and stereotypes about themselves and their group (David, 2009; Speight, 2007; Williams & Williams-Morris, 2000). While many Black and Hispanic individuals may resist stigmatization, they still must come to terms with experiencing a social environment that marginalizes ethnoracial minorities. Those who unwittingly internalize the perceived legitimacy of the larger social structure may come to believe they are inferior to the dominant group. For example, a Black or Hispanic person may believe he or she is incompetent when their performance points otherwise. CBT may be useful in challenging the irrational beliefs that stem from internalized stigma if the therapist is aware of how prejudice and discrimination influence the psychosocial functioning of clients. However, without the inclusion of cultural awareness, the experiences of historically disadvantaged populations are often misunderstood and their mental health needs continue unaddressed (David, 2009; Griner & Smith, 2006; Hall, 2001; Richardson & Molinaro, 1996).

Sociocultural barriers to mental health treatment, such as the inability to receive treatment from therapists who share the client’s ethnic background and who speak the same language (Griner & Smith, 2006), may in fact contribute to early treatment dropout among ethnoracial minorities (Robinson & Trochim, 2007; Sheikh, 2006). These sociocultural barriers are important to consider because research shows that treatment engagement and retention predict reduced substance abuse outcomes (Austin & Wagner, 2006; De Leon, 2001). Thus, the claim that CBT is an effective treatment for ethnoracial populations may be based on the individuals who remained in treatment, while overlooking those who dropped out because the interventions were not attuned to the cultural needs of clients.

Culturally Adapted Interventions

With the challenge of recruiting and retaining ethnoracial populations in mental health treatment and research, scholars recommend that mental health and substance abuse interventions be tailored to meet the specific needs of ethnoracial populations (Baquet, Commiskey, Mullins, & Mishra, 2006; Castro & Garfinkle, 2003; Institute of Medicine, 2002; Pinto, Campbell, Hien, Yu, & Gorroochurn, 2011). Researchers also suggest that CBT be adapted to address the concerns of ethnoracial minorities (David, 2009; Hinton, Rivera, Hofmann, Barlow, & Otto, 2012; Hodge, 2011) and lesbian, gay, and bisexual populations (Craig, Austin, & Alessi, 2013).

As discussed earlier, culturally adapted mental health interventions incorporate the client’s cultural values into the intervention (Flaskerud & Nyamathi, 2000; Rowe & Grills, 1993) and when possible match the clinician with the race/ethnicity or native language of the client (Lam & Sue, 2001; Sue et al., 2009). Studies indicate that Black individuals are more likely to remain in treatment when mental health interventions are congruent with Afrocentric values and beliefs (e.g., Banks, Hogue, Timberlake, & Liddle, 1998), and that clients of color paired with therapists by native language and ethnicity are less likely to stop treatment prematurely (Campbell & Alexander, 2002; Flaskerud, 1986; Sue, 1998). Moreover, two meta-analytic studies indicate that culturally adapted mental health interventions have an overall positive effect, and that culturally adapted mental health interventions may be more effective when these adaptations are specific or targeted to a particular racial/ethnic group (Benish, Quintana, & Wampold, 2011; Griner & Smith, 2006).

Studies also demonstrate that culturally adapted forms of CBT (ACBT) are more effective than non-adapted forms of CBT (e.g.: Kohn & Oden, 2003; Miranda et al., 2003). Miranda et al. (2003) found that participants in the ACBT group had lower dropout rates and greater improvement in symptoms and functioning than participants in the non-adapted CBT group. Likewise, Kohn and Oden (2003) found that the ACBT group produced a greater decrease in depressive symptoms when compared to demographically-matched African American women receiving non-adapted CBT in the same clinic. This finding indicates that ACBT worked twice as well as the non-adapted version. Unfortunately, there are relatively few studies that directly compare culturally adapted and non-culturally adapted forms of CBT (Sue et al., 2009), and among these studies the adaptations tend to focus on reducing symptoms of depression instead of substance use (Kohn & Oden, 2003; Miranda et al., 2006; Organista et al., 1994). Another critical gap in this literature is the biased nature of these studies samples since is relies on Black and Hispanic individuals who are willing to volunteer to participate in these studies. Volunteering to participate may make this group of Black and Hispanic subjects unrepresentative of the cultural uniqueness of Black and Hispanic populations that would motivate the need for adaption in the first place (Wolff, 2000). More exploratory research is needed to explore issues of recruitment, retention, and engagement.

Despite the call for research to further test the effectiveness of CBT in reducing substance use among Black and Hispanic individuals, there is a lack of randomized clinical CBT trials focusing on substance use among these populations. Given the serious health and mental health consequences of substance use and the empirical and clinical proliferation of CBT treatment for substance use disorders, the absence of meta-analytic studies comparing CBT’s effectiveness by race/ethnicity is surprising.

In response to this gap, we conducted a meta-analysis to examine whether CBT approaches were effective in reducing substance use among Black and Hispanic individuals as compared to their White counterparts. This study had three aims: (a) to examine the inclusion of Black and Hispanic populations in studies examining CBT effectiveness for reducing substance use; (b) to compare retention and engagement rates in CBT treatment for substance use between Black and Hispanic (BH) studies and non-Hispanic White (NHW) studies; and (c) to compare the effectiveness of CBT in reducing substance use between BH studies and NHW studies. Based on the existing literature, we hypothesized that CBT, when compared to existing treatments, will be less effective in BH studies than NHW studies.

Methods

Sample

We followed the recommendations of Johnson and Eagly (2000) in the selection of articles. These recommendations included: (a) the use of scientific databases (e.g., Web of Science, PsychINFO, and Medline) to identify studies examining the impact of CBT on the reduction of adult substance use (key words: substance, drug, abuse, dependence, addiction, Cognitive Behavioral Therapy, and CBT); (b) a search for unpublished articles and dissertations using ERIC database and the invisible college technique, that is, sending e-mails to the College on Problems of Drug Dependence and to the National Hispanic Science Network to request unpublished manuscripts; and (c) the use of the ancestry approach to identify new articles from the reference lists of articles identified in the search.

The search yielded 322 articles that were subsequently reviewed by three raters to ensure that these articles met the following inclusion criteria: (a) studies conducted in the United States; (b) studies published between 1990 and 2012; (c) samples from the adult population (those 18 and over); (d) the use of a randomized and controlled experimental design; (e) the inclusion of at least one substance use outcome variable; and (f) a sample racial distribution of at least 70% White or 70% Black and/or Hispanic. The raters independently assessed articles for inclusion and coded them to increase reliability. A coding tree was developed, and raters met three times to compare ratings, refine coding, and reach consensus.

The final meta-analytic sample comprised 15 research reports, describing 16 studies, and contributing 126 effect sizes resulting in an N = 3,784 individuals. Four of the 16 studies were published between 1997 and 2000 and were included in the NHW studies subgroup. All BH studies were published between 2001 and 2012. Figure 1 displays the sample selection process and Table 1 provides demographic information for studies included in the sample.

Figure 1.

Sample Selection

Table 1.

Study Characteristics

	All Studies (N=16)	NHW Studies (N=11)	BH Studies (N=5)
Date of publication
1997 – 2000	4	4	0
2001 – 2012	12	7	5
Studies Reporting Percentages of
Whites	15	11	4
Blacks	11	7	5
Hispanics	9	6	3
Other races	6	4	2
Sample recruitment setting
Medical setting	2	1	1
Substance use treatment	5	3	2
General population	9	7	2
Intervention Format
Group	5	4	1
Individual	9	6	3
Combination	1	0	1
Not reported	1	1	0
Interventionist qualification
Doctoral Level	1	0	1
Master level degree	4	3	1
Mixture of professionals	8	6	2
Not reported	3	2	1
Number of CBT sessions offered
3 to 12	10	5	5
13 to 24	4	4	0
25 to 48	2	2	0
Outcome differences between CBT and comparison at baseline
No differences	14	9	5
Differences	2*	2	0
Not reported	0	0	0
Demographic and Outcome differences between study dropouts and completers
No differences	13	4	5
Differences	2	2	0
Not reported	1	1	0
Study compared outcome across race
Yes	3	3	0
No	13	8	5
Substance use outcome
Cocaine	3	0	3
Marijuana	3	2	1
Alcohol	6	6	0
Alcohol & other drugs	4	3	1
Sample Demographics
Sample size (N)	3,784	3,272	512
Age	40 (06)	41 (05)	36 (08)
Mean % of sample male	81 (13)	77 (18)	83 (11)
Mean % of sample employed	71 (23)	70 (16)	45 (12)
Mean % of sample married	52 (18)	35 (18)	22 (18)
Mean % of studies racial distribution
White	63 (32)	81 (08)	14 (08)
Black	33 (35)	09 (04)	61 (33)
Hispanic	16 (23)	09 (05)	29 (40)
Other	03 (02)	03(01)	03 (04)

Note. NHW = Non-Hispanic White. BH = Black/Hispanic.

^*In one study, the CBT group had significantly more substance users at baseline and in the other study there were significantly more White participants in the CBT group.

Variable Extraction and Computation of Effect Sizes

The following data were extracted from each study and entered into SPSS 19: article type (i.e., peer reviewed, unpublished), year of publication, sample size, sample demographics (e.g. racial composition, mean age, gender), study type (efficacy versus effectiveness), rigor of methodology (e.g., types of measures used, whether clinician training was provided, study design), intervention description (i.e., length of intervention, delivery format), and outcome variables (e.g. decrease in substance use, retention). Three independent raters extracted all data, and disagreements among the raters were discussed until 100% agreement was reached.

Table 2 displays studies’ racial distribution percentages and outcomes used to calculate effect sizes. Effect sizes were calculated using Hedge’s g, that is, the difference between means of two groups divided by the pooled standard deviation (Hedges & Olkin, 1985). We used this approach to account for variability in sample size. In addition, Hedges and Olkin suggest pooling the standard deviations of the control and experimental groups to improve the accuracy of the population’s standard deviation since this approach accounts for the variance in both groups.

Table 2.

Racial distributions and Outcomes used to calculate effect sizes

Study	Racial distribution percentages		Study type, group comparison & results	Effect size comparison	Substance use outcomes	Follow up times from baseline
Anton 1999	Black Hispanic White Other	NR NR 86 NR	Effectiveness: Excluded from group comparison because it tested effect of Naltrexone.	CBT pre-posttest comparison	# drinks /drinking day Analog craving score	3 months
Anton 2005	Black Hispanic White Other	12 01 86 NR	Efficacy: Included CBT + placebo versus Motivational Interviewing + placebo. Four groups to evaluate Naltrexone. CBT did better.	CBT pre-posttest comparison	# drinks /drinking day	3 months
				CBT versus comparison group	# drinks /drinking day	3 months
Brown 2006	Black Hispanic White Other	11 13 75 NR	Efficacy: CBT versus 12 steps. CBT had better long term effects, though 12 steps had better post intervention effects. On average, 12 steps had a better overall effect.	CBT pre-posttest comparison	Days abstinent-multiple drugs	3, 6, 9, and 12 months
				CBT versus comparison group	Days abstinent-multiple drugs	3, 6, 9, and 12 months
Budney 2006	Black Hispanic White Other	NR NR 96 NR	Efficacy: CBT versus abstinence vouchers. CBT did better.	CBT pre-posttest comparison	Days of marijuana use in past 30 days	3, 6, 9, and 12 months
				CBT versus comparison group	Days of marijuana use in past 30 days	3, 6, 9, and 12 months
Carroll 2012	Black Hispanic White Other	64 12 19 5	Effectiveness: CBT versus Contingency Management. Contingency Management did better.	CBT pre-posttest comparison	N/A	N/A
				CBT versus comparison group	Days of marijuana use in past 30 days	3 months
Epstein2003	Black Hispanic White Other	70 NR NR NR	Effectiveness: CBT versus comparison. During active treatment, CBT did no better than comparison, but in 6 and 12 months follow-ups, CBT showed greater improvement on Addiction Severity Index.	CBT versus comparison group	% cocaine-negative urine	3, 6, and 12 months
Kennedy 2012	Black Hispanic White Other	100 000 000 000	Efficacy: CBT versus TAU. CBT did better.	CBT pre-posttest comparison	N/A	N/A
				CBT versus comparison group	# participants abstinent	1 month
Lydecker 2010	Black Hispanic White Other	NR NR 71 NR	Effectiveness: CBT versus 12 steps. CBT did better at posttest and had better long term effects.	CBT pre-posttest comparison	# days abstinent from alcohol and other drugs	3, 6, 9, 12, 15, and 24 months
				CBT versus comparison group	# days abstinent from alcohol and other drugs	3, 6, 9, 12, 15, and 24 months
Match 1997 treatment and after treatment studies	Black Hispanic White Other	06 12 80 01	Effectiveness: CBT versus MET and CBT versus 12 steps. MET and 12 steps did better.	CBT pre-posttest comparison	Drinking consequences GGT ASI psychological consequences	9 and 15 months
	Black Hispanic White Other	15 03 80 01		CBT versus comparison group	Drinking consequences GGT ASI psychological consequences	9 and 15 months
Morgenstern 2012	Black Hispanic White Other	10 13 74 03	Effectiveness: Excluded from group comparison because it tested effect of Naltrexone.	CBT pre-posttest comparison	Mean drinks per week	3 months
Oslin 2008	Black Hispanic White Other	NR NR 73 NR	Efficacy: CBT + placebo versus MET+ placebo. CBT did better.	CBT pre-posttest comparison	# drinking days past 30 days # heavy days drinking # drinks /drinking day	6 months
				CBT versus comparison group	# drinking days past 30 days # heavy days drinking # drinks /drinking day	6 months
Rowan-Szal 2005	Black Hispanic White Other	06 74 22 NR	Effectiveness: CM versus CBT. CM with rewards did better than all other groups, though CBT did better than comparison when CM was removed.	CBT versus comparison group	ASI drug and alcohol	15 weeks
Schmitz 2009	Black Hispanic White Other	71 NR 14 NR	Effectiveness: Excluded from group comparison because it tested effect of Naltrexone.	CBT pre-posttest comparison	# heavy drinking days in past 30 days	3 months
Shoptaw 2005	Black Hispanic White Other	03 12 80 04	Effectiveness: CBT versus CM. CBT did slightly better on this outcome, though CBT did worse on urine tests.	CBT pre-posttest comparison	# days of methamphetamine use in past 30 days	4, 6, and 12 months
				CBT versus comparison group	# days of methamphetamine use in past 30 days	4, 6, and 12 months
Stephens 2000	Black Hispanic 1>White Other	NR NR 95 NR	Efficacy: CBT versus brief counseling. There were no differences between CBT and comparison.	CBT pre-posttest comparison	# days substance use in past 30 days	4, 7.13, and 16 months
				CBT versus comparison group	# days substance use in past 30 days	4, 7.13, and 16 months

NR = Not Reported

To compute individual effect sizes from each study, we compared: (a) the group receiving CBT (hereafter referred to as the experimental group) with the group receiving the comparison treatment (e.g., 12 steps, treatment as usual- hereafter comparison group) for each substance use outcome at each follow-up point and (b) baseline with post treatment follow-up differences only for groups receiving CBT (hereafter pre-posttest comparison). For studies that included more than one follow-up (e.g. 3, 6, 9, and 12 months), we used the average effect size (Lipsey & Wilson, 2000). Three studies had to be excluded from the CBT versus comparison group analysis because these studies assessed the effect of medication or motivationally enhanced treatment in addition to CBT; thus both comparison and experimental groups received CBT. The comparison group analysis included 4 BH studies and 9 NHW studies (see figure 2). Six studies were excluded from the pre-posttest analysis because they did not include sufficient information to calculate the effect size. Therefore the pre-posttest analysis included 4 BH studies and 6 NHW studies (see figure 3). For studies that reported multiple substance use outcomes (e.g. number of days abstinent and number of negative urine tests), an aggregate effect size was calculated using the mean of the Hedges’ g effect size estimate. This reduced the artificial inflation/ deflation of effect sizes based on inclusion of multiple effect sizes from single studies (Lipsey & Wilson, 2000). For studies that did not report means and standard deviations, we calculated effect sizes based on the t-tests and p-values reported in these studies (Lipsey & Wilson, 2000). We measured engagement by the average percentage of CBT sessions attended and retention by the percentage of participants who completed the CBT treatment.

Figure 2.

High Resolution Plot for CBT versus Comparison Group Analysis

Figure 3.

High Resolution Plot for Pre-Posttest Comparisons

Data Analysis

The analysis examined descriptive statistics and compared substance use effect sizes. Effect size analysis was conducted using Comprehensive Meta-Analysis software (CMA) and was weighted by the inverse of its variance to account for sample size variability. Then, to examine the heterogeneity of effect sizes, we calculated I² and Q statistics. In order to assess and compare CBT’s effectiveness between White and Black and/or Hispanic samples, we divided the studies into two subgroups: Studies that had more than 70% of their sample self-identifying as White (NHW studies) and studies that had more than 70% of its sample self-identifying as Black and/or Hispanic (BH studies). Table 2 displays the sample racial breakdown for each study. We conducted a subgroup analysis using a mixed effects model in which we pooled within subgroups with a random effects model and conducted tests for significant differences between subgroups with the fixed effects model. A meta-regression analysis was conducted in CMA to examine the moderating impact of race, gender, employment, treatment format (group, individual, both), substance type, sample size, study type (efficacy versus effectiveness), number of CBT sessions, and use of intention to treat analysis on effect size. Finally, a funnel plot was created to explore the presence of publication bias on the meta-analytic sample.

Results

Study Characteristics

A total of 16 studies published in peer reviewed journals were included in the overall meta-analysis. Despite our efforts to include unpublished research, we could not identify any work that met the study’s criteria. Only 37.5% of studies specified the sample percentages of Black individuals, Hispanic individuals, or other racial groups, and only four studies included a sample of 70% Black and/or Hispanic individuals. All studies used a randomized, controlled design that compared the effectiveness of CBT with another treatment on at least one substance use outcome (including alcohol, marijuana, methamphetamine, cocaine, and stimulants). All studies used strong measures (e.g. standardized scales, biological markers), provided training to interventionists, and accounted for pre-treatment differences between comparison and experimental groups. Studies used a combination of statistical analyses (e.g., t-tests, ANOVA, GLM, ANCOVA, MANOVA, GEE, Latent Growth Analysis) to examine the data. Only three studies reported participants’ income, making it impossible to determine the mean income of the overall sample or to control for income across studies. Approximately 71% of the sample was employed and 52% were married or in a partnership. Males comprised the majority of the total sample (81%), with BH studies having lower representation of females (17%) compared to NHW studies (23%). Table 1 displays studies’ remaining demographic information.

Retention and Engagement in CBT and Follow-up

In order to examine differences in retention and engagement in CBT as well as follow-up completion rates between studies with respect to race (i.e., White vs. Black and/or Hispanic), we conducted a t-test analysis. The results revealed there was no significant difference between NHW and BH studies when it came to retention, engagement, and follow-up. Table 3 displays the t-test results.

Table 3.

Retention, engagement, and follow up completion comparisons

	Engagement Mean (SD)	Retention Mean (SD)	Follow up Completion Mean (SD)
N	16	14	12
Studies with predominantly White sample	75.4 (11.30)	71.8 (17.29)	87.2 (6.10)
Studies with predominantly Black and Hispanic sample	71.2 (27.12)	58.5 (26.28)	83.0 (13.5)
p value	.33	.14	.23

Substance Use Overall and Subgroup Effects in CBT versus Comparison Group Analysis

We included a total of 13 studies in this analysis (see Table 3). The overall effect size of CBT versus comparison groups on substance use outcomes was weak and failed to reach statistical significance (g < .01; 95% CI: −.09 – .10; p = .95). These studies had low heterogeneity (I²= .00). The mean effect size for BH studies was weak and failed to reach statistical significance (g = .03; 95% CI: −.25 – .31; p = .83). These studies had low heterogeneity (I² = 17.28). The mean effect size for NHW studies was weak and failed to reach statistical significance (g < −.01; 95% CI: −.10 – .10; p = .98). These studies had low heterogeneity (I² = 0.00). The differential effect size yielded a g = .02 (Q = .044; p =.83), indicating there was no significant difference in effect sizes between BH and NHW studies in the CBT versus comparison group analysis (see Figure 2).

Fixed effects meta-regression analysis failed to identify significant effects with respect to race, gender, employment at baseline, study type, treatment format, substance type, sample size, number of CBT sessions, and use of intent to treat analysis. Visual inspection of funnel plot of standard errors by Hedges’ g indicated that publication bias was not a concern in this sample.

Substance Use Overall and Subgroup Effects for CBT Pre-Posttest Comparison

The overall effect size of CBT on substance use outcomes had moderate heterogeneity (I²= 56.18) and was very strong, g = 1.19 (95% CI: 1.05 – 1.33; p = .00). The mean effect size for the BH studies was strong (g =.86) and had low heterogeneity (95% CI: .62 – 1.10; p < .001; I² = 0.00). The mean effect size for NHW studies was very strong and moderately heterogeneous with g = 1.36 (95% CI: 1.19–1.53; p < .001; I² = 42.12). The differential effect size using a random effects analysis yielded a g =.50 (Q= 11.04; p < .001), which indicates NHW studies had a significantly higher effect size than BH studies in the pre-posttest comparison (see Figure 3).

Random effects meta-regression analysis did not reveal significant effects with respect to treatment format, gender, type of substance, study type, number of CBT sessions, and use of intent to treat analysis on effect size. However, race (slope: −0.49; 95% CI: −.79 – −.20; p < .001), employment at baseline (slope: −0.01; 95% CI: .01 – −.02; p < .001), sample size (slope: .002; 95% CI: .00 – .00; p < .001), and study type (slope: −1.15; 95% CI: −1.39 – −.92; p < .001) were significantly associated with effect size. Visual inspection of the random effects funnel plot of standard errors by Hedges’ g indicated that publication bias is not a concern in this sample.

Discussion

The goal of this study was to (a) examine the inclusion of Black and Hispanic populations in studies examining CBT effectiveness for reducing substance use; (b) compare retention and engagement rates in CBT treatment for substance use between Black and Hispanic (BH) studies and non-Hispanic White (NHW) studies; and (c) compare the effectiveness of CBT in reducing substance use between BH and NHW studies. Overall, the results suggest that few studies (a) compared the effects of CBT across racial groups and (b) included a Black and/or Hispanic majority sample. This is a critical finding that highlights the lack of representativeness of the Black and Hispanic samples included in these studies and the need for increased engagement and retention of Black and Hispanic samples in research.

This meta-analysis failed to support our hypothesis that CBT, when compared to existing treatments, will be less effective in BH studies than NHW studies. While the comparison group analysis failed to identify significant differences in effect sizes between BH and NHW studies, we found preliminary evidence that NHW studies had a significantly higher effect size than BH studies in the pre-posttest comparison. This result suggests that researchers and treatment providers should continue to question whether non-culturally adapted forms of CBT are as effective in reducing substance use among Black and Hispanic individuals as compared with White individuals until more research examining CBT’s effectiveness among ethnically and racially diverse samples is available.

Substance Use Effects in CBT versus Comparison Group Analysis

Overall analysis results (that is, NHW and BH studies together) suggest CBT is an effective intervention for reducing substance use. These findings are consistent with existing literature, although we found a smaller effect of CBT on substance use outcomes (g = .01; 95% CI: −.09 – .12) compared to Magill and Ray (2009) (g = .14; 95% CI, .09–.1) and Irvin et al. (1999) (g = .28; 95% CI, .79–1.35). Moreover, findings indicated no significant CBT effect size difference comparing substance use outcomes in BH and NHW studies. The absence of significant differences between the effect sizes in BH and NHW studies within the posttest comparison analysis suggests that CBT worked as well as comparison treatments by approximately the same amount in both BH and NHW studies. It is important to consider, however, that the majority of studies included in the current meta-analysis relied on comparison treatment controls as opposed to true controls. Because it is unknown if these comparison treatments (contingency management, 12-step, motivationally enhanced therapy, psychoeducation, treatment-as-usual, methadone maintenance, and abstinence vouchers) work equally well for NHW and BH individuals, the current findings may not be replicated if the sample included studies that compared CBT to a true control. Some of the comparison group treatments in the current study included components of CBT (e.g., psychoeducation, motivational techniques, contingency management), making it harder to assess the true effect of CBT on the experimental group. Moreover, people who volunteer to participate in studies are often motivated towards the outcomes of the study (Wolff, 2000) (in this case, reduced substance use). This motivation may create a homogenization across non-Hispanic White, Black, and Hispanic participants, hence predicting the lack of difference between NHW and BH studies. More research is needed to examine recruitment differences (including motivations to participate) between non-Hispanic Whites, Blacks, and Hispanics. Finally, it is likely that the staff mix in BH studies reflected the client group characteristics, resulting in an unintended cultural adaptation of CBT. Thus, the findings presented here must be considered with caution.

Substance Use Effects for CBT Pre-Posttest Comparison

At pre-posttest comparison, the impact of CBT in reducing substance use among BH studies was strong, indicating that Black and Hispanic individuals in these studies’ samples benefited from traditional CBT. However, results from the pre-posttest comparison analysis indicated that CBT was significantly less effective in reducing substance use in BH studies than in NHW studies. This finding provides some support for our hypothesis. Without the use of comparison or true control groups, we cannot discern whether the difference in effect sizes between BH and NHW studies in the pre-posttest comparison results was due to CBT or another unknown variable. Thus, these findings must be interpreted with caution. Further research must be conducted to specifically examine the moderating impact of race in non- culturally adapted evidence based substance abuse treatment.

Retention and Engagement in CBT and Follow-up

Results from the t-test comparison analysis to assess differences in retention, engagement, and follow-up completion rates revealed that BH studies had lower percentages on all measures when compared to NHW studies. However, our retention and engagement analysis failed to reach statistical significance; perhaps due to the low power to detect difference among the small sample of studies. Our literature search only identified a few studies that were able to engage and retain ethnoracial participants in randomized clinical trials. This finding was consistent with existing literature (Alvidrez, Azocar, & Miranda, 1996; Austin & Wagner, 2006; Cooper, MacMaster, & Rasch, 2010; Mancino et al., 2010; Suinn, 2003; Voss Horrell, 2008). Likewise, the limited inclusion of women in clinical trials is alarming. Studies indicate that women face multiple barriers when accessing substance use treatment (Green, 2006). Possible explanations for the underrepresentation of ethnoracial minorities and women include lack of transportation or child care, distrust of clinical research initiatives, fear and shame, the belief that they can recover without help, and the lack of knowledge about available services (Caetano, 2003; Esser-Stuart & Lyons, 2002; Fuller et al., 2004; Longshore, 1999; Pinto, McKay, Wilson, et al., 2007).

Lower retention rates among Black and Hispanic individuals may suggest a conflict between the cultural views of Black or Hispanic populations and the theoretical framework of CBT as discussed in the literature review (David, 2009; Frank & Frank, 1993; Pinto, McKay, Baptiste, et al., 2007; Schmidt et al., 2006). Ignoring the unique context and experiences of historically disadvantaged populations may increase traditional CBT treatment dropout rates, since treatment engagement and retention are frequently linked with successful substance use outcomes (Oene, Schippers, De Jong, & Schrijvers, 2001; Simpson, Joe, Rowan-Szal, & Greener, 1995), low engagement and retention rates could contribute to health disparities between Black and Hispanic and White individuals (David, 2009; Hall, 2001; Richardson & Molinaro, 1996). The inclusion of a relevant cultural context within the framework of CBT may help to explain the thoughts, emotions, and behaviors of ethnoracial minority participants and therefore improve retention and increase CBT’s effectiveness with Black and Hispanic individuals (Frank & Frank, 1993). It will also be important for future research to develop and test new culturally grounded interventions and culturally-adapted versions of widely used substance abuse treatments, including contingency management, motivationally enhanced therapy, and CBT.

Higher dropout rates among ethnoracial minorities could potentially explain the lack of difference between comparison group effect sizes in BH and NHW studies. Those who find CBT and other/comparison treatments incompatible relative to their socio-cultural experiences may dropout, leaving only the ethnoracial participants who, for unknown reasons (e.g., education level, socioeconomic status, trauma history), yielded positive results. Moreover, scholars have reported that ethnoracial minority individuals performed as well or better than White individuals on substance use outcomes at post-treatment (Morgenstern & Bux, 2003; Tonigan, 2003). The current findings were consistent with this literature since the CBT versus comparison analysis indicated that CBT, when compared to other treatments, was equally effective in NHW and BH studies. However, given the limitations of these studies, the lack of meta-analyses examining the impact of CBT among ethnoracial minorities, and the methodological limitations of the current meta-analysis (e.g. sampling), it will be important to interpret these results with caution.

Analysis of Moderating Variables

A meta-regression analysis was conducted at CBT versus comparison analysis and at pre-posttest comparison analysis to examine the moderating impact of race, gender, employment, treatment format (group, individual, both), substance type, sample size, number of CBT sessions, type of study (efficacy versus effectiveness) and use of intention-to-treat analysis on overall effect size. While we did not find a significant moderating relationship in the CBT versus comparison analysis, we found that studies with higher numbers of White individuals, efficacy studies, and studies with a larger sample size had significantly larger effect sizes.

Surprisingly, studies with larger percentages of unemployed individuals were significantly associated with a larger effect size. We intended to control for socioeconomic status, but studies did not report this information, so we used employment as a proxy instead. Research shows that unemployed individuals have higher levels of substance abuse than those who are employed (Henkel, 2011). Thus, the effect of CBT on substance use outcomes may have been larger among those who were unemployed because their substance use may have been more severe than those who were employed. There is also evidence that individuals who are unemployed are more likely to engage in treatment than those who are employed (Mertens & Weisner, 2006). This may explain the increased reduction in substance use among those who were unemployed, though more research in this area is needed to better understand the relationship between unemployment and substance abuse treatment.

Surprisingly, we did not find a moderating effect of gender on substance use outcomes in the comparison and pre-posttest analysis. Gender is important to consider when examining effectiveness of CBT because research suggests that women are more likely to complete and benefit from treatment than their male counterparts (Green, 2006). This may be due to the low number of females included in the studies that we examined, especially among the BH studies. More research is needed to examine the intersection of gender and race/ethnicity as moderating variables on the impact of traditional CBT on substance use.

Limitations

This study had a number of limitations. First, only a few studies met the inclusion criteria and none of them provided analysis specific to Black or Hispanic individuals. Despite our effort to separate studies with a predominantly White sample from studies with a predominantly Black and/or Hispanic sample, all studies included White participants making it difficult to separate treatment effect by racial group. Thus, our findings should be interpreted with caution and be considered preliminary at best.

Due to the limited number of studies it was not possible to analyze the effect of CBT in reducing use of particular substances. Studies included a variety of follow-up points (e.g., 3 months, 12 months post baseline). When conducting a meta-analysis, one may select a fixed amount of time as follow-up (e.g., outcomes reported at 3 months post-baseline) or average the effect size for all follow-ups (Lipsey & Wilson, 2000). Since studies had a variety of lengths of intervention, we decided to average the follow-up effect sizes. However, this approach may fail to reflect an effect size that is apparent with a short follow-up, but not a longer follow-up.

Comparison group analysis assumes that CBT and comparison group outcomes were the same at baseline even though one of the studies included in the analysis (Shoptaw et al., 2005) revealed that the CBT group had significantly higher rates of substance use at pretest (see Table 1). The pre-posttest comparison analysis lacks a comparison or true control group, making it impossible to know whether the change was due to participation in CBT or other variables, such as the passage of time.

The lack of generalizability of the studies selected for Black and Hispanic individuals is another significant limitation of this meta-analysis. Only 5 of the 16 studies included a predominantly Black or Hispanic sample, and these studies had sample sizes ranging from 44 to193. Thus, the results cannot be generalized beyond the studies included in the analysis.

Conclusion and Recommendations for Future Research and Practice

Findings indicate that CBT was a useful intervention for reducing substance use among White and Black and Hispanic individuals who volunteered to participate in the clinical trials. However, the impact on BH studies was moderate, indicating room for improvement. Our findings suggest that more research is needed to assess the effect of CBT on reducing substance use among Blacks and Hispanic individuals. Future research should explore whether adapting culturally-sensitive CBT approaches could enhance the effectiveness of substance abuse treatment among Black and Hispanic individuals. Previous research highlighting the perspective of community members (Pinto, 2009) shows the importance of integrating the worldviews of ethnoracial minorities in the development of culturally-congruent health-related interventions (Dunlap, Golub, & Johnson, 2006; Schmidt et al., 2006; Windsor & Dunlap, 2010). Understanding clients’ needs, worldviews, strengths, as well as challenges can inform the development of interventions that are meaningful to participants and therefore improve recruitment and retention (Dunlap et al., 2006). Lastly, future research should systematically compare the effects of culturally adapted CBT models with traditional CBT methods on substance use treatment outcomes.