A comparison of treatment outcomes for adolescent community reinforcement approach participants with and without co-occurring problems

Abstract

This study examined the relationship between Adolescent Community Reinforcement Approach (A-CRA) participation with treatment engagement, retention, and satisfaction, and with substance use and emotional problem outcomes. Participants had substance use disorders (SUD) only or co-occurring substance use and psychiatric problems. Those with co-occurring problems reported more days of substance use and emotional problems at intake to treatment than those with SUD only. All groups received equivalent exposure to A-CRA during treatment implementation. At the 12-month follow-up, adolescents classiFIed as externalizers (n = 468) or those with both externalizing and internalizing problems (n = 674) had significantly greater improvement in their days of abstinence and substance problems relative to adolescents with substance use disorders only (n = 666). Additionally, adolescents reporting symptoms of internalizing (n = 154), externalizing, or both externalizing and internalizing disorders had significantly greater improvements in days of emotional problems relative to adolescents with SUD only.

1. Introduction

Studies of adolescents presenting for substance use treatment reveal that between 55% and 88% of youth meet criteria for a co-occurring psychiatric problem (Chan, Dennis, & Funk, 2008; Grella, Hser, Joshi, & Rounds-Bryant, 2001; Grella, Joshi, & Hser, 2004; Sterling & Weisner, 2005). The most common co-occurring problems (COPs) among this population are externalizing disorders, such as conduct disorder, with reported rates ranging from 50–80% (Diamond et al., 2006; Hser, Grella, Collins, & Teruya, 2003; Kaminer, Burleson, & Goldberger, 2002) and ADHD with rates ranging from 13–77% (Chan et al., 2008; Diamond et al., 2006; Grella et al., 2001); however, internalizing disorders are also quite common. Rates of depression have been reported from 14–50% (Bukstein, Glancy, & Kaminer, 1992; Diamond et al., 2006; Grella et al., 2001) and anxiety disorders from 7–40% (Chan et al., 2008; Clark et al., 1995; Diamond et al., 2006; Kaminer, 1996). Additionally, traumatic distress has been reported to occur at rates of 14–39% (Chan et al., 2008; Diamond et al., 2006). These adolescents' co-occurring problems result in greater overall problem severity when they enter substance use treatment, especially if they have both an internalizing and an externalizing disorder (Grella et al., 2001; Shane, Jasiukaitis, & Green, 2003).

There is also evidence that co-occurring internalizing and/or externalizing problems are moderators that affect adolescent treatment participation and outcomes. Adolescents with these problems are considered more difficult to engage and retain in treatment (Dobkin, Chabot, Maliantovich, & Craig, 1998; Kaminer, Tarter, Bukstein, & Kabene, 1992; Kazdin, 1996). Based on a review of several studies, it appears that adolescents with externalizing disorders were more likely to terminate from treatment early, whereas those with internalizing disorders had better retention rates (Flanzer, 2005); however, Rowe, Liddle, Greenbaum, and Henderson (2004) found comparable treatment engagement and retention for adolescents classified in three comorbid groups. Substance use outcomes also tend to be poorer for those with dual disorders (Couwenbergh et al., 2006). Grella et al. (2004) found that dually diagnosed adolescents reduced their substance use, but not as much as did adolescents with an SUD only, and those with COPs were more likely to relapse following treatment (Dakof, Tejeda, & Liddle, 2001; Grella et al., 2001; Shane et al., 2003; Tomlinson, Brown, & Abrantes, 2004). Other research has revealed that trajectories of change can vary by co-occurring group. Rowe et al. (2004) found that externalizers initially increased their substance use during treatment, while internalizers experienced a decrease; however, both groups regressed to their baseline levels of substance use by 12 months post-treatment. Randall, Henggeler, Pickrel, and Brondino (1999) found that externalizers had higher rates of substance use at 16 months post-baseline as compared to adolescents with SUD only; however, adolescents with both an internalizing and externalizing disorder had better 16-month outcomes.

To extend the field's knowledge about the relationships between psychiatric problems and treatment engagement and outcomes, Winters, Stinchfield, Latimer, and Stone (2008) examined the relationship between a 12-step program's treatment outcomes among subgroups of adolescents created using dimensional measures of internalizing and externalizing symptoms. They examined treatment completion and post-treatment drug use measures at 1, 4, and 5.5 year follow-up intervals after treatment admission. Results indicated that those adolescents classified in the externalizing group had poorer outcomes for treatment retention and for short- and long-term behavior outcomes when compared to those classified in the internalizing group. The researchers hypothesized that since the externalizing group had poorer retention, retention level was a potential mediator of drug behavior outcomes.

Given the high incidence of co-occurring problems among adolescents entering substance use treatment, and findings regarding the moderating impact of these problems on treatment outcomes, researchers have outlined the case for integrated treatments that address both the SUD and the co-occurring psychiatric problems (Armstrong & Costello, 2002; Bender, Springer, & Kim, 2006; Couwenbergh et al., 2006; Lamps, Sood, & Sood, 2008; Libby & Riggs, 2005; Riggs, Levin, Green, & Vocci, 2008). Specific treatment features recommended for an integrated approach to treating these adolescents include: a) validated intake assessments that adequately identify psychiatric and substance use problems (Bender et al., 2006; Couwenbergh et al., 2006; Riggs, 2003); b) engagement and retention techniques that rely on an empathic and non-confrontational approach (Lamps et al., 2008; Riggs, 2003); c) a fiexible clinical approach that includes input from participants when developing treatment plans (Bender et al., 2006); d) inclusion of family and available community resources (Riggs, 2003); e) developmentally appropriate and gender/culturally competent treatment (Armstrong & Costello, 2002; Bender et al., 2006); f) a multifaceted approach that addresses several domains including problem-solving and decision-making skills, affect regulation, impulse control, communication skills, and peer and family relations (Bender et al., 2006; Couwenbergh et al., 2006; Libby & Riggs, 2005; Riggs, 2003); g) homework assignments (Bender et al., 2006); and (h) paired medication monitoring and adherence procedures with medication (when indicated) (Riggs, 2003; Riggs et al., 2008). Experts also have noted the considerable need for training therapists to implement these assessment and treatment components (Couwenbergh et al., 2006; Riggs, 2003).

A-CRA is an empirically supported treatment (EST) that incorporates many components recommended for integrated treatment and has been widely implemented with a rigorous clinical training and certification process. It is an adaptation of the Community Reinforcement Approach (CRA) that was initially developed and tested with adults (Azrin, Sisson, Meyers, & Godley, 1982; Hunt & Azrin, 1973), and was then adapted, manualized, and clinically tested for use with adolescents (Dennis et al., 2004; Godley et al., 2001; Slesnick, Prestopnik, Meyers, & Glassman, 2007). A-CRA is based on the belief that it is critical to involve a substance using individual's “community” (i.e., family, friends, school, job, organizations) in the recovery process. The approach relies heavily on positive reinforcement and operant techniques, and specifically avoids confrontation (Azrin, 1976; Hunt & Azrin, 1973). Clinicians strive to increase an adolescent's access to social reinforcement through engagement in pro-social activities, which then compete with substance using behavior to increase abstinence. A-CRA includes 19 procedures (e.g., problem solving skills, communication skills, relapse prevention) from which clinicians can choose in order to address the immediate needs of their client during any given treatment session (Godley et al., 2001; Meyers & Smith, 1995). Medication monitoring/adherence is one of the procedures and is used to help facilitate compliance with prescribed medications for co-occurring psychiatric problems. There also are specific sessions designed for parents/caregivers. Importantly, the effectiveness of A-CRA has been supported in randomized clinical trials (Dennis et al., 2004; Godley, Godley, Dennis, Funk, & Passetti, 2007; Slesnick et al., 2007). Moreover, A-CRA has been shown to have relatively high and equivalent rates of treatment engagement, retention, satisfaction, and substance use recovery outcomes across gender and racial groups (Godley, Hedges, & Hunter, 2011).

Since prior research suggests that co-occurring problems moderate treatment engagement, retention, and substance use outcomes, and A-CRA includes many of the treatment features recommended for youth with comorbid psychiatric problems, the purpose of this study was to assess the relationship of A-CRA participation with treatment engagement, retention, satisfaction, and with substance use and emotional problem outcomes. A large SAMHSA/CSAT grant initiative to disseminate A-CRA in the United States with extensive attention to fidelity (Godley, Garner, Smith, Meyers, & Godley, 2011) provided a sufficient sample to divide adolescents into four subgroups for the analyses: non-comorbid, internalizers, externalizers, and a combined internalizing and externalizing group. As prior research has shown that those with co-occurring problems entered substance use treatment with the most severe problems, we hypothesized a) that adolescents in any of the three comorbid groups would present to treatment with more severe substance use and emotional problems, and those with both internalizing and externalizing symptoms would have the most severe problems. Since Rowe et al. (2004) found that adolescents participating in ESTs with COPs did not have different retention rates than adolescents with SUD only, we hypothesized (b) that all groups would have equivalent rates of A-CRA treatment engagement, retention, and satisfaction with treatment. Finally, since prior research has revealed different responses to treatment for adolescent substance users with and without co-occurring problems (Randall et al., 1999; Rowe et al., 2004), we hypothesized c) that the four groups would have different trajectories of change for substance use and emotional problems after participating in A-CRA.

2. Materials and methods

2.1. Sites

This study is based on a large dissemination project of A-CRA across 78 SUD treatment organizations that were implementing the treatment with funding provided by the Substance Abuse and Mental Health Services Administration's (SAMHSA) Center for Substance Abuse Treatment (CSAT). Data were collected as a requirement of each agency's grant for local program evaluation under their respective voluntary consent procedures. All data were de-identified prior to analysis. Each organization received grant funding specifically to implement A-CRA, and all clinicians participated in the same standardized training, cross-site supervision, and certification process based on individualized review and feedback provided by the treatment developers throughout the grant period (Godley, Garner, Smith, Meyers, & Godley, 2011).

Grantees were located in 26 different states and served diverse communities. The following regions of the country were represented: Northeast/Mid-Atlantic (5), Northeast/New England (10), Southeast/South Atlantic (17), Midwest (9), Southwest (11), West/Mountain (9), and West/Pacific (17). Forty-two agencies served youth living in urban settings, 8 served youth living in rural areas, and 28 served geographic areas that were mixed. The percentage of female participants at agencies ranged from 0% to 62%. Some agencies primarily served African American adolescents (11), Caucasian adolescents (33), Hispanic adolescents (28), and Native American adolescents (3). There also were sites that had an equal percentage of adolescents that were African American and Hispanic (1), African American and Caucasian (1), and Hispanic and Caucasian (1). Most of the organizations (64) were community-based not-for-profit organizations, five were led by university project directors who had partnered with community-based organizations for service delivery, one was a university treatment center, seven were state/county governmental units partnered with community-based organizations for service delivery, and one was a medical/health center. The intervention was delivered in different settings as follows: outpatient clinics (36), school-based (5), home-based (15), or a mixture of settings (22), with the latter primarily a combination of outpatient and home-based service delivery.

2.2. Participants

A sample of 5,150 participants from the 78 sites comprised the sample for the intake characteristic and treatment process measures analyses. To maximize the sample size for the analyses of the treatment engagement, retention, and satisfaction measures, we included participants who had been in treatment long enough to allow calculation of values for each measure. For example, based on the definitions of these measures provided in Section 2.3.3, participants would need to have been in treatment at least 14 days to have a value for initiation, 44 days to have a value for engagement. Consequently, the sample sizes for the treatment engagement, retention, and satisfaction variables ranged from 4,636 to 5,102.

Sites in this initiative were funded for 3 years and had staggered entry into the project from 2006 to 2010. Thus, participants who had not yet reached their 12-month follow-up due date were excluded from the outcome analysis, reducing the number of participants by 2,328 (45.2%). In addition, only sites achieving N 50% completed participant follow-up at 12 months were included, thereby decreasing the outcome analysis sample by an additional 338 (6.6%) participants. These exclusions resulted in an outcome analysis sample of 70 treatment sites and 2,484 participants with follow-up rates for the sample of 89.6% at 3 months, 85.3% at 6 months, and 79% at 12 months.

2.3. Measures

2.3.1. Co-occurring problem categories

Participants were classified into four mutually exclusive groups based on their endorsement of DSM-IV-R symptoms in the Global Appraisal of Individual Needs intake interview (GAIN; Dennis, Titus, White, Unsicker, & Hodgkins, 2003). Prior research has demonstrated that scores on the GAIN Internalizing and Externalizing scales have high agreement with independent clinical diagnoses based on other established diagnostic scales, clinical reports, and tests of construct/predictive validity (Chan et al., 2008; Conrad et al., 2010, 2012; Rush, Castel, Brands, Toneatto, & Veldhuizen, 2013; Rush, Dennis, Scott, Castel, & Funk, 2008; Shane et al., 2003; Subramaniam, Ives, Stitzer, & Dennis, 2010; Womack et al., 2004). The “non-comorbid” group included those participants who did not endorse symptoms that met criteria for any psychiatric problem. The “internalizing” group included participants endorsing DSM-IV symptoms used to indicate generalized anxiety, major depression, or traumatic stress. The “externalizing” group encompassed those endorsing symptoms used to indicate conduct disorder or attention deficit hyperactivity disorder (ADHD). The “mixed” group included participants endorsing symptoms that related to criteria for at least one externalizing and one internalizing problem.

2.3.2. Presenting Problem Severity Measures

Measures were collected during the GAIN intake interview. “Criminal justice involvement” is a dichotomous yes/no item indicating the adolescent's reported current involvement in the legal system for any criminal activity. “Single-parent custody” is a yes/no item based on whether an adolescent is in the legal custody of a single parent. Primary substance of referral was calculated based on the number of symptoms of abuse and dependence endorsed for each substance and weighted by the recency of use. In cases where two substances were tied on the number of symptoms of abuse and dependence, the substance with the most days of use out of the past 90 days was considered the primary substance of referral. The HIV Risk Scale represents the degree to which an individual is exposed to situations or engages in behaviors that facilitate the transmission of HIV. The scale ranges from 0 to 36, and the score is the average of items from its three subscales: Needle Problem Scale (Cronbach's alpha = .91; 9 items), Sex Risk Scale (Cronbach's alpha = .62; 12 items), and General Victimization Scale (Cronbach's alpha = .84; 15 items). The HIV risk scale has been validated using the RASCH measurement model (Conrad, Conrad, Dennis, Riley, & Funk, 2009) and has demonstrated good reliability with Cronbach's alpha = .83 (36 items). “Victimized in the past 90 days” is a dichotomous (yes/no) measure based on an endorsement by the adolescent of any kind of victimization in the last 90 days in answer to the question “When was the last time, if ever, you were attacked with a weapon, beaten, sexually abused, or emotionally abused?”

2.3.3. Treatment engagement, retention, and satisfaction measures

A-CRA has a planned duration between 12 to 14 weeks. Treatment measures were obtained from A-CRA session data that clinicians entered into a secure online database. Treatment initiation and engagement were calculated based on the definitions endorsed by the Washington Circle group (Garnick, Lee, Horgan, Acevedo, & the Washington Circle Public Sector Workgroup, 2009). Treatment participants met the criteria for initiation if they had received a second treatment session within 14 days of their first treatment session, and they met the criteria for engagement if they received 2 additional treatment sessions within 30 days of the initiation date. We also examined four session dosage measures: sessions with the adolescent only, with the caregiver only, family sessions (i.e., the adolescent and one or more caregivers), and total number of sessions. Treatment dosage measures were calculated based on treatment received during the 14 weeks after beginning treatment, which is the prescribed A-CRA treatment duration. Satisfaction with treatment services was assessed 3 months after treatment admission with the Treatment Satisfaction Scale (TxSS; Cronbach's alpha = .87) from the GAIN. The TxSS is a measure of general satisfaction with services and staff during treatment. The scale ranges from 0 to 14, with higher values indicating greater satisfaction with treatment services and staff.

2.3.4. Treatment outcome measures

Three measures from the GAIN intake and follow-up interviews (0, 3, 6, and 12 months) were used for outcome analysis (and are different measures than the ones used for group classification). The Emotional Problem Scale (EPS) is calculated as the proportional average of items measuring the recency (e.g., 1–3 months ago, 1– 4 weeks ago, 3–7 days ago, past 2 days) and number of days (during the past 90) of: a) being bothered by or kept from responsibilities because of emotional problems, b) being disturbed by memories, and c) having problems paying attention or with self-control (Cronbach's alpha = .79; 7 items; ranges from 0 to 1). Cut points for severity have been empirically derived to aid clinical interpretation of this scale: low = 0 to .13 (less than weekly problems); moderate = .14 to .50 (weekly problems); and high = .51 to 1.00 (daily problems). Scores greater than .13 indicate a degree of severity that warrants consideration in treatment planning (GAIN Coordinating Center, 2011). The Substance Problems Scale (SPS) is a count of past month symptoms of substance abuse, dependence, and substance-induced health and psychological disorders based on DSM-IV criteria (Cronbach's alpha = .90; 16 items; ranges from 0 to 16). Percent of days abstinent is a self-report of the proportion of days not using any alcohol or other drugs while in the community out of the past 90 days. Adolescent self-report measures of substance use from the GAIN have been shown to be consistent with collateral reports (kappa = .69–.92 and agreement = 90%–98%) and urine testing (kappa = .75–.90 and agreement = 88%–95%; Godley, Godley, Dennis, Funk, & Passetti, 2002).

2.4. Analytic plan

For all analyses, scores of each co-occurring problem group were compared to those of the non-comorbid group, which was used as a reference group. All analyses were conducted using HLM version 7.0 (Raudenbush, Bryk, & Congdon, 2011), which accommodates multilevel data by allowing the nesting of participants within agencies and partitioning the variance into, between and within agency variance. This addresses the problem of the violation of the assumption of independence of observations inherent in nested data. The current analysis focuses on explaining the variance within agency, or individual differences. First, multinomial HLM analyses were computed to compare intake characteristics of each comorbid group to those of the non-comorbid group. Missing data by item for intake measures were minimal (0–4%), and participants with missing data on particular items were dropped from the corresponding analysis. Due to the small rates of missing data and the very large sample, missing data were not replaced.

Second, to determine whether the different co-occurring problem groups received equivalent amounts of treatment during the A-CRA episode, as compared to the non-comorbid group, equivalence testing was carried out for the engagement and retention measures based on methods described in Rogers, Howard, and Vessey (1993). For dichotomous outcomes, δ₁ and δ₂ (δ is the minimum acceptable difference for equivalence in both a positive and negative direction) were calculated as 20% of the control group proportion. For continuous outcomes, δ₁ and δ₂ were calculated as .20 SD. These calculations were followed by two one-tailed z-tests that were used to determine significance of equivalence. There was no missing data for engagement or retention; however, the Treatment Satisfaction Scale was only completed by 77% of the eligible sample.

Third, participant outcomes were analyzed using a 2-level hierarchical multivariate linear model (HMLM2) with maximum likelihood estimation, where observations were nested within participants and participants within treatment sites, thus modeling participant trajectories of change across time while controlling for the clustering of the data within site. This allowed for the replacement of missing data for the 3- and 6-month follow-up for longitudinal analyses, while participants missing the 12-month follow-up were not included in the outcome analysis (n = 522). HLM version 7.0 (Raudenbush et al., 2011) was used for these analyses. While maximum likelihood estimation will replace entire missing waves of data in HLM, it will not replace individual missing items within a wave of data. Thus, missing items were replaced using the RMV function in SPSS, where the cases were sorted by observation wave, treatment agency, gender, race, and age, and the median of the four closest values was used to replace the missing value. Rates of missing data ranged from 0%–5%. The final analytic sample for the outcome analyses was composed of 1,962 participants.

Multilevel logistic regression analyses with participants nested within agencies were conducted to investigate whether there were any discrepancies between the retained outcome sample and the participants who were dropped from the outcome analyses either because they were treated at an agency with a 12-month follow-up rate less than 50% or because they did not complete a scheduled 12-month follow-up interview. To investigate the existence of discrepancies, a series of analyses were conducted using all of the intake characteristics included in Table 1 to predict attrition. Participants that were excluded from the outcome analyses for either of the aforementioned reasons differed from the retained sample only in regard to age; they were 3 months older on average (16.1 vs. 15.8 years old) than participants who were followed-up at 12 months (odds ratio = 1.09, 95% C. I. = 1.02, 1.17).

Table 1.

Results of HLM multinomial logistic regression analyses of treatment intake characteristics (N = 5,150).

Intake characteristic	Non-comorbid (n = 1,843)	Internalizing (n = 404)			Externalizing (n = 1,166)			Mixed (n = 1,737)
	M (SD)/N(%)	M (SD)/N(%)	OR	p	M (SD)/N(%)	OR	p	M (SD)/N(%)	OR	p
Co-occurring disorders
Major depressive disorder	0.0	290 (71.8%)	-	-	0.0	-	-	1457 (83.9%)	-	-
Generalized anxiety disorder	0.0	53 (13.1%)	-	-	0.0	-	-	542 (31.2%)	-	-
Traumatic stress disorder	0.0	185 (45.8%)	-	-	0.0	-	-	1059 (61.0%)	-	-
Conduct disorder	0.0	0.0	-	-	913 (78.3%)	-	-	1478 (85.1%)	-	-
ADHD	0.0	0.0	-	-	656 (56.5%)	-	-	1318 (76.2%)	-	-
Female	296 (16.1%)	165 (40.8%)	3.63	<0.001	191 (16.4%)	0.99	0.893	686 (39.5%)	3.31	<0.001
Single parent custody	845 (48.1%)	171 (44.1%)	0.86	0.042	562 (49.6%)	1.08	0.210	765 (46.0%)	0.93	0.216
Criminal justice involved	1255 (68.2%)	221 (54.7%)	0.51	<0.001	782 (67.1%)	0.91	0.297	1090 (62.8%)	0.71	<0.001
Primary substance of referral
Alcohol	254 (13.8%)	60 (14.9%)	1.07	0.639	153 (13.1%)	0.98	0.818	262 (15.1%)	1.24	0.030
Marijuana	1052 (57.1%)	187 (46.3%)	0.79	0.038	689 (59.1%)	1.21	0.023	861 (49.6%)	0.83	0.016
Amphetamine	219 (11.9%)	49 (12.1%)	0.96	0.795	103 (8.8%)	0.65	0.001	125 (7.2%)	0.54	<0.001
All others	318 (17.3%)	108 (26.7%)	1.85	0.001	221 (19.0%)	1.07	0.678	480 (28.2%)	2.16	<0.001
Age	15.9 (1.8)	16.0 (1.4)	1.09	0.061	15.6 (1.4)	0.88	<0.001	15.9 (1.3)	1.01	0.602
Race
Native Alaskan/American	69 (3.7%)	20 (5.0%)	1.29	0.400	31 (2.7%)	0.65	0.093	57 (3.3%)	0.84	0.528
Asian	17 (0.9%)	7 (1.7%)	1.40	0.469	17 (1.5%)	1.33	0.436	19(1.1%)	0.81	0.561
African American	381 (20.7%)	58 (14.4%)	0.71	0.035	201 (17.2%)	0.86	0.141	177 (10.2%)	0.53	<0.001
Hispanic	692 (37.6%)	122 (30.2%)	0.74	0.026	341 (29.2%)	0.75	0.003	489 (28.2%)	0.74	0.003
Mixed	201 (10.9%)	67 (16.6%)	1.48	0.012	172 (14.8%)	1.30	0.024	287 (16.6%)	1.40	0.001
White	469 (25.5%)	124 (30.7%)	1.18	0.204	401 (34.4%)	1.32	0.002	684 (39.5%)	1.53	<0.001
Other	12 (0.7%)	6 (1.5%)	1.92	0.200	3 (0.3%)	0.35	0.109	18 (1.0%)	1.30	0.504
% of days abstinent from AOD	71.9% (.32)	66.0% (.34)	0.59	0.002	59.0% (.36)	0.33	<0.001	51.4% (.37)	0.19	<0.001
Substance problem scale	1.43 (2.15)	2.56 (3.29)	1.17	<0.001	2.70 (3.11)	1.18	<0.001	4.22 (4.41)	1.31	<0.001
Emotional problem scale	0.10 (.10)	0.24 (.16)	9.91	<0.001	0.25 (.13)	11.58	<0.001	0.41 (.19)	34.63	<0.001
Victimized in past 90	143 (7.8%)	85 (21.1%)	3.12	<0.001	202 (17.4%)	2.40	<0.001	518 (30.0%)	4.88	<0.001
HIV risk scale	3.20 (2.93)	5.73 (3.90)	1.24	<0.001	4.79 (3.37)	1.16	<0.001	7.82 (4.44)	1.41	<0.001

Notes. OR = odds ratio; AOD = alcohol and other drugs. Missing data ranged from 0% to 4% across intake characteristics creating slight variation in sample sizes by intake characteristic.

3. Results

3.1. Presenting problem severity

Results for all HLM multinomial regression analyses with intake characteristics predicting group membership for participants with a COP relative to those with an SUD only are presented in Table 1. Participants who were female, had ‘other drugs’ as their primary drug of choice, were of mixed race, had more substance use and emotional problems, had more victimization in the past 90 days, and had higher HIV risk were more likely to have an internalizing problem instead of an SUD only. Participants who came from a single parent household, were involved in the criminal justice system, had marijuana as their primary drug of choice, were African American or Hispanic, and had more days of abstinence from substance use were less likely to have an internalizing problem instead of an SUD only. Participants who were of mixed race or Caucasian, had marijuana as their primary drug of choice, had more substance use and emotional problems, had more victimization in the past 90 days, and had higher HIV risk were more likely to have an externalizing problem instead of an SUD only. Participants who were older, Hispanic, had amphetamines as their primary drug of choice, and had more days of abstinence from substance use were less likely to have an externalizing problem instead of an SUD only. Female adolescents, those who had alcohol or ‘other drugs’ as their primary drug of choice, were of mixed race or Caucasian, had more substance use and emotional problems, had more victimization in the past 90 days, and had higher HIV risk were more likely to have both internalizing and externalizing problems instead of an SUD only. Participants who were involved with the criminal justice system, had marijuana or amphetamines as their primary drug of choice, were African American or Hispanic, and had more days of abstinence from substance use were less likely to have both internalizing and externalizing problems instead of an SUD only.

3.2. Treatment measures

Table 2 presents the results from the equivalence tests for the treatment engagement, retention, and satisfaction measures. When compared to the non-comorbid group, each of the co-occurring problem groups had significantly equivalent levels of treatment during their A-CRA treatment episode across all treatment measures (i.e., initiation, engagement, participant only sessions, caregiver only sessions, family sessions, total sessions, and TxSS). All effect sizes were less than .09, where an effect size of .20 is considered the minimum for a small effect.

Table 2.

Results of equivalence testing for treatment engagement, retention, and satisfaction measures.

Measure	Mean/%	SD/σ2	N	Mean/%	SD/σ2	N	SE	z1	z2	d
	Non-comorbid			Internalizing
Initiation	77%	0.2	1824	73%	0.2	398	0.02	−7.68	4.89	−0.08
Engagement	60%	0.3	1801	56%	0.3	395	0.03	−5.67	2.99	−0.07
Participant treatment sessions	6.9	3.4	1706	7.2	4.0	371	0.18	−1.80	4.88	0.09
Caregiver treatment sessions	0.7	1.0	1706	0.7	1.0	371	0.05	−2.74	4.15	0.04
Family treatment sessions	0.8	1.1	1706	0.7	1.2	371	0.06	−2.96	3.92	0.03
Total treatment sessions	8.3	4.1	1706	8.6	4.7	371	0.21	−1.75	4.99	0.09
Treatment Satisfaction Scale	13.4	1.7	1269	13.3	1.7	291	0.06	−5.50	4.56	−0.02
	Non-comorbid			Externalizing
Initiation	77%	0.2	1824	75%	0.2	1155	0.02	−10.40	8.54	−0.04
Engagement	60%	0.3	1801	57%	0.3	1146	0.02	−7.63	5.16	−0.05
Participant treatment sessions	6.9	3.4	1706	6.8	3.4	1088	0.12	−6.19	3.98	−0.04
Caregiver treatment sessions	0.7	1.0	1706	0.7	0.1	1088	0.03	−5.50	4.64	−0.02
Family treatment sessions	0.8	1.1	1706	0.8	1.2	1088	0.04	−2.92	7.06	0.08
Total treatment sessions	8.3	4.1	1706	8.3	4.2	1088	0.14	−5.54	4.62	−0.02
Treatment Satisfaction Scale	13.4	1.7	1269	13.3	1.8	834	0.07	−5.85	2.96	−0.06
	Non-comorbid			Mixed
Initiation	77%	0.2	1824	74%	0.2	1725	0.01	−12.28	8.83	−0.06
Engagement	60%	0.3	1801	57%	0.3	1719	0.02	−8.85	5.49	−0.06
Participant treatment sessions	6.9	3.4	1706	6.9	3.7	1636	0.11	−5.00	5.85	0.01
Caregiver treatment sessions	0.7	1.0	1706	0.7	1.0	1636	0.03	−5.69	5.77	0.00
Family treatment sessions	0.8	1.1	1706	0.8	1.1	1636	0.03	−5.15	6.28	0.02
Total treatment sessions	8.3	4.1	1706	8.3	4.5	1636	0.13	−4.91	5.95	0.02
Treatment Satisfaction Scale	13.4	1.7	1269	13.3	1.7	1264	0.06	−6.12	3.94	−0.04

Notes. Effect sizes can be interpreted as a Cohen's d, where a small effect ≥ .20; medium effect ≥ .50; and large effect ≥ .80.

Given that treatment received during the A-CRA outpatient treatment episode was equivalent across groups, further analyses were conducted using GAIN self-reported treatment services data at 6- and 12-month follow-up interviews to assess if there were differences in treatment received during the post A-CRA follow-up period. These included days of substance use treatment (the sum of the days the adolescent reported receiving any substance use treatment), and days of mental health treatment (the sum of the days the adolescent reported receiving any mental health treatment services). Additional equivalence tests revealed that there were no differences in mental health treatment services for any of the COP groups as compared to the SUD only group at either follow-up wave; however, participants with internalizing problems did not receive an equivalent number of days of treatment between months 10–12 compared to the SUD only group. On average, internalizers received .77 days less substance use treatment than the SUD only group (z₁ = − 2.61; z₂ = 1.20; d = − .08).

3.3. Treatment outcomes

Results of the HMLM2 growth models are presented in Table 3. All co-occurring groups had significantly smaller intercepts than the non-comorbid group for percent of days abstinent from alcohol and other drugs. In other words, participants with a co-occurring problem had a significantly higher percent of days using alcohol or any drug at intake. The externalizing and mixed groups had significantly larger slopes than the non-comorbid group, meaning that the externalizing and mixed groups had significantly larger rates of increase in percent of days abstinent from alcohol and other drugs as compared to the non-comorbid group. Fig. 1 depicts growth curves for percent of days abstinent from alcohol and other drugs by group. Effect sizes were calculated to quantify the magnitude of these differences. The effect size for externalizing compared to non-comorbid was .42 (approaching a medium effect), and the effect size for mixed compared to non-comorbid was .48 (approaching a medium effect). The internalizing group was not significantly different from the non-comorbid group with regard to the rate of change in percent of days abstinent from alcohol and other drugs.

Table 3.

Results of HLM growth model analyses for outcome measures (N = 1,962).

	Coefficient	SE	t-ratio	p
Percent of days abstinent from AOD
Intercept (non-comorbid)	0.74	0.02	47.44	<0.001
Internalizing	−0.07	0.03	−2.51	0.013
Externalizing	−0.10	0.02	−5.80	<0.001
Mixed	−0.16	0.02	−9.93	<0.001
Slope (non-comorbid)	0.01	0.00	7.47	<0.001
Internalizing	0.00	0.00	−0.05	0.964
Externalizing	0.01	0.00	3.22	0.002
Mixed	0.01	0.00	3.99	<0.001
Substance Problems Scale
Intercept (non-comorbid)	1.52	0.13	11.45	<0.001
Internalizing	0.88	0.25	3.52	<0.001
Externalizing	0.79	0.17	4.70	<0.001
Mixed	2.19	0.15	14.14	<0.001
Slope (non-comorbid)	−0.06	0.01	−5.04	<0.001
Internalizing	−0.03	0.03	−1.05	0.296
Externalizing	−0.04	0.02	−2.13	0.033
Mixed	−0.12	0.02	−6.76	<0.001
Emotional Problems Scale
Intercept (non-comorbid)	0.11	0.01	15.92	<0.001
Internalizing	0.13	0.01	10.99	<0.001
Externalizing	0.12	0.01	14.59	<0.001
Mixed	0.25	0.01	34.21	<0.001
Slope (non-comorbid)	0.00	0.00	0.07	0.942
Internalizing	0.00	0.00	−3.34	0.001
Externalizing	−0.01	0.00	−6.85	<0.001
Mixed	−0.01	0.00	−14.70	<0.001

Notes. Non-comorbid n = 666; internalizing n = 154; externalizing n = 468; mixed n = 674.

Fig. 1.

Results of HLM growth model analyzing group differences percent of days abstinent.

All co-occurring problem groups had significantly larger intercepts than the non-comorbid group for the Substance Problem Scale, meaning that all co-occurring problem groups had a significantly higher number of substance problems at intake. The externalizing and mixed groups had significantly larger slopes than the non-comorbid group, meaning that these groups had significantly larger rates of decrease in substance problems as compared to the non-comorbid group. Fig. 2 depicts the growth curves for the Substance Problem Scale by group. The effect size for externalizing compared to noncomorbid was .25 (small effect), and the effect size for mixed compared to non-comorbid was .72 (large effect). The internalizing group was not significantly different from the non-comorbid group with regard to their rate of change in substance problems.

Fig. 2.

Results of HLM growth model analyzing group differences for Substance Problem Scale.

All co-occurring problem groups had significantly larger intercepts than the non-comorbid group for the Emotional Problems Scale, meaning that these groups had a significantly higher severity of emotional problems at intake to treatment. The COP groups had significantly larger slopes than the non-comorbid group, meaning that these groups had larger rates of decrease in emotional problems as compared to the non-comorbid group. Fig. 3 depicts the growth curves for the Emotional Problem Scale by group. The effect size for internalizing compared to non-comorbid was .66 (medium effect), the effect size for externalizing compared to non-comorbid was .91 (large effect), and the effect size for mixed compared to non-comorbid was 1.77 (large effect).

Fig. 3.

Results of HLM growth models analyzing group differences for the Emotional Problems Scale.

4. Discussion

4.1. Reprise

The need for effective interventions for adolescents with COP has been called for in numerous published studies (Bukstein et al., 2005; Institute of Medicine (IOM), 2006; McGovern, Matzkin, & Giard, 2007; Substance Abuse and Mental Health Services Administration (SAMHSA) & Center for Substance Abuse Treatment (CSAT), 2005). While the secondary analyses reported here do not constitute an efficacy or effectiveness study, the findings nonetheless add to the knowledge base about the dissemination of an empirically supported treatment (A-CRA) to a large and diverse sample of youth with substance use disorders, the majority of whom (65%) had co-occurring problems as indicated by endorsement of DSM-IV symptoms.

Many differences in intake characteristics were observed between the COP groups and participants with an SUD only. Winters et al. (2008) reported that female adolescents were overrepresented in the internalizing group and male adolescents were overrepresented in the externalizing group, while Rowe, Liddle, and Dakof (2001) reported that female adolescents were overrepresented in the externalizing group and that male adolescents were overrepresented in the SUD only group. We found that being female increased the likelihood of having an internalizing problem or mixed problems as opposed to an SUD only. Male adolescents were overrepresented in all groups, but given the large sample size and the tendency of male adolescents to be overrepresented in substance use treatment, this finding is not surprising. In their sample, Rowe et al. (2001) found no race differences in their comorbid groups. Results of the current study indicate that adolescents of mixed race were more likely to have a co-occurring problem, and Caucasian adolescents were more likely to have an externalizing problem or both an internalizing and externalizing problem. Perhaps the large and diverse sample represented in this study allowed for the identification of racial differences among COP groups that were not possible with smaller samples. Similar to Rowe et al. (2001), we found that adolescents reporting symptoms of an externalizing problem were slightly younger on average than adolescents with an SUD only.

As hypothesized, our results suggest that participants with COPs enter treatment with more severity; that is, reporting more days of substance use and more substance use problems at treatment intake than non-comorbid adolescents. These results are similar to those reported by other researchers (Diamond et al., 2006; Grella et al., 2001; Tomlinson et al., 2004) who found that adolescents with a COP had significantly more days of marijuana use and more substance use problems at intake, more severe pre-treatment substance use and illegal activity, and more pre-treatment alcohol and stimulant use than non-comorbid youth.

Also as hypothesized, we found that adolescents with and without co-occurring problems had equivalent rates of initiation, engagement, and treatment retention during their A-CRA treatment episode. In contrast, several previous studies have found that adolescents with comorbid problems had more difficulty engaging in treatment and were more likely to terminate prematurely than non-comorbid youth (Dobkin et al., 1998; Kaminer et al., 1992). Our findings are consistent with those reported by Rowe et al. (2004), who also compared retention rates for youth with and without co-occurring problems who were participating in ESTs (multi-dimensional family therapy, and cognitive behavior therapy), and suggests that the implementation of EST may be related to improved retention rates for adolescents with co-morbid disorders.

With respect to our final hypothesis, there were differences in the trajectories of treatment outcome for the various groups. Adolescents with a co-occurring problem had an equivalent or greater rate of increase in abstinence, and greater rates of decrease in substance use and emotional problems, as compared to non-comorbid adolescents. While the internalizing group was not significantly different from the non-comorbid group in terms of the change in percent of days abstinent from alcohol and other drugs, the externalizing and mixed groups had significantly greater increases in days of abstinence. The same pattern of improvement was observed for substance use problems; the externalizing and mixed groups had significantly greater decreases in substance problems compared to the non-comorbid group, while the internalizing group was not significantly different from the non-comorbid adolescents. All COP groups had significantly greater decreases in emotional problems compared to the non-comorbid group. Even though each of the co-occurring groups consistently improved over the 12-month follow-up, the co-occurring groups each had an average score on the Emotional Problems Scale that indicated some persisting emotional problems.

While the co-occurring groups were more symptomatic of substance use and emotional problems at the 12-month follow-up than the non-comorbid group, these results should be considered in context. The externalizing and mixed groups had an overall magnitude of substance-related improvement that was greater than that of the non-comorbid group, thereby shrinking the gap in severity between non-comorbid adolescents and adolescents with these COPs. This finding is similar to Grella et al.'s (2001) from a study of 23 adolescent treatment programs. They reported that youth in a comorbid group had greater improvements than the non-comorbid youth in a number of problem behaviors, and yet the COP group still had significantly poorer 12-month outcomes than the non-comorbid youth for several substance use variables and for illegal activity. Substantial treatment progress also was found for the comorbid adolescent group in a study by Tomlinson et al. (2004), and yet their high relapse rate during follow-up exceeded that of the non-comorbid group. We also found that adolescents with co-occurring problems improved on all outcomes at a faster rate than non-comorbid adolescents; however, this finding contrasts with some previously published studies (Rohde, Clarke, Lewinsohn, Seeley, & Kaufman, 2001; Rowe et al., 2004).

Several studies have proposed that youth with externalizing disorders tend to prematurely leave treatment, have smaller improvement in substance use, and have faster post-treatment relapses compared to other COP groups (Crowley, Mikulich, MacDonald, Young, & Zerbe, 1998; Kaminer et al., 1992; Randall et al., 1999; Tomlinson et al., 2004; Winters et al., 2008). Additionally, adolescents with internalizing disorders sometimes have been shown to be less likely to relapse (Tomlinson et al., 2004), and the presence of an internalizing disorder for adolescents who also have an externalizing disorder (mixed condition) has been hypothesized to have a protective effect against the worse outcomes often associated with externalizing disorders (Randall et al., 1999; Tomlinson et al., 2004). In the current study, however, the externalizing group achieved at least as favorable an outcome (or better) for all three areas (substance use, substance-related problems, emotional problems) when compared to the internalizing and mixed groups, and had larger improvements overall when compared to the internalizing group. The externalizing group also had the highest rate of abstinence, lowest number of substance-related problems, and the lowest number of emotional problems at 12 months post-intake when compared to the internalizing and mixed groups.

Taken together, these results are encouraging for adolescents with externalizing problems because it demonstrates that they can achieve significant improvement in their substance use and related problems, and maintain these improvements out to 12 months post-intake. In contrast to the Randall et al. (1999) and Tomlinson et al. (2004) studies, we did not find evidence of a protective effect of internalizing problems for adolescents who also had externalizing problems, which is consistent with findings reported by Shane et al. (2003).

Although the mixed group in the current study had more substance-specific symptoms and mental health problems at 12 months compared to all other groups, the mixed group had the largest overall decreases in the three main outcome measures. These findings mirror the pattern reported by Shane et al. (2003), which showed that the mixed group had substantial reductions in substance use/problems up to the 6-month follow-up, but still had higher levels of certain drug use and more substance-related problems than the other three conditions (externalizing or internalizing, or non-comorbid) at 12 months. Furthermore, the mixed group regressed toward their intake levels of substance use from 6 to 12 months post-intake. Similarly, Rowe et al. (2004) found that a mixed group showed reductions in substance use through the 6-month follow-up, but then returned to baseline levels by 12 months. In contrast, the mixed group in the current study had a significant increase in abstinence, which peaked at 3 months post-intake and was maintained out to 12 months.

4.2. Strengths and limitations

A strength of the current study is the inclusion of a large, diverse sample of adolescents from SUD treatment organizations located across the U.S., thereby making the results relatively generalizable. The sample also allowed for the examination of a separate internalizing group, which has infrequently been possible in other studies (e.g., Rowe et al., 2004; Shane et al., 2003). Additionally, not only was this one of the few studies based on participation in empirically supported SUD treatment, but the participants were part of a large-scale dissemination and implementation effort of a single EST that included a rigorous training and fidelity monitoring process (Godley, Garner, Smith, Meyers, & Godley, 2011). Another strength of this study was the use of multi-level statistical analyses designed to decrease the risk of a type I error by controlling for the nested data within treatment organization.

In addition to these strengths, there were also limitations to this study. First, five treatment agencies were dropped from the analysis for having 12-month follow-up rates less than 50%, and an additional 10% of the sample was dropped due to uncompleted 12-month follow-up interviews. Dropping these agencies and participants could create a selection bias; however, attrition analyses conducted to investigate the effect of these exclusions indicated that the participants dropped for either of these reasons were not significantly different from the retained sample on any intake characteristic except for age. A second potential limitation is that the study design did not include corroboration of self-reported substance use through bio-marker or collateral reports. Nevertheless, adolescent self-report measures of substance use from the GAIN have been shown to be consistent with collateral reports and urine tests (Godley et al., 2002). Third, the results of this study may not be generalizable to adolescents receiving substance use treatment other than A-CRA. Finally, separate measures of specific symptoms of each psychiatric disorder at follow-up would have been preferable to a single measure of emotional problems, and yet time and participant burden constraints prevented them from being included in this project.

4.3. Future research and conclusions

Results of the current study indicate that adolescents with comorbid SUD and psychiatric problems presented to treatment with more severe substance use and emotional problems, but received equivalent treatment exposure during the A-CRA treatment episode compared to non-comorbid adolescents. Also, adolescents with co-occurring problems attained considerable improvement in their substance use and emotional problems, often at a higher rate than did those without a co-occurring problem; however, adolescents with a co-occurring problem did not achieve the same absolute level of improvement as did those without a co-occurring problem. Future research might determine whether there is a need to augment the procedures currently comprising the A-CRA treatment model in order to directly target specific comorbid symptoms. Nonetheless, the adolescents in the current study did experience notable improvements in the primary outcomes of interest, suggesting that A-CRA offers promise as a treatment option for adolescents with comorbid SUD and psychiatric disorders, and thus warrants further study for this population in randomized clinical trials.