Feasibility and impact of implementing motivational enhancement therapy-cognitive behavioral therapy as a substance use treatment intervention in school-based settings

Abstract

The expansion of behavioral health services to school-based health centers under the Affordable Care Act (Public Law 111–148) presents an opportunity to improve access to substance use disorders treatment for youth and reduce their substance use, and emotional, health, and school problems. We explore the feasibility of implementing five to seven sessions of motivational enhancement therapy–cognitive behavioral therapy (MET/CBT) in school settings relative to a matched cohort in community settings. Results indicate that MET/CBT in school settings is feasible, effective, and cost-effective. Moreover, it reaches youth earlier after the onset of substance use and has the possibility to reduce existing health disparities for girls and ethnic minorities.

1. Introduction

Only 1 in 19 adolescents (aged 12–17 years) with past-year substance use disorders (SUD) receive treatment; of those who do, over half leave treatment within six weeks and over half relapse within 90 days (Dennis, Clark, & Huang, 2014). While SUD is associated with a higher rate of dropout, bad grades, emotional problems, family problems, HIV-risk behavior, high health-care costs, fighting, and arrest – over 95% of adolescents with SUD are still attending school (Dennis et al., 2014). Thus, the unmet need for SUD treatment is both a public health and school problem. The expansion of behavioral health services to school-based health centers (SBHCs) under the Affordable Care Act presents an opportunity to improve access to SUD treatment for youth and reduce their substance use, and emotional, health, and school problems.

Conducted from 1997 to 2001, the Cannabis Youth Treatment experiments were one of the first large-scale (n = 600), long-term (12- and 30-month follow-up) comparative effectiveness studies of five evidence-based treatment protocols specifically targeting short-term (5–12 week) adolescent outpatient treatment (Dennis et al., 2002; Godley, Dennis, Godley, & Funk, 2004). While the interventions were similar in their clinical effectiveness, motivational enhancement therapy–cognitive behavioral therapy (MET/CBT) (Sampl & Kadden, 2001) was one of the two most cost-effective (Dennis et al., 2004) and the most beneficial in terms of reducing costs to society (French et al., 2003). Between 2003 and 2008, SAMHSA’s Center for Substance Abuse Treatment funded the Effective Adolescent Treatment (EAT) program to train clinicians in 37 SUD treatment sites across 21 states in the Midwest, Southwest, Southeast, Eastern, and Pacific regions of the USA on how to implement MET/CBT with the same level of quality and evaluate whether they could achieve similar outcomes using the same standardized assessment tool, the Global Appraisal of Individual Needs (GAIN) (Dennis, Titus, White, Unsicker, & Hodgkins, 2003). EAT demonstrated that given adequate training (e.g. coaching and feedback for interviewers to identify/address client-reported inconsistencies, live-time validity checks from the GAIN Assessment Building System [GAIN ABS] while conducting interviews, summary validity reports created by GAIN ABS) and resources and technical assistance, community-based programs could not only replicate the quality of implementation, but they could also actually achieve similar or better outcomes (Hunter et al., 2012). Other experimental and quasi-experimental replications of MET/CBT have confirmed its effectiveness (Godley, Jones, Funk, Ives, & Passetti, 2004; Hunter et al., 2012; Mason & Posner, 2009; Ramchand, Griffin, Suttorp, Harris, & Morral, 2011; Riley, Rieckmann, & McCarty, 2008; Stanger, Budney, Kamon, & Thostensen, 2009) and cost-effectiveness (Olmstead, Sindelar, Easton, & Carroll, 2007). In the most recent meta-analyses (Lipsey, Tanner-Smith, & Wilson, 2010; Tanner-Smith, Wilson, & Lipsey, 2013), MET/CBT5 was associated with greater reductions in substance use than no treatment or treatment as usual, and with equal or better reductions than most other evidence-based practices (EBPs).

With several years of robust evidence of MET/CBT5’s effectiveness and cost-effectiveness, and with a new momentum for expanding behavioral health care to SBHCs, we wanted to use existing data to compare treatment for SUDs using an EBP in school-based (SB) settings relative to traditional outpatient/community-based (CB) settings. Greater likelihood of mental/behavioral health service utilization in school over community settings has already been demonstrated (Anglin, Naylor, & Kaplan, 1996; Armbuster, Gerstein, & Fallon, 1997; Juszczak, Melinkovich, & Kaplan, 2003; Kaplan, Calonge, Guernsey, & Hanrahan, 1998; Kaplan et al., 1999). This means greater potential for reductions in substance use, and reduced substance use in turn is correlated with both: (1) reduced absenteeism, dropout, and suspension rates, as well as increased academic performance and educational attainment (Bradley & Greene, 2013; Chatterji, 2006a, 2006b; Coker et al., 2012; McCord, Klein, Foy, & Fothergil, 1993; Walker, Kerns, Lyon, Bruns, & Cosgrive, 2010) and (2) a range of clinical outcomes like reduced mental health problems, health problems, HIV-risk behaviors, etc. (Dennis et al., 2004; French et al., 2003). Furthermore, recent research indicating motivational interviewing is efficacious with middle- and high-school students’ cognitive and neurodevelopmental stages. Strait, McQuillin, and Englund (2012) provide further justification for validating an EBT-like MET/CBT in school settings.

Studies comparing the two settings, however, are limited to examining mental health interventions and outcomes pre–post instead of behavioral health/substance use outcomes (Armbuster & Lichtman, 1999; Weist, Myers, Hastings, Ghuman, & Han, 1999; Wong, 2005). Another limitation in these same studies is that EBT interventions such as MET/ CBT have not been used. The only study that used an EBT intervention used motivational interviewing, but only examined its use in a single group pre–post setting (Winters & Leitten, 2007). Thus, there is a need to fill a current gap in the literature with respect to using MET/CBT in an SB versus CB comparison condition. The importance of conducting such a study lies in the fact that schools, more so than community settings, may be the more practical setting in which to target youth with SUDs – and by doing so, opening the possibility of intervening early with greater proportions of youth. The present study addresses methodological and substantive gaps by comparing SUD treatment in school versus community settings; using an evidence-based intervention (MET/CBT) and an evidence-based assessment (GAIN); and adjusting for group differences at baseline using propensity score techniques to create a more rigorous quasi-experimental study. We examine differences in characteristics, service implementation, outcomes, and cost-effectiveness between school and community treatment settings.

2. Method

2.1. Study design

Data came from the 37-site EAT initiative in which all youth were supposed to receive MET/CBT (Sampl & Kadden, 2001), which was designed to include two individual sessions of motivational interviewing and three group sessions of CBT over five to six weeks and has been described at length elsewhere (Diamond et al., 2002). All youth were supposed to be assessed with the GAIN (Dennis et al., 2003) at intake 3, 6, and 12 months.

From the full EAT data set on 6397 youth, we excluded 2816 (44%) who were inappropriate for this analysis, including 1527 (24%) missing treatment data due to a computer error in data collection, 721 (11%) who were not due for a six-month follow-up by the end of the EAT program, 407 (6%) youth whose treatment setting could not be solidly classified as school or community (per classification described in the next paragraph), 141 youth (2%) who were not in school during the past year, 14(0.2%) who were in a level of care other than outpatient treatment, and 6 (0.1%) who did not enter into treatment for five sessions of MET/CBT. Sensitivity analyses were conducted to determine whether the excluded and included sample shared similar baseline characteristics. In general, the groups were fairly similar on gender, type of treatment, regular outpatient status, primary substance, victimization severity, and substance abuse (SA) and dependence, although differences in percentages of African-Americans, Hispanics, and intensive outpatient program membership between the samples were found.

Sites were classified based on where they said they were treating youth in SB (4 sites) or CB (17 sites) settings, or both (16 sites). The first two were each confirmed as providing over 75% of their treatment in the described setting and were used to classify all youth from their site. For the third group of sites that did both, we classified four additional sites as SB programs and included the youth treated in SB settings from there (49%–79% actual), but excluded the remainder (n = 313 initially assessed in CB setting). We classified 12 sites as CB and included youth treated in CB settings from there (82%–99% actual), but excluded the remainder (n = 94 initially assessed in SB settings). Of the remaining 3581 youth who were eligible and classified, we also had to drop 676 (19%) because they did not complete any follow-up interview. This left a final unweighted analytic sample of 492 youth entering into treatment in SB settings from 8 sites and 2413 youth entering into treatment in CB settings from 29 sites.

2.2. Data source and measures

All client characteristics were based on client self-report to in-person interviewers using the GAIN, a bio-psychosocial assessment instrument having broad utility within adolescent and adult school, substance, health, mental health, welfare, and justice systems (Dennis, White, & Ives, 2009; Ives, Chan, Modisette, & Dennis, 2010; Smith, Cleeland, & Dennis, 2010; Womack et al., 2004). The GAIN is designed to help clinicians diagnose for common psychiatric disorders based on the DSM-IV-TR (American Psychiatric Association, 2000), recommend levels of care based on the Patient Placement Criteria Version 2 (American Society of Addiction Medicine, 2001) treatment planning recommendations, as well as to map onto epidemiological, clinical, and economic outcomes (Dennis, Chan, & Funk, 2006). Validity for the GAIN has been documented in many prior studies using multiple methods (e.g. urine tests [Dennis et al., 2004; Lennox, Dennis, Ives, & White, 2006], collateral reports [Dennis, Funk, Godley, Godley, & Waldron, 2004; Godley, Godley, Dennis, Funk, & Passetti, 2001], Rasch measurement models, [Conrad, Dennis, Bezruczko, Funk, & Riley, 2007; Conrad et al., 2010; Conrad, Riley, Conrad, Chan, & Dennis, 2010; Conrad et al., 2012], and timeline follow-back [Dennis, Funk, Godley, Godley, & Waldron, 2004]). Copies of instruments, manuals, scoring syntax, norms by age, gender, and race on 100,000 + individuals from roughly 350 treatment sites are publicly available at www.gaincc.org.

2.2.1. Service measures

To identify services received in the first three months, we used 36 service variables from the three-month follow-up using the GAIN and the Treatment Transition Log (Dennis, Ives, White, & Muck, 2008), a separate Excel spreadsheet where staff documents a client’s treatment variables for each level of care.

2.2.2. Dependent measures

We utilized six outcomes, compared at intake and six months later: number of days out of the past 90 days in which the client reported: (1) any substance use, (2) any emotional problems, (3) any trouble with family, (4) any work/school trouble, (5) any illegal activity, and (6) being in a controlled environment. Additionally, health-care utilization cost and crime-related cost were compared between settings using previously published unit costs from economists (French et al., 2003; French, Popovici, & Tapsell, 2008; McCollister, French, & Fang, 2010) times the frequency of behavior from the GAIN and adjusted to 2012 dollars based on the consumer price index.

2.3. Analysis

An unweighted comparison indicated the SB and CB groups were different on 43 of the 90 baseline characteristics (see Table 1).

Table 1.

Unweighted characteristics at baseline by setting.

Characteristics and time period	School-based (n = 492)a	CB unweighted (n = 2413)a	Odds ratio (OR)/effect sizeb	95% CI (OR or d)
Demographics	N	N
Female	185 (38%)	696 (29%)	OR = 1.49*	(1.3 to 1.7)
Race
Caucasian	247 (50%)	1402 (58%)	OR = 0.73*	(0.5 to 0.9)
African-American	26 (5%)	198 (8%)	OR = 0.62	(0.2 to 1.0)
Hispanic	150 (30%)	405 (17%)	OR = 2.17*	(2.0 to 2.4)
Mixed/other	69 (14%)	408 (17%)	OR = 0.80	(0.5 to 1.1)
Single parent	247 (50%)	898 (37%)	OR = 1.70*	(1.5 to 1.9)
Age (capped)	14.86 (1.58)	15.76 (1.4)	d = −0.64*	(−1.4 to 0.1)
Substance use
Age of first use (capped)	12.55 (1.8)	13.21 (1.9)	d = −0.35*	(−1.1 to 0.4)
Used substances for 3 + years	202 (41%)	1067 (44%)	OR = 0.88	(0.7 to 1.1)
Weekly tobacco used	218 (44%)	1259 (52%)	OR = 0.73*	(0.5 to 0.9)
Weekly alcohol used	63 (13%)	342 (14%)	OR = 0.89	(0.6 to 1.2)
Weekly marijuana used	159 (32%)	1064 (44%)	OR = 0.61*	(0.4 to 0.8)
Weekly heroin used	3 (1%)	24 (1%)	OR = 0.61	(0 to 1.8)
Weekly cocaine used	10 (2%)	29 (1%)	OR = 1.71	(1.0 to 2.4)
Weekly other drug used	10 (2%)	33 (1%)	OR = 1.50	(0.8 to 2.2)
Lifetime dependence	199 (40%)	1196 (50%)	OR = 0.69*	(0.5 to 0.9)
Lifetime abuse	168 (34%)	852 (35%)	OR = 0.95	(0.7 to 1.2)
Past year dependence	173 (35%)	1025 (42%)	OR = 0.73*	(0.5 to 0.9)
Past year abuse	177 (36%)	934 (39%)	OR = 0.89	(0.7 to 1.1)
Past week withdrawal severity
Acute	9 (2%)	59 (2%)	OR = 0.74	(0 to 1.5)
Non-acute	94 (19%)	653 (27%)	OR = 0.64*	(0.4 to 0.9)
None	389 (79%)	1701 (70%)	OR = 1.58*	(1.3 to 1.8)
Prior substance abuse treatment	68 (14%)	597 (25%)	OR = 0.49*	(0.2 to 0.8)
Self-perceived substance problem	75 (15%)	489 (20%)	OR = 0.71*	(0.4 to 1.0)
Self-perceived need for treatment	285 (58%)	1583 (66%)	OR = 0.72*	(0.5 to 0.9)
Treatment Readiness Scale
Low	16 (3%)	142 (6%)	OR = 0.54	(0 to 1.1)
Moderate	300 (61%)	1467 (61%)	OR = 1.01	(0.8 to 1.2)
High – Reversed	176 (36%)	804 (33%)	OR = 1.11	(0.9 to 1.3)
Mental health problems
Co-occurring disordere	336 (68%)	1458 (60%)	OR = 1.41*	(1.2 to 1.6)
Mood disordere	187 (38%)	676 (28%)	OR = 1.58*	(1.4 to 1.8)
Generalized anxiety disordere	50 (10%)	199 (8%)	OR = 1.26	(0.9 to 1.6)
Homicidal/suicidal thoughtse	103 (21%)	414 (17%)	OR = 1.28*	(1 to 1.5)
Traumatic stress disordere	133 (27%)	441 (18%)	OR = 1.66*	(1.4 to 1.9)
Conduct disordere	237 (48%)	1080 (45%)	OR = 1.15	(1 to 1.3)
Attention deficit/hyperactivity Disordere	246 (50%)	1020 (42%)	OR = 1.37*	(1.2 to 1.6)
Prior mental health treatment	174 (35%)	1017 (42%)	OR = 0.75*	(0.5 to 1)
Health problems
High health problemsd	135 (27%)	729 (30%)	OR = 0.87	(0.7 to 1.1)
Pregnant or got someone pregnante	22 (4%)	105 (4%)	OR = 1.03	(0.6 to 1.5)
HIV risk
Sexual activityd	244 (50%)	1469 (61%)	OR = 0.63*	(0.4 to 0.8)
Multiple sexual partnersd	106 (22%)	623 (26%)	OR = 0.79	(0.6 to 1)
Unprotected sexual activityd	87 (18%)	544 (23%)	OR = 0.74*	(0.5 to 1)
Needle used	8 (2%)	20 (1%)	OR = 1.98*	(1.2 to 2.8)
School/vocational
Training Activity Scale
Low	5 (1%)	246 (10%)	OR = 0.09*	(0 to .98)
Moderate	286 (58%)	1165 (48%)	OR = 1.49*	(1.3 to 1.7)
High	201 (41%)	1002 (42%)	OR = 0.97	(0.8 to 1.2)
Trouble at work/schoold	434 (88%)	1836 (76%)	OR = 2.35*	(2.1 to 2.6)
Academic problemsd	339 (69%)	1569 (65%)	OR = 1.19	(1.0 to 1.4)
Behind > 1 year	299 (61%)	1115 (46%)	OR = 1.80*	(1.6 to 2)
Expelled or dropped outd	194 (39%)	972 (40%)	OR = 0.97	(0.8 to 1.2)
In schoold	488 (99%)	2200 (91%)	OR = 11.81*	(10.8 to 12.8)
Employedd	103 (21%)	922 (38%)	OR = 0.43*	(0.2 to 0.7)
Crime, violence and justice system involvement
Lifetime justice system involvement	316 (64%)	1979 (82%)	OR = 0.39*	(0.2 to 0.6)
Days of justice system involvementd	21.43 (36.2)	27.14 (38.2)	d = −0.15	(−0.7 to 0.4)
In a controlled environmentd	85 (17%)	453 (19%)	OR = 0.90	(0.6 to 1.2)
13 + days in controlled environmentd	29 (6%)	223 (9%)	OR = 0.62	(0.2 to 1.0)
Physical violencee	327 (66%)	1446 (60%)	OR = 1.33*	(1.1 to 1.5)
Illegal activitye	269 (55%)	1344 (56%)	OR = 0.96	(0.8 to 1.2)
Property crimee	198 (40%)	987 (41%)	OR = 0.97	(0.8 to 1.2)
Interpersonal crimee	183 (37%)	809 (34%)	OR = 1.17	(1.0 to 1.4)
Drug crimee	144 (29%)	875 (36%)	OR = 0.73*	(0.5 to 0.9)
Environment
Vocational Risk Index
Low	6 (1%)	24 (1%)	OR = 1.23	(0.3 to 2.1)
Moderate	228 (46%)	851 (35%)	OR = 1.59*	(1.4 to 1.8)
High	258 (52%)	1538 (64%)	OR = 0.63*	(0.4 to 0.8)
Social Risk Index
Low	4 (1%)	24 (1%)	OR = 0.82	(0 to 1.9)
Moderate	211 (43%)	899 (37%)	OR = 1.26*	(1.1 to 1.5)
High	277 (56%)	1490 (62%)	OR = 0.80*	(0.6 to 1.0)
Living Risk Index
Low	3 (1%)	34 (1%)	OR = 0.43	(0 to 1.6)
Moderate	313 (64%)	1575 (65%)	OR = 0.93	(0.7 to 1.1)
High	176 (36%)	804 (33%)	OR = 1.11	(0.9 to 1.3)
Lifetime homeless or runaway	129 (26%)	638 (26%)	OR = 0.99	(0.8 to 1.2)
Lifetime victimization	278 (57%)	1380 (57%)	OR = 0.97	(0.8 to 1.1)
Lifetime high severity victimization	203 (41%)	901 (37%)	OR = 1.18	(1.0 to 1.4)
Victimizationd	95 (19%)	492 (20%)	OR = 0.93	(0.7 to 1.2)
Count of 12 common past year problems c
No problems	38 (8%)	197 (8%)	OR = 0.94	(0.6 to 1.3)
1 problem	44 (9%)	285 (12%)	OR = 0.73	(0.4 to 1.1)
2 problems	66 (13%)	311 (13%)	OR = 1.05	(0.8 to 1.3)
3 problems	57 (12%)	319 (13%)	OR = 0.86	(0.6 to 1.2)
4 problems	61 (12%)	292 (12%)	OR = 1.03	(0.7 to 1.3)
5–12 problems	226 (46%)	1009 (42%)	OR = 1.18	(1.0 to 1.4)
Dependent variables at intake
Days of substance used	21.55 (27.5)	30.89 (29.9)	d = −0.31*	(−0.9 to 0.2)
Days of emotional problemsd	31.67 (33.3)	26.44 (32.6)	d = 0.16	(−0.4 to 0.7)
Days of trouble at work/schoold	14.57 (15.8)	11.86 (16.5)	d = 0.16	(−0.6 to 0.9)
Days of trouble with familyd	19.74 (28.3)	13.33 (22.5)	d = 0.28*	(−0.4 to 1.0)
Days in a controlled environmentd	3.07 (11.2)	4.96 (16.0)	d = −0.12	(−1.5 to 1.3)
Days of illegal activityd	3.9 (11.2)	4.93 (13.7)	d = −0.08	(−1.4 to 1.3)

^*p < 0.05.

^aCells give n and (%) for categorical variables or mean and (standard deviation) for continuous variables.

^bOdds ratio for school/CB settings for categorical variables or Cohen’s effect sizes d (school mean minus community mean divided by total standard deviation) for continuous variables.

^cCount of 12 common problems in the past year: alcohol use disorder, cannabis use disorder, other drug disorder, mood disorder, anxiety disorder, trauma disorder, attentive deficit/hyperactivity disorder, conduct disorder, suicidal thoughts, homelessness/runaway, victimization, illegal activity.

^dIn the past 90 days.

^eIn the past year.

2.3.1. Propensity score weighting procedure

To create a better quasi-experimental comparison group, we adjusted for these differences by using a propensity score weight (Lunceford & Davidian, 2004; Rubin & Thomas, 2000) to create a CB group matched in both characteristics and size to the SB group before exploring treatment and outcomes. Propensity scores were computed using logistic regression as the predicted probability (using the variables in Table 1 as predictors) that a respondent from the CB group could be part of the SB group. Two mean propensity scores, one for each group SB and CB were calculated and assigned to each member of the groups respectively. A weight, W₁, was calculated as the propensity score for that participant divided by the mean propensity score for that participant’s group. A second weight, W₂, was calculated as the N of the SB group (n = 492) divided by the N for CB group (n = 2413). A third weight, W₃, was calculated as W₁ multiplied by W₂ (and applied to those in the CB only). For those in the SB group, W₃ was set as 1. Applying a weight of W₃ to the data ensured the creation of equal samples sizes of n = 492 for both groups. Using this technique (described by Ives et al. [2010]) reduced the number of significant differences between groups from 43 to 3 differences (3.8% of 80 variables, excluding 10 redundant category variables), which is less than what would be expected by chance if randomly assigned. The weighted responses from the matched CB group were compared to the unweighted responses from the SB group on services received at three months and outcomes at six months.

2.3.2. Handling of missing data

As logistic regression uses list-wise deletion to prevent any bias or significant loss in sample size; median replacement via hot-deck imputation (Dennis, Lennox, & Foss, 1997; Schafer & Graham, 2002) using SPSS 19.0.0.1 (2010) Replace Missing Value (RMV) procedure was used to replace missing data for the 90 characteristics examined at intake. To create the hot deck, we sorted individual records by type of treatment (outpatient program vs. intensive outpatient program), treatment duration in days, and Global Individual Severity Scale (a total symptom count across GAIN domains of SUD, internalizing disorders, externalizing disorders, and crime and violence). We then replaced each missing value with the median of the four nearest valid answers (two above and two below current record) for that value in the ordered records. There were no change in differences between groups before and after utilizing the RMV procedure – the same 43 variables that were different pre-RMV procedure remained different post-RMV. Of the 36 service variables examined, those missing at 3-month follow-up (n = 219 cases, 7.5%) were replaced with the 6-month (n = 77), 9-month (n = 0), or 12-month (n = 142) version of the item. When data were still missing, the RMV procedure described above was utilized. For outcome analyses, in cases where a 6-month follow-up record was missing, outcomes were replaced in order of first available for 12-month and then 3-month records.

2.3.3. Measures of clinical significance

We measured clinical significance using odds ratios (OR) for dichotomous variables and Cohen’s effect size d for continuous variables. OR were calculated as the ratio of SB to CB settings, where values over 1 mean more common in SB and under 1 mean less common. Cohen’s d was used as the effect size for differences in characteristics and treatment/ service means, and also for differences in between-group means for outcomes as SB minus CB setting, where positive values indicate that SB is higher and negative values indicate it is lower. For within-group outcomes, Cohen’s d was calculated as the mean of follow-up minus baseline divided by the standard deviation at baseline, thus negative values indicate a reduction in values from baseline to follow-up and positive values indicate an increase.

3. Results

3.1. Differences between school- and community-based youth at intake

Table 1 shows the unweighted differences in intake characteristics of youth treated in SB and CB settings. Youth treated in SB settings were more likely to be female, Hispanic, under the age of 15, and with a single parent. SB youth were more likely than CB youth to report starting substance use at an earlier age but were less likely to report weekly use of tobacco and marijuana, have lifetime and past-year dependence, past-week withdrawal, prior SA treatment, perceive substance use as a problem, or feel they needed treatment.

Relative to CB youth, SB youth were more likely to report co-occurring disorders, including mood or trauma disorders, homicidal/suicidal thoughts, and ADHD, yet less likely to report prior mental health treatment. Unlike mental health, rates of medical problems in the past 90 days and pregnancy in the past year were similar between youth in both settings, although HIV/sexual risk was dissimilar. SB youth were less likely to report any sexual activity or unprotected sexual activity, but more likely to report needle use in the past 90 days.

SB youth were less likely to have lifetime justice system involvement, or to be involved in drug crime in the past year but were more likely to report being physically violent towards others during the past year. SB youth were less likely to report being employed in the past 90 days and more likely to being in school in the past 90 days, as well as being behind in school at least one year and being in trouble at school/work in the past 90 days. SB youth were more likely to report moderate levels of social and vocational risk and less likely to report high levels of social and vocational risk.

Table 1 shows that the levels of the six outcome variables at intake were similar for both groups except that SB youth reported fewer days of substance use and more days of being in trouble with family in the past 90 days. Number of problems was similar for youth in both settings.

3.2. Differences in intake characteristics after matching

After matching, the two groups were similar on 87 of 90 descriptive characteristics, but SB youth were still more likely than the CB youth to have Mood Disorder (38%–33%, OR = 1.26, p < 0.05) or Traumatic Stress Disorder (27%–23%, OR = 1.22, p < 0.05), and still less likely to have any lifetime criminal justice system involvement (64%–70%, OR = 0.77, p < 0.05).

3.3. Services/treatment received

Table 2 compares the services received in the first three months. The SB group did not significantly differ from the weighted CB group on 30 of 36 variables (83%). SB youth, however, were less likely to (1) initiate treatment within two weeks of the initial assessment, (2) engage in treatment, (3) had shorter lengths of stay, (4) received fewer external services, (5) had fewer urine/breath analysis tests, and (6) reported fewer days of structured activity without using substances.

Table 2.

Treatment and services by setting (after matching).

		School-based (n = 492)a N	CB matched (n = 492)a N	Odds ratio (OR)/effect sizeb	95% CI (OR or d)
Systems involvement	Initiation (initial interview conducted between 14 days before and 7 days after being admitted)	492 (80%)	492 (93%)	OR = 0.31*	(−0.1 to 0.3)
	System engagement (in treatment for six weeks across admits)	492 (75%)	492 (78%)	OR = 0.87	(0.6 to 0.9)
	Initial treatment engagement (in treatment 30 + days and 3 + sessions)	492 (65%)	492 (74%)	OR = 0.64*	(0.4 to 0.6)
	Continuing care (any treatment 90–180 days out)	492 (35%)	492 (39%)	OR = 0.81	(0.6 to 0.8)
	Positive discharge status (still in treatment/completed treatment)	492 (67%)	492 (69%)	OR = 0.91	(0.6 to 0.9)
	Completion of treatment	492 (66%)	492 (63%)	OR = 1.14	(0.9 to 1.1)
	Days from intake to discharge	62.8 (42.4)	80.3 (64.9)	d = −0.32*	(−0.4 to −0.3)
	Total days in index episode	103.8 (103.8)	109.4 (112.3)	d = −0.05	(−0.2 to −0.1)
Alliance Substance abuse treatment	Treatment Satisfaction Scale 3mc (α = 0.87)	13 (1.7)	12.9 (1.9)	d = 0.06	(−0.1 to 0.1)
	Nights in SA Residential 3mc (rs = 0.99)	0.8 (6.7)	0.9 (6.5)	d = −0.02	(−0.1 to 0)
	Times in SA ER 3mc (rs = 0.70)	0 (0.1)	0 (0.1)	d = 0.02	(−0.1 to 0)
	Days in SA IOP 3mc	0.2 (3.4)	0.2 (2.4)	d = 0.02	(−0.1 to 0)
	Times in SA OP 3mc (rs = 0.51)	5.3 (3.8)	6 (6.4)	d = −0.14	(−0.3 to −0.1)
	Days in other SA Tx 3mc	0 (0.4)	0.5 (4.2)	d = −0.15	(−0.3 to −0.1)
	Days on SA meds 3mc (rs = 0.58)	0 (0)	0.1 (1.5)	d = −0.05	(−0.2 to −0.1)
	Days in any SA treatmentc (rs = 0.66)	6.3 (8.4)	7.5 (10.1)	d = −0.13	(−0.3 to −0.1)
	Substance Abuse Tx Index	0.07 (0.1)	0.08 (0.1)	d = −0.13	(−0.3 to −0.1)
SA treatment content	Direct services received 3mc (α = 0.95)	5.3 (1.7)	5.4 (2.1)	d = −0.04	(−0.2 to 0)
	Family services received 3mc (α = 0.81)	1 (1.3)	0.9 (1.1)	d = 0.07	(−0.1 to 0.1)
	External services received 3mc (α = 0.92)	2.4 (1.9)	3.1 (2)	d = −0.37*	(−0.5 to −0.4)
	Treatment Received Scale 3mc (α = 0.97)	8.7 (3.9)	9.4 (4.1)	d = −0.19	(−0.3 to −0.2)
	Nights in MH hospital 3mc	0.1 (0.8)	0.2 (2.3)	d = −0.05	(−0.2 to −0.1)
Mental health treatment	Times in MH ER 3mc	0.02 (0.2)	0.01 (0.1)	d = 0.05	(−0.1 to 0)
	Times in MH OP 3mc (rs = 0.67)	0.7 (2.3)	1.3 (4.2)	d = −0.19	(−0.3 to −0.2)
	Days on MH meds 3m c	10.6 (27.7)	12.2 (29.5)	d = −0.06	(−0.2 to −0.1)
	Days of any mental healthc (rs = 0.67)	11 (28.1)	13 (29.8)	d = −0.07	(−0.2 to −0.1)
	Mental Health Tx Index	0.12 (0.2)	0.14 (0.2)	d = −0.07	(−0.2 to −0.1)
Interventions across systems (SA, MH, JJ)	Times urine/breath analysisc (rs = 0.78)	1.1 (2.5)	4.1 (6.1)	d = −0.62*	(−0.7 to −0.6)
	Any self-help received	0 (0.2)	0.1 (0.3)	d = −0.14	(−0.3 to −0.1)
	Days of self-helpc (rs = 0.95)	0.3 (4.2)	1.1 (6.9)	d = −0.13	(−0.3 to −0.1)
	Days of structured activity w/o substance usec (rs = 0.44)	10.6 (20)	15.2 (23.8)	d = −0.21*	(−0.3 to −0.2)
	Total days in a controlled environment (rs = 0.73)	2.2 (9.8)	3.8 (14.6)	d = −0.13	(−0.3 to −0.1)
	Days incarceratedc (rs = 0.40)	1.1 (6.7)	1.2 (7.1)	d = −0.02	(−0.1 to 0)
	Days in JJ/health inpatient	1.1 (6.7)	1.3 (7.1)	d = −0.02	(−0.1 to 0)
	Days of physical health Tx	8.6 (21.2)	8.3 (21.8)	d = 0.01	(−0.1 to 0)
	Physical Health Tx Index	0.1 (0.2)	0.1 (0.2)	d = 0.01	(−0.1 to 0)

Notes: r_s are test–retest Spearman Rho’s. SA, substance abuse; Tx, treatment; MH, mental health; ER, emergency room; OP, outpatient; IOP, intensive outpatient; JJ, juvenile justice.

^*p < 0.05.

^aCells give n and (%) for categorical variables or mean and (standard deviation) for continuous variables.

^bOdds ratio for school/CB settings for categorical variables or Cohen’s effect sizes d (school mean minus community mean divided by total standard deviation) for continuous variables.

^cIn the past 90 days.

3.4. Outcomes

Table 3 compares six-month outcomes; the within-group effect sizes and the between-group change effect sizes are provided. Only one between-group difference was statistically significant at the p = 0.05 level: SB youth had significantly fewer days in a controlled environment at six-month follow-up relative to the CB group at the same time interval (F = 5.33, p = 0.021). In terms of within-group differences, results indicate that there were small effects (ranging from −0.21 to −0.37) between intake and six-month follow-up for both SB and CB groups on days of substance use, days of emotional problems, days of work/school trouble, and days of family trouble indicating a reduction in days of problems in these areas (see Figure 1). There was no difference for either group between intake and follow-up on days of illegal activity or on days in a controlled environment, as indicated by the negligible within-group effect sizes (−0.2 < d < +0.2).

Table 3.

Outcomes by setting (after matching).

Outcomes	Mean school-based (n = 492)	Mean community-based matchedc (n = 492)	F	Sig.	Matched between group change effect size da
Days of substance use (Intake)	21.55	24.02	1.99	0.158	−0.09
Days of substance use (6-month)	15.88	15.26	0.14	0.709	0.02
Change in days of substance use	−5.68	−8.76	2.58	0.108	0.10
Within group effect size db	−0.21	−0.32
Days of emotional problems (Intake)	31.67	29.42	1.11	0.292	0.07
Days of emotional problems (6-month)	21.95	19.10	2.08	0.149	0.09
Change in days of emotional problems	−9.89	−10.25	0.03	0.870	0.01
Within group effect size db	−0.29	−0.31
Days of work/school trouble (Intake)	14.57	14.55	0.00	0.987	0.00
Days of work/school trouble (6-month)	9.03	8.08	1.07	0.302	0.07
Change in days of work/school trouble	−5.53	−6.47	0.50	0.479	0.05
Within group effect size db	−0.35	−0.37
Days of family trouble (Intake)	19.74	17.40	1.79	0.181	0.09
Days of family trouble (6-month)	11.08	11.37	0.04	0.841	−0.01
Change in days of family trouble	−8.97	−6.04	2.39	0.122	−0.11
Within group effect size db	−0.31	−0.23
Days of illegal activity (Intake)	3.07	3.61	0.51	0.477	−0.04
Days of illegal activity (6-month)	2.57	4.71	5.33	0.021	−0.13
Change in days of illegal activity	−0.49	1.09	2.41	0.121	−0.10
Within group effect size db	−0.04	0.08
Days in controlled environment (Intake)	3.90	4.25	0.22	0.639	−0.03
Days in controlled environment (6-month)	3.17	4.22	1.51	0.220	−0.07
Change in days of controlled environment	−0.75	−0.02	0.52	0.473	−0.04
Within group effect size db	−0.06	0.00

^aCohen’s d calculated as mean change in use over time in the SB setting minus mean change in use over time in the CB setting divided by the change in CB standard deviation.

^bCohen’s d calculated as mean at follow-up minus mean at baseline divided by the group standard deviation at baseline.

^cDue to weighting, the means for the baseline dependent variables differ from Table 1 for the CB group only.

Figure 1.

Change in days of problems between intake and six months.

Table 4 provides health-care utilization and crime-related costs in the year before and after the intake to treatment. There were no differences between the settings on any cost. However, there were significant decreases in health-care utilization costs in the year after intake for each setting (reduction of $356 per CB youth and $3141 per SB youth).

Table 4.

Average annual health-care costs and crime costs in the year before and after intake by setting.

		Community-based ($)	School-based ($)
Health-care utilization cost	Year prior	7522	9519
	Year after	7166a	6378a
	Raw change	−356	−3141
Crime-related cost	Year prior	217,042	303,073
	Year after	294,166	166,497
	Raw change	+77,124	−136,576

^aYear after is significantly lower than year before (p < 0.05).

4. Discussion

This study provided evidence for the feasibility of providing MET/CBT for youth with substance problems within a school setting by demonstrating that relative to a matched sample of CB youth, it achieved equivalent implementation on 80% of the service/process measures (this includes all the variables in Table 2 across Systems Involvement, Treatment Alliance, SA Treatment, SA Treatment Content, MH Treatment, and Interventions across Systems sections). The effectiveness and cost-effectiveness of MET/CBT was also similar by setting. The unweighted findings are important because they demonstrate that SB settings reach kids who are younger and earlier in their course of substance use, suggesting that SB clinics aid early identification and intervention for SUDs. They also appear to provide an opportunity to reduce existing health disparities by gender and ethnicity, as suggested by findings that SB youth were more likely to be girls and Hispanic.

Regarding the gender finding, it has been speculated (Weist et al., 1999), and we agree, that this could be because the threshold criteria for referral to and entry into SB clinics may not be as high as for CB settings. The latter typically results from youth getting into trouble, and males having greater propensity towards acting out. We further speculate that boys are under greater pressure to enter treatment by external sources such as family, work, and the justice system, whereas SB settings may reflect voluntary treatment-seeking (Godley, 2006). The finding on race is also consistent with previous research (Juszczak et al., 2003; Weist et al., 1999) that SB health care attracts minorities, because it is less expensive, more accessible, and perhaps has less attached stigma. Whatever the reasons, these findings lend support to offering alternative gateways into substance use treatment that would equilibrate disparities observed in traditional CB settings (Barber, Rosenblatt, Harris, & Attkinson, 1992).

SB youth also have a greater likelihood of mood and trauma disorders when presenting to treatment. Despite this, SB settings achieve comparable outcomes. Wong (2005), who found similar results, speculated that the greater frequency of mood/trauma disorders in the SB group might be due to youth presenting to treatment more immediately when in crisis than youth in the CB group and/or having a higher rate of females. In our sample, SB youth reported slightly more emotional problems at intake than CB youth, though the difference was not statistically significant.

Despite lower rates of implementation on 20% of the service indicators, SB youth showed a significant reduction in days of substance use, emotional problems, being in trouble with family, and being in trouble at school/work at six months post-intake. The fact that similar outcomes to a matched cohort in CB settings were achieved demonstrates the promise of treating youth in an SB setting and the possibility of even greater improvement in outcomes if implementation issues are addressed. There also appear to be significant reductions in service costs associated with SB settings in the year before and after intake. The $3141 per youth reduction in cost of health-care utilization in SB settings is substantially larger than the $1113–$1558 published costs of providing MET/CBT (French et al., 2003). This bodes well for justifying its expanded use as part of SBHCs.

In summary, our results indicate that treating youth in SB settings reaches more girls, ethnic minorities, first-time users of services, those younger, and those earlier in the life course of addiction, and that these same youth have had less treatment exposure, are less ready to enter treatment, have more health problems, and have greater severity of mood/ trauma disorders. Even with imperfect implementation, MET/CBT consistently led to reductions in several problem behaviors in school settings. Our study’s strengths include the use of two EBP’s, multisite data collection, and multiple sources of data on service utilization. Limitations include use of recruitment source and assessment location as an indicator that SB settings were being used for intervention delivery; client reports to assess the quality of the intervention received; and not assessing for socioeconomic status.

4.1. Conclusions and future directions

A resurgence of providing quality, affordable health care is underway as part of the Affordable Care Act. We have demonstrated that implementing MET/CBT in school settings is feasible/effective in improving problem outcomes and offers the potential to level disparities. We recommend a thorough exploration of barriers to implementation and treatment fidelity in future work.