Effectiveness of Treatment for Adolescent Substance Use: Is Biological Drug Testing Sufficient?

Abstract

Objective:

The purpose of this study was to compare the relative effectiveness of three treatment modalities for adolescent substance use: biological drug screening (BDS), Motivational Enhancement Therapy–Cognitive Behavioral Therapy (MET/CBT5), and BDS combined with MET/CBT5, relative to no treatment.

Method:

This study comprised 5,186 adolescents (70% male) enrolled in substance use treatment and tracked through the Substance Abuse and Mental Health Services Administration’s Center for Substance Abuse Treatment’s database (BDS = 1,110; MET/CBT5 = 784; BDS combined with MET/CBT5 = 2,539; no treatment = 753). Outcomes of interest were substance use frequency and severity of substance use problems at 3, 6, and 12 months, as measured by the Global Appraisal of Individual Needs survey. Propensity score weighting was used to adjust for pretreatment covariate imbalances between groups. Weighted generalized linear models were used to estimate the impact of treatment on outcomes at 3, 6, and 12 months.

Results:

BDS, alone or in combination with MET/CBT5, was associated with improved substance use and substance problems outcomes. Relative to youth reporting no treatment services, the BDS group reported significantly lower substance use at all visits, with the observed difference increasing over time. BDS alone was associated with significantly fewer substance problems than BDS combined with MET/CBT5 at all visits and significantly lower use at 12 months.

Conclusions:

Our results demonstrate significant improvement on substance use outcomes associated with BDS and offer preliminary evidence that BDS, particularly standalone BDS, may be an effective form of drug treatment for adolescents. Further work, including randomized studies, should explore the optimal format of administering BDS to adolescents to achieve maximum effectiveness.

Given the magnitude of the problem of adolescent substance use, identifying effective treatment programs and improving access to treatment is essential. Specifically, 16.8% of youth report past-month illicit drug use, and 6.9% of individuals ages 12–17 years meet Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV; American Psychiatric Association, 1994), criteria for a substance use disorder, yet only 10.5% of youth needing substance treatment receive formal treatment services (Johnston et al., 2013; Substance Abuse and Mental Health Services Administration [SAMHSA], 2012). There are numerous evidence-supported treatment programs for adolescents, including cognitive behavioral therapy (CBT), motivational enhancement therapy with cognitive behavioral therapy (MET/CBT), community reinforcement approach, functional family therapy, multidimensional family therapy, and multisystemic therapy (Stanger and Budney, 2010). However, continued research is needed given that even the most promising of these programs typically achieve only short-term modest reductions in substance use, rather than sustained abstinence (Grella et al., 2001; Hser et al., 2001; Stanger and Budney, 2010).

Biological drug screening (BDS) has long been a component of drug treatment, and regular drug screening was identified by the National Institute on Drug Abuse as one of 13 principles of drug treatment (National Institute on Drug Abuse, 2009). The effectiveness of contingency management (CM), a drug-screening protocol that provides contingencies, typically positive reinforcement, for negative drug screens (Higgins et al., 2008; Petry and Simcic, 2002), has been extensively studied among adult substance users; yet very little research has been conducted among adolescents. Research among adults indicates that CM is associated with decreased substance use, primarily during the period in which the contingencies are being offered (Dutra et al., 2008; Prendergast et al., 2006; Stanger and Budney, 2010). The few adolescent studies of CM have examined the effect of combining CM with evidence-based treatments and generally found support for this approach. Stanger et al. (2009) determined that youth receiving CM, based on urine drug screens, in combination with MET/CBT exhibited greater periods of abstinence from marijuana than youth who received only MET/CBT (study n = 69). Krishnan-Sarin et al. (2006) examined the effect of a school-based CM program targeting adolescent tobacco users, including both breath carbon monoxide and urine cotinine screening, and found that adolescents who received both CM and CBT had higher tobacco abstinence rates (53% after 4 weeks) than those who received only CBT (0% abstinence; study n = 28). Henggeler et al. (2012) found that youth participating in juvenile drug court who were randomized to receive CM, based on urine drug screens administered through the court, in combination with family engagement strategies had less marijuana use and committed fewer crimes than youth in the usual services group (study n = 104).

BDS may play an especially important role in the treatment of adolescents for several reasons. First, as evidenced by the high rate of external referrals to treatment by the juvenile justice system, education system, or family (Breda and Heflinger, 2004; SAMHSA, 2007; Stanger and Budney, 2010), many youth lack intrinsic motivation to reduce their substance use; being subject to drug testing may provide youth with crucial extrinsic motivation. In addition, adolescents may perceive drug testing as a relatively nonintrusive, objective treatment, especially for those who are unreceptive to psychotherapy. In a recent study, 59% of youth enrolled in outpatient substance treatment elected to participate in optional urine screening, with parental notification, suggesting that BDS is acceptable to youth (Levy et al., 2011). Finally, communication and trust between adolescents and parents is often strained when adolescents develop substance use problems; the objectivity of drug testing may validate an adolescent’s self-reported drug use, thereby helping to rebuild trust and facilitating recovery (Vakili et al., 2009).

Previous studies indicate that CM is an effective adjuvant treatment for adolescent substance use, yet it is not known whether more general forms of drug screening that do not adhere to a classical CM protocol may also provide such treatment benefits. Furthermore, it is not known whether drug screening may be effective at reducing substance use in its own right. The current study investigates both of these possibilities. Using a quasi-experimental design, we explored the association between substance use outcomes and biological drug testing, both alone and in combination with an evidence-supported treatment program, using a large observational cohort of adolescents enrolled in substance use treatment programs across the United States.

Method

Participants

Participants in this study were 5,186 adolescents from the larger Substance Abuse and Mental Health Services Administration’s Center for Substance Abuse Treatment’s (CSAT) 2007 adolescent treatment database that tracks outcomes for CSAT-sponsored providers. In our sample, 3,323 youth received Motivational Enhancement Therapy–Cognitive Behavioral Therapy 5 (MET/CBT5) (Sampl and Kadden, 2001) through the Effective Adolescent Treatment (EAT) program, a discretionary funding program that supported MET/CBT5 implementation (Dennis et al., 2004; Melchior et al., 2007; SAMHSA, 2003). MET/CBT5 is a widely used brief intervention comprising two sessions focusing on increasing desire to reduce substance use and three sessions teaching drug refusal skills, positive peer interactions, alternative social activities, and craving and relapse coping skills. The additional 1,863 youth were initially enrolled in a treatment program yet reported receiving either no treatment or only BDS during the first 90 days of follow-up. These 1,863 youth were enrolled in EAT (n = 892) or in one of eight other CSAT discretionary programs: (a) the Cannabis Youth Treatment experiment (n = 16; Dennis et al., 2004; Diamond et al., 2002) providing outpatient care; (b) the Adolescent Treatment Models program (n = 21; Dennis et al., 2003 a) providing community-based care; (c) the Adolescent Residential Treatment (n = 194; SAMHSA, 2002) providing continuing care for discharged youth; (d) the Strengthening Communities’ Youth program (n = 273; Dennis et al., 2008) aimed at building partnerships among community, school-based, and juvenile justice treatment services; (e) the Targeted Capacity Expansion program (n = 155; Wilson et al., 2005) providing intensive outpatient and inpatient services; (f) the Young Offenders Reentry Program (n = 176; SAMSHA, 2004) providing services to youth re-entering the community; (g) the Family and Juvenile Treatment Drug Court program (n = 46; SAMHSA, 2005) providing comprehensive services through drug courts; and (h) the Assertive Adolescent and Family Treatment program (n = 90; Godley et al., 2007) promoting family-centered services. Although recruitment strategies differed across programs, written informed consent from parents and assent from the adolescents were obtained, and institutional review boards at each site approved study protocol.

Measures

All youth in the study were evaluated with the Global Appraisal of Individual Needs (GAIN; Dennis et al., 2003b), a comprehensive structured clinical interview. The GAIN has modules assessing demographic characteristics; substance use severity and service utilization; physical health; mental health; risk behaviors; environment risk factors; legal status; and educational/vocational status. Race was defined by self-report, and adolescents identified themselves as White, Black, Hispanic, or other. The GAIN was administered at baseline and 3-, 6-, and 12-months visits.

Treatment groups

Treatment groups were defined in terms of services that youth reported receiving from baseline to 3 months, as measured by 30 items on the GAIN relating to drug treatment service utilization, 20 of which comprise the Substance Use Treatment Received Scale (α = .97; Dennis et al., 2010), as well as an indicator of whether the adolescent was enrolled in the EAT program (which administered MET/CBT5). We first identified two groups of interest—youth enrolled in the EAT study who reported receiving services and youth enrolled in any study participating in the CSAT database, including but not restricted to EAT, who reported that they had received no treatment services (other than potentially BDS). Each of these two groups was further divided into those who did report any BDS and those who did not, resulting in four treatment groups, described as follows: (1) no-treatment group (n = 753), (2) BDS group (n = 1,110), (3) MET/CBT5 group (n = 784), and (4) BDS combined with MET/CBT5 group (n = 2,539) (Figure 1). Youth in the no-treatment group received no services after treatment enrollment for a variety of possible reasons, including treatment refusal, dropout, treatment wait-listing, or inability to continue treatment because of incarceration or other factors; unfortunately, we do not have data as to why individual youth in this study reported receiving no services.

Figure 1.

Flow chart showing classification of the 5,186 adolescents in this study into the four treatment groups. MET/CBT5 = Motivational Enhancement Therapy–Cognitive Behavioral Therapy; BDS = biological drug screening.

Study outcomes

Substance use frequency during follow-up was assessed with the GAIN’s Substance Frequency Scale (SFS), an eight-item scale that assesses the average proportion of alcohol and other drug using days in the past 90, taking into account heavy use and problem days (Dennis et al., 2010). We rescaled this variable to represent number of days of use during the past 90 by multiplying the original scale by 90. We assessed severity of substance use problems with the past-month Substance Problem Scale (SPS; Dennis et al., 2010). This scale is a count of 16 possible symptoms in the past 30 days: the seven DSM-IV criteria for drug dependence, the four DSM-IV criteria for substance abuse, two items related to substance-associated health and psychological problems, and three items related to lower severity symptoms (e.g., hiding use, people complaining about use, and weekly use). Inter-item reliability for these scales is very good; among a similar population of adolescents, α = .80 for the SFS and α = .92 for the SPS (Dennis et al., 2010).

Given the potential suppression effect of institutionalization on substance use outcomes, we also considered institutionalization among youth in each treatment group. At each study visit, youth reported the number of days, of the past 90, that were spent in a controlled environment (e.g., jail, inpatient treatment, group homes, or probation camps) in which illicit drug use and liberty were substantially constrained for the whole day.

Statistical analyses

Our analytic approach was designed to answer the following question: What is the relative effectiveness of four different treatment modalities (no treatment, BDS, MET/CBT5, and BDS combined with MET/CBT5) for youth such as those in our sample? Given the nonrandomized nature of these data, we used propensity score methods to adjust for baseline differences among youth in the four treatment groups and create comparable groups (Imbens, 2000; Rosenbaum and Rubin, 1983). Propensity score weighting was used to re-weight each treatment group to match the baseline study population with respect to 34 pretreatment variables, including demographics and factors relating to substance use, mental health, legal involvement, living environment, and education/vocation (Table 1). Propensity score weights were estimated using generalized boosted modeling, a flexible, nonparametric estimation method that has been shown to outperform logistic regression for propensity score estimation (Lee et al., 2010; McCaffrey et al., 2004). Details on the methodology for estimating propensity score weights with more than two treatment groups are available in McCaffrey et al. (2013). Weights were estimated using the twang package in R (Ridgeway et al., 2012).

Table 1.

Propensity score weighted descriptive statistics , baseline (n = 5,186)

Baseline variable	No tx	BDS only	MET/CBT5	MET/CBT5 + BDS	Unweighted study	Max ASMD
Demographics
Age, in years	15.7	15.7	15.4	15.7	15.7	.17
Female	31.0%	27.4%	30.7%	29.1%	29.1%	.04
Race
White	48.4%	45.9%	50.5%	51.3%	49.3%	.07
Black	12.5%	14.9%	9.0%	10.0%	11.5%	.11
Hispanic	22.9%	23.9%	25.1%	21.4%	22.7%	.06
Other	16.1%	15.3%	15.4%	17.2%	16.5%	.03
Substance use
Daily substance use	27.0%	28.8%	26.4%	26.6%	28.1%	.04
In recovery	24.4%	26.5%	25.1%	24.9%	24.4%	.05
Substance Dependence Scale, past month	0.8	0.8	0.8	0.8	0.8	.06
Substance Dependence Scale, past year	2.2	2.2	2.2	2.2	2.2	.04
Days of substance use, past 90 days	9.2	9.4	9.4	9.2	9.6	.03
Substance Problem Scale, past month	2.5	2.3	2.5	2.5	2.6	.07
Substance Problem Scale, past year	5.8	5.9	5.9	6.0	6.1	.04
Lifetime substance use treatment	20.6%	25.4%	18.1%	24.3%	23.8%	.13
Treatment Motivation Index	1.7	1.7	1.7	1.7	1.7	.04
Mental and physical health
Suicidal thoughts, past 90 days	9.7%	8.3%	11.1%	8.8%	9.9%	.05
Emotional Problems Scale	0.21	0.20	0.20	0.21	0.21	.05
Behavior Complexity Scale	9.2	9.3	9.4	9.5	9.6	.04
Internal Mental Distress Scale	7.8	7.6	7.4	7.0	7.5	.07
Mental Health Treatment Index	0.01	0.01	0.01	0.01	0.01	.08
Health Problems Scale	0.10	0.10	0.10	0.11	0.11	.05
Physical Health Treatment Index	0.01	0.01	0.01	0.01	0.01	.06
Legal
Total arrests, past 90 days	0.2	0.2	0.2	0.2	0.2	.03
Criminal Justice System Index	0.3	0.4	0.3	0.4	0.4	.08
Crime and Violence Scale	6.0	6.2	5.9	6.2	6.3	.08
Illegal Activities Scale	0.1	0.1	0.1	0.1	0.1	.04
Drug Crime Scale	0.5	0.5	0.5	0.5	0.5	.06
Controlled environment days, past 90	9.4	9.5	6.4	7.7	9.5	.13
Environmental
Environmental Risk Scale	35.1	35.1	35.2	35.3	35.3	.03
Living Environment Risk Scale	10.3	10.2	10.4	10.3	10.3	.04
Social Environment Risk Index	12.8	12.9	12.9	12.8	12.9	.03
Lifetime homeless or runaway	23.1%	25.2%	26.0%	27.2%	27.1%	.04
Housing
House or apartment	85.7%	84.1%	85.7%	85.5%	84.7%	.03
Jail or correctional facility	6.1%	6.1%	6.0%	5.3%	5.8%	.02
Living elsewhere	8.1%	9.8%	8.2%	9.1%	9.5%	.05
Vocational
Attended any school in past 90 days	89.0%	89.2%	92.2%	90.9%	90.1%	.07
Worked at least 1 day in past 90 days	28.8%	28.2%	29.8%	33.2%	30.4%	.06
Training Activities Scale	0.6	0.7	0.7	0.7	0.7	.06
Employment Activities Scale	0.2	0.2	0.2	0.2	0.2	.06
Unweighted sample size	753	1,110	784	2,539	5,186
Effective sample size	636	866	619	2,396	4,517

Notes: Effective sample size is a conservative estimate of the sample size in the weighted sample. Tx = treatment; BDS = biological drug screening; MET/CBT5 = Motivational Enhancement Therapy–Cognitive Behavioral Therapy (MET/CBT5); ASMD = absolute standardized mean difference.

To show the comparability of the four groups after propensity score weighting, covariate balance across groups was assessed with the absolute standardized mean difference (ASMD). The appropriate ASMD for treatment group j when one is interested in making each group look like the overall sample is estimated by

where denotes the propensity score weighted mean for given variable among the jth treatment group (here, j = 1, 2, 3, 4), and and denote the unweighted mean and standard deviation in the baseline sample of all adolescents (McCaffrey et al., 2013). ASMD values close to 0 after propensity score indicate that groups are similar on a given covariate; ASMD values less than .20 are typically considered indicative of good balance (Cohen, 1992). In the context of multiple treatment groups, we consider a covariate well balanced after weighting if all four ASMD values are less than .20.

When estimating group differences, we used the survey package in R (Lumley, 2012) to fit propensity score weighted generalized linear models to outcomes at 3, 6, and 12 months. For each outcome at each follow-up visit, we fit two generalized linear models: (a) a model that only included treatment group indicators and (b) a “doubly robust” model with treatment indicators and all variables in the propensity score model. In general, the doubly robust results were similar to results with only treatment group; we present only the latter results.

Given that institutionalization rates differed notably across the four groups (Figure 4) and that restricted access to drugs while institutionalized may result in better outcomes on the SFS and SPS (Griffin et al., 2008; McCaffrey et al., 2007), we conducted additional analyses to estimate group differences in the absence of institutionalization. We used an artificial censoring approach that censored youth at the time they were institutionalized and then accounted for differences in pre-institutionalization characteristics between institutionalized (censored) and noninstitutionalized youth, with inverse probability of censoring weighting (IPCW) (Cain and Cole, 2009; Cole and Hernán, 2008; Cole et al., 2003). Institutionalization weights were estimated by modeling, separately within each treatment condition, the probability that an adolescent was institutionalized in the 90 days before each follow-up visit. To re-weight the data such that noninstitutionalized youth in the remaining sample were representative of all youth in the baseline sample, the institutionalization weights were multiplied by the original propensity score weights (Cole and Hernán, 2008), and these composite weights were used in the weighted generalized linear models to estimate group differences.

Figure 4.

Plot of propensity score weighted institutionalization rates at each follow-up visit for each of the four treatment conditions. Tx = treatment; BDS = biological drug screening; MET/CBT5 = Motivational Enhancement Therapy–Cognitive Behavioral Therapy.

Results

Participant characteristics

Each treatment group was weighted to match the characteristics of the overall study population of 5,186 youth (Table 1), such that treatment effects were estimated in reference to the overall study population. Among all youth, mean age is approximately 16 years, and nearly 70% are male. Approximately 49% of youth self-identify as White, 23% as Hispanic, 12% as Black, and 16% as other. Generally, at baseline youth report mild to moderate substance use and substance-related problems (Substance Frequency Scale and Substance Problem Scale) and meet few dependence criteria (Substance Dependence Scale); 75% report no prior substance use treatment. Youth report fairly low levels of mental and physical health problems, as measured by the Emotional Problems Scale, Internal Mental Distress Scale, and Health Problems Scale, among others.

Comparability of groups before and after propensity score weighting

Although adolescents across the treatment groups were generally similar before propensity score weighting, there were several notable differences across groups, specifically with regard to race/ethnicity and juvenile justice system–related variables. Adolescents in either MET/CBT5 group (with or without BDS) were more likely to be White and less likely to be Black (MET/CBT5: White = 51%, Black = 6%; BDS and MET/CBT5: White = 58%, Black = 8%) than adolescents in either the no-treatment group or the BDS group (BDS: White = 36%, Black = 18%; no treatment: White = 39%, Black = 18%). In addition, youth in either MET/CBT5 group (with or without BDS) had lower means on the Criminal Justice System Index than both the no-treatment group and the BDS group (MET/CBT5 = 0.21; BDS and MET/CBT5 = 0.33; no treatment = 0.35; BDS = 0.53). Similarly, adolescents in the BDS and no-treatment groups reported more controlled environment days than youth in either MET/CBT5 group (MET/CBT5 = 3; BDS and MET/CBT5 = 6; no treatment = 13; BDS = 19). Weighting successfully balanced treatment groups with regard to all baseline covariates included in the propensity score model at 3, 6, and 12 months (3-month balance table shown in Table 1; all ASMD < .20).

Approximately 25% (n = 1,294) and 53% (n = 2,738) of participants were lost to follow-up by 6 and 12 months, respectively. We calculated ASMD statistics for each covariate in Table 1 comparing youth remaining in the study at 6 months with the original sample and comparing youth at 12 months with the original sample; all ASMDs were smaller than .20 within each treatment group, indicating that youth at follow-up visits did not significantly differ from the original sample. Similarly, covariate balance was assessed as part of the institutionalization sensitivity analysis; adequate covariate balance (all ASMD < .20) was achieved at all study visits after IPCW.

Description of treatment groups

Treatment groups were defined based on services from baseline to 3 months. Youth in the BDS group reported an average of 5.0 biological drug tests (SD = 7.3) during these 3 months, which, if performed at regular intervals, is approximately one every 2.5 weeks. Similarly, youth in the BDS combined with MET/CBT5 group reported an average of 5.2 biological drug tests (SD = 6.7); by definition, youth in the MET/CBT5 and no-treatment groups reported 0. To assess treatment engagement for youth in both MET/CBT5 groups, we examined the average score on the GAIN’s Treatment Received Scale, which asks about 20 specific drug treatment services. Youth in the MET/CBT5 group reported receiving an average of 7.9 distinct services (SD = 4.3), and youth in the BDS combined with MET/CBT5 group reported receiving an average of 9.9 services (SD = 3.7) at 3 months. By definition, youth in the BDS and no-treatment groups reported exactly zero services on the Treatment Received Scale at 3 months.

Youth in this study received the majority of services before 3 months. Youth across all four groups had low Treatment Received Scale scores at 6 months (means ranged from MET/CBT5 = 0.7 to BDS combined with MET/CBT5 = 2.7) and even lower scores at 12 months (means ranged from MET/CBT5 = 0.5 to BDS combined with MET/CBT5 = 1.3). Trends in BDS across groups at 3 months were preserved at 6 and 12 months, with youth in the two BDS groups reporting more screenings than the MET/CBT5 and no-treatment groups, yet less than they reported at 3 months. At 6 months, youth in the BDS and the BDS combined with MET/CBT5 groups reported an average of 3.2 (SD = 8.0) and 2.9 (SD = 6.0) drug tests, respectively, in contrast to an average of 0.85 (SD = 5.1) and 0.95 (SD = 5.1) for youth in the MET/CBT5 and no-treatment groups, respectively. Biological drug testing had declined even more at 12 months; youth in the BDS and the BDS combined with MET/CBT5 groups reported an average of 1.6 (SD = 4.1) and 2.0 (SD = 7.0) drug tests, respectively, in contrast to an average of 0.49 (SD = 2.0) and 0.57 (SD = 1.7) for youth in the MET/CBT5 and no-treatment groups, respectively. Overall, adolescents received the majority of services by 3 months; services they received after 3 months were primarily consistent with their 3-month treatment classifications.

Estimated relative effectiveness of the four treatment modalities

Treatment effects with regard to substance frequency and substance problems were estimated at 3, 6, and 12 months. BDS, both alone and in conjunction with MET/CBT5, was associated with significant reductions in frequency of substance use among adolescents in this study (Table 2). At baseline, youth across all four groups reported approximately 9.2 days of use in the past 90 days. Of note, the standalone BDS group was the only treatment group that maintained decreased use throughout follow-up—they reported 5.4 days of use at 3 months, 4.9 days at 6 months, and 4.9 days at 12 months. All other treatment groups achieved lowest use at 3 months and then increased use by 6 and 12 months (Figure 2). Relative to the no-treatment group, the BDS group reported significantly fewer days of use at all follow-up visits, with the observed effect increasing over time (3 months = 1.3 fewer days, p = .01; 6 months = 2.0 fewer days, p = .003; 12 months = 2.7 fewer days, p = .004). By 12 months, the BDS group had a significantly lower use than all other groups—approximately 1.6 fewer days than both the MET/CBT5 group (p = .04) and the BDS combined with MET/CBT5 group (p = .003), and 2.7 fewer days than the no-treatment group (p = .004).

Table 2.

Estimated treatment effects on outcomes at 3, 6, and 12 months

Variable	Days of substance use			Substance Problems Scale
	Estimate	SE	p	Estimate	SE	p
3-month outcomes
BDS vs. no tx	-1.31	0.54	.01*	-0.28	0.15	.06
MET/CBT5 vs. no tx	-0.89	0.60	.14	0.08	0.17	.63
(MET/CBT5 + BDS) vs. no tx	-1.43	0.47	.002*,†	-0.02	0.13	.87
BDS vs. MET/CBT5	-0.42	0.52	.43	-0.36	0.16	.02*
BDS vs. (MET/CBT5 + BDS)	0.13	0.37	.73	-0.26	0.11	.02*
(MET/CBT5 + BDS) vs. MET/CBT5	-0.54	0.45	.23	-0.10	0.14	.47
6-month outcomes
BDS vs. no tx	-1.97	0.66	.003*,†	-0.34	0.16	.04*
MET/CBT5 vs. no tx	-1.34	0.70	.06	-0.03	0.19	.86
(MET/CBT5 + BDS) vs. no tx	-1.26	0.61	.04*	-0.04	0.14	.79
BDS vs. MET/CBT5	-0.63	0.54	.24	-0.30	0.16	.06
BDS vs. (MET/CBT5 + BDS)	-0.71	0.41	.09	-0.30	0.11	.008*,†
(MET/CBT5 + BDS) vs. MET/CBT5	0.07	0.47	.87	-0.01	0.15	.97
12-month outcomes
BDS vs. no tx	-2.74	0.96	.004*,†	-0.34	0.22	.13
MET/CBT5 vs. no tx	-1.19	1.08	.27	0.00	0.25	.99
(MET/CBT5 + BDS) vs. no tx	-1.16	0.92	.21	0.04	0.21	.83
BDS vs. MET/CBT5	-1.55	0.78	.04*	-0.34	0.21	.11
BDS vs. (MET/CBT5 + BDS)	-1.58	0.53	.003*,†	-0.38	0.15	.01*,†
(MET/CBT5 + BDS) vs. MET/CBT5	0.03	0.72	.97	0.04	0.19	.81

Notes: BDS = biological drug screening; tx = treatment; MET/CBT5 = Motivational Enhancement Therapy–Cognitive Behavioral Therapy.

^*Denotes p < .05;

^†denotes Bonferroni-corrected p < .05.

Figure 2.

Plot of average days of substance use, as assessed by the Substance Frequency Scale (SFS), for each treatment condition across the study period. Tx = treatment; BDS = biological drug screening; MET/CBT5 = Motivational Enhancement Therapy–Cognitive Behavioral Therapy.

BDS combined with MET/CBT5 was also associated with significant reductions in substance use frequency during follow-up. Relative to the no-treatment group, the BDS combined with MET/CBT5 group reported 1.4 fewer days of use at 3 months (p = .002) and 1.3 fewer days at 6 months (p = .04). This trend continued at 12 months, although the difference between groups was no longer statistically significant. The standard deviation of days of use across all groups was 9.8 at 3 months, 9.9 at 6 months, and 11.1 at 12 months.

Standalone BDS was also associated with significant reductions in substance problems among adolescents in this study (Table 2). At baseline, all four groups reported approximately 2.4 substance use problems on the SPS (maximum = 16) and exhibited similar trends in mean SPS scores across follow-up, with the largest reduction between baseline and 3 months, yet youth in the BDS group had the lowest mean SPS score at all follow-up visits (SPS = 1.3 at 3 months, 1.1 at 6 months, and 1.2 at 12 months; Figure 3). Of note, the BDS group reported significantly lower SPS scores than the BDS combined with MET/CBT5 group at all follow-up visits, with the difference increasing over time: 0.26 at 3 months (p = .02), 0.30 at 6 months (p = .008), and 0.38 at 12 months (p = .01). Relative to the no-treatment group, the BDS group showed significantly fewer problems at 6 months (difference = 0.34, p = .04), with a similar trend at both 3 months (difference = 0.30, p = .06) and 12 months (difference = 0.34, p = .13). The BDS group also reported significantly fewer problems than the MET/CBT5 group at 3 months (difference = 0.36, p = .02).

Figure 3.

Plot of mean levels of substance problems, as assessed by the Substance Problem Scale (SPS), for each treatment condition across the study period. Tx = treatment; BDS = biological drug screening; MET/CBT5 = Motivational Enhancement Therapy–Cognitive Behavioral Therapy.

Figure 4 shows the propensity score weighted rates of institutionalization in the 90 days before each follow-up visit for each treatment group. As shown, youth in both BDS groups had higher rates of institutionalization at each follow-up compared with youth in the no-treatment and MET/CBT5 groups. In light of these significant differences and institutionalization’s known potential for substance use suppression, we present additional results from an analysis using IPCW to balance groups on rates of institutionalization (Table 3). These results were generally consistent with the previous results, although power is lowered by the use of IPCW; thus, the reduction in statistically significant effects may be associated with reduced power. We observed very similar effect sizes and trends for youth in the BDS group relative to the no-treatment group, with youth in the BDS group reporting 1.9 fewer days of use at 6 months (p = .03) and 2.9 fewer days of use at 12 months (p = .01). In addition, compared with the BDS combined with MET/CBT5 group, the standalone BDS youth reported 1.7 fewer days of use at 6 months (p = .002) and 1.8 fewer days at 12 months (p = .003). Moreover, standalone BDS continued to be associated with significantly lower substance problems at every follow-up visit relative to the BDS combined with MET/CBT5 group; rather than attenuating, these group differences all increased after adjusting for institutionalization (-0.37 vs. -0.26 at 3 months, -0.42 vs. -0.30 at 6 months, -0.47 vs. -0.38 at 12 months).

Table 3.

Estimated treatment effects on outcomes at 3, 6, and 12 months, under no institutionalization during follow-up

Variable	Days of substance use			Substance Problems Scale
	Estimate	SE	p	Estimate	SE	p
3-month outcomes
BDS vs. no tx	-0.97	0.67	.15	-0.34	0.21	.11
MET/CBT5 vs. no tx	0.16	0.78	.84	0.28	0.26	.27
(MET/CBT5 + BDS) vs. no tx	-0.78	0.62	.21	0.03	0.21	.89
BDS vs. MET/CBT5	-1.13	0.67	.09	-0.62	0.21	.003*,†
BDS vs. (MET/CBT5 + BDS)	-0.19	0.47	.69	-0.37	0.14	.01*
(MET/CBT5 + BDS) vs. MET/CBT5	-0.94	0.62	.13	-0.25	0.20	.21
6-month outcomes
BDS vs. no tx	-1.94	0.88	.03*	-0.15	0.18	.42
MET/CBT5 vs. no tx	-0.89	0.97	.36	0.24	0.22	.27
(MET/CBT5 + BDS) vs. no tx	-0.27	0.86	.75	0.27	0.17	.11
BDS vs. MET/CBT5	-1.05	0.70	.13	-0.39	0.20	.06
BDS vs. (MET/CBT5 + BDS)	-1.66	0.53	.002*,†	-0.42	0.15	.01*
(MET/CBT5 + BDS) vs. MET/CBT5	0.61	0.67	.36	0.03	0.19	.87
12-month outcomes
BDS vs. no tx	-2.91	1.13	.01*	-0.29	0.24	.23
MET/CBT5 vs. no tx	-1.15	1.31	.38	0.18	0.32	.57
(MET/CBT5 + BDS) vs. no tx	-1.08	1.11	.33	0.18	0.23	.43
BDS vs. MET/CBT5	-1.77	0.93	.06	-0.47	0.28	.09
BDS vs. (MET/CBT5 + BDS)	-1.83	0.61	.003*,†	-0.47	0.17	.005*,†
(MET/CBT5 + BDS) vs. MET/CBT5	0.06	0.90	.94	0.00	0.27	1.00

Notes: BDS = biological drug screening; tx = treatment; MET/CBT5 = Motivational Enhancement Therapy–Cognitive Behavioral Therapy.

^*Denotes p < .05;

^†denotes Bonferroni-corrected p < .05.

Discussion

In this large, multisite observational study of adolescent substance users in treatment, we found that youth who received BDS achieved more favorable outcomes with regard to substance use frequency and substance use problems. Compared with youth in the no-treatment group, youth in the standalone BDS group reported significant reductions in substance use at each follow-up visit. Similarly, BDS in combination with MET/CBT5 was associated with significant reductions in use at 3 and 6 months relative to no treatment. Notably, BDS was associated with significantly fewer substance problems than BDS in combination with MET/CBT5 at all follow-up visits. A secondary analysis that accounted for the potential suppression effects of institutionalization was consistent with our key findings. In fact, evidence in favor of standalone BDS versus BDS in combination with MET/CBT5 increased after removing youth who were institutionalized and using IPCW to make the noninstitutionalized youth representative of all youth in the baseline sample.

This study is consistent with previous studies that examined the effectiveness of CM among adolescents yet unique in that it suggests potential treatment effects for BDS as more generally implemented than CM. Some adolescents reporting BDS in our study may have received CM, whereas others may have received other forms of drug testing. For example, instead of being rewarded for negative drug screens (typical CM protocol), some adolescents may have faced consequences for positive drug screens, a routine occurrence in the juvenile justice system. For other adolescents, particularly those receiving MET/CBT5, BDS may have been used primarily to monitor substance use and provide feedback to the adolescent. In addition, youth may have been subject to rewards or consequences based on drug test results through their parents. Finally, because our data represent community-based care, the fidelity of any CM received by youth in this study may have been lower than that of CM evaluated in previous randomized studies.

The consistent (albeit not all significant) reductions of substance use and substance problems associated with standalone BDS compared with MET/CBT5 suggest that BDS may be a preferable treatment modality for adolescents. BDS may be particularly effective, perhaps more so than MET/CBT5, at providing external motivation for behavior change. In addition, BDS may be an appealing and effective form of treatment for youth who are not particularly inclined to engage in therapy. The relative effectiveness of BDS in our study also may be associated with the relative lengths of MET/CBT5 and BDS. MET/CBT5 is a brief intervention comprising five sessions, whereas drug testing may be implemented for many months. Although MET/CBT5 has been demonstrated to show sustained effects on substance use after completion of therapy, these sustained effects may not be large as those of ongoing drug testing.

The finding of greater reductions in substance use and substance problems associated with BDS compared to BDS combined with MET/CBT5 may be due, in part, to differences in how BDS was implemented in the two groups. Specifically, youth in the standalone BDS group may have experienced more rigorous screening or stronger rewards or consequences, particularly if BDS was implemented primarily for deterrence. In contrast, for youth in the BDS combined with MET/CBT5 group, the MET/CBT5 sessions may have been viewed as the main intervention, with BDS used primarily for outcomes monitoring. If such implementation differences were present in our study, it is possible that BDS combined with MET/CBT5 would have shown an additive effect and better outcomes had BDS been implemented identically in both groups. It is also possible that BDS implementation was similar across groups, yet low motivation for MET/CBT5, resulting in a negative therapeutic alliance, may have diminished the effectiveness of BDS for some youth in the combined BDS and MET/CBT5 group. If so, this would suggest that standalone BDS may be a promising, even preferable, treatment option for adolescent substance users.

This study has a number of limitations, including the lack of detailed information available regarding the nature of BDS that youth received. Stanger and Budney (2010) reviewed five aspects that influence the effectiveness of CM (and perhaps more broadly BDS): “the schedule used to deliver consequences, the magnitude of the consequence, the choice of the target behavior, the selection of the type of consequence, and the monitoring of the target behavior.” The data in our study do not have information regarding these aspects, particularly the authority administering BDS or whether any consequences or rewards were associated with results. Developing a better understanding of the most effective authorities, structure, and contingencies related to BDS is crucial for future dissemination and implementation. In addition, all data are self-reported, so there is potential misclassification of adolescents with regard to treatment classification. Youth could potentially either underreport or overreport services; we have no reason to believe either type of misreporting would be more probable. Information on treatment compliance or treatment providers’ fidelity, which could be used to validate adolescents’ self-reported services, was not available in this study. Similarly, although the GAIN is a well-validated instrument for adolescents, substance use outcome data are self-reported and may be subject to inaccuracies or recall bias.

Moreover, 6- and 12-month analyses were not adjusted for any treatment services youth may have received after 3 months. As previously described, youth received the majority of services by 3 months; any continued services typically were similar to those received before 3 months, thereby preserving the interpretation of the treatment groups. Furthermore, the findings of our study may be specific to the study population and may not be fully generalizable to other populations of adolescents in substance treatment. Follow-up rates in this study are also far from ideal (75% at 6 months and 47% at 12 months), and although additional analyses showed that youth remaining in the study at both 6 and 12 months did not significantly differ from the original sample with respect to baseline characteristics, bias may still result if follow-up is related to outcomes, particularly if these associations vary by group. It would be impossible to know the direction of any potential bias without additional information on the (unobserved) outcomes of youth lost to follow-up.

Finally, as in any study using propensity scores to balance treatment groups, our quasi-experimental design is only able to ensure that our findings are robust to differences between the four groups on the observed characteristics used to estimate the weights. It is possible that unobserved characteristics confound our findings. Of note, we balanced the groups on more than 30 pretreatment characteristics (Table 1) that are commonly associated with both treatment assignment and adolescent substance use outcomes. In addition, our findings were robust to one important confounder of treatment effectiveness, namely institutionalization, diminishing concern that our results might be explained by higher rates of controlled environments among youth in the BDS groups. Nonetheless, it is still possible these youth may be under more judicial or police surveillance than youth in the other treatment groups in ways not directly measured in the GAIN.

Given the scarcity of research regarding BDS among adolescent substance users, this study provides an important and rigorous evaluation of the relative effectiveness of BDS, MET/CBT5, and BDS combined with MET/CBT5 among a large cohort of adolescents in substance use treatment. Our results are consistent with previous studies that have demonstrated effects of BDS (primarily CM); our results also offer preliminary evidence that standalone BDS may be an effective form of treatment. However, given the broad definition of BDS used in this study, future randomized studies should directly compare various implementation approaches to BDS to identify the formats (e.g., length, authority) and contingencies (e.g., rewards vs. consequences, magnitude of contingencies) that are most effective for adolescents. Establishing the effectiveness and optimal implementation of BDS, an intervention that can be easily implemented in a variety of settings (including a primary care setting and the school system), is imperative considering the large unmet treatment need for adolescent substance users. Previous studies have shown that BDS is a treatment modality that adolescents find acceptable (Levy et al., 2011). It may be time for treatment providers, clinicians, and policy makers to recognize drug testing as an effective form of drug treatment in its own right, rather than merely a monitoring tool or adjuvant treatment.