A randomized trial of an integrated cognitive behavioral treatment protocol for adolescents receiving home-based services for co-occurring disorders

Abstract

The current study conducted a preliminary test of whether community mental health clinic staff could implement a multicomponent cognitive behavioral treatment, developed for adolescents with substance misuse (alcohol and/or marijuana) and comorbid psychiatric symptoms. We randomized a total of 111 families, with an adolescent 12–18 years old, referred to a home-based services program for youth with co-occurring substance use and mental health problems, to receive treatment from either masters-level therapists who received intensive cognitive behavioral therapy (I-CBT) training or from masters-level therapists who took part in a typical brief continuing education–style CBT workshop (treatment as usual, or TAU). Each family’s therapist and insurance company determined the frequency and intensity of treatment. We administered follow-up assessments at 3, 6, and 12 months. There was a small, but not statistically significant, reduction in the percent days of heavy drinking and marijuana use over time for both conditions, with the overall effect across the three follow-up points favoring the I-CBT condition. There were no differences on alcohol use days or other drug use. There was also a small, but nonsignificant, positive effect over time on externalizing symptoms, depressed mood, and anxiety, favoring the I-CBT condition. Youth in the I-CBT condition relative to TAU had significantly fewer juvenile justice contacts, while the pattern of costly service use varied, with higher rates at 6-month and lower rates at 12-month follow-ups. If therapists pay greater attention to parent training and provide more parent-adolescent communication sessions, outcomes may improve above standard community care. Training enhancements, to better meet the needs of community therapists and their clinic settings, may also produce better overall results for parents and adolescents.

1. Introduction

Psychiatric disorders frequently co-occur among adolescents with substance use disorders (SUDs). Externalizing (oppositional defiant, conduct, and attention deficit hyperactivity) disorders are most commonly diagnosed in adolescents, followed by internalizing (mood and anxiety) disorders (Armstrong & Costello, 2002; Chan, Dennis, & Funk, 2008). In a comprehensive review of community-based studies, on average, 60% of adolescents with an SUD had a comorbid psychiatric disorder (Armstrong & Costello, 2002). Comorbidity rates are even higher in SUD treatment settings, ranging from 62% to 88% in outpatient settings and as high as 80% in inpatient and residential settings (Dennis et al., 2004; Grella, Joshi, & Hser, 2004; Mason & Posner, 2009; Rowe, Liddle, Greenbaum, & Henderson, 2004). The presence of more than one comorbid psychiatric disorder is also common. One study using an outpatient SUD treatment sample found that 20% of adolescents had one comorbid psychiatric disorder, 24% two, 17% three, and 26% four or more disorders (Rowe et al., 2004). In an inpatient SUD treatment sample, 68% had two or more psychiatric disorders (Langenbach et al., 2010). Rates of comorbidity increase with severity of the SUD (Roberts, Roberts, & Xing, 2007).

Not surprisingly, comorbidity is generally associated with a more severe course of illness, including suicide attempts/death (Brent et al., 1993; King et al., 1996), as well as aggressive and high-risk criminal behavior in early adulthood (Clingempeel, Britt, & Henggeler, 2008). In addition, comorbidity complicates treatment and has been associated with poorer treatment outcomes in terms of both substance use and psychiatric outcomes across outpatient, intensive outpatient, and inpatient/residential settings (Cornelius et al., 2004; Rowe et al., 2004; Shane, Jasiukaitis, & Green, 2003; Tomlinson, Brown, & Abrantes, 2004; White et al., 2004). Adolescents with comorbid SUD, externalizing and internalizing disorders, and those with a higher number of diagnoses in total tend to have the worst outcomes (Rowe et al., 2004; Shane et al., 2003; Tomlinson et al., 2004).

SUD treatments that address comorbid psychiatric disorders are needed for patients to achieve optimal outcomes. Unfortunately, substance use and mental health treatment programs have historically been discrete entities with minimal coordination or collaboration between them. Standardized measurement of dual diagnosis capability in 256 community programs across the United States revealed that only 18% of addiction treatment programs and 9% of mental health programs were capable of dual diagnosis services (McGovern, Lambert-Harris, Gotham, Claus, & Xie, 2014). A smaller scale investigation of 30 community programs in California showed 43% of programs met or exceeded measurement criteria for dual diagnosis capability. However, these programs tended to score higher on domains of staffing, training, and assessment, and scored lower on criteria related to treatment, program structure, and continuity of care, suggesting a lag between service potential and actual clinical practice (Padwa, Larkins, Crevecoeur-MacPhail, & Grella, 2013). Thus, integration of systems and the use of evidence-based SUD protocols that address comorbid psychiatric disorders are sorely needed.

There is limited research that examines SUD interventions specifically designed to treat co-occurring psychiatric disorders. The few integrated treatment protocols that have been tested in randomized clinical trials were delivered by study staff rather than in real-world clinical settings. One set of studies integrated behavior therapy for SUDs with medication for psychiatric disorders. Cornelius et al. (2009) compared 12 weeks of motivational enhancement therapy (MET) plus cognitive behavioral therapy (CBT), with fluoxetine or placebo, for adolescents with an alcohol use disorder and major depressive disorder. Riggs et al. (2007) compared 16 weeks of CBT, with fluoxetine or a placebo, for adolescents with major depressive disorder, a lifetime history of conduct disorder, and an SUD. Overall, across both studies, CBT or CBT/MET, with or without fluoxetine, led to improvements in self-reported alcohol/substance use symptoms and psychiatric symptoms (depressive and/or conduct) and there was no difference in substance use outcomes for those who did or did not receive medication. A study that compared stimulant medication for attention-deficit hyperactivity disorder with CBT for SUDs found similar results (Riggs et al., 2011).

The psychosocial therapy literature has shown that a few ecological family-based and multicomponent SUD treatments yield reductions in co-occurring externalizing problem behavior (see Hogue, Henderson, Becker, & Knight, 2018; Hogue, Henderson, Ozechowski, & Robbins, 2014 for reviews). Family-based interventions may be particularly important for adolescent populations due to the embeddedness of youths in the family and the need to address contextual factors in the family that may exacerbate presenting problems. To our knowledge, only one family-based integrated treatment protocol has been tested in a randomized clinical trial that targets adolescent SUDs as well as co-occurring externalizing and internalizing psychiatric symptoms. Esposito-Smythers, Spirito, Hunt, Kahler, and Monti (2011) compared an integrated cognitive behavioral treatment for adolescents with an SUD and suicidality (I-CBT) to enhanced treatment-as-usual (E-TAU) in the community. I-CBT resulted in lower rates of suicide attempts, psychiatric hospitalizations, emergency department visits, arrests, binge drinking days, and marijuana use days relative to E-TAU, but not suicidal ideation or drinking days at 18-month follow-up.

Though deemed successful in efficacy trials, an ongoing problem for evidence-based, psychosocial treatments has been transporting these treatments to the community and obtaining the same effects as those obtained in research settings (e.g., Clarke et al., 2005; Southam-Gerow et al., 2010). There are a variety of reasons why this may be the case. First, the wide range of comorbidities (e.g., Chu, Merson, Zandberg, & Margaret, 2012), as well as cultural and socioeconomic factors (Southam-Gerow, Rodríguez, Chorpita, & Daleiden, 2012) typically encountered in the community, complicate the transfer to the real world of treatments tested in diagnostically homogeneous research samples. Second, the selection of therapists in treatment trials—often doctoral students who have received intensive training in, and may have an allegiance to, a particular therapy approach, and who have small caseloads—differ from therapists in the community who tend to have a wide range of backgrounds and training with large caseloads. And third, there is much greater attention to supervision focused on fidelity to treatment protocols in treatment trials (Henggeler, Pickrel, & Brondino, 1999) than in community care (e.g., Schoenwald, Sheidow, & Chapman, 2009).

1.1. Current study

The current study sought to determine whether a modified multi-component cognitive behavioral treatment developed for adolescents with SUDs (alcohol and/or marijuana) and comorbid psychiatric symptoms (Esposito-Smythers et al., 2011) could be conducted in the community. Specifically, this trial was conducted in a mental health clinic that delivered intensive outpatient, home-based, behavioral health services to adolescents with co-occurring substance use and mental health problems and their families. We hypothesized that, when conducted in the community with masters-level clinic staff who received intensive training and ongoing supervision, use of this protocol would be associated with greater reductions in alcohol use, marijuana use, and psychiatric symptoms, relative to a TAU control condition in which therapists received a brief continuing education (CE)-style CBT workshop. This hypothesis was based on the premise that greater depth of training, including repetition and practicing skills, would lead to better outcomes than standard care where there is typically more breadth than depth of skill practice (Beidas & Kendall, 2010; Herschell, Kolko, Baumann, & Davis, 2010).

2. Material and methods

2.1. Participants

Participants included 111 adolescents (ages 12–18) and their families who presented for treatment at a community mental health clinic in the northeast region of the United States. Adolescents were eligible for the study if they were enrolled in the intensive outpatient, home-based program (IOP) for co-occurring substance use and mental health problems at the clinic and: (a) were 12–18 years of age; (b) reported alcohol and/or other substance use in the prior three months; and (c) were English speaking. Participants were excluded from the study if they: (a) had serious psychotic symptoms (e.g., hallucinations) or a primary diagnosis of an eating disorder or obsessive-compulsive disorder, or (b) were acutely suicidal or homicidal. Table 1 presents the demographic characteristics of the sample.

Table 1.

Socio-demographics by treatment condition.

Variable	(n = 111)
	EXP (n = 61)		TAU (n = 50)
	n	%	n	%
Sex
Male	33	54.10%	31	62.00%
Female	28	45.90%	19	38.00%
Sexual orientation
Heterosexual	30	69.77%	23	74.19%
Gay or lesbian	1	2.33%	1	3.23%
Bisexual	7	16.28%	4	12.90%
Not sure	1	2.33%	2	6.45%
None of the above	4	9.30%	1	3.23%
Race
White	46	77.97%	32	66.67%
Black or African-American	4	6.78%	8	16.67%
Other	2	3.39%	2	4.17%
Multiracial	7	11.86%	6	12.50%
Ethnicity
Latinx	20	33.90%	14	28.57%
Year in school
8th	3	5.17%	1	2.04%
9th	17	29.31%	17	34.69%
10th	12	20.69%	15	30.61%
11th	12	20.69%	11	22.45%
12th	14	24.14%	5	10.20%
Retained a year in school	15	25.86%	15	30.61%
Parental income
0 to $25,999	27	45.76%	21	42.86%
$26,000 to $49,999	19	32.20%	13	26.53%
$50,000 or more	13	22.03%	15	30.61%
Parental employment
Mother employed outside of the home	35	59.32%	34	68.00%
Father employed outside of the home	28	62.22%	31	79.49%
Mother language preference
English all the time	48	84.21%	42	84.00%
English most of the time	2	3.51%	1	2.00%
English and other language equally	6	10.53%	2	4.00%
Other language most of the time	1	1.75%	5	10.00%
Father language preference
English all the time	33	73.33%	29	70.73%
English most of the time	8	17.78%	2	4.88%
English and other language equally	4	8.89%	5	12.20%
Other language most of the time	0	0.00%	5	12.20%

Note: EXP = experimental treatment condition; TAU = treatment-as-usual condition.

There were no statistically significant differences between treatment conditions. N per variable may not equal total sample N due to participants choosing not to answer a question.

2.2. Procedure

All adolescents, and their parent or legal guardian, who presented for treatment at the IOP were invited to participate in a two-group randomized parallel trial, if they met the inclusion/exclusion criteria described. Research assistants described the study to families and then obtained written informed consent and assent from parents and adolescents, respectively. We then administered a baseline assessment battery, consisting of a structured interview and self-report instruments (described below), which took approximately 4 h to complete. Research assistants who were unaware of the treatment condition administered follow-up assessments at 3, 6, and 12 months, which each took 4 h to complete. Participants received compensation for each assessment visit as well as money to cover transportation costs to assessments, when necessary (up to $375 each for the parent and adolescent if they completed all four assessments). The hospital human subjects’ protection committee approved the study, and we conducted the study from October 2012 to July 2016 with the last follow-up interviews completed in April 2017. After completion of the baseline assessment, families were assigned to their treatment condition, on a 1:1 randomization schedule, using a computerized urn randomization procedure that the statistician prepared (Stout, Wirtz, Carbonari, & Del Boca, 1994). The variables that we used in the urn randomization included whether the adolescent was referred to the IOP immediately following residential treatment, a diagnosis of a mood disorder, and number of self-reported risk factors (i.e., above/below 14). Agency clinicians asked parent/guardians about 23 risk factors, adapted from the Stress Index (Attar, Guerra, & Tolan, 1994) and Risk and Protective Factors Scale (Looper & Grizenko, 1999) as part of the intake process. The risk factors are listed in Table 2. The allocation sequence was concealed in the electronic program from the research assistant, who enrolled and randomized participants.

Table 2.

Baseline risk factor endorsement by treatment condition.

Variable	EXP		TAU
	n	%	n	%
Teen’s mother has mental health issues?	37	61.67	22	44.90
Teen’s father has mental health issues?	32	54.24	21	45.65
Teen ever failed a grade?	25	40.98	21	47.73
Violence in the home?	25	40.98	18	36.73
Sexually abused?	8	13.11	7	14.29
Physically abused?	7	11.48	6	12.24
Teen lived in foster care or placement?	15	24.59	11	22.45
Teen ever lived with single parent?	47	77.05	37	75.51
Moved more than three times in teen’s life?	35	57.38	26	57.78
Changed school more than four times?	17	28.33	13	30.95
Teen family on welfare?	16	27.59	13	28.89
Family have social worker?	34	55.74	29	60.42
Teen in gang?	4	6.56	3	6.12
Teen ever gone to court?	33	54.10	32	65.31
Teen’s mother ever in jail?	25	40.98	16	36.36
No family transportation?	13	21.31	6	12.50
Gunfire in neighborhood?	8	13.33	9	18.75
Drug use in neighborhood?	41	67.21	29	59.18
Gangs in neighborhood?	19	31.67	11	22.45
Teen truant?	22	36.07	16	33.33
Teen on probation?	13	21.31	15	31.25
Teen mandated in treatment by court?	21	34.43	15	31.25
Special education services?	27	44.26	22	44.90

Note: EXP = experimental treatment condition; TAU = treatment-as-usual condition.

There were no statistically significant differences between treatment conditions.

2.3. Treatment conditions

In both treatment conditions, session length and frequency varied based on clinical presentation and insurance coverage. Sessions ranged from 30 min to 3 h, when dictated by the insurance company for certain home-based services, and therapists met with families up to 3 times/week. For example, some insurers required 6 h of weekly in-home treatment in order to receive reimbursement, and this could be delivered in 2–3 home visits. Insurers approved treatment, on average, for 12 weeks. In both conditions, consistent with standard clinic practices, we assigned two therapists to each family (one for the adolescent; one for the parent). At times, however, the clinic was understaffed and we assigned only one therapist for a period of time. Therapists in both conditions met weekly with the same onsite masters-level supervisor who provided both clinical supervision and case management oversight (e.g., insurance company re-authorization). Supervision from the clinic director focused on eclectic therapy techniques that were not specific to a particular manual or model and consultation around case management. Following standard clinic procedures, adolescents could be referred and evaluated for medication by one of two clinic psychiatrists and received random urine drug screens over the course of treatment. Individual therapists could also choose to request drug screens for individual patients based on their own clinical judgement.

2.3.1. Integrated-cognitive behavioral therapy (I-CBT)

We grounded the I-CBT in social cognitive learning theory (Bandura, 1986). According to this theory, substance use and mental health problems may result in part from prior learning histories, especially the learning of social behaviors and core beliefs. To effectively address these problems, CBT targets the maladaptive behaviors and beliefs that underlie problems in this area, such as ineffective problem-solving, cognitive appraisal, affect regulation, and communication skills. We also integrated motivational interviewing into the protocol to improve motivation to change substance use and enhance treatment engagement.

We manualized the protocol and included individual adolescent, individual parent, and family modules (Esposito-Smythers et al., 2011). We assigned two therapists to each family, one that worked individually with the adolescent and one that worked with the parents. The parent therapist led any family sessions, but both therapists participated in these joint sessions. Some individual sessions were “core” sessions (problem-solving, cognitive restructuring, behavioral activation, and affect regulation) that all families received. In addition, specific substance use sessions for adolescents (e.g., motivational interview, coping with substance use urges, substance use refusal skills, as well as parent sessions on monitoring adolescent substance use) were administered routinely. Others were “supplemental” and only used as needed (e.g., an exposure session for adolescents with social anxiety).

Therapists began treatment with a review of assessment results and corresponding recommendations for treatment, including psycho-education around the importance of medication for some presentations. Therapists used all individual modules with the adolescent or parent to address individual skill improvement, for example, affect regulation. Therapists also designed modules specifically for parents to help parents to improve their parenting skills (e.g., contingency management and attending to the positive). Adolescent sessions had common components, including a safety check-in (around suicidal ideation, self-harm, substance use), adherence to prescribed medications, homework review, agenda setting, new skill introduction/skill practice, agenda discussion, homework assignment, and parent-teen check-in.

Although therapists taught core skills in the first portion of the treatment, the protocol was flexible and allowed for tailoring of the treatment and management of crises while staying “in-protocol”. Therapists could, and often did, choose to practice a previously learned core skill rather than teach a new skill. Further, though there were many sessions, most built on core skills (e.g., problem-solving, cognitive-restructuring, affect regulation) to address different problem areas. For example, problem-solving was used in the increasing social support module to help adolescents generate options for adding to their support network. We built on core skills deliberately, to facilitate adolescent and parental skill acquisition and generalization, but also to ease the training for study therapists and increase the likelihood of future dissemination of skills training. Therapists could interweave motivational interviewing sessions and techniques throughout sessions as needed, to enhance treatment engagement.

We emphasized parental monitoring in parent sessions because it is such a strong predictor of adolescent substance use (Barnes, Hoffman, Welte, Farrell, & Dintcheff, 2006; Macaulay, Griffin, Gronewold, Williams, & Botvin, 2005) and, therefore, important to address. As might be expected, some adolescents demonstrated resistance to increased parental monitoring. Consequently, parent-teen communication typically proceeded monitoring sessions to lay the groundwork for monitoring. Individual sessions with teens were also used to prepare for this work (e.g., cognitive restructuring around why parents monitor, pros and cons of parental monitoring for that particular teen). Also, parents often paired monitoring with a behavioral contingency contracts that included rewards as well as consequences for a specified behavior, which helped to increase buy-in. These contracts often helped to decrease conflict at home because rules were entirely clear to both parent and teen, and parents were guided to provide appropriate consequences for misbehavior the teen could live with. For example, before treatment it was not uncommon for parents to take away all privileges (e.g., phone, computer, socializing with friends, etc.) for an extended period of time when a misbehavior occurred rather than an enforce an appropriate consequence for a limited amount of time that was logically linked with the specified misbehavior.

2.3.2. Training, fidelity, and competency in I-CBT

The authors trained community clinicians through didactic instruction in a day-long workshop. Two additional clinicians, hired about midway through the study, viewed the videotape of the day-long training. All clinicians reviewed videotaped training sessions, and role-played protocol sessions with a supervisor. The total duration of this “real world” training was more consolidated than previous tests of this protocol, which were provided over several weeks, followed by intensive supervision and feedback. Once trained, therapists received weekly supervision. We audio recorded all intervention sessions with study participants. Each therapist saw all recordings for the first two participant cases, and four of the authors reviewed a random selection of 20% of subsequent recordings to rate fidelity and provide detailed feedback and supervision.

We created adherence checklists, adapted from a prior clinical trial for the I-CBT condition (Esposito-Smythers et al., 2011), for each module and included components common (e.g., set agenda, reviewed homework) and unique (e.g., provided rationale for problem solving) to each session. The number of items ranged from 9 to 23 across sessions. Each item on the checklist was rated “0” for nonadherence, “0.5” for partial adherence, or “1” for full adherence. We deemed adherence to ≥80% of session content acceptable. We rated competency using the Cognitive Therapy Rating Scale (CTRS; Young & Beck, 1980), a scale with strong psychometric properties (Vallis, Shaw, & Dobson, 1986), with total scores ≥44 deemed acceptable for sessions.

Three of the authors provided weekly supervision. They reviewed protocol implementation and ratings from selected taped sessions. Supervision also incorporated a review of the adolescents’ weekly assessments of mood, suicidal ideation, and substance use. Authors also provided individual supervision when emergent situations arose. If therapist drift was evident through any method of supervision, the supervisors processed obstacles to adherence, generated solutions, and conducted role-playing to enhance training.

2.3.3. Treatment-as-usual (TAU)

We based TAU on practitioner-based principles. These included the use of eclectic, flexible treatment, such as supportive therapy, person-centered approaches, advice, skill discussion, establishing a positive relationship, and case management. A previous study indicating that therapists prefer using a broad range of eclectic strategies at a low level of intensity (Garland et al., 2010) and a survey of mental health and substance use practitioners indicating that only 10% of providers followed a structured protocol to treat dually diagnosed adolescents (Lichtenstein, Zimmermann, & Spirito, 2010) support this approach. Therapists in this condition could use CBT techniques, such as those taught in the continuing education CBT workshop that we describe. Practitioners who had a background in motivation enhancement techniques occasionally incorporated this style into their sessions and any other techniques they deemed appropriate, but they did not follow a standard protocol.

2.3.4. Training–TAU

Researchers provided basic instruction to therapists in the I-CBT protocol in a brief, 4-hour, didactic workshop, with a focus on core sessions (i.e., problem-solving, cognitive restructuring, affect regulation, and behavioral activation). Therapists also had access to the I-CBT treatment manual, which they could use as needed with families. However, the researchers did not provide any follow-up training or supervision to therapists in this condition. This training was intended to mimic the typical CE instruction provided in the community. Therapists in TAU received supervision by their masters-level clinical supervisor for their cases, who also attended the 4-hour CBT workshop, but who had an eclectic therapy orientation. TAU clinicians met with the clinic supervisor and received feedback on case management as well as use of eclectic therapeutic techniques that were not specific to one particular manual or model. To be comparable to training received in most clinics, therapists could also seek out any additional training that they desired in any treatment approach, which they could use in sessions at their discretion, but under supervision.

We asked therapists in this condition to record sessions for the research project. The clinical supervisor also reviewed the recordings on occasion. One of the authors and another doctoral-level coder reviewed a random selection of tapes to determine whether therapists were using CBT skills from the I-CBT protocol. The author and coder used an adherence checklist to determine if therapists in the TAU condition were using specific components of the I-CBT protocol.

2.4. Measures

We administered the substance use outcome measures at baseline, 3, 6, and 12 months. We administered the behavior problem rating scale and secondary, psychiatric symptom outcome variables at 6 and 12 months. We administered the diagnostic interview at baseline only.

2.4.1. Adolescent Drinking Questionnaire (ADQ)

The ADQ (Jessor, Donovan, & Costa, 1989) consists of 4 items, rated on an 8-point scale, that assess recent drinking frequency (days per month), quantity (drinks per occasion), frequency of heavy drinking (≥5 drinks per occasion), and frequency of intoxication (feeling “drunk” or “very, very high”). Survey research has widely used the ADQ, and it has demonstrated sensitivity to change (e.g., Spirito et al., 2004; Spirito, Simon, et al., 2011; Spirito, Sindelar-Manning, et al., 2011).

2.4.2. Drug Use Questionnaire (DUQ)

Participants reported on the number of days they used substances, including nicotine, marijuana, cocaine, LSD, PCP, inhalants, etc., over the prior 30 days. Test/retest reliability for mean number of days each substance was used has been shown to average 0.83 from 3- to 6-month follow-up and 0.94 from 6 to 12-month follow-up (Spirito et al., 2004).

2.4.3. Timeline Followback (TLFB)

The TLFB (Sobell & Sobell, 1995) is a calendar-assisted daily drinking and drug use estimation method that has also been used to measure alcohol and/or cannabis use in randomized clinical trials with youth (e.g., Waldron, Slesnick, Brody, Turner, & Peterson, 2001). The TLFB has shown satisfactory reliability and validity across numerous studies (see Sobell & Sobell, 2003 for a review).

2.4.4. Child Depression Inventory—2 (CDI-2)

The CDI-2 (Kovacs, 2010) is a 28-item self-report measure of depressive symptoms over the prior two weeks, rated on a 3-point Likert scale. The total raw score ranges from 0 to 56, with higher scores indicative of greater depressive symptom severity. The CDI-2 has demonstrated good reliability and validity in clinical samples (Figueras, Amador-Campos, Gómez-Benito, & del Barrio Gándara, 2010). In the current study, baseline internal consistency was high (α = 0.91).

2.4.5. Screen for Child Anxiety Related Disorders (SCARED)

SCARED (Birmaher et al., 1997, 1999) is a 41-item self-report measure of anxiety, which we used to measure anxiety over the prior three months. All items are rated on a 3-point Likert scale. The total score ranges from 0 to 82, with higher scores indicative of greater anxiety symptom severity. A total score ≥25 may indicate the presence of an anxiety disorder. This measure has demonstrated good internal consistency and discriminant validity (Birmaher et al., 1997, 1999). In the current study, baseline internal consistency was high (α = 0.95).

2.4.6. The Kiddie Schedule for Affective Disorders and Schizophrenia-Present Version (K-SADS-P)

K-SADS-P (Kaufman et al., 1997) is a semistructured diagnostic interview. Clinicians administered it separately to parents and adolescents at baseline. A second clinician rated 20% of all K-SADS interviews and reliability was excellent. There was 100% agreement on six diagnoses, including major depression, while agreement ranged from 89% to 95% for the remaining diagnoses.

2.4.7. The Child Behavior Checklist–6/18 (CBCL)

We used the CBCL (Achenbach & Rescorla, 2001) to measure adolescent externalizing problems at baseline, as rated by a parent, over the prior 6 months. CBCL t-scores of 65–69 indicate borderline levels and 70 or greater indicate clinical levels of externalizing symptoms. Only externalizing behaviors are reported here because parents have been shown to more reliably rate observable than internalizing behavior (De Los Reyes et al., 2015). Externalizing scales have demonstrated excellent internal consistency in treatment-seeking samples (Nakamura, Ebesutani, Bernstein, & Chorpita, 2009; Van Meter et al., 2014).

2.4.8. The Child and Adolescent Services Assessment (CASA)

CASA (Burns, Angold, Magruder-Habib, Costello, & Patrick, 1997) is a clinician-administered interview used to assess service use for mental health problems. Research has demonstrated adequate reliability (ICCs = 0.74–0.76) (Farmer, Angold, Burns, & Costello, 1994). We calculated two scores from the CASA: juvenile justice contacts, including jail or probation involvement; and use of costly services, including inpatient psychiatric admissions, in-home treatment, and emergency department visits. The parent completed the CASA for all mental health services that the adolescent received.

2.5. Analytic strategy

2.5.1. Power

The study was powered for 80 participants per group based on data regarding a reduction of substance use disorders in the pilot study of I-CBT, indicating a minimum detectable difference of 0.604. Unfortunately, due to clinic reorganization, secondary to financial issues in the 4th year of the study, the caseload in the treatment program dropped significantly. The final sample size consisted of 61 participants in the I-CBT and 50 in the TAU conditions. Thus, the focus of study results is on effect sizes and direction of effects rather than statistical significance. When appropriate, we used Cohen’s d as a measure of effect size to improve ease of interpretation of the magnitude of effects. For dichotomous variables, we used Odds Ratios to interpret strength of effect. Positive Cohen’s d for treatment condition comparisons indicates an effect favoring the experimental condition while negative Cohen’s d favor the comparison condition.

We set the significance level at p < .05 for all analyses, and we used Type III tests for all hypotheses. We used means and standard deviations to describe continuous variables, and frequencies to describe categorical variables. In our postbaseline analyses, we examined outcomes at months 3, 6, and 12 in a single analysis. For these longitudinal group comparisons of continuous variables, we used hierarchical linear modeling (HLM) for repeated measures (Raudenbush & Bryk, 2002) since it can accommodate some missing data, covarying the baseline score. We did not covary variables used in the urn randomization (Stout et al., 1994) because preliminary analyses indicated that these had no significant effects on the outcome variables. For categorical variables, we used a parallel set of analyses, employing generalized estimating equations (GEE), co-varying the baseline score of the dependent variable, for longitudinal analyses of months 3, 6, and 12. An odds ratio (OR) >1 indicates a result in favor of the I-CBT intervention.

Due to missing follow-up data, we also conducted analyses using multiple imputation for missing values (Little & Rubin, 2002). In these analyses, we employed 100 imputed samples, and used these to repeat all substance use, psychiatric symptoms, and service use outcome analyses. We assured proper convergence of the imputation process by inspection of diagnostic plots and tests for monotone trends in outcome estimates. We also imputed values for the dichotomous variables, ADQ and DUQ. We imputed missing values only for cases that had a non-missing baseline score and at least one nonmissing score for months 3, 6, and/or 12.

We dichotomized variables if there were high levels of skewness (e.g., heavy drinking days, service use). We calculated intraclass correlation coefficients (ICC) estimates and their 95% confident intervals using SPSS statistical package version 25 (SPSS Inc., Chicago, IL) based on a mean-rating (k = 2), absolute-agreement, 2-way mixed-effects model.

3. Results

3.1. Baseline assessment

As shown in Table 1, there were no differences between treatment conditions on any of the sociodemographic variables. With respect to clinical characteristics of the sample, there were high rates of background risk factors, most notably, maternal and paternal mental health concerns, violence in the home, teen school failure, teen special education placement, frequent family moves, social service involvement, and juvenile justice involvement. However, there were no significant differences between conditions on any of the individual risk factors (Table 2). There was also no difference between the I-CBT (M = 8.59, SD = 3.26) and TAU (M = 8.12, SD = 4.46) conditions on the total number of risk factors, t (1,109) = 0.63, p = .74.

As can be seen in Table 3, marijuana abuse and dependence, as diagnosed using DSM-IV criteria on the K-SADS-P, were the most frequent substance use disorders in the sample, with alcohol abuse and dependence much less frequent. Overall, there were high rates of substance use, internalizing and externalizing disorders in the study sample, with no statistically significant differences between treatment conditions on any K-SADS-P diagnoses, though a trend was evident for conduct problems occurring more frequently in TAU. There was also no statistically significant difference across conditions on externalizing behavior on the CBCL. Notably, neither parental report of severity of externalizing symptoms (as per the CBCL) nor adolescent report of severity of recent depressed mood and anxiety, on average, were in the clinical range (per the CDI-2 and SCARED; see Table 3).

Table 3.

Baseline clinical characteristics by treatment condition.

Variable	EXP		TAU		X 2
	n	%	n	%
KSADS DSM-IV diagnoses
Major depressive episode-current	36	59.02	29	58.00	0.01
Generalized anxiety disorder	19	31.15	18	36.00	0.29
Social anxiety disorder	10	16.39	12	24.0	0.32
Attention deficit hyperactivity disorder	35	57.38	25	51.02	0.44
Oppositional defiant disorder	36	59.02	35	74.47	2.81+
Conduct disorder	30	49.18	32	66.67	3.35+
Alcohol abuse	14	22.95	7	14.58	1.21
Alcohol dependence	8	13.11	9	18.75	0.65
Marijuana abuse	14	22.95	13	27.08	0.25
Marijuana dependence	38	62.30	29	60.42	0.04
Other substance abuse	2	3.28	2	4.17	0.06
Other substance dependence	8	13.11	7	14.58	0.05
Post-traumatic stress disorder	14	22.95	8	16.33	0.75
	EXP		TAU		t
	n	M(SD)	n	M(SD)
CBCL - externalizing problems t score	59	66.98 (7.89)	48	68.65 (9.75)	0.98
Children’s Depression Inventory – 2	61	17.13 (10.21)	49	16.09 (10.04)	0.54
Screen for Child Anxiety-Related Disorders	60	21.53 (17.26)	47	24.21 (16.29)	0.82

Note: EXP = experimental treatment condition; TAU = treatment-as-usual condition; KSADS DSM – IV permits the simultaneous diagnosis of both conduct disorder and oppositional defiant disorder; CBCL = Child Behavior Checklist.

Note: df range from 105 to 108 due to some missing data.

⁺.05 > p < .10; no other comparisons were statistically significant.

There was no statistically significant difference across treatment conditions on psychotropic medication use at baseline (I-CBT = 21%, TAU = 8%), X² (1, n = 111) = 3.75, p = .053, though we found a trend for higher use for I-CBT. There was also a trend toward a statistically significant difference between treatment conditions on psychotropic medication use over the course of treatment, X² (1, n = 106) = 3.64, p = .06. An additional 50% of the I-CBT sample was placed on medication (total = 71%) and an additional 45% of the TAU sample was placed on medication (total = 53%) during treatment. Data were not available on adherence to medication.

3.2. Attrition analyses

To examine attrition at the follow-up points, we created a “missingness” variable across the entire follow-up period. If participants had any nonmissing substance use data at a follow-up time point, we coded them as having nonmissing outcome data; each time point is scored as missing only if all three instruments were missing. A total of 87 participants (78%) had complete substance use data, while 7 participants had (6%) no outcome data at any follow-up. There were 12% of participants missing at the 3-month follow-up, 14% at the 6-month follow-up, and 15% at the 12-month follow-up. Though not statistically significant, there was a trend for the I-CBT condition to have a higher percentage of follow-up interviews than TAU, 85% versus 70%, X² (1, n = 111) = 3.77, p = .052, raising the possibility that the missing at random assumption made in the hierarchical analyses might be questionable; we conducted multiple imputation analyses to determine if results of the main analyses differed after accounting for missing data.

We then examined baseline variables to determine what variables were related to attrition at follow-up. Adolescents in grades 11 and 12 had lower rates of missing data compared to those in grades 8–10, X² (1, n = 111) = 4.10, p = .043. In addition, lower risk factor total scores were related to higher follow-up interview rates, t(108) = 2.06, p = .042.

3.3. Therapist fidelity and competency

3.3.1. I-CBT condition

Four of the authors rated approximately 14% of taped sessions (n = 307) for fidelity and competency, including 16% of teen sessions and 13% of parent/family sessions. This included the first two cases that each therapist saw and a random selection of other tapes. For teens, 83.9% of rated sessions met the competency criterion of a score ≥44 on the CTRS (M = 46.49, SD = 5.62, range = 16–58). For parent/family sessions rated, approximately 73% of rated sessions met this criterion (M = 45.09, SD = 5.92, range = 16–58). Of the teen sessions, 83.8% rated met the 80% criterion on the adherence checklists for fidelity (Mean fidelity score = 88.92%, SD = 13.85%). Of the parent/family sessions, 65% met the 80% criterion (Mean fidelity score = 82.14%, SD = 17.67%).

Of the tapes rated for fidelity and competency, a second rater related approximately 10% (n = 30) to calculate inter-rater reliability. The ICC for adherence/fidelity ratings was 0.85 (95% CI = 0.67–0.93), indicating good reliability. The ICC for CTRS/competency ratings was 0.91 (95% CI = 0.82–0.96), indicating excellent reliability.

3.3.2. TAU condition

Two doctoral psychologists trained in CBT rated tapes of sessions (n = 59) that the three TAU therapists conducted. The TAU therapists attempted to teach a CBT skill to adolescents in slightly more than half of the sessions overall (Therapist 1 = 57%; Therapist 2 = 63%; Therapist 3 = 74%). Of the sessions in which therapists taught skills, the doctoral students rated 59% as fully teaching the skill and 41% as partially teaching the skill. The most commonly taught skill was problem-solving (19% of the time), followed by affect regulation (13%), communication (13%), cognitive restructuring (10%), and contingency management (10%). The three therapists only rarely used any of the other components of the structured protocol used in the I-CBT condition. Specifically, therapists assessed mood at the beginning of the session in 18% of the sessions reviewed, therapists reviewed an agenda with the adolescent in 30% of the sessions, and assigned homework 14% of the time.

3.4. Primary analyses: past month substance use reported on the TLFB

Findings regarding substance use obtained using an interview format with the TLFB are reported here.

3.4.1. Alcohol use days

On the percent days of alcohol use in the past month on the TLFB (see Table 4), there was a nonsignificant difference between conditions across months 3–12, F (1, 97) = 0.02, p = .90 (d = −0.02, 95% CI = −0.35 to 0.31). The time effect was not significant F(2,178) = 1.94, p = .15 (d = −0.12, 95% CI = −0.58 to 0.34); the estimated reduction from months 3–12 in days of alcohol use was 0.94 (or 9.4%). The time by treatment condition interaction was also nonsignificant, F(2,178) = 0.72, p = .49.

Table 4.

Percentage of days in past month that alcohol use, heavy alcohol use, and marijuana use on the Time Line Follow-Back (TLFB) interview by treatment condition.

TLFB variable	Baseline				3 month follow-up				6 month follow-up				1 year follow-up
	EXP		TAU		EXP		TAU		EXP		TAU		EXP		TAU
	n	M (SD)	n	M (SD)	n	M (SD)	n	M (SD)	n	M (SD)	n	M (SD)	n	M (SD)	n	M (SD)
Alcohol use	61	0.07 (0.17)	49	0.09 (0.20)	58	0.05 (0.14)	39	0.04 (0.05)	55	0.02 (0.04)	41	0.03 (0.06)	55	0.04 (0.06)	38	0.05 (0.12)
Heavy alcohol use	61	0.06 (0.17)	49	0.08 (0.20)	58	0.01 (0.04)	39	0.02 (0.04)	55	0.01 (0.02)	41	0.02 (0.05)	55	0.02 (0.04)	38	0.04 (0.11)
Marijuana use	61	0.36 (0.38)	49	0.39 (0.38)	58	0.24 (0.34)	39	0.39 (0.44)	55	0.28 (0.37)	41	0.40 (0.42)	55	0.43 (0.42)	38	0.42 (0.41)

EXP: experimental treatment condition; TAU: treatment-as-usual condition; M = unadjusted mean; SD = standard deviation.

3.4.2. Heavy alcohol use days

On the percent days of heavy alcohol use in the past month on the TLFB (see Table 4; Fig. 1), there was a nonsignificant difference between treatment conditions across months 3–12, F (1,97) = 2.92, p = .09, and a nonsignificant time by treatment condition interaction, F(2,178) = 0.40, p = .67. The overall between condition effect size estimate was d = +0.41, 95% CI = −0.07 to 0.88, consistent with a small-to-medium effect favoring the I-CBT condition. The estimated reduction from months 3 to 12 in days of heavy alcohol use was 1.21 (or 12.1%), F(2,178) = 2.13, p = .12, (d = −0.34, 95% CI = −0.80 to 0.13), consistent with a small-to-medium effect.

Fig. 1.

Mean percent of days of heavy drinking, from TLFB.

3.4.3. Marijuana use days

On the percent of days of marijuana use in the past month on the TLFB (see Table 4: see Fig. 2), there was no significant difference between treatment conditions, F (1,97) = 1.55, p = .22, nor was there a statistically significant time by treatment condition interaction, F (2,178) = 2.39, p = .09. The overall between treatment condition effect size was d = +0.28, 95% CI = −0.17 to 0.74, consistent with a small effect in favor of the I-CBT condition. Our examination of adjusted means indicated that there was an effect size of d = +0.40 at the 6-month follow-up in favor of the I-CBT condition, which was no longer evident at the 12-month follow-up. The estimated reduction from months 3 to 12 in days of marijuana use was 0.31 (or 3.1%), F(2,178) = 2.74, p = .07 (d = −0.01, 95% CI = −0.47 to 0.45).

Fig. 2.

Mean percent of days of marijuana use, from TLFB.

3.5. Primary analyses

Past three-month substance use, as assessed via self-report questionnaires, is reported here.

3.5.1. Number of drinking days

On the number of drinking days in the past 3 months on the ADQ analyzed as a categorical outcome by GEE (see Table 5), there was no significant difference between treatment conditions across months 3–12, X²(1) = 0.24, p = .63 (OR = 1.17, 95% CI = 0.64 to 2.14). The time by treatment condition interaction was not significant, X²(2) = 2.40, p = .30. The main effect for time was not significant, X²(2) = 2.39, p = .30.

Table 5.

Adolescent self-report of past 3 month alcohol use, heavy drinking, marijuana use, and other drug use days by treatment condition.

Variable	Baseline				3 month follow-up				6 month follow-up				1 year follow-up
	EXP		TAU		EXP		TAU		EXP		TAU		EXP		TAU
	n	%	n	%	n	%	n	%	n	%	n	%	n	%	n	%
ADQ - alcohol use days
Did not drink in past 3 months	26	42.62%	13	26.53%	26	49.06%	12	34.29%	32	65.31%	23	57.50%	30	54.55%	19	51.35%
Less than monthly	11	18.03%	14	28.57%	6	11.32%	7	20.00%	7	14.29%	5	12.50%	8	14.55%	10	27.03%
Monthly	11	18.03%	3	6.12%	5	9.43%	2	5.71%	3	6.12%	4	10.00%	3	5.45%	2	5.41%
Less than once per week, more than once per month	4	6.56%	7	14.29%	6	11.32%	4	11.43%	5	10.20%	4	10.00%	9	16.36%	4	10.81%
Weekly	2	3.28%	5	10.20%	3	5.66%	4	11.43%	0	0.00%	1	2.50%	4	7.27%	1	2.70%
2–3 days per week	5	8.20%	6	12.24%	5	9.43%	4	11.43%	2	4.08%	3	7.50%	1	1.82%	1	2.70%
4–6 days per week	0	0.00%	0	0.00%	1	1.89%	2	5.71%	0	0.00%	0	0.00%	0	0.00%	0	0.00%
Every day	2	3.28%	1	2.04%	1	1.89%	0	0.00%	0	0.00%	0	0.00%	0	0.00%	0	0.00%
ADQ - any heavy drinking days
Yes	29	47.52%	25	51.02%	16	30.77%	16	45.71%	13	26.53%	14	35.00%	19	35.19%	12	32.43%
DUQ – any other drug use days
Yes	17	28.33%	17	34.69%	9	16.98%	7	20.00%	2	4.35%	5	13.16%	7	13.73%	5	13.51%
	EXP		TAU		EXP		TAU		EXP		TAU		EXP		TAU
	n	M (SD)	n	M (SD)	n	M (SD)	n	M (SD)	n	M (SD)	n	M (SD)	n	M (SD)	n	M (SD)
DUQ - marijuana use # of days	61	37.39 (35.11)	49	35.08 (30.96)	54	27.17 (32.48)	35	33.34 (32.85)	49	24.57 (31.10)	40	39.50 (35.19)	55	36.55 (37.81)	37	42.57 (36.40)

EXP: experimental treatment condition; TAU: treatment-as-usual condition; ADQ = Adolescent Drinking Questionnaire; DUQ = Drug Use Questionnaire. M = unadjusted mean; SD = standard deviation.

3.5.2. Any heavy drinking

We used a logistic GEE model to analyze the effect of treatment on a binary outcome of any heavy drinking episodes across months 3–12. There was no significant difference between treatment conditions, X²(1) = 0.036, p = .85 (OR = 1.41, 95% CI = 0.73 to 2.73), suggesting a small effect favoring I-CBT. There was not a statistically significant time by treatment interaction X²(2) = 2.60, p = .27. The main effect for time was not significant, X²(2) = 1.42, p = .49 (Fig. 3).

Fig. 3.

Mean percent of participants drinking heavily, from ADQ.

3.5.3. Number of marijuana use days

On the number of days of marijuana use in the past 3 months on the DUQ (see Table 5), an HLM analysis indicated that there was not a significant difference between treatment conditions across months 3–12, F (1,99) = 3.16, p = .08, though the small-to-medium effect favored the I-CBT condition (d = + 0.41, 95% CI = −0.05 to 0.87). The time by treatment condition interaction was not significant, F(2,164) = 0.56, p = .57. The main effect for time was not significant, F(2,164) = 2.53, p = .08.

3.5.4. Other drug use

We used a logistic GEE analysis to analyze the effect of treatment on the use of a binary score of any use vs. no use of any other drug across months 3–12. We did not find effect of treatment condition to be statistically significant, X²(1) = 0.004, p = .95 (OR = 1.04, 95% CI = 0.30 to 3.60), consistent with little, if any, effect. The time by treatment interaction was also not significant, X²(2) = 2.22, p = .33. The main effect for time was not significant, X²(2) = 4.17, p = .12 (see Table 5).

3.6. Secondary analyses

3.6.1. Mood and behavior ratings

Clinicians administered the CBCL at only the 6- and 12-month follow-up periods. HLM analyses did not indicate a significant treatment condition main effect on the externalizing subscale, F(1,92) = 1.02, p = .31, d = +0.29, CI = −0.28 to 0.96, but there was a small effect favoring the I-CBT condition. The effect of time was not significant, F (1,80) = 0.33, p = .57, d = −0.23, CI = −0.24 to 0.69, but suggested a small decrease in externalizing symptoms over time. The time by treatment condition interaction, F(1,80) = 0.72, p = .40, was also not significant.

HLM analyses found that there was no significant treatment condition main effect on the CDI-2 over months 3–12, F (1,99) = 1.86, p = .18, d = +0.29, CI = −0.13 to 0.71, but there was a small effect favoring the I-CBT treatment condition. There were slightly greater average scores in E-TAU than I-CBT, 12.2 versus 10.8 adjusted mean scores. There was a significant time effect, F(2,168) = 4.63, p = .011, consistent with a small decrease in depressive symptoms over time. The time by treatment condition interaction, F(2,168) = 0.22, p = .80 was not significant.

HLM analyses found that there was no significant treatment condition main effect on the SCARED over months 3–12, F(1,95) = 1.51, p = .22, d = +0.21, CI = −0.13 to 0.55, but there was a small effect favoring the I-CBT treatment condition. These findings suggest slightly greater average SCARED scores in E-TAU than I-CBT, 22.8 versus 20.1 adjusted mean scores. There was no significant effect for time F (2,154) = 0.20, p = .82. The time by treatment condition interaction was not significant, F (2,154) = 0.20, p = .82.

3.6.2. Service use

We conducted a series of logistic GEE analyses to examine binary use vs. nonuse measures of three categories of service use on the CASA, covarying the baseline score for each.

Across months 3–12, the effect of treatment on juvenile justice contacts was statistically significant, X²(1) = 6.66, p = .036, with a medium-to-large effect in favor of the I-CBT condition (OR = 3.53, 95% CI = 1.52 to 8.18). There was not a statistically significant difference for time [X²(2) = 1.383, p = .50 (OR = 1.62, 95% CI = 0.72 to 3.63)], but the findings indicate a small reduction in juvenile justice contacts over time. The time by treatment interaction, X² (2) = 1.63, p = .44, was not significant.

The effect of treatment on the rate of CASA costly services was complex. First, there was a statistically significant treatment main effect across months 3–12, X²(1) = 5.67, p = .034, (OR = 0.64, 95% CI = 0.29 to 1.44), with a marginally higher adjusted mean rate of use for I-CBT of 0.85 vs. TAU at 0.78. (Note: Higher odds ratios here indicate lower costs for I-CBT, a more favorable clinical outcome). However, this overall mean difference must be interpreted in the context of a strong time by treatment interaction, X²(2) = 16.11, p = .0003. We examined simple main effects at three time points. At month 3 there was no significant treatment effect but the I-CBT condition had a slightly higher estimated mean, X²(1) = 1.32, p = .25, OR = 0.36, 95% CI: 0.06 to 2.05. By month 6, a significant treatment effect emerged, with the I-CBT group having a higher rate of use of costly services, X²(1) = 7.13, p = .0076, OR = 0.28, 95% CI: 0.11 to 0.71. However, at month 12, a treatment effect in the opposite direction emerged, with TAU reporting more costly services, X²(1) = 4.50, p = .034, OR = 1.55, 95% CI: 1.44 to 3.48.

3.7. Multiple imputation analyses

Multiple imputation analyses resulted in somewhat smaller effect sizes for most of the comparisons studied, though not by enough to suggest revisions in the conclusions regarding treatment effects. Furthermore, due to the potential for bias in our results because of covariates potentially related to missing data, we also conducted a set of analyses focused on baseline correlates of missing data. We found two baseline variables to be related to missingness at p < .05 (not corrected for multiple comparisons) in preliminary analyses: grade in school and RFS total score. However, when we included these variables in analyses of our primary TLFB outcomes along with treatment effects, none of the three significantly predicted any of the outcomes, nor did the inclusion of these covariates materially affect any main effect or treatment by time conclusions.

4. Discussion

The purpose of the current study was to test whether staff employed in a community agency in the context of a randomized pilot trial could implement a multicomponent cognitive behavioral treatment developed for adolescents with substance misuse (alcohol and/or marijuana) and comorbid psychiatric symptoms. Our data on fidelity and competency indicate that we had reasonable success in training clinicians in a community clinic to use the I-CBT protocol. The hypotheses that we tested in this study were that the I-CBT protocol—in which clinicians received an intensive day-long training followed by weekly supervision—would result in greater reductions in adolescent alcohol use, marijuana use, and drug use (the primary aim), and greater reductions in psychiatric symptoms and service use (the secondary aim), than would a “CBT exposed” protocol, in which clinicians received standard community clinic continuing education; i.e., a TAU condition. However, difficulty with recruitment compromised our power to detect statistically significant effects. Thus, the focus of this study is on effect sizes and direction of effects rather than statistical significance. Consequently, the findings must be considered preliminary. Overall, the results on substance use and psychiatric symptoms favored the I-CBT condition over E-TAU, with small-to-medium effects, both between conditions and across time, but they were nonsignificant. Though our primary hypotheses were not statistically significant, there was a small reduction in the percent days of heavy drinking (12.1%) and marijuana use days (3.1%) overall for both conditions across the three follow-up points but that favored the I-CBT condition. There was no effect on days of alcohol use. Though reported alcohol use was low in the sample as a whole, findings regarding heavy drinking days can be extrapolated to approximately one versus two and one-half heavy drinking episodes per year in favor of I-CBT. Given the potential negative consequences associated with heavy drinking, the reduction in one episode is potentially important from clinical and public health perspectives. Notably, small-to-medium effects are also commonly found in other well-established treatments for youth substance use (Hogue et al., 2014, 2018). Specifically, group and individualized CBT show a moderate effect size in treating marijuana use in efficacy trials (Bender, Tripodi, Sarteschi, & Vaughn, 2011; Hendriks, van der Schee, & Blanken, 2011), and several meta-analyses of family-based therapies have found a small effect size for these approaches compared to TAU and other manualized treatments (Baldwin, Christian, Berkeljon, & Shadish, 2012; Tanner-Smith, Wilson, & Lipsey, 2013). These results should also be considered in light of the fact that clinicians administered random urine drug screens to all enrolled adolescents during the course of treatment, which was typically about 3 months for most participants. Thus, any effects that we found at 3-month follow-up were above and beyond those that may have been achieved via the use of drug screens.

Findings related to the intervention on other drug use were not significant. Drug use was very low in the sample as a whole, making it difficult to find a significant result at follow-up. We had also hypothesized that I-CBT, designed to address both mood/behavior an6d substance use problems, would have stronger effects on psychiatric symptoms. However, given that we required participants to use substances to be enrolled in the treatment program, substance use was the primary focus of treatment for most families. There was a small, but nonsignificant, positive effect over time on externalizing symptoms, depressed mood, and anxiety, favoring the I-CBT condition. With insurance dictating treatment length and intensity, clinicians may not have had adequate time to dedicate to co-occurring psychiatric symptoms to achieve a larger effect. This effect may also be due in part to the fact that self-reported psychiatric symptom severity ratings were lower than anticipated in the current sample, despite relatively high rates of psychiatric disorders at baseline. With less severity and range in psychiatric symptoms, it is not surprising that we did not find larger effect sizes. Nonetheless, these findings are consistent with results of previous trials that show a minimal effect of substance focused treatments on symptoms of internalizing disorders (Hersh, Curry, & Kaminer, 2014).

The strongest findings favoring the I-CBT condition were related to juvenile justice contacts and additional psychiatric service use. There was a significant and large effect on juvenile justice contacts; adolescents in the I-CBT condition were less likely to have juvenile justice contacts over the 12-month follow-up period than those in TAU. Though youth in TAU had marginally higher rates of conduct disorder, there were equal rates of youth on probation and mandated to treatment by the courts across conditions at baseline. The I-CBT protocol emphasizes work with parents that is not typical of community treatment (Spirito, Simon, et al., 2011), which may account for this finding: more work with parents on behavioral management of their adolescent, even if imperfect in the I-CBT condition (as noted, with respect to fidelity and competency), may have reduced adolescent behavior that would have increased encounters with the juvenile justice system compared to TAU. I-CBT also impacted use of costly psychiatric services (e.g., in home and/or crisis related services), which were more common in the I-CBT condition at the 6-month follow-up but less common at 12 months. These findings may indicate that many families across conditions needed more than the typical three months of treatment that insurance companies commonly allot. This is not surprising given the very high number of risk factors reported at baseline in this sample. However, clients and/or therapists in the I-CBT condition may have recognized when clients needed additional services and sought them more quickly than those in E-TAU, which resulted in less need for service utilization in the longer term.

4.1. Limitations

This study has several limitations. First, the results should be considered preliminary due to difficulties that we encountered in recruitment because of administrative changes in the community clinic. Due to limited power and the importance of minimizing Type II error in this pilot trial, we did not apply statistical correction to data analyses. Therefore, the potential of Type I error limits our study results. In addition, the follow-up rates were lower in the TAU condition than the I-CBT condition, which may have affected results. Since the number of risk factors at baseline was negatively related to follow-up interview completion, it is possible that substance use and other outcomes in favor of the I-CBT condition may have been stronger if we were able to complete interviews with TAU participants lost to follow-up. Second, the different substance use assessment approaches, self-report and interview, resulted in somewhat different findings. Differential findings may have been related to response bias of the assessment method (interview versus self-report) and also recall time period (1 versus 3 months). We intentionally limited the TLFB to one month to minimize recall difficulties. Despite some differences, the pattern of results was similar regardless of assessment method and time period. The restricted range of alcohol use, regardless of assessment approach, had a greater effect on the ability to detect differences. Third, though exposing therapists and the on-site clinic supervisor in the TAU arm to a brief continuing education–style CBT workshop that mimics what commonly happens in community care settings, it did lead to some contamination of the I-CBT treatment protocol across conditions, thus potentially attenuating I-CBT treatment effects. Of the randomly selected tapes reviewed, though TAU therapists rarely followed the structure of the CBT sessions (i.e., mood check-in, agenda setting, homework assignment, etc.), they did attempt to teach a core CBT skill in more than half their sessions with 59% rated as fully and 41% rated as partially adherent. Fourth, we considered evaluating how various clinician-level factors related to treatment outcome, but the small number of clinicians did not allow for stable comparisons of clinician-level variables. Fifth, we considered randomly assigning therapists to conditions. However, our experience supervising community clinicians indicated that clinician beliefs regarding implementing a manualized protocol can have a large effect on the quality of the treatment they deliver. A substantial body of literature, which found that large therapist effects reduce the ability to find treatment outcome differences across different therapies, supports this practical experience (Miller, 2010). The first two clinic therapists for the I-CBT condition expressed an interest in learning a structured protocol, but due to staff turnover, clinical administrators assigned the last two therapists to the I-CBT condition. Consequently, therapist ability varied across the trained therapists. Sixth, a large proportion of adolescents in both conditions were taking medication so we cannot rule out the role of medication in treatment outcomes. We did not control medication prescribing in the study, nor did clinicians collect any clinical data on adherence, so interpreting the effects of medications is challenging. Another limitation of our study is that our fidelity was weaker for the parent and family sessions than for the adolescent sessions. The manual and training program may need to be restructured for community clinicians to achieve higher fidelity and quality of parent training and family sessions since fidelity has been shown to be correlated with outcomes in a number of community studies (e.g., Henggeler et al., 1999). Future trials should enhance fidelity within these sessions, and strengthening the training protocol (e.g., lengthening the training period, incorporating more role plays and feedback for parent training and family sessions) and/or supplementing explicit protocols designed to improve parent-adolescent relationships and family communication could enhance the current intervention. We should also note that there were somewhat higher proportions of adolescents with conduct problems in the TAU condition, which may have affected our results. And, finally, the adolescents in this study had high rates of substance use and psychiatric disorders, and they came from families with a large number of risk factors for impaired functioning. Consequently, the findings reported here may not be generalizable to other samples.

5. Conclusion

This study generated some preliminary data suggesting that, overall, I-CBT relative to TAU resulted in small-to-medium, but non statistically significant, effects on percent days of heavy drinking, marijuana use days, externalizing symptoms, depressed mood, and anxiety. Reductions in these outcomes at the three follow-up points overall favored the I-CBT condition. There was no effect on days of alcohol use or other drug use. The strongest findings favoring the I-CBT condition were related to reduced juvenile justice contacts.

By using masters-level therapists treating patients with a wide range of psychopathology and substance use, this study tested the effect of the I-CBT protocol under real world conditions. The level of impairment that parents reported and the adverse socioeconomic conditions that most families experienced made working with parents in community clinics that much more challenging for therapists and may suggest the need for more intensive training and supervision for therapists working with parents of teens, as well as treatment for parents’ own symptomatology. We added a more structured supervision component than is found in typical community care because studies transporting evidence-based treatments (EBTs) to the community have found that poor treatment adherence decreases efficacy. With community clinics being encouraged to adopt EBTs, more easily digestible treatment protocols, coupled with more intensive, innovative and context-responsive supervisory approaches and retraining procedures, may be required to produce stronger outcomes when transferring I-CBT, and similar structured protocols, to community clinics. When working with community clinicians, supervisors may need to place a greater emphasis on the clinician’s working with parents than we did in the current study. Also, after treatment ended, adolescents in both conditions tended to gradually increase their use of substances. Regularly scheduled booster sessions with these families for an extended period following completion of treatment may also be necessary to maintain any positive effects initially obtained in treatment.