Motivational Interviewing with and without Normative Feedback for Adolescents with Substance Use Problems: A Preliminary Study

Abstract

Background

Many adolescents in need of substance use disorder treatments never engage in treatment. Further, the most promising interventions that could be adapted to target treatment engagement often use normative feedback (NF) despite concerns about its appropriateness for adolescents. This preliminary study will inform a larger trial designed to isolate whether NF is an inert, helpful, or harmful active ingredient within pre-treatment Motivational Interviewing (MI) interventions designed to increase treatment engagement.

Methods

Adolescents (n=48) presenting for treatment intake assessments were randomized to receive MI (n=22) or MI +NF (n=26) immediately following their assessments. Three-month outcomes included the percentage of youth engaged in treatment, the percentage of youth reporting past month binge drinking, and the percentage of days of abstinence.

Results

Treatments were delivered with high fidelity, and a high proportion of eligible participants were recruited and retained in this study. Participants significantly increased their percentage of days of abstinence by approximately 10% at follow up (d=.32, p =.03), with no significant differences between groups. Fifty-five percent of youth in MI and 41.7% of youth in MI+NF engaged in treatment (OR=.60, ns, 95% CI [.136 – 2.68]).

Conclusions

Larger trials should test whether NF is an active ingredient in adolescent MI interventions, and also determine the mechanisms through which MI+NF may produce effects.

INTRODUCTION

Over the past 25 years, in nationally representative surveys, only 6–10% of adolescents in need of substance use disorder (SUD) treatments report receiving such treatments.¹ Adolescents referred for SUD assessments at public-not-for profit settings are a high risk population with multiple comorbid conditions,² and frequently do not engage in SUD treatments. Only 50 to 60% of these adolescents are successfully engaged in SUD outpatient treatments, defined as attendance of one to three post-assessment treatment sessions.^3–5 Studies find even lower estimates of treatment engagement (40%) when using statewide agency records.⁶ This could have potentially devastating effects, as treatment engagement predicts decreased odds of using alcohol (OR = .63) at six months.⁷ Thus, additional research on adolescent treatment engagement is critical given that some researchers have found that failing to engage adolescents with SUDs in treatment may extend their substance use by many years⁸ and expose them to multiple public health risks. That is, adolescent alcohol use predicts higher blood pressure and obesity in young adulthood,⁹ the risk of contracting a sexually transmitted disease or HIV,^10,11 the risk of a premature and accidental death,¹² involvement in crime and interpersonal violence,^13,14 and increased risks for depression during emerging adulthood.¹⁵

As little is known about how clinicians engage adolescents in SUD treatments during initial assessments, we designed and tested an intervention targeting adolescent treatment engagement. The intervention, CHOICE (Compassionate Helpers Openly Inviting Client Empowerment), is based on Motivational Interviewing (MI),¹⁶ an empirically-supported approach that targets client ambivalence about behavior change.¹⁷

The goals of this preliminary study were to 1) obtain data on the feasibility of conducting a larger trial (i.e., recruitment, follow-up retention, fidelity), and 2) report treatment engagement and substance use outcomes for youth receiving two different versions of MI. By randomly assigning youth to two versions of MI, one with and one without normative feedback (NF; see below), we obtain preliminary data on whether or not NF is an active ingredient, or an integral component of the MI intervention. Studies using this methodology, called dismantling designs, are very rare with adolescents ¹⁸. It is imperative that more dismantling studies are conducted with adolescents with SUDs in order to identify which intervention components are inert and which drive change. This research will guide intervention refinements.¹⁸ In the remainder of this section we discuss the limited research on adolescent treatment engagement, as well as the importance of isolating whether Normative Feedback is an active ingredient in MI-based models.

Adolescent Treatment Engagement Research

We are only aware of four randomized and two quasi-experimental studies using adolescent substance use treatment engagement as a primary outcome. These studies have used motivational phone call reminders¹⁹, conjoint ^20,21 and unilateral family therapy, ²² streamlined admissions procedures,²³ and Motivational Interviewing.³ One study found increased treatment engagement attendance for those adolescents who received motivational phone calls versus treatment as usual¹⁹. Studies utilizing family therapies (e.g. Strategic Structural Systems Engagement and Community Reinforcement and Family Training (CRAFT)) found that adolescents assigned to these interventions were more likely to engage in treatment than those who were assigned to treatment as usual conditions.^20,21 Interestingly, one study²³ found that using a streamlined admissions process (e.g. same day assessment, same day treatment assignment, and a set of admission tracking procedures) did not result in more treatment engagement than the assessment as usual.

Motivational Interviewing, a client-centered and directive intervention designed to reduce ambivalence about change, may be an especially promising intervention to encourage adolescents to attend treatment because it encourages adolescents to articulate the potential benefits of change and respects individuals’ autonomy.^3,24–26 Only one non-controlled process study provided MI immediately following an initial assessment, and it found higher treatment engagement for adolescents at higher levels of MI treatment integrity.³ In this study, MI was conceptualized as something that occurred before specialized SUD treatment started, whereas in all other adolescent studies, the MI session was the first treatment session. Although MI has extensive empirical support for use with adolescents, there is very limited research on using MI to target treatment engagement or overall session attendance.^25,27,28

Normative Feedback: Description and Use with Adolescents

Normative feedback interventions contrast an individual’s substance use against that of a large reference sample in an attempt to quantify ones’ substance use in relation to others’, correct misperceptions of how frequently other individuals use substances, and trigger reductions in use. Normative feedback interventions are empirically supported for reducing college student drinking, and are often used in conjunction with MI.^29,30 Studies show that norm correction--altering the erroneous belief that one’s personal use is no different from that found in the general population--is one process through which the intervention affects use.³⁰

Extending this research on NF to adolescents is important for two reasons. First, it is commonly assumed that NF is beneficial for adolescents, despite limited testing and multiple developmental differences between adolescents and college-age populations. Adolescents represent a substantially different population than college students developmentally,^31,32 with different patterns of treatment response ³³ and different reasons for quitting³⁴ relative to college-age populations. So, some of the mechanisms of change that are assumed to occur with older drinkers (i.e., norm correction) may not apply to adolescents. In short, extending the generalizability of NF research to adolescents is warranted given the myriad developmental ³⁵, social ³⁶ and executive functioning³⁷ differences between adolescents and college-age individuals.

Second, anecdotal concerns suggest that adolescents may react more negatively to normative feedback due to their higher psychological reactance compared to adults.²⁵ For example, researchers have suggested that adolescents may be skeptical about NF, questioning its accuracy.³⁸ These anecdotal concerns and the lack of efficacy data have prompted some researchers to exclude NF from their MI interventions.²⁷ One major concern is that if adolescents do react negatively to NF, it will take the form of personalized counterarguments against the feedback, or sustain talk, which has been shown to predict poorer outcomes for adolescents.³⁹ It is possible that if contrasts between personal use and data from normative samples are extreme, adolescents may be especially prone to engaging in sustain talk. For example, the author of the largest adolescent study of MI+NF plus cognitive behavioral therapy⁴⁰ noted that adolescent substance use was often in the 99^th percentile among youth referred to treatment (Personal Communication, Michael L. Dennis, November 7^th, 2013). These severe contrasts may be unpalatable for adolescents and result in (avoidable) arguments with therapists that could suppress treatment outcomes. Thus, a dismantling study that tests whether NF is an active, inert, or harmful ingredient has important implications for refining efficacious MI interventions that are delivered to adolescents.²⁵

Summary & Hypotheses

In summary, lack of treatment engagement is a widespread problem in the adolescent substance use disorder treatment field. In this study, our goal was to determine whether NF is an active ingredient within an adaptation of MI designed to increase treatment engagement. Based on prior anecdotal concerns described above about using NF with adolescents, we hypothesized that the MI+NF condition would have lower treatment engagement and a lower percentage of days of abstinence relative to the MI condition. Finally, we hypothesized that treatment engagement would be positively associated with the percentage of days of abstinence at the three-month follow-up.

METHODS

Human Subjects Protection

All procedures were approved by the University of Illinois at Urbana-Champaign’s institutional review board, and this is a registered clinical trial (ClinicalTrials.gov Identifier: NCT02083523). A parental consent waiver was granted for situations where staff documented that collecting it would put teens at risk or teens would opt out if parental consent was required. Otherwise, all participants assented to treatment and research procedures, and parents consented to youth participation.

Study Design and Setting

The feasibility of delivering two versions of CHOICE and its preliminary efficacy were investigated among adolescents in a two-arm randomized controlled trial. Participants (n=48) at two not-for profit treatment agencies in Chicago and Urbana, IL were randomly assigned to receive MI+NF (n = 26) or MI (n = 22). Data were collected in-person, using the Global Appraisal of Individual Needs-Quick (GAIN-Q3; version GQ.3.2.0)⁴¹ at baseline (i.e., treatment intake assessment) and then again three months following the baseline assessment. Whereas a prior report from this study investigated whether MI or MI+NF impacted post-session readiness to change,⁵ this study focused on three-month findings.

Participants

Participants (n = 48) were adolescents referred for SUD assessments at two not-for-profit treatment clinics in the U.S. Midwest (n_site1=23, n_site2=25) from March, 2013 through August, 2013. At both sites adolescents were randomized to either receive MI+NF or MI only. Sites were state-funded, not-for-profit outpatient substance use treatment agencies that provide drug and alcohol assessments and treatment for adolescents referred by the juvenile justice system, schools, and parents. Adolescents were eligible for the study if they were 13–19 years of age, spoke English, and either scored 2+ on the CRAFFT ⁴² or reported 13+ days of substance use in the past 90 days. The CRAFFT (Car, Relax, Alone, Forget, Family or Friends, Trouble) is a validated screener with high sensitivity and specificity in predicting the presence of SUDs in adolescents at a cutoff of two or higher. Participants received 10$ US for completing the initial assessment and 20$ US for the three-month follow-up assessment. In total, each participant could earn 30$ US. Adolescents were excluded if they met the Common Rule (45 CFR § 46) definition of being a prisoner at the time of recruitment, reported a diagnosis of schizophrenia, or exhibited signs of cognitive impairment (i.e., scored 10+ on the Global Appraisal of Individual Needs’ (GAIN; Cognitive Impairment Scale).⁴³

Procedures

Prior to the start of the study, participant identification (ID) numbers were pre-randomized by a research staff member with no participant contact who randomly drew half of all ID numbers and assigned them to the intervention conditions prior to the start of the study. These ID numbers were then assigned to participants (in exact chronological order) consecutively as they were screened and enrolled in the study (see below).

All adolescents referred for SUD assessments at participating clinics were screened to determine if they met the study inclusion criteria. If so, the study was described, and they were encouraged to meet with a research staff member before their initial assessment appointment (i.e., baseline assessment). Adolescents (and parents when available) then met with a research staff member who reviewed informed consents, answered questions about study procedures, and collected assent and consent. Participants also provided their contact information in order to facilitate completion of three-month follow-up assessments. Finally, once participants assented to the study they scheduled baseline assessment appointments.

Participants completed the 45 minute GAIN-Q3 assessment, which assesses a range of psychological, substance use, and social aspects of the participant’s life (described below). Immediately following the assessment, an opaque envelope that revealed their pre-randomized treatment condition was opened. The staff member then proceeded to complete either MI+ NF or MI. After receiving MI or MI+NF, participants could decide whether to attend any post-intake assessment SUD treatment sessions.

Research assistants, who were blind to condition assignment, met with participants to administer the GAIN-Q3 three months after the baseline assessment. All GAIN-Q3 administrators were trained and monitored, which involved attending a didactic training, receiving feedback on audio taped interviews until certification was achieved, and receiving periodic fidelity checks on data collection procedures.⁴¹

Conditions

Both interventions (i.e., MI or MI + NF) were facilitated by a computer-generated report and included: a brief orientation of what would occur in the session, discussion of the participants’ strengths, an agenda setting procedure that prioritized discussion topics and gave the participant choice about what to discuss first, a review of concerns and referral suggestions, and a session summary. Sessions are designed to take approximately 30–45 minutes. Though structured, therapists use MI throughout by conveying empathy, emphasizing reflections and open questions, and eliciting and reinforcing change talk about the target behaviors.¹⁷

Participants in the MI+NF condition also received NF, during which the therapist contrasted the participants’ days of alcohol and marijuana use with U.S. norms (N=35,191) available from Chestnut Health Systems (CHS),⁴⁴ the organization that developed the GAIN and GAIN-Q3. CHS has served as a data clearinghouse for multiple studies using these instruments. Because the goal was to increase treatment engagement, we chose to contrast the participants’ substance use with adolescent treatment sample norms, rather than general population norms. Age specific norms and those for youth admitted to different levels of care (e.g., residential, outpatient) were available, and therapists used whichever norm provided a greater contrast with participants’ use.

Therapist characteristics and training

Five therapists provided both versions of the intervention (e.g. MI+NF or MI only). Therapists were mostly female (80%) and master’s level clinicians (80%). They had an average of 5.7 years (SD = 5.4) of counseling experience, with 3.1 years (SD = 2.3) specific to treating SUDs. Therapists received a 12-hour interactive MI training, completed mock role plays that were reviewed for adherence prior to enrollment, and continued to receive feedback on session tapes during the study period (M = 3.0; SD = 1.0, range: 2–4). Fifteen tapes (31.2%) were reviewed by the lead author using the Motivational Interviewing Treatment Integrity Scales (MITI, 3.1.1),⁴⁵ a reliable coding instrument for establishing MI fidelity.⁴⁶

Performance indices on the MITI included the ratio of reflections to questions (2:1 is recommended), the ratio of MI Adherent behaviors to total MI Adherent and Non-adherent behaviors (100% adherent is recommended), a global spirit rating that averaged mean ratings on five-point Likert scales of Autonomy Support, Collaboration, and Evocation (4.0 or higher is recommended), the percentage of open questions out of the sum of all open and closed questions (70% open is recommended), and the percentage of complex reflections out of all reflections (1:1 is recommended).

Measures

GAIN-Q3

The GAIN-Q3 (Version Q3 2.0_MI) is a 30–45 minute semi-structured assessment designed to assess a wide range of life problems among adolescents such as substance use problems, school problems, and psychological problems (see GAIN Q3 manual for more detailed information), based on the reliable and valid parent instrument called the GAIN I.^47–49 It addresses multiple adolescent life domains such as school problems, work problems, physical health, sources of stress, risk behaviors (i.e. sexual risks, needle use), mental health, substance use, and crime and violence. All dependent variables described below were derived from the GAIN-Q3.

PERCENT DAYS ABSTINENT (PDA)

A single-item PDA measure was derived by subtracting the number of days of substance use (alcohol and other illicit drugs) in the 90 days prior to the assessment while residing in the community (e.g. not housed in jail, prison, or inpatient hospital) from 90, and then dividing the days of abstinence by 90 days to obtain the proportional measure.

ANY BINGE DRINKING (ABD)

To measure binge drinking, we used a single, self-reported item where youth reported the number of days in the past 90 days on which they had consumed four (females) or five (males) drinks on a single occasion. Because responses to this continuously-scaled item were highly skewed, we dichotomized this variable to indicate if the youth had reported any binge drinking (ABD) in the past 90 days (0 = no, 1 = yes).

PERCENT ENTERING TREATMENT (PET)

To measure PET, at three-months post intake assessment we asked youth on how many days they received any substance use treatment sessions (e.g. any professionally-led individual, group or family counseling sessions for SUD) out of the past 90 days. As in other studies⁴ treatment engagement was defined as attending a minimum of three post-assessment treatment sessions, resulting in a dichotomized PET measure (0 = not engaged, 1 = engaged).

Data Analysis

Descriptive analyses focusing on study feasibility issues examined the percentage of eligible participants recruited into the study, the percentage of participants retained for research assessments, average fidelity of MI and MI+NF sessions, and the typical session length. These are all important indices for determining the feasibility of conducting a larger clinical trial.⁵⁰

Additionally, independent samples t-tests and chi-square analyses were used to determine whether the participants differed across conditions on numerous demographic, substance use, and psychosocial variables at baseline. We examined whether PDA increased among the entire sample from baseline to three months by using paired samples t-tests. To test group differences on PDA (a normally distributed continuous variable), we regressed condition (0 = MI, 1 = MI+NF) on three-month PDA, while controlling for baseline values of PDA and site (0 = Chicago, 1 = Urbana), and condition. Finally, for the dichotomous PET and ABD variables, we used separate logistic regression models, controlling for site, condition, and baseline values of the dependent variables for each model. All analyses were intent to treat analyses, including participants regardless of their exposure to the intervention.

RESULTS

Participant Characteristics

Study participants were mostly male (77.1%), were racially diverse (36.2% Bi- or multi-racial, 36.2% African-American, 23.4% White, non-Hispanic 2.1% Hispanic/Latino, 2.1% Other), and, on average, were 16.3 years of age (SD = 1.4). Most (85.4%) participants met DSM 5 ⁵¹ criteria (M = 2.1, SD = 1.3) for past year SUD, even though they were not asked the full criteria set on the GAIN-Q3 assessment. Few participants had received any substance use treatment in the 90 days prior to the baseline assessment (14.5%). No significant differences existed between conditions for demographic characteristics or 13 different indices of clinical severity such as days of substance use in the past 90 days (M = 56.9, SD = 30.1), number of unprotected sex acts (M = 4.5, SD = 13.6), days bothered by mental health problems in the past 90 days (M = 27.2, SD = 34.6), or days spent in a controlled environment (i.e., jail, residential treatment) (M = 11.2, SD = 23.7). In terms of normative feedback comparisons, on average, youth in the NF condition exceeded the outpatient treatment sample norms for past 90-day alcohol use by 2 days, marijuana use by 15 days, and other drug use by 3 days.⁵ Additional detail on participant characteristics is reported elsewhere.⁵

Treatment Sites

No differences existed between participants by treatment condition, but there were differences by site. In short, participants at the Urbana site were less likely (p < .05) to be racial minorities (59% vs. 92%), report less days of being bothered by mental health problems (14.4 vs. 39.0, respectively), report more days of substance use treatment (11.1 vs. 0.28, respectively), and have more days of being on probation in the past 90 days (32.9 vs. 8.7, respectively). Thus, all outcomes analyses controlled for site.

Recruitment, Retention, and Intervention Fidelity

This study recruited a high proportion (71%) of all participants that were eligible for study participation (See Figure 1). Only three individuals (6.2%; MI_n = 2, MI +NF_n = 1) did not complete assigned treatments (e.g., MI or MI+NF). Overall, follow-up retention was 87.5%, with 81.8% (n = 18) of MI and 92.3% of MI+NF youth (n = 24) completing follow up assessments (p > .05). There were no differences between those retained (M = .463, SD = .33) and not retained (M = .457, SD = .45) in terms of baseline percent of days abstinent. All attriters were male, and they were more likely (p < .05) to be White (60%, n = 3) versus from a minority racial background (40%, n = 2).

Figure 1.

CONSORT Participant flow diagram

Session ratings made by the first author using the MITI coding instrument indicated good adherence to the MI model per the recommended proficiency standards appearing in parentheses below.⁴⁵ Furthermore, there were no differences between the sites, or between MI and MI+NF conditions on any of these MITI indices. We found a mean MI spirit rating of 4.11 (SD = .45; 4.0 is recommended), a reflection to question ratio mean of 1.74 (SD =1.15; 2.0 is recommended), a percent of MI Adherent responses of 99% (SD = 3.0; 100% is recommended), a percent of complex reflections at 52% (SD = 12.0; 50% is recommended), and a mean percent of open questions of 62%, (SD = 23.0; 70% is recommended).⁴⁵

Finally, mean session length was 26.80 minutes (SD=11.07, Range=5.8–51.7). There were no significant differences in session length (p = .369) between the MI (M=27.94, SD=14.18) and MI+NF (M=24.80, SD= 6.39) conditions. Overall, however, we note that these average session lengths were shorter than planned (i.e., 30–45 minutes).

Substance Use Outcomes

Table 1 shows the means (SD) and percentages for all dependent variables at baseline and three-months, as well as the regression summaries. For the overall sample, there was a significant increase in PDA (n = 42, t = −2.96, p < .01) over time with the average participant reporting 50.3% days of abstinence at follow up versus 37.4% days at baseline. There were no significant differences in PDA between MI and MI+NF (β = −.079, 95% CI [−.264 – .106]; p = .39) at three-months, with those receiving MI+NF experiencing 7.9% fewer days of abstinence (d = .174) at follow up. Similarly, there were no statistically significant differences between MI and MI+NF participants for the ABD (OR = .58, 95% CI [.14 – 2.36]; p = .45) outcome at follow up, with a lower percent of youth receiving MI+NF (d = −.33) having any episode of binge drinking at follow-up.

Table 1.

Baseline and Post-test Means (SD) and Regression Summaries

	MI (M, SD) or % (n)	MI+NF (M, SD) or % (n)	p	d
PDA-baseline	.30(.27)	.37(.33)	.15	.23
ABD-baseline % (n)	54.5(12)	69.2(18)	.29	−.38
P90 days of Treatment-baselinea.	4.6(13.9)	6.3(18.9)	.73	.12
PDA-3 months	.52(.41)	.49(.40)	.78	−.07
ABD-3 months % (n)	50.0(9)	45.8(11)	.79	.09
PET-3 Months % (n)	55.6(10)	41.7(10)	.37	−.34
			β/OR	95% CI		p
				LB	UB
Model 1: Percent Days Abstinent (PDA) (df:3, 38; R2 =.507, p<.001)b.
Constant			.096	−.24	.43	.56
Condition			−.079	−.264	.106	.39
Site			.072	−.114	.258	.44
PDA at baseline*			.907	.602	1.21	.00
Model 2:Percent Engaged in Treatment (PET) (df:5;Hosmer & Lemeshow = 8.4, p =.13)
Constant			−.195			.06
Condition			.604	.136	2.68	.51
Site			16.2	2.95	89.8	.00
Baseline-Days of treatment (past 90)			1.07	.951	1.20	.26
Model 3:Any Binge Drinking (ABD) (df:8,;Hosmer & Lemeshow = 5.85, p =.44)
Constant			.372			.19
Condition			.580	.142	2.37	.45
Site			1.09	.283	4.21	.90
ABD at baseline*			6.49	1.58	27.8	.01

^aNumber of days of treatment received in 90 day period prior to baseline assessment.

^bModel 1 is an OLS regression model, and Models 2–3 are Logistic Regression models. Odds Ratios are presented in the β/OR column. Condition is coded as (1 = MI+NF, 0 = MI only).

^*p<.05

Treatment Engagement

For the overall sample, 47.6% of adolescents across conditions engaged in treatment. However, we found large site differences in the percent of youth engaged in treatment. Whereas the PET was 16.7% at one site, it was 70.8% at the other (p < .05). In logistic regression analyses (OR = .60, 95% CI [ 0.14 – 2.68];, p = .51, d = −.31) controlling for site, there were non-significant differences between treatment conditions, with 55% of youth in the MI condition and 41.7% of youth in the MI+NF condition engaging in treatment. There were no significant differences (d =.24, p = .47) in PDA between youth who engaged in treatment (14% increase) and those that did not engage in treatment (7.2% increase).

DISCUSSION

Although NF is efficacious for reducing college drinking,²⁹ and one dismantling study with college students found MI+NF to be superior to MI alone,⁵² there is little research on the use of NF with adolescents with probable SUDs. This study found no significant differences in outcomes among youth receiving MI + NF or MI immediately after an initial assessment. Although not statistically significant, adolescents in the MI + NF group had poorer treatment outcomes for two of three primary outcomes: percent engaged in treatment (PET) and percentage of days of abstinence (PDA). This preliminary study echoes some anecdotal concerns about using normative feedback as a treatment engagement strategy for adolescents, and supports the feasibility of doing a larger trial to examine the impact NF has on treatment engagement and post-assessment substance use outcomes. That is, Barnett and colleagues (2012)²⁵ hypothesized that NF may not be suitable for adolescents due to their higher psychological reactance when compared to adults. Although others have posited concerns about whether norm contrasts are too drastic for some high-risk adolescents, and have been reluctant to use NF,²⁶ our study is the first to empirically assess this question.

Our findings suggest the need to study how the proposed mechanisms of change operating in MI+NF interventions may work differently during this developmental stage or environmental context (i.e., not-for-profit settings with high clinical severity and racial diversity). For example, one recent study found that electronically delivered normative feedback to reduce risk factors for drinking and alcohol related consequences did not impact perceived norms regarding beliefs in peer drinking or quantity of weekly drinks among 9^th graders.⁵³ The authors suggested that because of the low alcohol use frequency, the normative comparison may have had a lower impact on changing norms due to potential floor effects. In our context, the current study’s sample would likely not suffer from that problem, as their use would be much higher than that found for adolescents in nationally representative surveys. Future studies should test whether norm correction, or the process of reducing the misperception that other individuals use as heavily as oneself, is a mechanism of change operating in NF interventions for heavy substance-using adolescents.

Another key finding in our study was the low overall treatment engagement for adolescents in these settings (47.6%). In two other studies using the same definition of engagement, approximately 60% of youth engaged in treatment.^6,54 Our treatment engagement was highly variable between the two sites, which we think is explained by the extra treatment resources one site had as part of a treatment demonstration project funded by the Substance Abuse and Mental Health Services Administration (SAMHSA) at the time of the study. In future studies, treatment resources (e.g., outreach and case management time) should be rigorously tracked in multi-site treatment engagement studies. Nevertheless, we controlled for site differences in analyses, and these findings suggest that treatment engagement in some public not-for-profit agencies is problematic and worthy of additional study. Lack of engagement limits adolescents’ level of exposure to various evidence-based treatment procedures that predict outcomes.⁵⁵ Unfortunately, many youth who fail to engage in treatment will never receive such optimal doses of empirically-supported treatments in publicly funded treatment centers.

Strengths and Limitations

This study had a number of strengths, notably the “real world” population and setting, and the training and fidelity of the MI intervention. Further, our study successfully recruited 71% of all adolescents eligible to participate, and retained 87.5% of our enrolled participants at three-months. Although retention was good at the Urbana site (81.8%), it would likely be increased if a full-time staff member could be fully dedicated to follow-up tracking versus half-time graduate assistants. This may elevate retention rates to above 90% for both sites, which could increase the statistical power in a larger trial.

It also had a number of limitations, including the small sample size (n = 48) and limited follow-up period. The small sample size limited our ability to reliably detect differences between the MI and MI+NF conditions. That is, we estimate that achieving 80% power in a two-group design with two-tailed tests (α = .05) would require total sample sizes of 262, 198 and 175 for the PDA (d = .174), PET (OR = .60, d = −.31) and ABD (OR = .58, d = −.33) outcomes, respectively. Further, effect size estimates from small studies such as this one are noted to be unreliable, and some have questioned their use in informing larger scale trials.⁵⁶ Thus, our study findings should be viewed with substantial skepticism and should be replicated in larger trials. Also, this study was not powered to test whether various mechanisms of change such as norm correction or change talk⁵⁷ would mediate outcome differences between the MI and MI+NF conditions. Replication studies with longer follow up times are also needed, as they would permit testing whether the impact of pre-treatment MI on alcohol and drug use outcomes are mediated by treatment receipt. Here we found non-significant trends showing that adolescents engaged in treatment had higher PDA at follow up. As this pre-treatment version of MI targets treatment engagement, its effects on outcomes are expected to be fully or partially mediated by subsequent treatment receipt. Thus, these data also support the feasibility of a trial that investigates whether treatment engagement interventions improve longitudinal substance use outcomes through increasing engagement.

It is also worth noting that the length of our MI session was relatively short compared to other MI+NF interventions discussed in the literature. For instance, in a dismantling study of MI and NF among college drinkers, the average length of the MI and MI+NF sessions were 40 and 50 minutes, respectively, in contrast to our 27- and 24-minute sessions. Although audiotape reviews revealed that intervention fidelity was as good in both conditions, it is possible that the shorter sessions used here may have accounted for discrepancies in findings from this and other studies. This implies that session length should be closely monitored in a larger trial to ensure that adolescents receive an adequate dose of the treatment. Furthermore, if clinicians cannot deliver this amount of MI between the initial assessment and first treatment session, researchers may consider adding technological solutions to MI that increase the model’s dosage (e.g., MI-based text message appointment reminders). In addition, although college student studies have generally found normative perceptions to mediate the effect of NF, most studies have included other elements besides NF in their report, such as money spent on alcohol, caloric equivalent, and other risk factors.²⁹ It is possible that one of these elements, not featured in our report, accounts for the effects seen in the college literature. Finally, our treatment engagement measure was based on self-report data. Although reliability of self-report may have affected outcomes, self-reported treatment engagement did converge with agency records in one study.⁵⁸

Conclusion

Although small in size, this randomized study provided novel data that is consistent with anecdotal concerns about using normative feedback with heavy substance-using adolescents for two of three outcomes. Normative feedback interventions are widely available for use with adolescents. Thus, it is imperative that larger replication studies 1) test the efficacy of adolescent normative feedback interventions for increasing treatment engagement and 2) investigate mechanisms of change that may account for outcomes.