Normative Feedback and Adolescent Readiness to Change: A Small Randomized Trial

Abstract

For adolescents with substance use problems, it is unknown whether the provision of normative feedback is a necessary active ingredient in motivational interviewing (MI). This study investigated the impact of normative feedback on adolescents’ readiness to change and perceptions of MI quality. Adolescents referred for substance use disorder (SUD) assessments were randomized to MI with normative feedback (NF; MI + NF, n = 26) or MI only (MI, n = 22). There were no significant differences between the MI + NF or MI conditions with reference to changes in readiness, and although not significant, there was a decline in readiness for the overall sample. Treatment satisfaction and ratings of MI quality were generally high with no between-group differences. Post hoc analyses revealed a nonsignificant trend where race interacted with treatment condition. Larger replication studies are needed to further study the effects of NF and potential NF by participant characteristic interactions.

Adolescent alcohol and drug use is a major public health concern and one of the most prevalent and leading causes of adolescent morbidity and mortality in the United States (Branigan, Schackman, Falco, & Millman, 2004; Johnson, O’Malley, Bachman, & Schulenberg, 2012; Sussman, Skara, & Ames, 2008). Several studies have shown that substance use typically begins during adolescence and is a major predictor of developing a substance use disorder (SUD) later in life (Gil, Wagner, & Tubman, 2004; Grant, Scherrer, Lynskey, 2006; King & Chassin, 2007; Windle & Wiesner, 2004). More specifically, Gil et al. (2004) found that individuals who use early in life were 1.5 times more likely to experience alcohol abuse and two times more likely to experience marijuana abuse or be diagnosed as having an SUD. Similarly, King and Chassin (2007) reported adolescents with an onset of use before the age of 13 were 3.16 times more likely to develop a drug dependency during their young adult (18–25) years. Nearly 7% of adolescents have an SUD, and adolescents aged 12–17 represent 7.2% of publicly funded treatment admissions in the United States (Substance Abuse and Mental Health Services Administration, 2012).

Readiness to change (RTC) often impacts an adolescent’s decision to make changes with or without professional treatment. For example, most youth who could benefit from treatment do not believe they need treatment, with 60% of teens with marijuana dependence and 48% with alcohol use disorders saying they had no problems due to their substance use (Smith, Hall, Arndt, & Jang, 2009). As adolescent RTC predicts engagement in treatment (Clair et al., 2011) and subsequent reductions in drug use (Breda & Heflinger, 2007; Cady, Winters, Jordan, Solberg, & Stinchfield, 1996; Maisto et al., 2011), it is important to study interventions that target RTC. The purpose of this randomized pilot study was to compare the effects of motivational interviewing (MI) with and without normative feedback (NF) on adolescents’ RTC.

MI

MI is an empirically supported, nonconfrontational intervention that attempts to resolve ambivalence about change through empathic communication and directive reflective listening skills that elicit and reinforce individuals’ prochange statements or change talk (Miller & Rollnick, 2012). MI has been studied in over 200 clinical trials (Miller & Rose, 2009), with a wide range of behavior change targets (i.e., health promotion, substance use, gambling). Meta-analyses show small to medium effect sizes favoring MI (Burke, Arkowitz, & Menchola, 2003; Hettema, Steele, & Miller, 2005; Lundahl, Kunz, Brownell, Tollefson, & Burke, 2010; Rubak, Sandbaek, Lauritzen, & Christensen, 2005). Furthermore, there is an active body of research on the mechanisms of change operating within MI, with multiple studies showing that MI’s effects are mediated by how well counselors evoke change talk or in-session prochange language articulated by individuals (Apodaca & Longabaugh, 2009; Baer et al., 2008; Moyers, Martin, Houck, Christopher, & Tonigan, 2009). Therapists also try not to elicit or reinforce sustain talk or language favoring prosubstance using behaviors (Miller & Rollnick, 2012). For adolescents, specifically, Baer and colleagues (2008) found that reasons for change voiced by adolescents positively predicted 1-month substance use outcomes and conversely that statements made about not wanting to or being able to change were strong negative predictors. In summary, there is strong evidence for the efficacy of MI with substance-using populations and a growing body of process research identifying specific mechanisms of action operating within MI.

MI and adolescents

As MI is nonconfrontational, supports clients’ autonomy (Miller & Rollnick, 2012; Walker, 2011), and is conducive to harm reduction goals (Naar-King & Suarez, 2011), it is thought to be a good developmental fit for adolescents. Further, notwithstanding cognitive impairments associated with drug use or other medical problems common among substance-using adolescents, it is likely to be appropriate for the level of abstract thinking of which adolescents are generally capable (Strait, McQuillan, Smith, & Eglund, 2012). Although there is comparatively less research on MI with adolescents having substance use problems, numerous studies support MI’s efficacy with this population. One meta-analysis found MI had small posttreatment effects (d = .173) for adolescents (Jensen et al., 2011), and another review reported positive findings in 67% of adolescent MI studies (Barnett, Sussman, Smith, Rohrbach, & Spruijt-Metz, 2012).

NF: Description, Theoretical Foundations, and Outcomes

NF interventions contrast an individual’s substance use to available norms for one’s reference group (i.e., descriptive norms). For example, a sample feedback statement could be “You reported drinking five or more drinks on the same occasion on 12 days in the past month. Only 3% of students on this campus reported drinking this many days.” The NF interventions are based on research findings that individuals, including adolescents, commonly misperceive how frequently individuals in their peer reference group (i.e., norm misperception) are using substances (Pedersen et al., 2013; Walker, Neighbors, Rodriquez, Stephens, & Roffman, 2011). Interpreting findings through the lens of the Theory of Planned Behavior, subjective substance use norms are thought to influence individuals’ behavioral intentions, which in turn should predict actual substance use (Ajzen, 1991, 2002). Thus, perceptions that behaviors are nonnormative would influence substance use (see subsequently).

NF research with college students

There is a large body of research with college students supporting the use of NF interventions for reducing drinking. Walters and Neighbors (2005) review of college drinking studies found that NF interventions were successfully delivered in individual sessions (Baer et al., 1992; Borsari & Carey, 2000, Dimeff & McNeely, 2000; Larimer et al., 2001; Marlatt et al., 1998; Murphy et al., 2001, 2004; Neal & Carey, 2004), electronically (Neighbors, Larimer, & Lewis, 2004) or via mailed handouts (Agostinelli, Brown, & Miller, 1995; Collins, Carey, & Sliwinski, 2002; Walters, 2000, Walters, Bennett, & Miller, 2000, Walters & Woodall, 2003). Their review suggested additional work was needed using dismantling designs to test on which specific components of multicomponent feedback interventions were responsible for change in alcohol use. A more recent review included 31 additional studies published after Walters and Neighbors (2005) review, finding additional support for such models and recommending additional research on which intervention components are active ingredients, who derives the most benefits, and which mechanisms of change are essential for producing optimal outcomes (Miller et al., 2013).

The small body of mechanisms of change research with college students finds that the effects of feedback interventions on subsequent alcohol use are mediated by the extent of norm correction (Carey, Henson, Carey, & Maisto, 2010; Walters, Vader, Harris, Field, & Jouriles, 2009) as well as increases in client change language within NF sessions (Vader, Walters, Prabhu, Houck, & Field, 2010). Norm correction refers the reduction in the distance between what an individual perceives as typical substance use (aka norm misperception) and what epidemiological surveys reveal. College students are frequently found to overestimate the amount that typical students drink, which is a robust correlate of alcohol use. Again, change talk refers to the articulation of prochange statements by the intervention recipient during a session.

Use of NF interventions with adolescents

Although there is a small and growing body of literature that is starting to manipulate the content of feedback interventions with college student populations, there is none with adolescents (Barnett et al., 2012). That is, NF interventions are only delivered to adolescents when embedded in stand-alone multicomponent MI models or combined MI plus (MI+) behavioral treatment interventions. Barnett and colleagues (2012) reviewed 39 adolescent MI studies and found that 43.5% of all MI studies included some form of personalized feedback and 35.9% of studies provided MI+. Stand-alone MI models typically include multiple activities such as completing a structured decisional balance activity where clients weigh the pros and cons of continued substance use, receipt of NF, and substance use reduction goal-planning activities (Clair et al., 2013). These MI models have been successfully used in schools (Winters, Fahnhorst, Botzet, Lee, & Lalone, 2012), medical centers (Spirito et al., 2011), and criminal justice settings (Clair et al., 2011; Stein et al., 2006) and in contrast with combined MI+ models do not regularly assume that effects are mediated by subsequent treatment receipt. Examples of MI+ models include those that combine MI with other treatment modalities such as additional adolescent cognitive behavioral groups (Dennis et al., 2004) or parent sessions (Winters et al., 2012). For example, Dennis and colleagues (2004) provided three different versions of MI+ that varied based on the number of cognitive behavioral skills training groups (e.g., 3 in motivational enhancement therapy [MET]/cognitive behavioral therapy [CBT]5 and 10 in MET/CBT12) and the level of family involvement (e.g., MET/CBT12 plus family involvement), testing whether there were improved outcomes with additional treatments. The least intensive model, MET/CBT5, had equivalent 12-month outcomes compared to other treatments. In short, most MI studies with adolescents have never disentangled the effects of NF from MI or other model components, so it is currently unclear whether there is an additive benefit of using NF with adolescents.

Additionally, in reviewing the existing body of research with adolescents, additional research is needed on implementing MI as a pretreatment engagement intervention with adolescents, as community providers are often charged with providing treatments that are longer than typical research protocol treatments and justified by patient placement criteria (Mee-Lee, Gartner, Miller, Shulman, & Wilford, 2001). This treatment contrasts with both stand-alone MI and MI+ models in that the goal of a pretreatment MI model is to reduce client ambivalence about both making changes and attending treatment. This can be an important part of an MI model when there is heavy use or substance dependence, and in this context MI has been suggested as a strategy to refer adolescents to treatment (Levy, Winters, & Knight, 2010). This contrasts with stand-alone MI models that do not explicitly focus on ambivalence surrounding treatment attendance as well as MI+ models, where the MI sessions occur after a client has agreed to enroll in treatment (i.e., MET/CBT5). There is very limited research on using MI with treatment-referred adolescents who likely have higher severity than seen in universal pediatric screening populations and for whom referral to treatment should be the focus of the MI intervention.

Importance of dismantling effects of MI + NF interventions for adolescents

There are reasons why it is important to disentangle the effects of combined MI + NF interventions for adolescents such as the potential for activating process that can diminish the intervention’s effectiveness (i.e., increased sustain talk) and also reactivity from diverse adolescent populations. That is, some anecdotal concerns suggest that adolescents may react more negatively to NF due to their higher psychological reactance compared to adults (Barnett et al., 2012), which have prompted some researchers to not to include NF in their MI interventions (Winters et al., 2012). One major concern is that if adolescents do react negatively to NF, it will take the form of personalized counterarguments against the feedback, or sustain talk, which has been shown to predict poorer outcomes among adolescents (Baer et al, 2008). Adult research is mixed as to whether NF conditions result in more versus less change talk, with one college student study showing increased change talk in the NF condition (Vader et al, 2010) and one process study showing change talk was the lowest during the section of the session when NF was delivered (Amrhein, Miller, Yahne, Palmer, & Fulcher, 2003). So, it is plausible that this treatment component may be working against the other curative active ingredients when included in MI interventions, and there are anecdotal concerns that this may be more prominent for adolescents. Second, there are important contextual variables that may diminish the effectiveness of NF with adolescents, as brief feedback interventions have rarely been tested in public not-for-profit settings serving racially diverse clients with higher psychiatric comorbidity (Worden & McGrady, 2013). One particular problem with using NF with racial minorities is that they may not believe that norms are specific to their group. The college student literature shows that students can better estimate racial group–specific substance use norms (Larimer et al., 2011) and that stronger group identification moderates the association between perceived norms and substance use (Neighbors et al., 2010). One study reported that race did not interact with NF condition in predicting alcohol outcomes but had a low percentage of minorities (i.e., 15% in Walters et al., (2009) dismantling study). In short, it is possible that using nonrace-specific norms during NF sessions may have less impact on racial minority adolescents’ substance use due to the lack of credibility.

Relevance of Study Outcome Variables

Few studies have investigated the impact of MI models on adolescent RTC and treatment satisfaction, and no existing studies have used adolescent report measures of MI quality. Subsequently, we briefly discuss the salience of these outcome variables in the present study.

In some studies, RTC predicts treatment outcomes (Cady et al., 1996; Callaghan et al., 2005), but there are few examples of full mediation models where the treatment-impacted RTC (i.e., mediator), which, in turn predicted outcomes. Barnett and colleagues (2012) reported that out of 39 adolescent MI studies, 7 nontobacco studies investigated MI’s effect on RTC. Findings were mixed, with some showing positive effects of MI on RTC and some not. However, upon our reexamination of these studies, it appears that the nonfindings may possibly be due to lower treatment integrity in the studies with nonfindings. That is, of the three studies using rigorous treatment coding (D’amico, Miles, Stern, & Meredith, 2008; Mason, Pate, Drapkin, & Sozinho, 2011; Thush et al., 2009), two (67%) reported positive findings. However, of the two studies finding no significant outcomes for MI’s effectiveness on RTC, only one (50%; Thush et al., 2009) coded MI using the widely used Motivational Interviewing Treatment Integrity (MITI) Scales (Moyers, Martin, Manuel, Miller, & Ernst, 2010). Our study will add another study using rigorous training and therapist adherence–monitoring procedures to possibly assist in clarifying these mixed results by providing data for use in future meta-analytic reviews.

Impact of MI on treatment satisfaction or adolescent perceptions of MI quality

In Barnett and colleagues’ (2012) review of adolescent MI studies, only 15.3% (6 of 39 studies) reported treatment satisfaction outcomes. Of these, only one study compared treatment satisfaction outcomes between adolescents receiving different aftercare treatments that included MI (Kaminer, Burleson, & Burke, 2008). They found no significant differences between patient condition assignments. Furthermore, client-report measures of MI quality have only recently been developed, and we believe this is the first to use the Client Evaluation of Motivational Interviewing (CEMI; Madson et al., 2013) with adolescent populations. The CEMI measures both the relational (i.e., empathic stance) and the technical (i.e., directive focus on behavior change) components of MI. Due to the lack of research on NF with adolescents, and anecdotal concerns that adolescents may argue against substance use norms, we reasoned that NF may have a negative impact on both treatment satisfaction and perception of MI quality. The use of the CEMI and focus on treatment satisfaction ratings is an additional contribution of this study.

Summary and Hypotheses

This article addresses some major gaps in the literature on MI with adolescents with substance use problems. First, to our knowledge, this is the first pilot study to attempt to isolate the effects of a specific intervention frequently used alongside MI called NF (see subsequently). The study design used here tests whether MI alone is sufficient to impact adolescent readiness to change or if there is an additive benefit to using NF. This is important because there is but one college study showing MI + NF to be superior to MI only (Walters et al., 2009), and there are anecdotal concerns about using NF with adolescents (Barnett et al., 2012; Winters et al., 2012). This pilot study will take an important initial step in resolving this debate. We also test whether treatment satisfaction or client perceptions of MI quality differ between those receiving MI and MI + NF, which will inform the research community on whether NF has any negative or positive impact on indices of the therapeutic relationship. Additionally, we add to the limited number of studies that have tested the impact of MI on RTC. Also, this pilot study provides preliminary information on whether the study question warrants additional attention and can inform power analyses for future trials. Finally, we also note the importance of replicating work in this area with a sample of adolescents seen in public not-for-profit settings that are known to serve racially and socioeconomically diverse clients with high psychiatric comorbidity (Dennis, White, & Ives, 2009; Worden & McGrady, 2013). As there is no data to guide hypotheses for adolescent samples, we refrained from making directional hypotheses about how NF would impact RTC, treatment satisfaction, or client-perceived quality of MI.

Method

Procedure and Participants

Procedures were approved by the institutional review board at the lead author’s university. Adolescents referred for SUD assessments at two not-for-profit agencies from March to August 2013 were invited to participate if they were between 13 and 19 years of age, scored 2+ on the CRAFFT (Knight, Sherritt, Harris, Gates, & Chang, 2003) and/or reported 13+ days of substance use in the past 90 days, and spoke English. The CRAFFT was used due to its high sensitivity in predicting which youth would have probably SUD (Knight et al., 2003), and the substance use frequency criterion permitted us to also ensure that we selected adolescents whose frequency of use was similar to those referred for outpatient treatments (Dennis et al., 2009). This criterion distinguishes this study from other MI studies in universal prevention contexts that had lower participant inclusion thresholds. A parental consent waiver was granted for adolescents for whom obtaining an assessment was deemed to put them at risk, but otherwise parental consent was collected for all participants. Adolescents were assessed with the Global Appraisal of Individual Needs–Quick version (GAIN-Q; GQ.3.2.0_MI; Titus et al., 2012), a reliable and valid assessment. They were then randomized to receive either MI (n = 22) or MI + NF (n = 26). Randomization was done prior to assessment by having a staff member with no participant contact draw half of the identification numbers and designate them for the NF condition. Treatment agency staff members conducted assessments while blind to the treatment allocation, as allocation was concealed in an opaque envelope until baseline measures were complete.

After finishing baseline assessments and the intervention, adolescents completed posttest measures immediately after the intervention, which were sealed in an envelope by participants and returned to researchers. Posttest RTC measures were available for 45 clients, with data missing from three individuals (MI_n = 2, MI + NF_n = 1) who dropped out of treatments. Figure 1 displays participant flow through the study (Moher, Schulz, Altman, & for the CONSORT Group, 2001).

Figure 1.

CONSORT participant flow diagram.

Participants

On average, study participants were 16.3 years of age (standard deviation [SD] = 1.4) and mostly male (77.1%). Participants self-reported their racial backgrounds as bi- or multiracial (36.2%), African American (36.2%), White, non-Hispanic (23.4%), Hispanic/Latino (2.1%), or other race (2.1%). Although only four (Diagnostic and Statistical Manual of Mental Health Disorders (Fifth Edition; DSM 5) SUD criteria are included on the GAIN-Q, 85.4% of participants met DSM 5 (APA, 2013) criteria required for past year SUD. Table 1 displays the characteristics of participants in the MI and MI + NF conditions. No significant differences between conditions existed for demographic characteristics or on indices of clinical severity.

Table 1.

Baseline Characteristics of Youth Randomized to MI + NF or MI-NF.

	MI (n = 22), M (SD) or % (n)	MI + NF (n = 26), M (SD) or % (n)	Total(n = 48)a
Age, in years	16.4 (1.5)	16.3 (1.3)	16.3 (1.4)
% Female	27.3% (6)	19.2% (5)	22.9% (11)
Last grade in school	9.59 (1.5)	9.56 (1.3)	9.57 (1.4)
Racial background
African American	31.8% (7)	40% (10)	36.2% (17)
Biracial/multiracial	40.9% (9)	32.0% (8)	36.2% (17)
White	18.2% (4)	28.0% (7)	23.4% (11)
Latino	4.5% (1)	0.0% (0)	2.1% (1)
Other	4.5% (1)	0.0% (0)	2.1% (1)
Substance use and problems
Days of substance use (past 90)b	60.2 (28.1)	54.2 (31.9)	56.9 (30.1)
Days of alcohol use	9.8 (18.2)	7.9 (13.8)	8.8 (15.8)
Days of binge alcohol use	5.2 (11.1)	6.1 (13.0)	5.7 (12.1)
Days of marijuana use	47.3 (32.9)	41.6 (34.6)	44.2 (33.6)
Substance use disorder screenerc	3.1 (1.4)	3.4 (1.4)	3.23 (1.3)
Days of SUD treatment	4.6 (13.9)	6.3 (18.9)	2.4 (16.7)
Risk behaviors
Times of unprotected sex	2.1 (5.2)	6.6 (17.8)	4.5 (13.6)
Mental health (MH) problems
Days of externalizing problems	31.6 (35.4)	33 (34.7)	32.4 (34.6)
Days of internalizing problems	39.3 (37.7)	41.7 (42.1)	40.6 (39.7)
Days bothered by MH problems	22.3 (29.2)	31.3 (38.7)	27.2 (34.6)
Days of MH treatment	9.8 (26.1)	14.0 (31.4)	12.1 (28.9)
Criminal justice involvement
Days on probation	22.1 (38.2)	18.7 (34)	20.3 (35.6)
Days in a controlled environment	10.4 (24.5)	11.9 (23.4)	11.2 (23.7)

Note. DSM = Diagnostic and Statistical Manual of Mental Disorders; MI = motivational interviewing; NF = normative feedback; SD = standard deviation; SUD = substance use disorder.

^aFor all contrasts, p > .05.

^bPast 90 days of use was calculated as the past 90 days living in the community if the youth reported greater than 13 days in controlled environments in the past 90-day period. All other “days of” and “times of” questions range from 0 to 90 and are out of the 90-day period prior to assessment.

^cA 5-item screener including 1 item about weekly use and four DSM Fifth Edition SUD criteria. Scores of 2 or higher indicate a high likelihood of the presence of a DSM Fourth Edition SUD (see Dennis et al., 2006).

Treatment sites

Sites included state-funded, not-for-profit SUD treatment agencies in Illinois (Chicago, IL and Urbana, IL). Although no differences existed between participants by treatment condition, participants at the Chicago site were more likely (p < .05) to be minorities (92% vs. 59%), report more days of being bothered by mental health problems in the past 90 days (39.0 vs. 14.4 days), report fewer days of SUD treatment in the past 90 days (0.28 vs. 11.17 days), and have fewer days of being on probation in the past 90 days (8.7 vs. 32.9 days). Due to these findings, all analyses controlled for site.

Treatment conditions

The Compassionate Helpers Openly Inviting Client Empowerment (CHOICE) model (Smith, 2012) is a multiple target behavior adaptation of MI completed when reviewing results from a standardized assessment called the GAIN-Q (Titus et al., 2012). Thus, CHOICE happens prior to treatment admission. CHOICE sessions (15–30 min) are facilitated by a computer-generated report and include an orientation to the session, discussion of participants’ strengths, an agenda-setting procedure, a review of concerns and referral to treatment, and a session summary. Although it is a structured session, therapists are instructed to use MI throughout by conveying empathy, using mainly reflections to propagate the discussion, and eliciting and reinforcing change talk about the selected target behaviors. For a full description of CHOICE, see Smith (2012).

Both conditions received all procedures described earlier, and participants in the MI + NF condition also received NF when reviewing substance use concerns. Specifically, participants’ days of marijuana and alcohol use were compared to two sets of norms available from the Chestnut Health Systems (2011), the developers of the main outcome instrument. Norms were available for youth completing this assessment nationwide (N = 35,191). Age-specific norms as well as those for youth admitted to different levels of care (e.g., residential, outpatient) were available, and therapists used whichever norm provided a greater contrast with participants’ use.

Therapist characteristics and training

The five therapists providing the intervention were mostly female (80%) and master’s-level clinicians (80%) and had an average of 5.7 years (SD = 5.4) of counseling experience, with 3.1 years (SD = 2.3) specific to substance use treatment. Therapists provided both the MI and the MI + NF treatments. They initially received a 12-hr interactive training, observing exemplar use of MI, role-playing skills, and receiving feedback. Before open enrollment, therapists completed mock role-plays and received feedback from the lead author who coded sessions using the MITI Scales (3.1.1; Moyers et al., 2010), a reliable observational coding instrument used to monitor the quality of MI (Moyers, Martin, Manuel, Hendrickson, & Miller, 2005). After starting the trial, all sessions were taped, and the lead author continued to supervise therapists. Fifteen tapes (31.2% of total N) were reviewed during the trial, with each therapist receiving an average of 3.0 (SD = 1.0, range: 2–4) tapes reviews/coaching calls.

Supervisor ratings indicated good adherence to the MI model. We looked at several indices of MI performance available on the MITI, including the global MI spirit rating, the percentage of MI-adherent responses, the ratio of reflections used to questions used, the percentage of open questions out of all questions asked, and the percentage of complex reflections of all reflections used. The global MI spirit rating measures clinicians’ ability to evoke and reinforce change talk, collaborate with the client, and also respect their autonomy. It is rated on a 1–5 Likert-type scale, with scores of 4 or above as the recommended threshold for MI competency (Moyers et al., 2010). The percentage of MI-adherent behaviors (i.e., count of affirmations, autonomy enhancing statements, statements asking permission to give advice/concerns, and support statements) is the total number of these adherent behaviors and nonadherent behaviors (e.g., confrontations, unsolicited advice), with 100% recommended as the competency threshold. Finally, as open questions and complex reflections are preferred to closed questions and simple reflections, respectively, the reflection to question ratio and percentage of complex reflections provide additional info on clinician skill level with MI, with 2% and 50% recommended as competency thresholds, respectively. Finally, another MI competency threshold per the MITI is for 70% of all questions to be open questions. We found a mean MI spirit rating of 4.11 (SD =.45), a reflection to question ratio mean of 1.74 (SD =1.15), a percentage of MI-adherent responses of 99 (SD =3.0), a percentage of complex reflections at 52 (SD =12.0), and a mean percentage of open questions of 62 (SD = 23.0; Moyers et al., 2010). There were no significant differences between the sites or between MI and MI +NF conditions on these indices.

Measures

Readiness to change

Participants responded (0–100%) to the item “How ready are you right now to make changes in your behavior related to your use of alcohol or other drugs?” at baseline and posttest. Notwithstanding concerns about reliability and validity, similar single-item measures have been found to have higher predictive utility for substance use outcomes when compared to multi-item measures of RTC (Hoeppner, Kelly, Urbanoski, & Slaymaker, 2011; Maisto et al., 2011). To controlfor baseline RTC while preserving degrees of freedom in this small pilot study, we subtracted pretest RTC from posttest RTC. Scores could range from −100.00 to 100.00, with positive scores indicating increased posttest RTC and a score of 0.0 indicating no change.

Session satisfaction and client-perceived MI quality

Two measures gauged the impact of NF on adolescents’ session quality ratings. First, participants rated their level of agreement (1 = very dissatisfied, 5 = very satisfied) with the item “Overall, how satisfied are you with this discussion you just had about your assessment results?” Additionally, we administered the CEMI, a reliable and valid measure of client perceived quality of MI (Madson et al., 2013). The CEMI contains 16 items rated on a Likert-type scale (1 =not at all, 4 =a great deal). In addition to a total scale (CEMI Total Scale, range =16–64, α=.72), it contains subscales for the technical aspects of MI (i.e., CEMI Technical) that pertain to talking to behavior change in a manner consistent with MI (8 items, range = 8–32, α = .89, e.g., “help you recognize the need to change your behavior”) as well as relational aspects of MI (i.e., CEMI Relational) that may not be unique to MI (8 items, range = 8–32, α = .68, e.g. “push you forward when you became unwilling to talk about an issue further”; reverse scored). Higher scores indicate a counseling style more consistent with MI. Analyses of CEMI Scales are based on 38 (79.2%) participants, due to a copying mistake (n = 7) and dropout prior to posttest completion (n = 3), which was equally distributed across conditions.

Data Analysis

Initially, distributions were analyzed for missing data and out of range responses. A small percentage (average percent missing = 1.95%) of missing data was replaced (i.e., 596 observed values and 12 imputed values) for CEMI items using simple mean replacement within treatment site. Preliminary analyses revealed that site did not account for variance in treatment integrity but that participant characteristics did differ. For normally distributed continuous outcome variables (e.g., change score for RTC, CEMI Total), we used regression models entering condition and site as predictors. We used Mann–Whitney U and Wilcoxon Sign-Rank tests for nonnormally distributed variables (e.g., CEMI technical, CEMI relational, and treatment satisfaction). All analyses presented are intent-to-treat analyses that include all participants completing follow-up assessments regardless of whether they attended the services provided in the assigned condition.

Results

Readiness to Change

Overall, participants reported a mean decrease of −2.04 (SD = 23.9) in RTC from baseline to posttest, but this change was not statistically significant (Wilcoxon T = 101.5, standard error = 28.6, p = .625). Different readiness patterns were apparent, with 53.4% (n = 24) of the sample reporting no change in RTC from baseline to posttest, 22.2% (n = 10) reporting an increase in RTC and 24.4% (n = 11) of participants reporting decreased readiness after their MI session. Among participants reporting no change in readiness, many (n = 10) reported baseline and posttest RTC scores of 100. Table 2 presents the pre–post differences in RTC by condition, treatment site, and minority status (racial minority = 1).

Table 2.

Baseline and Posttest Readiness to Change Scores.

	Baseline, M (SD)	Posttest, M (SD)	p	d
MI + NF	74.1 (27.2)	71.2 (31.7)	.43	−.098
MI	62.7 (37.3)	59.5 (41.6)	.86	−.081
Site 1	75.6 (29.7)	78.8 (34.8)	.57	.10
Site 2	62.8 (34.2)	55.4 (34.8)	.31	−.21
Minority	70.8 (30.9)	63.4 (35.6)	.24	−.22
White	58.2 (37.6)	72.0 (40.5)	.14	.22

Note. MI = motivational interviewing; NF = normative feedback; SD = standard deviation.

Neither treatment condition (β = −5.2, 95% confidence interval [CI]: [−14.1, 37.7], p = .48) nor site (β = −7.1, 95% CI: [−19.8, 9.5], p = .33) was significant predictors of posttest change in RTC. Although nonsignificant, coefficients indicated that participants in MI + NF (Cohen’s d = −.14) and those at the Chicago site had greater declines in RTC relative to those receiving MI and at the Urbana site. Table 3 displays the model summary for this main effects model (i.e., Model 1).

Table 3.

Model Summaries.

	β	95% CI		p
		LB	UB
Model 1: (df: 2, 42; R2 = .03, p = .52)a
Conditionb	−5.2	−14.1	37.7	.48
Site	−7.1	−21.7	7.6	.34
Model 2: (df: 6, 37; R2 = .11, p = .59)
Conditionb	−8.2	−23.9	7.6	.30
Site	−6.6	−25.3	12.2	.48
Minority	−14.9	−35.9	5.9	.16
Days of MH problems	.12	−.11	.35	.29
Days of SA TX	−.50	−.61	.51	.86
Days on probation	−.02	−.27	.24	.89
Model 3: (df: 4, 39; R2 = .084, p = .48)
Conditionb	−9.5	−44.7	25.8	.59
Site	−2.6	−18.9	13.6	.75
Minority	−17.2	−50.1	15.7	.29
Condition by minority	3.8	−35.4	43.0	.85

Note. CI = confidence interval; df = degrees of freedom; LB = lower bound; MH = mental health; MI = motivational interviewing; NF = normative feedback; SA TX = substance abuse treatment; UB = upper bound

^aThe dependent variable for all regression models was the difference in score in readiness to change (e.g., posttest minus pretest). Negative β coefficients indicate a decrease in readiness from pretest to posttest.

^bCondition is coded as (MI = 0, NF =1). Negative coefficients indicate a decrease in readiness to change from baseline to posttest for the MI + NF condition.

To further clarify what variables accounted for site differences, we completed exploratory post hoc analyses by entering variables for which there were significant site differences into our original regression model (Models 2 and 3). These variables included minority status, days on probation, days of mental health problems, and days of prior SUD treatment. Reductions in the coefficient for site would indicate that this effect on posttest readiness was explained by differences in participant characteristics at these sites. In this exploratory analysis, we found that site (β = −6.6, 95% CI: [−25.3, 12.2], p = .48) differences diminished and condition effects (β = −8.2, 95% CI: [−23.9, 7.6], p = .30) increased. One variable, minority status (β = −14.9, 95% CI: [−35.9, 5.9], p = .16) had a larger effect on postsession readiness than either site or condition. Because of this strong, albeit marginally significant effect, for race in the second model, we calculated a third model where we regressed condition and site on outcomes while entering an interaction term for minority status and condition. In Model 3, the coefficient for condition (β = −9.5, 95% CI: [−44.7, 25.8], p = .59) became stronger in magnitude, indicating an even more negative effect for MI + NF when this strong confounding factor was controlled. Furthermore, the interaction term showed that minority participants in MI + NF had increases in RTC (β = 3.8, 95% CI: [−35.4, 43.0], p = .85) compared to minority participants assigned to MI.

Participant Satisfaction and Client-Perceived Competence in MI

Mean treatment satisfaction across conditions was 4.0 (SD =1.41), with a modal reply (55.5%) of very satisfied. There were no statistically significant differences (Mann–Whitney U = 240.0, p = .94) in treatment satisfaction between MI (mean = 3.94, SD = 1.47) and MI + NF (mean = 4.04, SD = 1.39) participants.

Treatment condition was not significantly associated with higher CEMI technical (Mann–Whitney U = 243.00, p = .06) or CEMI relational (Mann–Whitney U = 159.00, p = .58) scores. On average, scores on the CEMI Technical Scales were 25.11 (SD = 5.35) and 28.29 (SD = 3.55) for MI and MI + NF participants, respectively, representing a marginally significant trend favoring the MI + NF condition (d = .72; p = .06). CEMI Relational Scale scores were similar for both MI (mean = 24.41, SD = 4.15) and MI + NF (mean = 23.9, SD = 3.94) participants. As the CEMI total score was normally distributed, we used regression modeling, controlling for site. In this model, neither treatment condition (β = 2.76, 95% CI: [−1.77, 7.30], p = .224) nor treatment site (β = −.201, 95% CI: [−4.37, 4.77], p = .93) were significant predictors of CEMI Total scores. The positive coefficient indicated that the adjusted mean difference of 2.76 favored MI + NF and approached a moderate effect size (d = .43). The effect, however, was statistically nonsignificant at p < .05. Figure 2 presents unadjusted means for treatment satisfaction and the CEMI Scales for both the MI and the MI + NF groups. In summary, there were no statistically significant differences in any of these treatment satisfaction and client-report MI competency measures, and nonsignificant trends on two measures favoring the MI + NF group.

Figure 2.

Treatment satisfaction and Client Evaluation of Motivational Interviewing (CEMI) Scale scores for motivational interviewing (MI) and MI + normative feedback (NF) participants.

Discussion

The limitations of this pilot study, especially sample size, preclude making definitive statements about whether NF is an important active ingredient in broader MI or combined MI and behavioral interventions. However, the purpose of this pilot study was to generate pilot data to stimulate additional inquiry in larger trials surrounding the controversy over whether NF is an active ingredient in adolescent substance use interventions. Several trends emerged that appear important to study in such trials. First, in all three outcomes models we presented those that varied on the number of confounds for which we adjusted, and adolescents randomized to MI + NF had reduced posttest RTC relative to those randomized to MI. Here, regression-adjusted standardized mean differences in RTC between MI and MI + NF represented small effects (range of Cohen’s d’s: .14–.26), which increased when additional confounds were statistically adjusted. These findings are the first preliminary data available on this topic and resonate with prior anecdotal concerns about using NF with adolescents. On the other hand, there was also a trend for those receiving MI + NF to rate their clinician performance of MI, especially the technical aspect of discussing change, much higher than those receiving MI. Further, treatment satisfaction was similar between groups, suggesting NF did not negatively impact client-rated session satisfaction. Although limited statistical power diminishes confidence in these findings, they are important preliminary data, as effect sizes represented small effects. Subsequently, we discuss these findings further and make suggestions for future research.

As RTC does predict subsequent service use (Broome, Simpson, & Joe, 1999; Joe, Simpson, & Broome, 1999) and reduced substance use (Bailey, Baker, Webster, & Lewin, 2004; Cady et al., 1996; D’Amico et al., 2008; Grenard et al., 2007; Mason et al., 2011) in some studies, future studies should replicate our findings and determine whether NF lowers RTC. As these effect sizes may be unstable due to our small sample size (Kraemer, Mintz, Noda, Tinklenberg, & Yesavage, 2006), it is important to replicate findings with larger and racially diverse samples. If future research determines that NF does indeed have a negative effect on RTC, studies should also test whether these negative effects are mediated by reduced adolescent change talk (Baer et al., 2008) or incredulity at the information being presented (Kubris, Schaumberg, Davis, Hall & Morgenstern, 2014). That is, current anecdotal claims that youth may not always believe the NF presented, which may diminish its effectiveness, should be rigorously studied in larger trials. Although there is one college study showing that the positive impact of NF on alcohol use outcomes was mediated by increased change talk (Vader et al., 2010), this may not be the case for adolescents. A better understanding of this is important, since findings have implications for whether multicomponent MI models for adolescents could be even more effective if they remove NF.

In addition to the main findings contrasting treatment conditions, it is curious that RTC decreased for both treatment groups. This finding is likely not due to poor treatment integrity, as MI was implemented here with high fidelity, a departure from previous adolescent literature where nonfindings may be related to treatment integrity procedures. What then can account for these reductions in RTC? First, it is possible that the large proportion of participants reporting no net change in readiness limited the ability to detect pre-post differences in RTC (i.e., range restriction). For example, 22% (n = 10) of all participants reported 100% RTC at both baseline and posttest. (However, findings from post hoc analyses that excluded these individuals were consistent with analyses presented here and are available upon request from the first author.) Future studies may screen for readiness prior to randomization and consider using this as blocking variable or an eligibility criterion. Second, test reactivity may also have influenced findings. That is, participants may have been keenly aware that they were repeating the same questions they had on a baseline assessment that occurred an hour or so prior to the posttest. Also, participants may have been more comfortable reporting lower readiness at posttest, as these answers were anonymously collected in envelopes. Thus, youth reporting decreases in RTC may have given socially desirable and inflated estimates of RTC at baseline, which may have influenced findings. Notably, minorities had pre–post decreases in RTC and Whites had increases. It could be that public stigma and stereotypes for people with multiple stigmatized identities (e.g., African American substance users; Luoma et al., 2013) may have influenced the reactions of minority youth to questions about their RTC. Finally, we must consider the possibility that pretreatment MI simply may not impact RTC, as found in some MI studies (Barnett et al., 2012). However, we note that these findings are preliminary and our study did not include an assessment as usual treatment condition, which would have permitted us to examine whether either MI condition performed better than typically available assessment procedures in terms of influencing RTC. Future studies should include an assessment as usual condition.

Potential Interactions of NF With Participant Race

The influence of racial differences in RTC on the treatment main effect estimates was demonstrated by the increased coefficient for condition when participant race was included in the second and third models. Additionally, our third post hoc analysis considered whether race interacted with treatment condition. Interestingly, although race had a negative impact on RTC outcomes in all three models, the interaction term in the third model was positive. This indicated that racial minorities (non-White participants) assigned to MI + NF reported a 3.8-point increase in RTC. Given the strong negative association between race and postsession RTC, it is somewhat encouraging that NF had a small positive effect on racial minorities’ postsession RTC. This finding is somewhat counterintuitive and difficult to explain for a couple of reasons. First, we would expect that to protect against self-devaluation from stigma, minorities presented with information about their use relative to typical adolescents their age may feel threatened (Luoma et al., 2013). Second, some studies with college students have found that group identification, or a strong sense of affiliation with a proximal group, was associated with normative associations of alcohol use (Neighbors et al., 2010). When college-age students are asked to indicate their drinking in relation to distal groups, they mostly overestimate the drinking and behaviors of the “other” groups (based on age, sex, or race; Larimer et al., 2011). Thus, in our study, it would be reasonable to assume that racially diverse youth may not believe that norms applied to their racial group, resulting in a frustrating experience for them and materializing in greater declines in RTC. However, our findings ran contrary to this assumption since minority youth assigned to NF experienced increases in RTC.

Other studies have not found significant differences by race when using it as a moderator of MI (Peterson, Baer, Wells, Ginzler, & Garrett, 2006) and NF’s effects (Walters et al., 2009), and although we collected a smaller sample, this study had far more racial diversity. Past studies have also called for research to understand the mechanisms by which either MI or NF can have most effect with minority youth and adolescents (Neighbors et al., 2010; Naar-King & Suarez, 2011; Larimer et al., 2011). Thus, future studies are needed to determine whether there are Race × NF interactions.

NF’s Impact on MI’s Quality and Treatment Satisfaction

This study found preliminary evidence that NF does not appear to negatively influence treatment satisfaction or client-perceived MI quality. To the contrary, adolescents receiving MI + NF rated their clinicians as more competent on both the technical aspect of discussing behavior change and the overall quality of MI. Although nonsignificant, we found moderate effect sizes for these variables. So, on these scales at least, it does not appear that youth rate their counselors as any more coercive if they use NF, which allays some anecdotal concerns about using NF with adolescents. It may be that since MI is a highly empathic treatment experience, the NF subcomponent was not a sufficiently negative experience. Additionally, treatment satisfaction scores were similar for both youth in the MI and the MI + NF conditions; so whether replication studies find positive or negative effects for using NF, it seems unlikely that any outcome differences will be mediated by treatment satisfaction or perceived MI quality. This is consistent with prior work showing treatment satisfaction is largely unrelated to adolescent treatment outcomes (Kaminer et al., 2008; Tetzlaff et al., 2005). It may be that since treatment satisfaction is a weak predictor of substance use treatment outcomes, future studies should focus more on whether RTC, treatment engagement, and alcohol and drug outcomes vary for youth receiving MI interventions with and without NF.

Limitations

The main limitation of this pilot study was the low statistical power to detect the nonsignificant trends found here. This pilot study sought preliminary data on a novel research question in order to determine whether the expense of a larger trial would be justified. The low statistical power of this study beckons us to be conservative about conclusions we make and suggests a need for large replication studies to test whether findings here were spurious. For example, although it is important for social workers to know whether interventions work equally well for diverse clients, low statistical power prevents this study from adequately addressing that question. This is a common problem in most studies that have tested whether race moderates treatment effects (Miller, Villanueva, Tonigan, & Cuzmar, 2007). Additionally, although we checked for baseline differences in potential confounding variables, low power also limited our ability to detect baseline differences between treatment conditions. It is unknown how such potential confounds may have affected findings, underscoring the need for replication studies. Second, the absence of an assessment as usual control group rendered this study unable to answer the question of whether MI or MI + NF would be superior to typical assessment processes. Third, there may have been some variation in the delivery of the NF that could have influenced the findings. That is, therapists could select different sets of norms based on the largest discrepancies with the participants substance use. Although general MI adherence was good and equivalent between conditions, variation in NF delivery could have influenced findings. One further limitation was the inclusion of participants who reported 100% RTC at baseline that could not show improvements in readiness. Finally, as there are mixed findings on whether RTC predicts actual change in substance use, future studies should also investigate substance use outcome variables.

Conclusion

Notwithstanding study limitations, this study was the first to test the impact of NF on RTC outcomes, with racially diverse adolescents presenting for initial SUD assessments. Although we found no significant differences between youth randomly assigned to MI or MI + NF on postsession RTC, these preliminary trends showing that NF may have some negative impact on RTC echo previous anecdotal concerns about using NF with adolescents. There is a poignant need to replicate these findings as part of broader efforts to maximize the effectiveness of multicomponent MI models for adolescents presenting for substance use assessments.