Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Mar 24.
Published in final edited form as: Prof Psychol Res Pr. 2020 Feb;51(1):16–24. doi: 10.1037/pro0000270

Motivational Interviewing Skills as Predictors of Change in Emerging Adult Risk Behavior

Elisa C DeVargas a,1, Elizabeth A Stormshak a
PMCID: PMC7989792  NIHMSID: NIHMS1050755  PMID: 33767528

Abstract

Emerging adulthood is a unique developmental stage during which significant transitions in living environment, social networks, personal responsibilities, and identity development occur. Stress associated with these transitions relates to increases in health-risk behaviors, such as substance use and high-risk sexual behavior. This research examined health-risk behavior outcomes associated with the Young Adult Family Check-Up (YA-FCU). The YA-FCU comprises three sessions: an initial interview, an ecological assessment, and a feedback session that integrates motivational interviewing (MI) techniques. This study measured treatment fidelity of the YA-FCU and the extent to which therapists adhered to principles of MI during feedback sessions. Therapists included both licensed psychologists and trainees. The study also examined the relationship between therapists’ MI fidelity and client change talk (CT), in order to determine if MI fidelity and client CT predicted postintervention health-risk behaviors among emerging adults who participated in the YA-FCU. Measures of health-risk behaviors were collected pre- and post- intervention. Results indicated overall adequate treatment fidelity. MI fidelity was positively related to client CT. Several indicators of MI fidelity predicted decreases in emerging adults’ health-risk behaviors. For example, therapist ratio of reflections to questions predicted a decline in emerging adults’ alcohol use frequency and marijuana use quantity. These results have important implications for YA-FCU training and implementation and indicate that MI consistent skills might be a mechanism of change in the YA-FCU intervention.

Introduction

Emerging adulthood is a developmental period between adolescence and adulthood spaning ages 18–29 years, with some emerging adults transitioning to adulthood at 25 years of age (Arnett, 2015). The term “emerging adulthood” was derived in response to shifts in historical and social trends of industrialized countries that led to a delay in attainment of normative adult responsibilities (Arnett, 2000). Emerging adulthood is often marked by frequent changes in living situation, social networks, and personal responsibilities (Arnett, 2015). Many of the stressors experienced by emerging adults are linked to increased health-risk behaviors. For example, emerging adults (EAs) are at greater risk of meeting criteria for an alcohol or other substance use disorder than are older and younger age groups (Johnston, O’Malley, Bachman, Schulenberg, & Miech, 2015), and they have the highest age-specific prevalence of sexually transmitted diseases (STDs; Centers for Disease Control, 2015). Although many researchers attribute spikes in health-risk behaviors to the college experience, increases in substance use and other health-risk behaviors have been found among EAs regardless of whether they attend college (White et al., 2006). Despite such high prevalence rates of health-risk behaviors among EAs, very few receive treatment. In a study by Pedrelli, Nyer, Yeung, Zulauf, and Wilens (2015), only 5% of college students with an alcohol or substance use disorder received treatment. Given the prevalence and lack of engagement in treatment to address these behaviors, interventions that meet the unique needs of EAs are warranted. Therapist training and education in brief models of treatment could help address disparities in EA treatment engagement.

Brief intervention (psychotherapy involving only 1-3 hour-long sessions) can lead to decreases in health-risk behaviors with as little as one session (Neighbors et al., 2015). Brief interventions are also effective at decreasing treatment attrition (Mistier, Sheidow, & Davis, 2015). Among EAs, participation in one to two sessions might seem more manageable and could fit the lifestyle of those who are experiencing significant stressors. The Young Adult Family Check-Up (YA-FCU; Stormshak et al., 2019) and motivational interviewing (MI) have evidence supporting their efficacy at preventing and reducing health-risk behaviors among adolescent and EA samples, and both have been designed for use as brief treatment models (Caruther, Van Ryzin, & Dishion, 2014; Cushing, Jensen, Miller, & Leffingwell, 2014; Fosco, Frank, Stormshak, & Dishion, 2013).

The Family Check-Up and Motivational Interviewing

The Family Check-Up (FCU) is an evidence-based intervention designed to reduce youth problem behavior and improve mental health (Dishion & Stormshak, 2007). It uses a comprehensive multimethod approach to assess family functioning and includes three sessions: initial interview, ecological assessment, and feedback. Assessment-driven feedback is used to motivate parents to improve parenting practices, particularly in the areas of supervision, positive parenting, and management of their child’s behavior.

The Young Adult Family Check-Up (YA-FCU) was adapted from the FCU to target parenting skills during emerging adulthood and prevent the escalation of substance use and other risky behaviors. The YA-FCU utilizes a feedback structure that embraces the family system as a contextually important influence on motivation and behavioral change, yet also recognizes the developmental need for autonomy and EAs’ capacity to be a change agent. The feedback session of the YA-FCU is split into two parts; a family feedback session in which parents and EAs receive norm-based feedback on family interactions and functioning, and an individual feedback in which the EA meets individually with the therapist to receive norm-based feedback on their behaviors, support networks, and risk behavior. For the purposes of the current study, the individual feedback session will be the focus.

Motivational interviewing.

The YA-FCU incorporates MI consistent skills to provide feedback, with the goal of enhancing readiness to change. MI comprises a set of therapeutic techniques that are used in a manner that elicits and attends to client language about change (Miller & Rollnick, 2013). The goal is to minimize arguments that maintain the status quo (sustain talk; ST) and deepen arguments for change (change talk; CT). Therapeutic techniques drawn from MI, such as open-ended questions, affirmations, reflections, and summaries (OARS), are significantly more likely to elicit CT (Miller & Rollnick, 2013). The FRAMES (feedback, responsibility, advice, menu of options, empathy, and self-efficacy) model, which is inherent to the YA-FCU intervention, includes the use of OARS as a means of communicating with clients and responding to CT in an empathic, client-centered manner that reflects “MI spirit” (Miller & Rollnick, 2013). MI-spirit is composed of four elements: partnership, acceptance, evocation, and compassion (Miller & Rollnick, 2013). For more information on OARS, FRAMES, and MI spirit, please refer to Miller and Rollnick (2013).

Few studies have investigated FCU treatment fidelity, and the focus of those conducted was on ways in which feedback was delivered (e.g., Smith et al., 2013). However, no studies have investigated how FCU therapists’ in-session verbal responses to clients influence outcomes, or how core elements of MI, such as OARS skills and MI spirit, are used in the FCU to promote behavior change. Treatment fidelity is important to study because it allows for client outcomes to be differentially attributed to the intervention in question, provides information about how interventions are implemented across different sites and therapists, and informs the training of therapists (Waltz et al., 1993). Despite scant FCU fidelity studies, various MI fidelity studies have demonstrated a significant relationship between specific MI consistent skills, client language, and improvements in client health-risk behaviors (Barnett, Moyers, et al., 2014; Barnett, Spruijit-Metz, et al., 2014; Houck & Moyers, 2015; Miller & Rollnick, 2013). Thus, rendering MI important to investigate within the YA-FCU.

Current Study

In this study, we evaluated the extent to which YA-FCU therapists maintain MI fidelity to reduce health-risk behaviors among EAs during the YA-FCU feedback session. Therapists were expected to reach fair fidelity scores and use more MI adherent skills than MI non-adherent skills. We examined the relationship between use of MI and client CT during the YA-FCU feedback session. Higher levels of MI fidelity were hypothesized to relate to higher proportion of CT during the YA-FCU feedback sessions. Moreover, we investigated whether therapist MI fidelity and client CT in the YA-FCU feedback sessions are related to reductions in health-risk behaviors among EAs at post-test one year later. Higher levels of MI fidelity and proportion of CT were expected to predict decreases in EA health-risk behaviors.

Method

Sample

Participants comprised a subsample (n = 134) of a randomized, controlled trial (Project Alliance 2; Stormshak, HD 075150) that examined the efficacy of a family-centered substance use intervention delivered throughout adolescence and during emerging adulthood. The sample was drawn from an urban Pacific Northwest population. Participants and their caregivers were recruited from three socioeconomically and ethnically diverse public middle schools and were followed longitudinally through emerging adulthood. At initial enrollment, participants were in sixth grade (Mage = 11.87 years, SD = 0.46). Survey data were collected at two time points during emerging adulthood; prior to intervention (Mage = 19.82, range: 17.175 – 21.33), and 1 year after intervention (Mage = 21.38, range: 20 - 23.17). At initial recruitment, 386 of the 593 families were randomized to the treatment. During the emerging adulthood wave, 273 families were retained in the treatment condition and 134 completed the YA-FCU (49%). Data collection for this study was approved by the Institutional Review Board at the University of Oregon.

Our study used EA time point data only (T0, preintervention; T1, intervention; and T2, 1-year postintervention), which included youth self-report data about a range of behaviors. Participants included in the sample must have completed the feedback session of the YA-FCU at T1. Of the subsample, 95% were retained from T0 (N = 134) to T2 (N = 126). Males and females represented 48.7% and 51.3% of the study sample, respectively. The sample was culturally diverse: 58% European American/White, 33.6% African American/Black, 17.6% Hispanic/Latino, 7.6% Asian American, 5% Pacific Islander/Native American, and 2.5% other. In addition, 84.9% of the EAs in the study reported having graduated from high school or having received their general educational development certificate. At the time of intervention, 63% of the sample was enrolled in some college or a vocational training program and 37% were non-students.

Feedback session recording and coding.

Of the 134 participants who completed feedback sessions, 133 were video recorded and one session was not recorded. Three videos were excluded because the recording did not have working audio. An additional four videos were excluded because an individual feedback was not conducted with the EA; only the family feedback was available. Therefore, 126 videos were coded for therapist MI fidelity and client CT.

Procedure

Prior to the initial visit, participants were mailed questionnaires and either returned the completed documents to the lab at the initial visit (T0). Therapists met individually with the EAs to complete a feedback session focused on risk behavior, daily living, family relationships, daily stressors, physical/emotional health, and coping/self-esteem (Tl). At T2, questionnaires were mailed to participants with a self-addressed envelope so they could return the surveys by U.S. postal service.

Measures

Self-reported measures included demographics (age, gender, level of education, and employment status), and alcohol use, marijuana use and sexual behavior. An adapted measure of substance use was completed by EAs at T0 and T2 to assess frequency and quantity of alcohol, marijuana, and other drug use during the past 3 months (SUBSTS; Stormshak, Fosco, & Dishion, 2010). Observational data were coded for MI fidelity and client CT.

Alcohol use.

Items measuring alcohol use were asked in three sections (beer, wine/wine cooler/malt liquor, and hard liquor) and include, “How often did you drink beer in the last 3 months?” and “When you drank beer in the last 3 months, how much did you usually drink?” Items were rated on a scale ranging from 0 (never) to 8 (2–3 times a day or more), and 0 (less than one can) to 4 (four to five cans), respectively (α = .87). For the purposes of analysis, the three sections assessing alcohol use were combined to create two separate composite variables (alcohol frequency, alcohol quantity). A mean score was computed to represent alcohol frequency by averaging the reported scores for each of the alcohol frequency items across the three sections of alcohol use. A sum score was computed to represent alcohol quantity by adding the responses for each of the alcohol quantity measures across the three sections of alcohol use.

An additional set of variables was computed with the alcohol frequency and quantity variables to categorize participants into low- and high-risk groups for T0. Dichotomous items were computed by recoding Likert-type responses from the variables described previously to 0 (low risk) and 1 (high risk). The dichotomous variable for alcohol frequency was computed by coding participants who reported a 3 (once every 2–3 weeks) or lower for alcohol frequency as 0, and participants who reported a 4 (once a week) or higher as 1. Likewise, the dichotomous variable for alcohol quantity was computed by coding participants who reported a 3 (three standard drinks) or lower for alcohol quantity as 0, and participants who reported a 4 (4–5 standard drinks) or higher as 1.

Marijuana use.

The following two items were used to assess marijuana use: “How often did you use marijuana in the last 3 months?” and “When using marijuana, how much did you usually smoke?” Items were rated on a scale ranging from 0 (never) to 7 (2–3 times a day or more) and 0 (1–2 hits) to 5 (more than 2 bowls or joints), respectively (α = .91). Following the same procedure used to create the alcohol use variables, mean and sum scores were used to represent marijuana frequency and marijuana quantity, respectively.

Dichotomous variables were computed to indicate high- and low-risk groups by recoding Likert-type responses from the variables listed previously to 0 (low risk) and 1 (high risk) for T0. Participants who reported a 3 (once every 2–3 weeks) or lower for marijuana frequency were coded as 0, and participants who reported a 4 (once a week) or higher were coded as 1. Similarly, participants who reported a 4 (two or more bowls or joints) or lower for marijuana quantity were coded as 0, and participants who reported a 5 (more than two bowls or joints) were coded as 1.

Sexual behavior.

At T0 and T2, participants completed the Teen Interview (CINT) developed by researchers at the Child and Family Center to assess a variety of youth behaviors, including sexual behavior (Child and Family Center, 2001). CINT items were adapted to reflect the developmental changes that occur during emerging adulthood. Two items were used in our study to assess EA sexual risk behavior including, “Altogether during the last 3 months how many different people have you had as sexual partners (this includes intercourse and/or anal sex)?” and “In the last 3 months how many people have you had sex with and not used a condom?” The first item listed above represents number of sexual partners and is rated by a self-reported count. The second item listed above represents frequency of condomless sex and is reported on a 10-point scale. Response options range from 0 (0 times) to 9 (41 or more).

Dichotomous items were computed to indicate high- and low-risk groups by recoding counts and Likert-type responses from the variables listed previously to 0 (low risk) and 1 (high risk) for T0. To create the sexual partner risk groups, participants who reported two or fewer sexual partners were coded as 0, and participants who reported three or more sexual partners were coded as 1. Likewise, to create the frequency of condomless sex risk variable, participants who indicated a zero frequency of condomless intercourse were coded as 0 and those who endorsed any frequency of intercourse without a condom were coded as 1.

Therapist motivational interviewing fidelity.

Random 20-minute video segments from EA feedback sessions were coded for therapist MI fidelity using the Motivational Interviewing Treatment Integrity (MITI 4.1) coding protocol. Video segments were randomly selected using a randomization chart generated by Microsoft Excel. The MITI is a behavioral coding system that focuses on therapists’ verbal responses and includes both global ratings and behavior counts (Moyers et al., 2014). Four global ratings are used to capture MI spirit, each is coded using a 5-point Likert scale with a minimum of 1 and a maximum of 5. Global codes reflect the holistic evaluation of therapist behaviors and are separated into two domains: relational codes and technical codes (Moyers et al., 2014). The relational global codes measure the extent to which therapists engage in collaborative processes with clients (Partnership) and the extent to which therapists convey empathy (Empathy). The technical global codes measure the extent to which therapists effectively respond to both CT (Cultivating Change Talk) and ST (Softening Sustain Talk; Moyers et al., 2014). Behavioral counts measure the frequency of the following 10 therapist behaviors: giving information, persuading, persuading with permission, questioning, simple reflection, complex reflection, affirming, seeking collaboration, emphasizing autonomy, and confronting. Summary scores are computed after behavior counts and global codes are obtained and are recommended for analyses as they represent the critical indices of MI (Moyers et al., 2014). The summary scores include the following: technical (average of scores on Cultivating Change Talk and Softening Sustain Talk codes), relational (average of scores on Partnership and Empathy codes), ratio of reflections to questions (R:Q; total Reflections divided by total Questions), percent complex reflection (%CR; total complex reflections divided by the sum of the total complex and simple reflections codes), Mi-adherent (MIA; the sum of the seeking collaboration, affirming, and emphasizing autonomy codes), and MI non-adherent (MINA; the sum of the confront and persuade codes). Global scores are impacted by the manner in which therapists exhibit each behavior. For example, when therapists use reflections (both simple and complex), questions, and affirmations in a manner that strengthens and deepens CT, the technical score increases. Whereas, when therapists use reflections, questions, and affirmations in a way that strengthens or deepens ST, technical scores decrease. Similarly, the use of behaviors such as persuade with permission, seeking collaboration, and emphasizing autonomy increased relational scores. Whereas, the use of behaviors such as persuade and confront decreased relational scores. The MITI is one of the most extensively used measures of MI fidelity (Apodaca et al., 2016; Moyers et al., 2009; Moyers et al., 2016).

Training of therapists.

Four therapists provided the YA-FCU intervention and were assessed for MI fidelity: 1 graduate student, 1 post-doctoral research assistant, and 2 licensed psychologists. Therapists were trained in the YA-FCU through a 4-day workshop. MI training content included a 1-day intensive training focused on OARS, supporting self-efficacy, developing discrepancy, eliciting CT, and expressing empathy. After training, therapists observed three live YA-FCUs (all three sessions: initial interview, ecological assessment, and feedback session) and were subsequently observed leading two YA-FCUs. The final step before being authorized to lead the YA-FCU independently was to have the two observed YA-FCUs coded using the COACH rating system (Smith et al., 2013a). Therapists’ COACH ratings had to be within the satisfactory range (minimum score of 5). The COACH assesses five dimensions of therapist skill and an assessment of client engagement in the session (Smith et al., 2013). The five dimensions of the COACH represent the therapist’s conceptual accuracy of the FCU implementation, observant and responsive approach to clients’ context and needs, active structuring of sessions to optimize effectiveness, careful approach to teaching and providing feedback, and promotion of hope and motivation (Smith et al., 2013). COACH ratings of fidelity are indicated on a 9-point scale (1-3 = needs work, 4-6 = acceptable work, 7-9 = good work), with a score of 5 indicating sufficient fidelity to the FCU model (Smith et al., 2013). Weekly group supervision meetings were held throughout the duration of the study, and focused on consolidation of assessment data for the delivery of feedback to maximize treatment fidelity. Therapists did not receive Mi-focused coaching or feedback with the MITI 4.1 before or during the study.

Client change talk.

The CLEAR (Glynn & Moyers, 2012) coding system classifies and quantifies client language as either favoring (change talk) or countering (sustain talk) behavioral change. To measure client CT and ST, coders listened to the complete EA feedback session and tallied each occurrence of CT and ST using the CLEAR to yield total counts for each type of utterance. Examples of CT include, “Weed is expensive” and “I can quit if I want to.” Examples of ST include, “Drinking is fun” and “It doesn’t feel good to use condoms.” To better represent the proportion of CT used by clients during the YA-FCU feedback session, CT and ST counts were transformed into a composite variable by dividing total CT counts by the sum of CT and ST (CT / (CT + ST)).

Training of coders.

Five graduate students coded the observational data for this study. Three coders were assigned to code with the CLEAR, and two coders were assigned to code with the MITI 4.1. Coder training followed recommendations in the CLEAR and MITI 4.1 manuals (Moyers et al., 2014). Training included self-review of MI textbooks, video learning tools, participation in a 4-hour online MI training course developed by the Addiction Technology Transfer Center Network (ATTC, 2015; http://tourofmi.com), and an 8-hour in-person training on the MITI 4.1 and CLEAR coding. Coding was completed using only the audio from YA-FCU feedback sessions. Coders engaged in approximately 40 hours of practice coding to reach interrater reliability (IRR). IRR was considered poor for intraclass correlation (ICC) values of .40, fair for values of .41–.59, good for values of .60–.74, and excellent for values of .85–1.0 (Cicchetti, 1994). All CLEAR coders were within the good to excellent (.66-.98) range of IRR. MITI coders were within the fair to good range (.51-.91) for each of the MITI 4.1 summary scores except for the percentage of complex reflections (%CR), which fell in the poor range (.30). As a result, %CR was excluded from all the analyses. Intraclass correlations were conducted with the first author servings as the expert to which coders were compared for reliability. Expert status was established by engaging in a 3-day MITI 4.1 and CLEAR coding training and by completing coding with the MITI 4.1 and CLEAR until an IRR of .75 was reached in comparison to expert coders (co-author of the MITI 4.1 and CLEAR coding protocols - Denise Ernst, PhD). Frequent consultation and reliability checks were done throughout data collection. Coder drift was addressed through weekly coding meetings, and through conducting weekly ICCs for each coder.

Results

Therapist Treatment Fidelity (Aim 1)

Group means for the technical (M = 3.15, SD = 0.82), relational (M = 3.81, SD = 0.86), and R:Q (M = 1.49, SD = 1.12) codes were at or above the “fair” benchmark of fidelity. Therapists reached fair fidelity (fair = 3; good = 4) for the technical global. Therapist group means were approaching good fidelity for the relational global (fair = 3.5; good = 4) and R:Q score (fair = 1:1; good = 2:2). Benchmarks for fidelity were drawn from the MITI 4.1 manual and are based upon expert opinion rather than normative data (Moyers et al., 2014). Overall group means of MIA exceeded those of MINA.

Relationship Between MI Fidelity and Change Language (Aim 2)

Correlations between therapist MI fidelity and proportion of CT were examined to determine the relation between therapist and client behaviors. Spearman’s rank-order correlations were run to assess the relationship between proportion of CT and R:Q and MINA because those two MITI 4.1 summary score variables did not meet assumptions of normality. All other variables were normally distributed and therefore were analyzed with Pearson correlations. Based on Pearson correlations, there was a statistically significant positive correlation between the proportion of CT and technical score. Similarly, a statistically significant positive correlation was found between the proportion of CT and relational score. Relationships between the proportion of CT and R:Q, MIA, and MINA were not significant. See Table 1 for a summary of correlation results.

Table 1.

Therapist MI-Adherent Variables and Client Change Talk Correlations (N = 126).

Measures 1 2 3 4 5 a
1. Prop CT –
2. Technical .24** –
3. Relational .20* .61** –
4. MIA −.08 .17 .20* –
5. MINAa −.11 −.38** −.46** −.05 –
6. R:Qa −.03 .17 .25** −.08 −.14
Open in a new tab

Note. R:Q = ratio of reflections to questions, MIA = motivational interviewing-adherent, MINA = motivational interviewing nonadherent, Prop CT = proportion of change talk.

a

Spearman’s rho (rs) correlation coefficients.

*

p < .05

**

p < .01

***

p < .001.

Predictions of EA Postintervention Health-Risk Behaviors (Aim 3)

Poisson and negative binomial regressions were conducted to determine the amount of variance explained by MI fidelity (based on the MITI 4.1 summary scores) and proportion of CT for each of the following dependent variables at T2: alcohol use frequency, alcohol use quantity, marijuana use frequency, marijuana use quantity, number of sexual partners, and frequency of condomless sex. Goodness of fit was determined by a deviance/df ≤ 1.50. Specifically, if a model demonstrated inadequate model fit with Poisson regression, then Negative Binomial regressions were conducted. To account for multiple comparisons and type I error, Bonferroni corrections were made using a p ≤ .01. Table 2 provides a summary of the regression results.

Table 2.

Poisson and Negative Binomial Regressions Predicting Postintervention Emerging Adult Health-Risk Behavior

Health-risk behavior β (SE)
Variable/category Alcohol frequency1 Alcohol quantity2 Marijuana frequency2 Marijuana quantity2 Number of sexual partners1 Frequency of condomless sex1
Risk group
  Low-risk (0) −0.70 (0.18)* −0.93 (0.21)* −0.85 (0.24)* 1.22 (0.28)* −0.98 (0.25)* 0.66 (0.20)*
  High-risk (1) – – – – – –
Technical −0.12(0.11) −0.19 (0.17) −0.39 (0.19)† −0.47 (0.22)† −0.10(0.14) −0.34 (0.16)†
Relational 0.23 (0.11)† 0.16 (0.17) 0.22 (0.20) 0.26 (0.22) 0.16(0.15) 0.28 (0.16)
R:Q −0.04 (0.06) −0.02 (0.10) −0.14 (0.12) −0.45 (0.17)* 0.03 (0.08) 0.03 (0.08)
MIA −0.03 (0.02) −0.02 (0.02) −0.01 (0.06) −0.04 (0.03) −0.02 (0.02) 0.02 (0.02)
MINA −0.00 (0.03) −0.05 (0.05) −0.02 (0.06) 0.03 (0.05) −0.04 (0.04) 0.03 (0.04)
Prop CT −1.54 (0.54)* −1.33 (0.82) −0.86 (0.88) −2.93 (1.02)* −0.60 (0.65) −0.73 (0.76)
Likelihood ratio χ2 30.26* 27.10* 22.97* 36.92* 15.83† 17.59*
Degrees of 7 7 7 7 7 7
freedom
Number of cases 126 126 126 126 126 126
Open in a new tab

Note. “Risk group” represents the risk group variable that was computed individually for each of the health-risk behaviors: alcohol use frequency risk, alcohol use quantity risk, marijuana use frequency risk, marijuana use quantity risk, sexual partner risk, and condom use frequency risk. R:Q = ratio of reflections to questions, MIA = motivational interviewing-adherent, MINA = motivational interviewing nonadherent, Prop CT = proportion of change talk.

1

Poisson regression.

2

Negative binomial regression.

†

Numbers in bold and marked with have a p < .05.

*

Numbers in bold and with have a p ≤ .01.

Alcohol use.

Beginning with alcohol frequency as the count variable, Poisson regression was conducted. The likelihood ratio chi-square test indicated that the full model was significant (p < .001). Baseline (T0) alcohol frequency risk (p < .001) and proportion of CT (p < .01) were statistically significant predictors of alcohol frequency at T2, and therapist relational score was a marginally significant predictor (p < .05). Based on these data, we predict that initially low-risk participants (T0: alcohol use frequency risk = 0) would have a decrease in alcohol frequency at T2 by a factor of 0.50, and higher proportion of CT would predict a decrease in alcohol frequency at T2 by a factor of 0.22. Additionally, higher relational scores would predict an increase in T2 alcohol frequency by a factor of 1.26. All other indicators of MI fidelity were not significant predictors of T2 alcohol frequency.

Negative binomial regression was conducted to examine relations among initial risk, MI fidelity, proportion of CT, and T2 alcohol quantity. The likelihood ratio chi-square test indicates that the full model was significant (p < .001). TO alcohol quantity risk was the only significant predictor (p < .001). Based on these results, we predict that initially low-risk participants would have a decrease in alcohol quantity at T2 by a factor of 0.40. MI fidelity and the proportion of CT were not significant predictors of T2 alcohol quantity.

Marijuana use.

Negative binomial regression was conducted to examine relations amongst initial risk, MI fidelity, the proportion of CT, and T2 marijuana frequency. The likelihood ratio chi-square test indicates that the full model was significant (p < .01). T0 marijuana frequency risk was the only statistically significant predictor (p < .001), and therapist technical score was a marginally significant predictor (p < .05). That is, we predict that initially low-risk participants would have a decrease in marijuana frequency by a factor of 0.43 at T2. Similarly, higher therapist technical scores predicted a decrease in marijuana frequency at T2 by a factor of 0.68. All other indicators of MI fidelity and the proportion of CT were not significant predictors of T2 marijuana frequency.

Negative binomial regression was conducted to determine how much of the variance in T2 marijuana quantity is accounted for by predictors. The likelihood ratio chi-square test indicates that the full model was significant (p < .001). T0 marijuana quantity risk (p < .001), therapist R:Q (p = .01), and the proportion of CT (p < .01) were significant predictors of T2 marijuana quantity. Therapist technical score (p < .05) was also a marginally significant predictor of T2 marijuana quantity. Based on these results, we predict that initially low-risk participants would have an increase in marijuana quantity at T2 by a factor of 3.40. Higher therapist R:Q scores predicted a decrease in marijuana quantity at T2 by a factor of 0.64, higher therapist technical scores predicted a decrease in marijuana quantity at T2 by a factor of 0.63, and higher proportion of CT predicted a decrease in marijuana quantity at T2 by a factor of 0.05.. All other indicators of MI fidelity were not significant predictors of T2 marijuana quantity.

Sexual behavior.

Poisson regression was conducted to determine how much of the variance in T2 number of sexual partners is accounted for by predictors. The likelihood ratio chi-square test indicates that the full model was marginally significant (p < .05). T0 sexual partner risk was the only statistically significant predictor (p < .001). Based on these results, we would predict that initially low-risk participants would have a decrease in number of sexual partners at T2 by a factor of 0.38. MI fidelity and proportion of CT were not significant predictors of T2 number of sexual partners.

Poisson regression was conducted to determine how much of the variance in T2 frequency of condomless sex is accounted for by the predictors. The likelihood ratio chi-square test indicates that the full model was significant (p = .01). T0 condom use risk (p < .001) was the only statistically significant predictor, and therapist technical score was a marginally significant predictor (p < .05). Based on these data, we would predict that initially low-risk participants would have an increase in frequency of condomless sex at T2 by a factor of 1.99. In contrast, higher therapist technical scores predicted a decrease in frequency of condomless sex at T2 by a factor of 0.71. All other indicators of MI fidelity and proportion of CT were not significant predictors of T2 condom use frequency.

Discussion

Therapists conducted the YA-FCU with fair fidelity to MI. Global indicators of MI fidelity (i.e., technical and relational scores) were positively related to the proportion of CT in YA-FCU feedback sessions. Some indicators of MI fidelity and the proportion of CT predicted changes in post-intervention health-risk behaviors; higher technical scores predicted decreases in marijuana frequency and quantity and decreases in condomless sex. Similarly, higher R:Q predicted decreases in marijuana quantity and higher proportions of CT predicted decreases in alcohol frequency and marijuana quantity. Lastly, higher relational scores predicted an increase in alcohol frequency.

Therapist group means on the MITI 4.1 met the minimum “fair” competency benchmark, and therapists used more frequent MIA behaviors than MINA behaviors during feedback sessions. These findings support previous research suggesting therapists with previous clinical experience have higher proficiencies in MI than community provider samples (Darnell et al., 2016; Miller et al., 2004). These results bring to question whether therapists’ MI fidelity could be even further improved through more comprehensive MI training and targeted feedback and coaching of MI skills. Previous research recommends an eight-stage model for learning MI (Miller & Moyers, 2006) and studies investigating MI training suggest incorporating feedback and coaching of MI skills to reach maximum proficiency (Madson, Loignon, & Lane, 2009). Given that initial YA-FCU training only covered some aspects of MI (e.g., OARS, expressing empathy, and eliciting CT) and the weekly supervision in which therapists participated was focused on consolidation of assessment data for the delivery of feedback, therapists’ MI fidelity might have been limited.

Partially consistent with previous research and this study’s hypothesis, global indicators of MI fidelity had a positive correlation with the proportion of CT (Cushing et al., 2014; Darnell et al., 2016). However, there were no significant correlations between MIA, MINA, R:Q and the proportion of CT, which suggests that merely utilizing MI consistent skills may not be enough to increase client CT. Perhaps, MI spirit and the manner in which MI consistent skills are applied is most important when eliciting CT. Literature on MI practices highlights that MI is more of a conversational style rather than a set of skills that can be applied, and MI spirit is one of the key factors that distinguishes MI from other interventions (Miller & Moyers, 2006; Miller & Rollnick, 2013; Moyers et al., 2009).

Initial risk status across each type of health risk behavior was the greatest predictor of T2 health risk behavior, and our predictions regarding MI fidelity and CT were only partially supported. When participants are initially classified as low-risk at T0, they are significantly more likely to have a decrease in health risk behavior at T2, except when predicting marijuana quantity and frequency of condomless sex. Initial low-risk for marijuana quantity predicted increases in marijuana quantity at T2, and initial low-risk condom frequency predicted increases in frequency of condomless sex at T2. Increases in marijuana quantity might reflect recent changes in local laws governing the recreational use of cannabis that went into effect in July, 2015 (as data for T2 were collected between 2014-2016). Easier access to cannabis and its decriminalization might have contributed to changes in marijuana quantity. However, the same phenomenon was not observed for marijuana frequency. Tolerance might explain this discrepancy in that, over time, marijuana users might increase quantity of use due to decreased intoxication effects; thus, using higher quantities per occasion to gain the same effects (Babor, Mendelson, Greenberg, & Kuehnle, 1975). Increases in sexual intercourse without a condom might be reflective of normative EA development. As EAs get older, they are more likely to enter into committed relationships with a single partner, rendering the use of condoms less important (O’Sullivan, Udell, Montrose, Antoniello, & Hoffman, 2010). Additionally, it is possible that EAs, upon entering a committed relationship, rely on other forms of contraception (e.g., birth control pills) as the risk of contracting an STI is perceived as less threatening (O’Sullivan et al., 2010).

MI fidelity and the proportion of CT inconsistently predicted post-intervention risk behaviors. Therapist technical skills were marginally predictive of marijuana use and condom use. Whereby, high technical scores predicted reductions in marijuana frequency and quantity and reductions in frequency of condomless sex. R:Q significantly predicted reductions in marijuana quantity but did not predict changes in any other risk behaviors. Similarly, the proportion of CT significantly predicted decreases in alcohol frequency and marijuana quantity but did not predict changes in other risk behaviors. Whereas, relational skills only marginally predicted alcohol frequency, and did so in the opposite direction than expected. The mean age of participants at T2 was 21 years (SD = 0.62, range = 20-23). As such, reaching the legal age to consume and purchase alcohol might have played a role in the minimal treatment effects on alcohol use. Study participants are still in a stage of life where normative drinking is generally higher than that of older adults (Miech et al., 2015), and EAs might not experience as much ambivalence about their drinking because it is more socially acceptable. MI aims to highlight discrepancies in clients’ own arguments for and against behavior change in order to elicit CT (Miller & Rollnick, 2013). Therefore, these alcohol use findings might indicate a lack of social consequences and therefore less ambivalence about drinking behaviors in the current sample. Additionally, researchers find that normative alcohol use trajectories begin to level off as EAs approach the mid- to late-20s (Ashenhurst, Harden, Corbin, & Fromme, 2015). It is possible that a change in drinking behaviors will occur naturally as a reflection of personal characteristics (e.g., initial alcohol use behaviors) rather than a reflection of intervention effect.

Furthermore, although therapists used MIA more than MINA, the frequency with which MIA was used might not have counterbalanced the times when MINA was used. As aforementioned, research suggests that avoiding MINA is highly influential in predicting positive client outcomes (Gaume et al., 2009). In contrast, R:Q influences therapists’ technical scores because those skills are used to draw out and strengthen CT as well as to soften and reduce ST. With this in mind, and taking into account research suggesting that CT is a causal mechanism of change (e.g., Moyers et al., 2009), it is not surprising that R:Q and technical scores were significant predictors in at least a few of the regression models. Furthermore, given that CT is a causal mechanism of MI (Moyers & Martin, 2006; Moyers et al., 2009), it is surprising that the proportion of CT was only predictive of two health risk behaviors. When coding for CT and ST for the current study, utterances were not flagged or separated by the target change behavior (i.e., type of health risk behavior). For example, if a client stated, “I hate feeling hung over after drinking,” this statement was coded generically as CT. Therefore, the proportion of CT variable could comprise a majority of CT and ST statements about one health risk behavior (e.g., marijuana use), and not take into account other target change behaviors.

Implications

Although prior work supports the efficacy of the FCU model on reduction of risk behaviors (Stormshak et al., 2019), current findings add to the literature by documenting the influence of MI consistent skills on FCU client outcomes, specifically within the context of predicting marijuana use. These findings provide a potential focus for FCU therapist training and supervision. FCU therapists in this study maintained fidelity to MI. However, there is still room for improvement and a need for more careful MI training and monitoring of fidelity. Targeted feedback and coaching could prove to be useful for further improving treatment integrity. Monitoring of treatment fidelity over time is recommended to prevent drift from fidelity (Chiapa et al., 2015) and is particularly relevant to the current study, given that data were collected longitudinally. The MITI 4.1 is a useful tool that therapists can use independently to monitor their treatment fidelity and it can be used as a supervision tool for structuring feedback (Moyers et al., 2014; Moyers et al., 2016). Evidence suggests that higher fidelity to both MI and the FCU model significantly predicts improvements in client outcomes (Gaume et al., 2009; Smith et al., 2013), and the FCU intervention, in and of itself, is efficacious at producing positive changes in families and youth, without accounting for specific MI techniques (Caruther et al., 2014). The current findings suggest that MI fidelity might be a mechanism of change within the FCU, specifically with regard to targeting changes in frequency of marijuana use. Moreover, the effect of therapist technical scores and R:Q on client health risk behaviors suggest a potential benefit of therapists’ attention to client language, and the use of specific MI consistent skills (i.e., reflections and open-ended questions) could be an area of focus in FCU therapist training.

Limitations and Future Directions

There are a number of limitations to consider when interpreting results. Achieving IRR on the MITI 4.1 proved to be difficult. Specifically, the ICCs for technical and MINA scores in our study are lower than those found by Moyers et al. (2016) in a study assessing the preliminary reliability and validity of the MITI 4. However, our study ICCs for the technical score were comparable to those of the two best coders in the Moyers et al. (2016) study. Furthermore, coders in our study and in the study by Moyers et al. (2016) were unable to reach a satisfactory IRR on the %CR score. Additionally, coding for the current study involved the use of more than one target behavior. Moyers et al. (2014) recommend that target behaviors are identified prior to coding, but typically only one target behavior is used. Relatedly, the CLEAR coding did not flag which utterances related to specific target behaviors. Therefore, the measures of CT could be a representation of language that is not generalizable across each of the domains of health risk behavior that were measured in the current study. Future research should identify one primary target behavior as those are the recommendations in the MITI 4.1 and CLEAR coding manuals (Glynn & Moyers, 2012; Moyers et al., 2014). If more than one target behavior is warranted, delineating what client utterances pertain to specific target behaviors could be useful when running and interpreting analyses.

In addition, therapist characteristics may have influenced client outcomes. Future studies of FCU treatment fidelity should consider examining the data using multilevel modeling (MLM), with clients nested within therapist to determine if significant effects of therapist on client outcomes exist. Moreover, self-selection might have impacted the internal validity of the study, given that this study’s sample was only 49% of the original sample. Finally, this study highlights the utility of the YA-FCU in addressing certain health risk behaviors among EAs but does not address challenges with engaging EAs in treatment. Due to the brief nature of the YA-FCU, this might increase engagement; however, future studies should explicitly examine factors related to engagement to determine if the YA-FCU does in fact address some of the barriers to treatment.

Conclusions

Despite these limitations, our study adds important information to the current literature base about MI and YA-FCU interventions. To find significant results from only one therapist session is remarkable and lends important implications for intervening in the lives of EAs. Furthermore, this study is unique in that the sample is comprised of both college-attending and non-college-attending EAs. Prior research with individuals in this developmental stage has focused predominantly on students at universities. Yet, there is evidence to suggest that EAs who are not in college experience similar elevated levels of substance use and risky behaviors. Results from this study suggest that the YA-FCU has the potential to meet the unique needs of the EA population and could aid in the reduction of health risk behaviors during this stage of life.

Acknowledgments

This research was funded by NIDA grant DA 018374 and NICHD grant HD 075150, both to the second author.

The authors whose names are listed above certify that they have NO affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers’ bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript.

Biosketches

Elisa DeVargas received her MS in counseling psychology and human service and PhD in counseling psychology from the University of Oregon. She is currently a post-doctoral fellow at the New Mexico Veteran Administration Healthcare System in Albuquerque, NM. Her clinical interests include child and adolescent development, parent skills training, and family therapy. Her research interests include substance use prevention, treatment outcomes, and Latinx mental health.

Elizabeth Stormshak received her MS and PhD in child clinical psychology from Penn State University. She is currently the director of the Prevention Science Institute, has expertise in the area of prevention, including prevention of substance use, problem behavior, and later mental health problems in children and youths. Her research focuses on the development of family-centered, model-driven interventions designed to reduce problem behavior and promote successful developmental transitions. She has served as the principal investigator on multiple grants, including randomized trials that tested the efficacy and effectiveness of family-centered models of prevention to reduce risk behavior in early childhood, in school-age children, and in adolescents, with a primary focus on enhancing parenting skills and behavioral management.

References

  1. Apodaca TR, Jackson KM, Borsari B, Magill M, Longabaugh R, Mastroleo NR, & Barnett NP (2016). Which individual therapist behaviors elicit client change talk and sustain talk in motivational interviewing? Journal of Substance Abuse Treatment, 61, 60–65. doi: 10.1016/j.jsat.2015.09.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Arnett JJ (2000). Emerging adulthood: A theory of development from the late teens through the twenties. American Psychologist, 55(5), 469–480. doi: 10.1037/0003066X.55.5.469. [DOI] [PubMed] [Google Scholar]
  3. Arnett JJ (2015). Identity development from adolescence to emerging adulthood: What we know and (especially) don’t know. In McLean KC & Syed M (Eds.), Oxford library ofpsychology. The Oxford handbook of identity development (pp. 53–64). New York, NY, US: Oxford University Press. [Google Scholar]
  4. Ashenhurst JR, Harden KP, Corbin WR, & Fromme K (2015). Trajectories of binge drinking and personality change across emerging adulthood. Psychology of Addictive Behaviors, 29(4), 978–991. doi: 10.1037/adb0000116. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Babor TF, Mendelson JH, Greenberg I, & Kuehnle JC (1975). Marijuana consumptions and tolerance to physiological and subjective effects. Archives of General Psychiatry, 32(12, 1548–1552. doi: 10.1001/archpsyc.1975.01760300086007. [DOI] [PubMed] [Google Scholar]
  6. Barnett E, Moyers TB, Sussman S, Smith C, Rohrbach LA, Sun P, & Spruijt-Metz D (2014). From counselor skill to decreased marijuana use: Does change talk matter? Journal of Substance Abuse Treatment, 46, 498–505. doi: 10.1016/j.jsat.2013.11.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Barnett E, Spruijt-Metz D, Moyers TB, Smith C, Rohrbach LA, Sun P, & Sussman S (2014). Bidirectional relationship between client and counselor speech: The importance of reframing. Psychology of Addictive Behaviors, 28(4), 1212–1219. doi: 10.1037/a0036227. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Caruthers AS, Van Ryzin MJ, & Dishion TJ (2014). Preventing high-risk sexual behavior in early adulthood with family interventions in adolescence: Outcomes and developmental processes. Prevention Science, 75(1), S59–S69. doi: 10.1007/s11121-013-0383-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Centers for Disease Control (CDC). Sexually transmitted disease surveillance 2015. Atlanta, GA: U.S. Department of Health and Human Services. [Google Scholar]
  10. Chiapa A, Smith JD, Kim H, Dishion TJ, Shaw DS, & Wilson MN (2015). The trajectory of fidelity in a multiyear trial of the Family Check-Up predicts change in child problem behavior. Journal of Consulting and Clinical Psychology, 83(5), 1005–1011. doi: 10.1037/ccp0000034. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Child and Family Center. (2001). Teen Interview (CINT). Unpublished instrument, Child and Family Center, University of Oregon, Eugene, OR. [Google Scholar]
  12. Cicchetti DV (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290. [Google Scholar]
  13. Cushing CC, Jensen CD, Miller MB, & Leffingwell TR (2014). Meta-analysis of motivational interviewing for adolescent health behavior: Efficacy beyond substance use. Journal of Consulting and Clinical Psychology, 82(6), 1212–1218. doi: 10.1037/a0036912. [DOI] [PubMed] [Google Scholar]
  14. Darnell D, Dunn C, Atkins D, Ingraham L, & Zatzick D (2016). A randomized evaluation of motivational interviewing training for mandated implementation of alcohol screening and brief intervention in trauma centers. Journal of Substance Abuse Treatment, 60, 36–44. doi: 10.1016/j.jsat.2015.05.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Dishion TJ, & Stormshak EA (2007). Intervening in children’s lives: An ecological, family-centered approach to mental health care. Washington, DC: American Psychological Association. [Google Scholar]
  16. Fosco GM, Frank JL, Stormshak EA, & Dishion TJ (2013). Opening the “Black Box”: Family check-up intervention effects on self-regulation that prevents growth in problem behavior and substance use. Journal of School Psychology, 51, 455–468. doi: 10.1016/j.jsp.2013.02.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Gaume J, Gmel G, Faouzi M, & Daeppen J-B (2009). Counselor skill influences outcomes of brief motivational interventions. Journal of Substance Abuse Treatment, 37(2), 151–159. http://dx.doi.org.libproxy.uoregon.edu/10.1016/j.jsat.2008.12.001. [DOI] [PubMed] [Google Scholar]
  18. Glynn LH, & Moyers TB (2012). Manual for the Client Language EAsy Rating (CLEAR) Coding System: Formerly “Motivational Interviewing Skill Code (MISC) 1.1.” Retrieved from http://casaa.unm.edu/codinginst.html.
  19. Houck JM, & Moyers TB (2015). Within-session communication patterns predict alcohol treatment outcomes. Drug and Alcohol Dependence, 157, 205–209. doi: 10.1016/j.drugalcdep.2015.10.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Lee DC, Schlienz NJ, Peters EN, Dworkin RH, Turk DC, Strain EC, & Vandrey R (2019). Systematic review of outcome domains and measures used in psychosocial and phrarmacological treatment trials for cannabis use disorder. Drug and Alcohol Dependence, 194, 500–517. 10.1016/j.drugalcdep.2018.10.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Madson MB, Loignon AC, & Lane C (2009). Training in motivational interviewing: A systemic review. Journal of Substance Abuse Treatment, 36, 101–109. doi: 10.1016/j.jsat.2008.05.005. [DOI] [PubMed] [Google Scholar]
  22. Miech RA, Johnston LD, & O’Malley PM (2015). Bachman JG, Schulenberg JE Monitoring the Future National Survey Results on Drug Use 1975-2014. Ann Arbor, Institute for Social Research, The University of Michigan. [Google Scholar]
  23. Miller WR, Yahne CE, Moyers TB Martinez J, & Pirritano M (2004). A randomized trial of methods to help clinicians learn motivational interviewing. Journal of Consulting and Clinical Psychology, 72(6), 1050 – 1062. DOI: 10.1037/0022-006X.72.6.1050. [DOI] [PubMed] [Google Scholar]
  24. Miller WR & Moyers TB (2006). Eight stages in learning motivational interviewing. Journal of Teaching in the Addictions, 5(1), 3–17. DOI: 10.1300/J188v05n01_02. [DOI] [Google Scholar]
  25. Miller WR, & Rollnick S (2013). Motivational interviewing: Helping people change (3rd ed.). New York, NY: Guilford Press. [Google Scholar]
  26. Mistler LA, Sheidow AJ, & Davis M (2015). Transdiagnostic motivational enhancement therapy to reduce treatment attrition: Use in emerging adults. Cognitive and Behavioral Practice, 23(3), 368–384. doi: 10.1016/j.cbpra.2015.09.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Moyers TB, Manuel JK, & Ernst D (2014). Motivational Interviewing Treatment Integrity coding manual 4.1. Unpublished manual. Retrieved from http://casaa.unm.edu/codinginst.html. [DOI] [PMC free article] [PubMed]
  28. Moyers TB, Martin T, Houck JM, Christopher PJ, & Tonigan JS (2009). From in-session behaviors to drinking outcomes: A causal chain for motivational interviewing. Journal of Consulting and Clinical Psychology, 77(6), 1113–1124. doi: 10.1037/a0017189. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Moyers TB, Rowell LN, Manuel JK, Ernst D, & Houck JM (2016). The Motivational Interviewing Treatment Integrity Code (MITI 4): Rationale, preliminary reliability and validity. Journal of Substance Abuse Treatment, 65, 36–42. doi: 10.1016/j.jsat.2016.01.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Neighbors C, Rodriguez LM, Rinker DV, Gonzales RG, Agana M, Tackett JL, & Foster DW (2015). Efficacy of personalized normative feedback as a brief intervention for college student gambling: A randomized controlled trial. Journal of Consulting and Clinical Psychology, 53(3), 500–511. doi: 10.1037/a0039125. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. O’Sullivan LF, Udell W, Montrose VA, Antoniello P, & Hoffman S (2010). A cognitive analysis of college students’ explanations for engaging in unprotected sexual intercourse. Archives of Sexual Behavior, 39(5), 1121–1131. doi: 10.1007/s10508-009-9493-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Pedrelli P, Nyer M, Yeung A, Zulauf C, & Wilens T (2015). College students: Mental health problems and treatment considerations. Academic Psychiatry: The Journal of the American Association of Directors of Psychiatric Residency Training and the Association for Academic Psychiatry, 39(5), 503–511. Doi: 10.1007/s40596-014-0205-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Stormshak EA, Caruthers A, Chronister KM, DeGarmo DD, Stapleton J, Falkenstein C, DeVargas E, & Nash W (2019). Reducing risk behavior with family-centered prevention during the young adult years. Prevention Science, 2(3), 321–330. doi: 10.1007/s11121-018-0917-2 PMC6310108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Smith JD, Dishion TJ, Moore KJ, Shaw DS, & Wilson MN (2013). Effects of video feedback on early coercive parent-child interactions: The intervening role of caregivers’ relational schemas. Journal of Clinical Child and Adolescent Psychology, 42(3), 405–417. doi: 10.1080/15374416.2013.777917. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Stormshak EA, Fosco GM, & Dishion TJ (2010). Implementing interventions with families in schools to increase youth school engagement: The Family Check-Up model. School Mental Health, 2(2), 82–92. doi: 10.1007/s12310-009-9025-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Waltz J, Addis ME, Koerner K, & Jacobson NS (1993). Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence. Journal of Consulting and Clinical Psychology, 61(4), 620–630. 10.1037/0022-006X.6L4.620 [DOI] [PubMed] [Google Scholar]
  37. White HR, McMorris BJ, Catalano RF, Fleming CB, Haggerty KP, & Abbott RD (2006). Increases in alcohol and marijuana use during the transition out of high school into emerging adulthood: The effects of leaving home, going to college, and high school protective factors. Journal of Studies on Alcohol, 67(6), 810–822. doi: 10.15288/jsa.2006.67.810. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES