A randomized evaluation of motivational interviewing training for mandated implementation of alcohol screening and brief intervention in trauma centers

Abstract

The American College of Surgeons has mandated that Level I and Level II trauma centers implement universal alcohol screening and brief intervention (SBI) for injured patients. This study was a secondary analysis of a national, 20-hospital, cluster-randomized implementation trial focusing on practical issues of training and supervising alcohol SBI providers in Motivational Interviewing (MI). The purpose of this study was to examine whether real-world trauma center providers can be trained to provide higher quality counseling using MI as part of brief interventions for alcohol and whether MI skills can be maintained over time. Sites were randomly assigned to receive a one day workshop training in MI for alcohol SBI or not, and all providers regardless of training completed up to seven standardized patient assessments of MI fidelity over 27 months. Six domains on the Motivational Interviewing Treatment Integrity (MITI) coding system were assessed and compared to proficiency criteria. Providers in the intervention training group showed substantially improved MITI scores over the course of the 27-month time period. Domains that had particularly strong improvement were MI Spirit and Empathy; however, despite the overall improvement in the intervention group scores, expert-derived proficiency criteria were attained only for the global scores. Routine trauma center providers who receive MI training can deliver higher quality counseling in alcohol brief interventions, but may not, however, attain previously derived proficiency standards. Future implementation efforts in real-world acute care medical settings could further elucidate provider characteristics that predict training response and also strive to demonstrate that higher quality alcohol SBI implementation is associated with improved patient-level outcomes.

1. Introduction

Each year in the United States (U.S.), over 30 million individuals present to acute care medical settings for the treatment of traumatic injury, and 1.5–2.5 million Americans are so severely injured that they require inpatient surgical hospitalization (American College of Surgeons, U.S. Department of Health and Human Services, & Department of Transportation, 2010; American College of Surgeons Committee on Trauma, 2006; Bergen & National Center for Health Statistics (U.S.), 2008; Bonnie, Fulco, Liverman, & Control, 1999; National Center for Injury Prevention, 2012). Alcohol use problems are endemic among injured trauma survivors (Soderstrom et al., 1997; Zatzick et al., 2012). Thus, the widespread implementation of high quality alcohol screening and brief intervention (SBI) into acute injury care has the potential to markedly increase the population impact of prevention efforts and has been a long-standing international public health objective (Jonas et al., 2012; Koepsell, Zatzick, & Rivara, 2011; Rehm et al., 2009; Room, Babor, & Rehm, 2005; Rose, 1992). Over the past decade, the American College of Surgeons has established mandates for alcohol screening and brief intervention at trauma centers; currently the College requires universal alcohol screening and intervention for Level I and II trauma centers nationally (American College of Surgeons Committee on Trauma, 2014; Terrell et al., 2008). This represents the first ever U.S. policy mandate for the integrated treatment of alcohol use problems in a general medical setting. The American College of Surgeons’ mandate allows specific alcohol SBI procedures and SBI provider preparation and training to be left to the discretion of each trauma center and initial reports suggest that the mandate is implemented with marked variability (Terrell et al., 2008). There is a risk that lower quality alcohol SBI counseling and procedures could become the default standard of trauma center care nationwide. The routine use of an evidence-based counseling approach, such as Motivational Interviewing (MI), which emphasizes patient autonomy and exploration of patients’ reasons and motivation for change (Miller & Rollnick, 2014), could help to insure that the mandate is implemented with sufficient quality.

Given that MI is consistent with recommendations for how to conduct brief interventions by the College (American College of Surgeons et al., 2010), MI is commonly used in alcohol SBI across general medical settings (Field, Hungerford, & Dunn, 2005; Miller & Rollnick, 2014). Brief interventions using MI appear to be generally effective in trauma center, emergency department, medical inpatient, and primary care general medical settings, although treatment effects vary across settings and patient populations (Bertholet, Palfai, Gaume, Daeppen, & Saitz, 2014; Bray, Cowell, & Hinde, 2011; Cunningham et al., 2009; D’Onofrio et al., 2012; D’Onofrio et al., 2010; Field, Baird, Saitz, Caetano, & Monti, 2010; Havard, Shakeshaft, & Sanson-Fisher, 2008; Jonas et al., 2012; Madras et al., 2009; Nilsen et al., 2008; Vasilaki, Hosier, & Cox, 2006; Whitlock, Polen, Green, Orleans, & Klein, 2004). More specifically, with injured patients admitted to trauma centers, brief interventions using MI have been shown to improve alcohol use problems (Field et al., 2014; Field, Caetano, Harris, Frankowski, & Roudsari, 2010; Gentilello et al., 1999; Schermer, Moyers, Miller, & Bloomfield, 2006).

The large body of literature on MI training points to factors that may influence the ability to teach and learn high quality MI-based brief alcohol interventions (Barwick, Bennett, Johnson, McGowan, & Moore, 2012; de Roten, Zimmermann, Ortega, & Despland, 2013; Madson, Loignon, & Lane, 2009; Schwalbe, Oh, & Zweben, 2014; Soderlund, Madson, Rubak, & Nilsen, 2011). This literature suggests that specific training models are associated with improved training results. MI training routinely includes a 1–3 day workshop consisting of didactics, demonstrations, and active learning exercises (Madson et al., 2009), with shorter one day trainings more common in healthcare settings (Soderlund et al., 2011). Although these workshops can improve immediate MI skills (Baer et al., 2009; Miller & Mount, 2001), randomized controlled trials have documented improved learning and skill retention when workshops are followed by ongoing feedback and coaching (Miller, Yahne, Moyers, Martinez, & Pirritano, 2004; Moyers et al., 2008; Smith et al., 2012).

Our literature review revealed a paucity of comprehensive training studies examining SBI performed in the unique acute care trauma medical center mandated implementation context. Therefore, the current investigation was a secondary analysis of a larger cluster randomized implementation trial of alcohol SBI using MI with predominantly acute care nurses and social workers at 20 U.S. trauma centers (Zatzick et al., 2014). The intervention group providers were given a workshop with feedback and coaching in concert with the American College of Surgeons’ guidelines (American College of Surgeons Committee on Trauma, 2014) and then followed longitudinally using a novel 27-month standardized patient design (Zatzick et al., 2013). Findings from the trial indicated that alcohol SBI delivered by study-trained providers was associated with modest reductions in alcohol use problems among patients, particularly those without traumatic brain injury.

A crucial implementation issue in this mandated regulatory context is that trauma centers typically designate current employees to be SBI providers without regard for counseling skill or trainability (Zatzick et al., 2014; Zatzick et al., 2013). These service delivery characteristics raise questions for nationwide trauma center implementation efforts in the wake of the American College of Surgeons’ alcohol SBI mandate. First, can routine trauma center providers randomized to a one day workshop and follow-up feedback and coaching demonstrate sustainable gains in MI skills over a 27-month period? Second, can SBI providers be trained to MI proficiency criteria?

2. Material and Methods

2.1 Design

The current investigation is a secondary analysis of training implementation and outcomes from the Disseminating Organizational Screening and Brief Intervention Services (DO-SBI) study (Zatzick et al., 2014; Zatzick et al., 2013), a cluster randomized implementation trial of the effects of a multilevel (i.e., trauma center and provider levels) intervention targeting the delivery of high quality alcohol SBI services at 20 U.S. Level I trauma centers. The goal of the DO-SBI study was to harness the opportunity afforded by the American College of Surgeons’ mandate by testing the delivery of evidence-based alcohol SBI at U.S. trauma centers. In the longest follow-up period for a MI training evaluation yet, the present analysis includes longitudinal data collected prior to and 27-months following randomization and training. This allowed for the detection of transient training effects versus potentially sustained gains in MI proficiency as a result of a workshop followed by feedback and coaching.

2.2 Participants and Procedure

The University of Washington’s and each participating site’s Institutional Review Board approved all study procedures prior to full protocol initiation. The study included American College of Surgeons Level I trauma centers classified as middle adopters (Zatzick et al., 2013). Middle adopters meant that the centers demonstrated an interest in readiness to implement evidence-based alcohol SBI procedures, but without well-established SBI services (Rogers, 1995). Additionally, these sites had never received grant funding for alcohol SBI related research or service development and were deemed unlikely to implement high quality services without additional training. Site randomization to intervention (n = 10) and control (n = 10) occurred in a 1:1 ratio. A series of blocks of either two or four sites were generated using a random number generator by the investigation’s statistician. Once generated, intervention and control site assignments were entered into 20 sequentially numbered envelopes. Site randomization was then conducted by a blinded research coordinator. Despite randomization, a greater percentage of providers were nurses in the control group and social workers in the intervention group (see Table 1).

Table 1.

Characteristics of study providers

		No. (%)
Characteristics	All (N = 40)	Intervention (n = 16)	Control (n = 24)
Gender
Male	2 (5.0)	0 (0.0)	2 (8.3)
Female	38 (95.0)	16 (100.0)	22 (91.7)
Race
White	36 (90.0)	15 (93.8)	21 (87.5)
Black	2 (5.0)	0 (0.0)	2 (8.3)
Hispanic	2 (5.0)	1 (6.3)	1 (4.2)
Job type*
Nurse	19 (47.5)	3 (18.8)	16 (66.7)
Social worker	15 (37.5)	11 (68.8)	4 (16.7)
Othera	6 (15.0)	2 (12.5)	4 (16.7)
Years since receipt of professional degreeb
<1	1 (2.5)	1 (4.2)	0 (0.0)
1–10	15 (37.5)	10 (41.7)	5 (31.3)
11 – 20	11 (27.5)	4 (16.7)	7 (43.8)
21 – 30	7 (17.5)	5 (20.8)	2 (12.5)
>31	6 (15.0)	4 (16.7)	2 (12.5)

Note. M = mean, SD = standard deviation, MI = Motivational Interviewing.

^*Between group difference statistically significant at p < .05.

^aOther = chemical dependency professional (n = 1), physicians assistants (n = 4), and respiratory therapist (n = 1).

^bMeans and standard deviations for years since receipt of professional degree: intervention group, M = 16.5, SD = 11.4, and control group, M = 17.0, SD = 13.0.

Provider recruitment (N = 40) for the trial began in 2008 and included nurses (n = 19), social workers (n = 15), physicians assistants (n = 4), one chemical dependency professional, and one respiratory therapist, who were predominantly White and female (see Table 1). The training background of nurses included in the study were split between having a bachelor’s degree or less (58%) and master’s degree or more (42%). The DO-SBI trial aimed to have one provider designated to conduct alcohol SBI at each of the 20 trauma center sites. At 12 sites (five intervention) there was one SBI provider. At the remaining sites, staff coverage and turnover dictated that more than one provider conducted SBI. Six sites had two SBI providers, and two sites had three. The total number of providers included in this study is greater than the number included in the main outcomes paper (Zatzick et al., 2014), which included only providers who had actually intervened with patients during the trial. Researchers obtained providers’ professional degree and the number of years since receipt of the degree by provider self-report (see Table 1). Providers in both groups were to perform bedside SBI for alcohol with real trauma patients, beginning immediately after joining the study in the case of control providers. Intervention group providers began seeing patients after completing a 6-month SBI training (see below; see Zatzick et al., 2014 for demographic information about patients enrolled in the study).

2.2.1 Intervention Group Training

After the American College of Surgeon’s original SBI mandate (American College of Surgeons Committee on Trauma, 2006), the College in collaboration with the National Highway Traffic Safety Administration and the Centers for Disease Control, funded a series of one day SBI workshop trainings to which any trauma center could send their SBI providers or program managers. These workshops were offered intermittently around the U.S. during the years of the study. Miller et al. (2004) found that a two day MI workshop plus feedback and coaching resulted in improved training outcomes among substance abuse counselors seeking MI training. Accordingly, the DO-SBI study team adopted a similar feedback and coaching design to follow a one day on-site workshop; the one day duration was thought to be more feasible for busy medical providers.

In the workshop, providers were taught how to conduct brief motivational interventions to address risky alcohol use based on methods developed by the study team with front-line trauma center providers over the past decade (Field et al., 2005; Gentilello et al., 1999; Zatzick et al., 2013; Zatzick et al., 2004; Zatzick et al., 2013). The trainer (CD) is a Motivational Interviewing Network of Trainers member and one of the national trainers for the College workshops mentioned above. MI promotes collaborative conversations that aim to strengthen an individual patient’s motivation and commitment to drink less or abstain (Miller & Rollnick, 2012). The workshop training taught a 20–30 minute brief intervention that included a series of counseling tasks to be readily delivered to patients using the style and method of MI by the full spectrum of trauma center providers (e.g., social work and nursing providers; Dunn, 2003). These brief intervention tasks included providing feedback to patients about their blood alcohol test results, exploring patients’ views on the importance of changing and their confidence to do so, evoking patients own reasons for change, and planning how to change if patients were ready (Dunn, 2003). Each patient’s motivation for counseling and readiness to change (Prochaska & DiClemente, 1986) determined when and how many of these tasks were to be discussed.

MI didactics included review of the spirit of MI, defined as an interaction style with patients emphasizing partnership, acceptance, compassion, and evocation of the patients’ perspective about behavior change (Miller & Rollnick, 2014), and the importance of MI spirit in creating the context in which patient ambivalence about changing risky alcohol use can be explored. Specific skills demonstrated and practiced during the workshop with the trainer included asking open-ended questions, communicating affirmations, making simple and complex reflections of patient’s perspectives, making summaries, and eliciting statements in favor of change (i.e., “change talk”) from the patient. Providers were taught to use 0–10 scaling questions which ascertain how important change is to patients and how confident they were in their ability to change their behavior as techniques for eliciting change talk. Providers then practiced summarizing responses to the scaling questions.

The on-site workshop was followed over the next six months by feedback and coaching delivered by the trainer based on the provider’s performance on the first four standardized patient role-plays, which were also used to assess provider MI performance (described below). The trainer listened to audio-recordings of standardized patient role-plays, emailed the provider feedback about the things they did well and strategies for improving their MI performance, and conducted a 30-minute telephone coaching call with the provider, which may have included practicing MI and SBI skills. The actual training dose varied due to provider availability and turnover. Some providers received more than four coaching calls when they requested additional practice with the trainer (see Table 2).

Table 2.

Motivational Interviewing skill assessment and training process data

		No. (%)
Training Process	All (N = 40)	Intervention (n = 16)	Control (n = 24)
Total number of completed SPsa
3	2 (5.0)	0 (0.0)	2 (8.3)
4	2 (5.0)	0 (0.0)	2 (8.3)
5	7 (17.5)	4 (25.5)	3 (12.5)
6	3 (7.5)	0 (0.0)	3 (12.5)
7	26 (65.0)	12 (75.0)	14 (58.3)
Total written feedback on standardized patient assessments
2 SPs	--	4 (25.0)	--
3 SPs	--	11 (68.3)	--
4 SPs	--	1 (6.3)	--
Total coaching calls
2 CCs	--	3 (18.1)	--
3 CCs	--	1 (6.3)	--
4 CCs	--	6 (37.5)	--
5 CCs	--	3 (18.8)	--
6 CCs	--	3 (18.8)	--

Note. M = mean, SD = standard deviation, SP = standardized patient, CC = coaching call. Between group difference for total number of standardized patients completed was not statistically significant.

^aMeans and standard deviations for total number of completed SPs: intervention group, M = 6.5, SD = 0.9, M = 6.0, SD = 1.4. Frequencies indicate there were missing SP data for 4 intervention and 10 control providers due to uncompleted SPs.

2.2.2 Control Group: Training-as-Usual

Control trauma centers were permitted to use any available, non-study alcohol SBI training resources such as published guidelines, electronic or print training material, or private, state, or federal SBI training. Researchers did not track data regarding SBI training of control sites. The American College of Surgeon’s sponsored SBI trainings included the same SBI content described above regarding intervention training, which is consistent with the recommendations by the College (American College of Surgeons et al., 2010). Recommendations for brief interventions include giving feedback or information about screening results, understanding patient’s reasons to change, and giving clear and respectful professional advice to quit or cut down.

2.3 Motivational Interviewing Skills Assessment

The study design compares provider performance in MI skills between a group of formally-trained alcohol SBI providers (intervention group) to those who have not received any formal training (control group). To assess provider skill, both intervention and control providers participated in up to seven audio-recorded 20-minute standardized patient (i.e., actor) telephone role-play alcohol brief interventions. These standardized patient role-plays incorporated a graded sequence of clinical difficulty ranging from early role-plays, in which the actor role-played patients ready and willing to change alcohol use, to final role plays, in which the actor role-played more challenging patients in the pre-contemplation stage of change. For each assessment point, telephone appointments were made by the standardized patient actor, who was blinded to SBI provider intervention and control group status. Intervention and control group providers received identical role-play scenarios and the same actor role-played all of the patient scenarios. Standardized patient interviews were completed with all 20 sites at baseline, prior to randomization, then again at 1-week and 1-, 4-, 7-, 17-, and 27-months after the study team conducted an on-site workshop, which in the case of the intervention group, included training in study recruitment and data collection procedures, as well as conducting SBI using an MI counseling style. In the case of the control group, the standardized patient telephone role-plays were used to assess MI skills only. In the case of the intervention group, the first four standardized patient role-plays, including the baseline pre-randomization assessment, were used both for training purposes (described above), as well as an assessment of MI skill. Therefore, in the intervention group, the 1-week standardized patient role-play represents post-workshop MI performance. The 7-month standardized patient interview represents immediate post-training performance (including the workshop and six month period of feedback and coaching), and the 17- and 27-month interviews represent long-term post-training performance. Completion of standardized patient interviews varied due to provider availability and modest turnover. Sixty-five percent completed the full sequence and 95% completed at least the four standardized patient interviews that were part of the six month training period. Due to turnover, there was a tendency for control group providers to complete their 17- and 27-month standardized patient interviewers earlier than intervention providers (see Figure 2).

Figure 2.

Intervention and control group means and individual provider scores at each standardized patient assessment (1-week and 1-, 4-, 7-, 17-, and 27-month) after the baseline assessment. MI = intervention group (received training in brief motivational interventions for alcohol SBI). In the color version of the figure, blue corresponds to the intervention group and red corresponds to control. Dotted lines indicate expert-derived thresholds for basic proficiency (lower line) and competency (upper line). In the intervention group, providers received coaching and feedback up to the fourth assessment. The first assessment represents post-workshop MI skill and the sixth and seventh assessments represent post-training MI skill for the intervention group.

Each interview was coded using the Motivational Interviewing Treatment Integrity (MITI) system version 3.1.1 (Moyers, Martin, Houck, Christopher, & Tonigan, 2009; Moyers, Martin, Manuel, Hendrickson, & Miller, 2005) by coders trained by the study expert MITI coder (CD), including global scores evaluating the entire session (i.e., MI Spirit, Empathy) and frequencies of individual provider behaviors (i.e., open-ended questions, simple and complex reflections, MI-adherent and non-adherent behaviors). Two coders rated every 5^th audio recording to obtain a sample for estimating inter-rater reliability. The Intraclass Correlation Coefficients for these codes ranged from .62 to .95, which are considered to be in the “good” reliability range according to Cicchetti’s (1994) guidelines.

To observe provider skill across multiple MI domains, the current investigation utilized the five suggested MITI summary scores plus the global item Empathy (scores range from 1 to 5), which measures the quality and depth of the counselor’s understanding of the patient. Summary scores included MI Spirit (an average of three global Likert scales: Collaboration, Autonomy, and Evocation; scores range from 1 to 5) and ratio scores derived from counts of counselor behaviors: percentage of total reflections that are complex (versus simple reflections; Percent Complex Reflections), percentage of total questions that are open questions (versus closed questions; Percent Open Questions), the ratio of total reflections to total questions made by the interventionist (Reflection-to-Question Ratio), and percentage of MI adherent responses (ratio of adherent responses to the sum of adherent and non-adherent responses; Percent MI-Adherent). The MITI manual includes expert-suggested thresholds for basic proficiency (lower standard) and competency (higher standard) for these MITI scores, and our results are reported relative to these two thresholds. The MITI has established reliability and validity, and MITI coding procedures have been manualized (Moyers et al., 2005).

2.4 Data Analysis Plan

The goal of the current study was to examine whether it is possible to train routine care trauma center providers to deliver high-quality alcohol SBI using MI. Analyses focused on intervention (i.e., MI training) versus control (i.e., training-as-usual) group comparisons on MITI scores, from pre-workshop to post-workshop, and then post-workshop through training and post-training phases. Statistical analyses used linear mixed models (also called multilevel or hierarchical linear models) to control for the nested nature of the data due to repeated measures and providers (Hedeker & Gibbons, 2006). Moreover, given the relatively small number of providers, data visualizations (Cleveland, 1993) were used to contrast group means and individual variability, as well as compare means and individual scores to expert-derived MI proficiency criteria. Due to the length of the overall study and natural turnover of providers, there was some missing data on MI training outcomes (i.e. < 7 standardized patients, see Table 2). Mixed models provide unbiased estimates as long as missing data can be considered missing at random, which seems a reasonable assumption with the current provider data. All analyses were done using R v3.1.2 (Statistical Package, 2009).

3. Results

Table 3 presents descriptive statistics and results of linear mixed models for comparing pre- to post-workshop MITI scores between intervention conditions. As seen in Table 3, the workshop significantly improved MI skills for all summary measures (though the difference between the intervention and control conditions at post-workshop for MI Spirit was only p = .06). Effect sizes for intervention conditions at post-workshop represent large effects. Figure 1 presents group means as well as individual provider scores as a slopegraph (Tufte & Graves-Morris, 1983). Dotted lines indicate expert-derived thresholds for basic proficiency (lower line) and competency (upper line). Figure 1 provides important context and nuance to the statistical analyses of group means. Although the workshop did improve provider MITI scores on average, there is notable individual variability around these means. Moreover, particularly for the summary scores based on behavior counts, very few providers performed above the basic proficiency benchmarks. In contrast, for the global measures (i.e., MI Spirit, Empathy), a majority of providers who received the workshop training scored above the basic proficiency threshold.

Table 3.

Means, standard deviations, and effect size estimates for differences between intervention and control conditions from pre- to post-workshop

	Pre-Workshop				Post-Workshop
	Intervention		Control		Intervention		Control
	M	SD	M	SD	M	SD	M	SD	da
MI Spirit	3.0	0.8	2.6	0.9	4.0	0.5	3.0	0.7	1.6
Empathy	2.9	0.9	2.8	0.9	4.1	0.5	3.0	0.8	1.6
Reflection-to-Question Ratio	0.3	0.2	0.4	0.3	0.7	0.3	0.4	0.2	1.2
% Complex Reflections	8.0	12.4	8.3	11.3	20.7	13.3	8.4	9.4	1.1
% Open Questions	16.9	8.6	17.9	9.0	31.8	13.7	16.1	9.5	1.4
% MI Adherent	21.6	32.7	27.0	38.1	79.7	34.0	31.1	38.5	1.3

Note. Based on linear mixed models, treatment by training period interactions were statistically significant at p < .01 for all outcomes, except MI Spirit (p = .06).

^aBetween groups effect size estimates.

Figure 1.

Slopegraph of intervention and control group means and individual provider scores from pre- to post-workshop training. MI = intervention group (received training in brief motivational interventions for alcohol SBI). In the color version of the figure, blue corresponds to the intervention group and red corresponds to control. Dotted lines indicate expert-derived thresholds for basic proficiency (lower line) and competency (upper line).

Focusing on provider performance post-workshop, Figure 2 presents a similar set of figures for the standardized patient assessments from 1-week to 27-months post-workshop. In the intervention condition, providers received coaching and feedback for the first three assessments as shown in Figure 2. Coaching and feedback was not routinely provided as part of the final three assessments (at approximately 7-, 17- and 27-months). On average, Figure 2 suggests that providers receiving workshop training, feedback, and coaching were able to maintain higher MI skills over time relative to those in the control condition, which was also confirmed by linear mixed models (for all training effects p < .01). Several parameterizations for time were examined (e.g., linear, quadratic, piecewise linear). The substantive findings were similar across different models for time, and hence, reported results using a simple linear parameterization of time. Skills were relatively constant over time, on average, though linear mixed models revealed significant decreases over time for Percent Complex Reflections, Empathy, and Reflection-to-Question Ratio. Within-provider variability in MI skills over time is also apparent. Finally, similar to what was seen at post-workshop, relatively few providers passed basic proficiency thresholds, except for global assessments of MI Spirit and Empathy.

4. Discussion

To our knowledge this is the first study of MI training implementation in an inpatient hospital acute care medical setting utilizing a randomized, longitudinal, standardized patient design. We observed that routine trauma center providers receiving evidence-based alcohol SBI workshop training, feedback, and coaching consistently performed higher quality counseling during alcohol brief interventions as evidenced by sustained improvements in MI skills. The observed training effect sizes were relatively large and similar to other randomized controlled trials with a no-training control group (de Roten et al., 2013). However, statistically significant decreases over time were observed for Percent Complex Reflections, Empathy, and Reflection-to-Question Ratio, a decrease most visibly pronounced for Reflection-to-Question Ratio (see Figure 2). Overall, our findings indicate that MI training is associated with improved MI skills among routine providers required to implement alcohol SBI following the American College of Surgeons’ policy mandate.

Our findings have important implications for the trainability of routine care providers in the mandated context. First, observed effects of training on MI skills are highly encouraging for ongoing dissemination of mandated trauma center alcohol SBI. An important caveat is that trained providers did not uniformly attain expert-derived standards for basic proficiency across MI domains. Although on nearly all MITI domains assessed, trained providers’ scores on global indices were one point higher and twice as high on summary scores of behavior counts, the mean level of skill met the basic proficiency threshold only for global scores of MI Spirit and Empathy. Providers appeared to have most difficulty improving and sustaining skills in open questions and complex reflections, skills known to be challenging for providers across MI training studies (Baer et al., 2004; Forsberg, Forsberg, Lindqvist, & Helgason, 2010; Miller et al., 2004; Mitcheson, Bhavsar, & McCambridge, 2009; Moyers et al., 2008; Schoener, Madeja, Henderson, Ondersma, & Janisse, 2006). The fact that global scores improved most readily is consistent with the proposed stages of learning MI by Miller & Moyers (2006) in which the person-centered counseling style (i.e., MI Spirit) is acquired before the core skills (e.g., open questions, reflections). Unfortunately, the clinical significance associated with varying degrees of MI proficiency is unclear given the lack of empirical data relating proficiency standards to client behavior change (Miller & Rollnick, 2014). Despite these remaining questions regarding MI proficiency standards, our finding that trained providers sustained improvement in MI-adherent behaviors is quite promising given the evidence for the beneficial impact of MI-consistent therapist behavior on intermediate client outcomes (e.g., client “change talk” during a motivational interview; Apodaca & Longabaugh, 2009; Gaume, Bertholet, Faouzi, Gmel, & Daeppen, 2013; Moyers et al., 2009; Vader, Walters, Prabhu, Houck, & Field, 2010).

Our results on provider proficiency contribute to an emerging theme in the MI training literature, that providers in studies with community-based, routine care providers for whom counseling is not a primary aspect of their professional role may be less likely to demonstrate post-training MI proficiency than those with a counseling background and who self-select to learn MI (Bohman, Forsberg, Ghaderi, & Rasmussen, 2012; Forsberg et al., 2010; Moyers et al., 2008; van Eijk-Hustings, Daemen, Schaper, & Vrijhoef, 2011). The differential impact of training may be due to incoming skill level. For instance, providers in the present study had lower skills at baseline than the substance abuse counselors included in Miller et al.’s (2004) randomized controlled trial. In fact, providers in our study attained a level of skill after training that is similar to what providers in the Miller et al. study started with pre-training. Additionally, the attainment of skills post-training among our study providers is similar to that of another group of routine care trauma nurses also trained in response to the American College of Surgeons’ mandate who had similarly low baseline MI skill (London, Dunn, & Utter, 2011).

In contrast to findings from the randomized controlled trial on which the training paradigm was based (Miller et al., 2004), feedback and coaching did not appear to help providers advance to proficiency and, over time, often did not even sustain the post-workshop improvements. It is possible that we did not observe steady improvements over time among trained providers because our standardized patient design intentionally utilized increasingly challenging role-plays, a design that was selected for training purposes with the intervention group. This interpretation of the data is consistent with previous research indicating that client motivation and substance use behavior is related to therapist competence in the use of MI skills (Imel, Baer, Martino, Ball, & Carroll, 2011). Alternative methods, such as the Objective Structured Clinical Examinations, which are assessments of clinical skill competence within medical settings used commonly in medical education (Newble, 2004), may be more familiar and realistic to routine trauma center providers. It is also possible that the routine care trauma center providers require more intensive training and supervision than the substance abuse counselors recruited into Miller et al.’s trial. Whether some providers are unable to learn MI to proficiency regardless of the amount of training offered is an important implementation research question that warrants future study.

These findings have implications for how the American College of Surgeons’ mandate could be expanded or refined. Typically, College mandates will suggest procedures for following the national guidelines, such as the one day workshops made available by the College following the initial mandate. However, it is clear that workshops, and even several hours of post-workshop feedback and coaching, are not sufficient for most routine care providers to reach proficient levels of MI. Given that in the mandated services implementation context, trauma centers must identify staff to deliver alcohol SBI and frequently select those that already have roles within the center (Zatzick et al., 2014), alternative approaches to a uniform, one day training may be needed. As an example, the criterion-based stepwise approach put forth by Martino and colleagues (2011), which provides additional training to those providers whose skills are insufficient, may be incorporated into the guidelines. Such an approach would require methods for assessing provider skills. Such assessments of skill would require the use of audio recordings of patient interactions (e.g., standardized patients) which are then coded by using human coders. In the real-world implementation context, human coding may be impractical as it requires intensive resources. Coding of MI skills using computer software (Atkins, Steyvers, Imel, & Smyth, 2014) could be honed and refined for large-scale such implementation efforts using such training methods.

4.1 Limitations & Considerations

The present multisite study provides important data about the capacity for high quality alcohol SBI within the unique implementation context of Level I and II trauma centers that must respond to the American College of Surgeons’ alcohol SBI mandate. Findings do not necessarily generalize beyond this setting, or beyond trauma nurses and social workers, who were the predominant providers elected by their organizations to train in and conduct SBI. Given that 69% of providers in the intervention group were social workers and 66% of providers in the control group were nurses, findings regarding trainability are more certain for social workers than nurses and we cannot rule out that professional degree and background account for observed gains from the intervention workshop training. It is possible that social workers, who made up a large proportion of trained providers, are more likely than nurses to benefit from training given the greater exposure to and practice with counseling skills in social work degree programs and professional roles. Additionally, trauma center nurses included in the study have a broad range of training backgrounds and may have held diverse hospital roles requiring diverse skill sets, which could influence trainability in alcohol brief intervention counseling skills. Examination of moderators of the effect of MI training, including providers’ professional training background, is needed to shed light on provider trainability and guide decisions as to which providers may be best positioned to provide high quality alcohol SBI counseling. As noted, a remaining research question is whether providers trained in counseling in SBI that does or does not meet proficiency standards impacts patient outcomes.

In total, the study had 40 providers and training occurred among only the 16 providers in the intervention group, which is a relatively small sample size for between group analyses and the observation of interaction effects. Despite the relatively small sample of intervention providers, our descriptive analyses and data visualization benefited from a relatively large number of assessments over the course of the 27-month standardized patient design. Related, a limitation of this implementation spectrum trial design was the greater rate of provider turnover in the control condition. We cannot rule out the possibility that the greater intervention team engagement of providers at intervention sites contributed to enhanced engagement and diminished turnover in the intervention condition.

The present study utilized standardized patients to objectively assess provider skill in MI over time, with higher MITI scores indicating greater capacity for high quality counseling during alcohol brief interventions. Actual performance of brief interventions with real patients seen by the study providers was not assessed. Performance on standardized patients and real patients are known to have low to moderate correlations, indicating that providers perform differentially with real patients; usually more variably (Decker, Carroll, Nich, Canning-Ball, & Martino, 2013; Imel et al., 2014). Therefore, our data speaks to trainability, but not provider performance of MI with actual patients. The use of standardized patients provides greater control over the assessment process, allowing researchers to maximize follow-up rates and internal validity of the training paradigm. Previous MI training studies document the challenge of gathering work samples from providers asked to generate these samples from their own caseload (Baer et al., 2004; Miller et al., 2004; Moyers et al., 2008). Standardized patients also provide the benefit of high quality audio-recordings, which can facilitate the use of cutting edge, technology-enhanced methods for measuring MI fidelity (Atkins et al., 2014).

5. Conclusion

This investigation contributes to an evolving literature on the widespread implementation of alcohol SBI in the acute care medical, trauma center setting, which is occurring in a unique “make-it-happen” regulated implementation context; this is in contrast to the “negotiated” implementation context in many other general medical settings (Greenhalgh, Robert, Macfarlane, Bate, & Kyriakidou, 2004). The American College of Surgeons has demonstrated its commitment to using empirical data derived from pragmatic trials to inform policy targeting the integration of alcohol SBI at U.S. trauma centers (American College of Surgeons Committee on Trauma, 2006, 2014). The results of this secondary analysis of the larger cluster randomized implementation trial suggest that routine trauma center care providers can be trained to attain higher quality brief motivational interviews with a non-resource intensive, standard workshop followed by feedback and coaching training model.

Highlights.

• Training can improve the quality of brief motivational interviews delivered by routine trauma center providers

• Providers more readily improve in the skills of Empathy and MI Spirit

• Trained providers do not attain pre-established MI proficiency criteria for most MI skills