Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2012 Jun 1.
Published in final edited form as: J Subst Abuse Treat. 2011 Feb 11;40(4):357–365. doi: 10.1016/j.jsat.2010.12.004

A Criterion-Based Stepwise Approach for Training Counselors in Motivational Interviewing

Steve Martino 1, Monica Canning-Ball 1, Kathleen M Carroll 1, Bruce J Rounsaville 1
PMCID: PMC3104507  NIHMSID: NIHMS289387  PMID: 21316183

Abstract

This pilot study evaluated a criterion-based stepwise approach for training counselors in motivational interviewing (MI). Three sequential steps of training in MI were provided to 26 counselors who worked within U.S. State of Connecticut Veterans Administration addiction treatment programs: a distance learning Web course (step 1); a skill-building workshop (step 2); and competency-based individual supervision (step 3). Counselors first participated in the course and then only received the next step of training if they failed to meet an independently rated criterion level of adequate MI performance. The results showed that counselors who showed inadequate MI performance immediately after taking the Web course and who subsequently participated in a workshop or supervision improved their adherence to fundamental MI strategies over time, whereas those who performed MI adequately following the Web course continued to demonstrate similar levels of fundamental and advanced MI adherence and competence over a 24-week period without additional training. The pilot study’s findings, though preliminary, suggest that different counselors likely require different types and amounts of training in order to perform a behavioral treatment well.

Keywords: criterion-based training, dissemination, implementation, motivational interviewing, substance abuse treatment

1. Introduction

Multiple strategies for training counselors in behavioral treatments have been described in the literature (Beidas & Kendall, 2010; Martino, in press). These strategies have included distance learning methods (Shafer, Rhode, and Chong, 2004), skill-building workshops (Walters, Matson, Baer, & Ziedonis, 2005), and competency-based supervision (Falender & Shafranske, 2007). Often these strategies are blended to provide counselors with comprehensive training to learn behavioral treatments (Cucciare, Weingardt, & Villafranca, 2008). The extent to which counselors require all components of a blended training approach to reach a criterion-based level of performance is unclear and the subject of this pilot study.

Individual counselors may vary considerably in their response to training and need different types and amounts of training to use a treatment adequately. Some counselors might more readily “take” to one treatment or another (e.g., spiritual orientation for twelve-step) and be able to learn it quickly with less intensive training approaches, whereas others may lack certain capacities (e.g., limited pre-training empathic abilities for motivational interviewing) that might necessitate more intensive and extensive training efforts (Miller, Sorensen, Selzer, & Brigham, 2006). An adaptive training approach (cf. Collins, Murphy, & Bierman, 2004) might be useful such that counselors receive only the type and amount of training necessary to perform a treatment adequately rather than using a uniformly comprehensive and more costly blended training approach with all counselors.

Evidence has begun to accumulate about the effectiveness of counselor training strategies. Initial randomized controlled trials of distance learning methods for teaching behavioral treatments (typically computer-assisted and Web-based training) suggest they have promise. In a 3-week training trial, Sholomskas and Carroll (2006) showed counselors who used a twelve-step facilitation manual (Norwinski, Baker, & Carroll, 1992) plus a computer-assisted, multimedia CD-ROM designed to teach twelve-step facilitation skills significantly improved their treatment performance and knowledge compared to counselors who only used the manual. Likewise, Sholomskas and colleagues (2005) demonstrated that Web-based cognitive behavioral therapy training significantly improved counselors’ skills over a 12-week follow-up period relative to a manual only counselor training approach, though not as effectively as an intensive workshop coupled with follow-up supervision.

Research on the effectiveness of workshops (expert facilitated didactics and skill-building activities delivered in a group format) shows counselors consistently improve their attitudes, knowledge, and confidence, but immediate skill gains resulting from the training diminish quickly, sometimes in as little as 2–3 months (Baer et al., 2004; Miller et al., 2004; Mitcheson, Bhavsar, & McCambridge, 2009; Walters, Matson, Baer, & Ziedonis, 2005). In contrast, competency-based supervision approaches attempt to identify the knowledge and skills each counselor needs to master and uses targeted learning strategies and evaluation procedures to develop the counselors’ treatment skills (Falender & Shafranske, 2007). Direct observation of counselors’ sessions, use of treatment integrity rating-based performance feedback, and individualized coaching often are key components of this form of supervision (Baer et al., 2007; Carroll, 1997). Several studies have shown that competency-based supervision improves counselors’ addiction treatment skills and the percentage of them who perform the treatment with sufficient proficiency (Miller, Yahne, Moyers, Martinez, & Pirritano, 2004; Sholomskas et al., 2005).

This pilot study evaluated a criterion-based stepwise approach for training counselors in motivational interviewing (MI; Miller & Rollnick, 2002), an empirically supported treatment for addictions and several other problem areas (Lundahl, Kunz, Brownell, Tollefson, & Burke, 2010). In the pilot study, three sequential steps of training in MI were provided to counselors, namely, a distance learning Web course (step 1), a skill-building workshop (step 2), and competency-based individual supervision (step 3). All counselors first participated in the course. Subsequently, counselors only received the next step of training if they failed to meet a criterion level of adequate MI performance in client sessions that has been used in several past MI effectiveness trials (Ball et al., 2007; Carroll et al., 2006; 2009). Training counselors to a criterion-based level of MI performance offers the advantages of setting clear training objectives, using limited training resources efficiently, and identifying counselors who might be more likely to achieve improved client outcomes with the new treatment. We hypothesized that only counselors who received the workshop or supervision would show increased MI adherence and competence over time because of the gains conferred by the additional training steps, whereas the performance of those who reached adequate MI standards quickly via the Web course would remain steady.

2. Methods

2.1. Participant counselors

Participating clinicians were required to 1) be employed ≥ 20 hours/week, and 2) treat English-speaking substance-using clients. Clinicians were excluded if they 1) had received distance learning or workshop MI training 3 months prior to pilot study initiation, a timeframe in which MI skills are likely to diminish (Baer et al., 2004; Miller et al., 2004; Mitcheson et al., 2009; Walters et al., 2005), or 2) had ever received competency-based supervision in MI. All counselors worked within Veterans Administration substance abuse treatment programs located in the U.S. State of Connecticut.

2.2. Measures

2.2.1. Clinician Survey

This baseline survey (Ball et al., 2002) evaluated a broad array of counselor characteristics (e.g., demographic, educational and professional experiences, recovery status, and counseling orientation). It has been used in several other MI trials (Ball et al., 2007; Carroll et al., 2006; 2009).

2.2.2. Independent Tape Rater Scale

The ITRS assesses counselors’ adherence and competence in implementing MI and a variety of other strategies inconsistent with MI (e.g., direct confrontation) or common to drug counseling (e.g., assessing substance use) in audiotaped client sessions. This pilot study used 15 ITRS items: ten evaluated strategies and interventions that characterize MI and five that were inconsistent with MI. For each item, raters evaluated the clinicians on two dimensions using a 7-point Likert scale. First, they rated the extent to which the clinician delivered the intervention (adherence; 1 = not at all, to 7 = extensively). Second, they rated the skill with which the clinician delivered the intervention (competence; 1 = very poor, to 7 = excellent). Past confirmatory factor analyses have supported a two-factor model for the 10 MI consistent items (corresponding to “fundamental” and “advanced” MI strategies) (Martino et al., 2008; Santa Ana et al., 2009): five fundamental MI strategies that underpin the client-centered stance of MI (e.g., reflective listening) and five items that involved advanced strategies for evoking client motivations for behavior change (e.g., heightening discrepancies). Each scale was averaged to obtain adherence and competence scores. In addition, the five MI inconsistent items (unsolicited advice, therapeutic authority, direct confrontation, emphasizing total abstinence, asserting disease concepts of addiction) were averaged to determine the impact of training on this area.

2.2.3. Training Evaluation Form

This 12-item form asks counselors to rate the extent to which different skills were covered during the training (10 MI consistent items paralleling those in the ITRS and 1 MI global rating of attention paid to avoiding MI inconsistent strategies), the overall skill of the trainer, the helpfulness of the training for learning MI, and overall satisfaction. Each item is rated on Likert-type scales, with higher ratings assigned to increased levels on each dimension.

2.3. Procedures

2.3.1. Recruitment

Research staff informed counselors about the opportunity to participate in the pilot study at on-site clinical staff meetings. Program directors told counselors that their participation was completely voluntary, had no bearing on their worksite performance evaluations, and that information about their pilot study involvement would not be shared with directors or supervisors. Thereafter, a research assistant met with counselors who expressed interest in participation, screened them for eligibility, and obtained written informed consent approved by Human Studies Subcommittee of the Veterans Administration Healthcare System in West Haven, Connecticut.

2.3.1. Training steps

All counselors received the 2002 MI textbook (Miller & Rollnick, 2002) and the Center for Substance Abuse Treatment MI manual (Miller, 1999). Thereafter, three steps of MI training were available to the counselors: Web course (step 1), skill-building workshop (step 2), and competency-based supervision (step 3). All counselors received the first step of training but only received the next training step if they failed to demonstrate adequate standards of MI performance in a sample audiotaped client sessions. Counselors independently evaluated as having at least half of the ITRS MI consistent scale items rated average or above in terms of adherence and competence were deemed as having performed MI adequately, the same criterion that has been used three large-scale effectiveness trials involving different applications of MI implemented by community program counselors in which MI had improved either client retention (Carroll et al., 2006) or substance use treatment outcomes (Ball et al., 2007; Carroll et al., 2009). Once counselors met this criterion, no further training was provided to them. Counselors received continuing education credits for each completed step of training. Monetary reimbursement for pilot study participation did not occur.

Step 1 - Web course

This Web MI course, developed by the Mid-Atlantic Addiction Technology Transfer Center (http://www.attcnetwork.org/regcenters/distanceeducation.asp), contained four 1-hour modules that could be accessed on demand once counselors had been enrolled in it. Module 1 defined the approach, its origins, and style or spirit of interaction. Module 2 presented how MI fits into the stages of change model (Prochaska, DiClemente, & Norcross, 1992). Module 3 described four basic principles of MI, how to form reflections, and ways to handle client resistance skillfully. Module 4 reviewed how to use fundamental skills (open questions, affirmations, reflective listening, and summaries), recognize “change talk”, and avoid communication traps that might interfere with motivational enhancement. Each module had recommended supplemental readings and “discussion assignments” in which participants were asked to consider an issue or practice a skill and then post their responses/experiences to a discussion board managed by the course instructor (SM). The instructor then responded to these postings and encouraged participant exchanges in between modules. Discussions were not “live”, meaning the instructor and counselors could read and respond to the postings at their convenience rather than being required to complete the modules and postings simultaneously. Counselors were encouraged to complete one module per week over four weeks.

Step 2 – Skill-building workshop

Counselors participated in a skill-building, face-to-face workshop (delivered by SM) that used procedures consistent with those recommended by the Motivational Interviewing Network of Trainers (MINT; Miller & Rollnick, 2002). The workshop occurred over four consecutive weekly 2-hour sessions. Session 1 included a review of the MI style or spirit of interacting with clients and gave counselors practice using open questions. Session 2 focused on developing the counselors’ reflective listening, affirmation, and summarization skills. Session 3 taught counselors how to recognize change talk and use open questions and reflections to elicit these statements from clients. Session 4 gave counselors role-play practice using all the MI skills.

Step 3 – Competency-based supervision

Counselors were supervised using the National Institute on Drug Abuse – Substance Abuse and Mental Health Service Administration Blending Product called Motivational Interviewing Assessment: Supervisory Tools for Enhancing Proficiency (Martino et al., 2006). Counselors conducted MI with clients who had substance use problems and audiotaped these sessions (signed consent obtained). Counselors selected clients from their caseloads based on their clinical judgment about the clients’ suitability for MI and willingness to be audiotaped. The supervisor (SM) rated these sessions for MI adherence and competence, completed rating feedback forms and reviewed them with the counselors, and then had counselors practice skills to improve their performance. Each supervision meeting was 45 minutes. Counselors were supervised in person once per month for two months.

2.3.2. Assessment

Assessment of the counselors’ MI performance occurred at baseline and at the end of each of the 8-week MI training step periods (weeks 8, 16, and 24). At each point, counselors submitted audiotaped 40-minute client sessions (signed consent obtained) in which they tried to motivate clients to reduce or stop substance use. As described above, counselors self-selected clients for these audiotaped sessions. Counselors were aware that these audiotapes were being used to assess their MI skills in clinical practice and to determine if they would receive the next step of training.

2.3.3. Independent tape rating

Six raters, blind to training condition and assessment points, rated the audiotaped sessions. All raters attended a didactic seminar in which they learned how to rate the ITRS items. Following this training, each rater completed ratings for an identical set of 6 calibration tapes randomly selected from a pool of MI session tapes compiled in another MI training study (Martino et al., in press), to evaluate interrater reliability using Shrout and Fleiss (1979) intraclass correlation coefficients (ICC’s) two-way mixed model (3.1). Across ITRS mean scores, ICCs showed good to excellent inter-rater reliability for the respective adherence and competence scores: fundamental MI strategies = .93 and .93; advanced MI strategies = .69 and .82; MI inconsistent strategies = .82 and .84. All counselor audiotapes were rated in this pilot study. Audiotapes from counselors who had just completed a training step were rated first to determine if they had performed MI adequately or if they would continue with further training.

3. Results

3.1. Participants

Thirty-four counselors were screened for the pilot study and all were found eligible. Eight of them dropped out after reporting having had difficulty obtaining audiotaped client sessions as part of their baseline assessment, leaving a total of 26 counselors. The counselors who dropped out before training began did not differ demographically as a group from those who received the training. Counselors were primarily female (58%), Caucasian (69%), and about 49 years old on average. They had about 12.5 years of counseling experience and most endorsed a cognitive behavioral counseling orientation (73%). Two thirds of the counselors (69%) had no prior substance abuse history. About half had at least a master’s degree (54%) and professional license (58%). Most worked as therapists (62%) or case managers (27%) primarily in outpatient treatment settings (69%). In the year prior to pilot study involvement, clinicians had little training in MI (see Table 1).

Table 1.

Counselor demographic variables and experience levels

Variable
Female, n (%) 15 (58%)
Mean (SD) age 48.8 (9.7)
Ethnicity, n (%)
 African-American 4 (15%)
 Caucasian-American 18 (69%)
 Hispanic-American 2 (8%)
 Caribbean-American 1 (4%)
 Unspecified Other 1 (4%)
Mean (SD) Years Education 17.2 (2.7)
Highest Degree Earned, n (%)
 Doctorate 5 (19%)
 Master’s 9 (35%)
 Bachelor’s 3 (12%)
 Associates 4 (15%)
 High School 5 (19%)
Discipline, n (%)
 Case Management 7 (27%)
 Alcohol/Drug Counseling 3 (12%)
 Social Work 6 (23%)
 Psychology 5 (19%)
 Occupational Therapy 2 (8%)
 Nursing 3 (12%)
Licensure Status, n (%)
 Alcohol/Drug Certification 7 (27%)
 Licensed Professional 15 (58%)
Primary Role, n (%)
 Therapist 16 (62%)
 Case Manager 7 (27%)
 Program Manager/Supervisor 3 (12%)
Level of Clinical Care, n (%)
 Outpatient/intensive outpatient 18 (69%)
 Residential/partial hospital 6 (23%)
 Inpatient 2 (8%)
Mean (SD) Years Counseling Experience 12.5 (11.7)
Counseling Orientation, n (%)
 Cognitive behavioral 19 (73%)
 Twelve step/disease model 12 (46%)
 Rogerian/client-centered 12 (46%)
 Solution focused 9 (35%)
 Psychodynamic 8 (31%)
 Motivational interviewing 7 (27%)
 Family/marital 7 (27%)
 Reality therapy 7 (27%)
 Gestalt therapy 5 (19%)
Past Substance Abuse Problem/In Recovery 8 (31%)
Past Year MI Training
 Mean (SD) hours 1.2 (3.8)
 Lecture/presentation, n (%) 4 (15%)
 Workshop, n (%) 3 (12%)
 General supervision, n (%) 2 (8%)
Read 2002 MI Textbook, n (%) 4 (15%)
Used MI Manuals, n (%) 7 (27%)
Open in a new tab

Note: For counseling orientation, participants indicated the extent to which they adhered to different therapeutic orientations along 5-point Likert scales (1 = no adherence, to 5 = strong adherence). Counselors were categorized as oriented toward these approaches if they rated their adherence at a 4 or 5 level.

3.2. Training steps received

The flow of training step participation and pilot study retention is presented in Figure 1. All 26 counselors enrolled in the Step 1 Web course. Nineteen counselors (73%) completed the entire four-module course, 2 (8%) completed three modules, 2 (8%) completed 2 modules, and 3 (12%) completed none of them. Five of the 26 counselors did not provide audiotaped sessions for review upon completing Step 1 training and dropped out of the pilot study, all citing they were too busy for further participation.

Figure 1.

Figure 1

Open in a new tab

CONSORT Flowchart

Of the 21 counselors who provided audiotapes at the 8-week assessment point, 11 counselors (52%) met adequate standards of MI performance at the end of the course and 10 did not, thereby making them eligible for the Step 2 skill-building workshop. Two of these counselors then dropped out of the pilot study because they indicated they were too busy to receive additional training. Thus, 8 counselors participated in the workshop. Six received all four sessions. Two received only the first two sessions.

Five of these 8 counselors (62%) met adequate standards of MI performance after receiving workshop training (16-week assessment point). Three counselors became eligible for Step 3 competency-based supervision. One counselor had two supervisions and the other 2 received one (because they did not submit a second audiotaped session for supervisory review). An additional counselor dropped out of the pilot study at the 24-week assessment point due to an unexpected medical leave, resulting in 18 counselors (69%) retained for the pilot study’s duration.

3.3. Training satisfaction

For each training step, the participants indicated that the trainer covered specific MI strategies quite a bit to a considerable level and that the trainer was quite skilled in teaching the material (see Table 2). Similarly, participants indicated they were moderately to mostly satisfied with each step and found the training helpful in learning MI. Analyses of ratings by counselors who received more than one step of training indicated they evaluated the different trainings in similarly positive ways. No significant differences existed between the two groups in how they evaluated the initial Web course.

Table 2.

Coverage of MI strategies and participant satisfaction with Web course, skill-building workshop, and competency-based supervision

Training Step
MI Skill Course n = 21 Workshop n = 8 Supervision n = 3
MI spirit 5.6 (1.4) 6.1 (1.1) 6.0 (1.0)
Open questions 5.7 (1.5) 6.1 (0.8) 6.3 (0.6)
Reflections 5.7 (1.5) 6.0 (0.8) 6.3 (0.6)
Affirmations 5.6 (1.5) 6.0 (0.9) 6.7 (0.6)
Fostering collaboration 5.7 (1.4) 6.1 (0.7) 5.8 (1.1)
Motivation for change 5.5 (1.7) 6.0 (1.1) 6.0 (1.0)
Client-centered discussion & feedback 5.4 (1.5) 5.6 (.09) 6.5 (0.7)
Exploring pros/cons/ambivalence 5.5 (1.5) 5.6 (1.2) 6.5 (0.7)
Developing discrepancies 5.5 (1.5) 5.5 (1.3) 6.0 (1.0)
Change planning discussion 5.2 (1.5) 5.0 (1.7) 6.0 (1.0)
Reducing MI inconsistencies 5.3 (1.7) 5.2 (1.8) 6.3 (1.2)
Overall satisfaction 5.5 (1.6) 6.4 (0.7) 6.0 (0.0)
Helpfulness of training approach 3.8 (1.4) 4.4 (0.9) 5.0 (0.0)
Overall trainer skillfulness 4.0 (1.4) 4.9 (.04) 5.0 (0.0)
Open in a new tab

Note. Coverage was rated 1=not at all, 2=a little, 3=infrequently, 4=somewhat, 5=quite a bit, 6=considerably, 7=extensively. Overall satisfaction was rated 1=not at all, 2=very little, 3=a little, 4=somewhat, 5=moderately, 6=mostly, 7=completely., Helpfulness of training approach and overall trainer skillfulness were rated 1=not at all, 2=a little, 3=somewhat, 4=quite a bit, 5=very.

3.4. MI adherence and competence

Overall, counselors typically used fundamental MI strategies about twice as often as advanced ones and with adequate levels of competence (see Table 3). MI inconsistent strategies seldom occurred.

Table 3.

Counselor MI adherence and competence at different assessment points

Variable Training Steps Receive
Web Course Only Additional Workshop or Supervision
M SD n M SD n
Fundamental MI Strategies
 Adherence
  Baseline 4.4 1.3 15 4.0 1.1 10
  8 weeks 5.2 0.5 11 3.7 1.1 10
  16 weeks 4.3 1.2 8 4.5 0.9 8
  24 weeks 4.9 0.5 10 4.5 1.1 5
 Competence
  Baseline 4.1 0.8 15 3.8 1.2 10
  8 weeks 4.7 0.7 11 3.5 1.0 10
  16 weeks 4.6 1.1 8 4.8 0.7 8
  24 weeks 3.9 0.8 10 4.6 0.7 6
Advanced MI Strategies
 Adherence
  Baseline 2.9 1.1 15 2.5 1.2 10
  8 weeks 3.5 0.9 11 2.0 0.6 10
  16 weeks 2.3 1.1 8 2.8 1.0 8
  24 weeks 2.2 1.0 10 2.5 0.7 6
 Competence
  Baseline 3.9 0.6 14 3.8 1.2 8
  8 weeks 4.5 0.8 11 3.3 0.8 10
  16 weeks 4.7 1.0 7 4.6 1.0 8
  24 weeks 4.6 0.5 9 4.6 0.5 6
MI Inconsistent Strategies
 Adherence
  Baseline 1.9 1.0 15 1.9 0.8 10
  8 weeks 1.7 0.8 11 1.8 0.5 10
  16 weeks 2.2 1.1 8 1.2 0.3 8
  24 weeks 1.6 0.5 10 1.5 0.6 6
 Competence
  Baseline 3.7 0.9 9 3.5 1.5 8
  8 weeks 3.9 0.7 7 3.2 1.2 8
  16 weeks 4.3 0.8 6 4.0 1.4 2
  24 weeks 4.2 1.1 7 3.7 0.6 3
Open in a new tab

Note. Adherence was rated 1=not at all, 2=a little, 3=infrequently, 4=somewhat, 5=quite a bit, 6=considerably, 7=extensively. Competence was rated 1=very poor, 2=poor, 3=acceptable, 4=adequate, 5=good, 6=very good, 7=excellent.

Because of the very small sample sizes among those who received training steps 2 (n = 8) or 3 (n = 3) and the overlap of participants in them, these groups were combined for data analyses. MI adherence and competence was compared over time between those who only received the first step of training (Web course = 1) and those who had subsequent training steps (additional workshop or supervision = 0) using a random effects regression model for the six primary outcome variables (fundamental MI, advanced MI, and MI inconsistent adherence and competence) to allow the use of all available data and accommodate participants who had missing follow-up data points by estimating individual change trajectories (Bryk & Raudenbush, 2002). These analyses were conducted for the full sample (n = 26), which included five counselors who took the course but did not provide session samples to determine their need for additional steps of training and then dropped out, and the subsample excluding these counselors to examine training effects only for those who participated in the criterion-based stepwise MI performance assessment. There were no significant condition differences in baseline levels of fundamental and advance MI or MI inconsistent strategy adherence and competence in either sample.

The results of the random regression analyses (see Table 4) show that in the full sample, counselors who only took the Web course had significantly higher overall fundamental and advanced MI strategy adherence than those who participated in the workshop or supervision (ps < .05). Across time, counselors significantly increased their fundamental MI strategy adherence (p < .05) and competence (p < .01) and their advanced MI strategy competence (p < .01). No changes occurred in the counselors’ adherence or competence using MI inconsistent strategies.

Table 4.

Effects of training condition on MI adherence and competence over time for full sample and counselors who had a criterion-based stepwise assessment: results of random regression analyses

Fundamental MI Strategies Advanced MI Strategies MI Inconsistent Strategies
Adherence Competence Adherence Competence Adherence Competence
T P T p T p T p t p t P
Full Sample
Intercept 12.09 .000 14.95 .000 7.74 .000 14.02 .000 9.02 .000 9.95 .000
Condition 2.58 .013 1.77 .083 2.54 .015 1.94 .590 .006 .995 0.85 .402
Time 2.10 .045 3.22 .004 0.65 .523 3.42 .001 −1.64 .108 0.60 .551
Condition by Time −1.54 .135 −1.11 .277 −0.97 .339 −1.40 .167 0.66 .510 0.33 .740
Criterion Assessed
Intercept 12.96 .000 15.25 .000 7.61 .000 13.62 .000 8.93 .000 9.81 .000
Condition 3.41 .001 2.42 .020 3.07 .004 2.42 .020 0.50 .622 0.95 .349
Time 2.12 .043 3.24 .004 0.73 .475 3.47 .001 −1.63 .109 0.61 .459
Condition by Time −2.02 .053 −1.50 .147 −1.37 .182 −1.85 .072 0.26 .800 0.23 .879
Open in a new tab

Note. The full sample size contained 26 counselors. The criterion assessed subgroup included only the 21 counselors who provided session samples for stepwise MI performance assessment upon completing Step 1 training. For condition, web-based course = 1, and additional workshop or supervision = 0.

In the subsample of counselors who completed the criterion-based assessment, those who met adequate standards of MI performance after taking the course and received no further training had significantly higher overall fundamental and advanced MI strategy adherence (ps < .01) and competence (ps < .05) than those who required additional training. As in the full sample, counselors significantly increased their fundamental MI strategy adherence (p < .05) and competence (p < .01) and their advanced MI strategy competence (p < .01) across time. In addition, consistent with the pilot study’s hypothesis, counselors who fell below adequate standards and received additional steps of training significantly increased their fundamental MI strategy adherence over time (p = .05), whereas those who only had the course stayed the same on this performance dimension. MI inconsistent strategy adherence and competence did not vary significantly.

3.5. Condition differences in counselor characteristics

Chi-square likelihood ratios and independent t-tests were used to evaluate baseline characteristic differences between the groups of counselors in both conditions. Those who required extra steps of training following the course were older (53.5 vs. 45.8 years; t(24) = 2.10; p = .05), less likely to have graduate degrees (30% vs. 69%; X2 (1) = 3.80, p = .05), and more likely to be certified alcohol and drug counselors (50% vs. 12%; X2 (1) = 4.37, p = .04) and to have more years of counseling experience (19.8 vs. 10.8 years; t(24) = 2.87; p = .008). The alcohol and drug counselors were more likely to be older (57.6 vs. 47.5; t(24) = 3.34; p = .003) and have more years of counseling experience (23.1 vs. 8.5; t(24) = 3.35; p = .003), with the latter two characteristics of age and experience being positively associated (r = .56; p = .003). No other characteristics overlapped significantly.

3.6. Baseline MI Performance Adequacy

We repeated the random regression analyses for the full sample to explore the extent to which meeting adequate MI performance standards at baseline (i.e., prior to receiving any training) influenced the outcomes of criterion-based stepwise training over time and in interaction with training condition. Counselors who demonstrated adequate performance at baseline had overall significantly higher levels of fundamental adherence (t(27) = 3.59, p = .001) and competence (t(33) = 3.29, p = . 002) and advanced adherence (t(30) = 2.56, p = .016) and competence (t(54) = 3.21, p = .002) across assessment points than those who began training below criterion. However, only those below criterion at baseline showed significant improvements in fundamental adherence (t(23) = 3.19, p = .006) and competence (t(60) = 2.90, p = .009) and advanced adherence (t(22) = 2.76, p = .012) and competence (t(41) = 3.66, p = .004) with training over time. MI inconsistent strategy adherence and competence were unaffected.

4. Discussion

This pilot study has three main findings. First, counselors who showed inadequate MI performance immediately after taking a Web course and who subsequently participated in a workshop or supervision improved their adherence to fundamental MI strategies over time. Second, counselors who performed MI adequately following the Web course continued to demonstrate similar levels of fundamental and advanced MI adherence and competence over a 24-week period without additional training. Third, counselors who showed inadequate MI performance before training were more likely to improve their adherence and competence with training than those who already performed MI adequately.

As hypothesized, the criterion-based stepwise approach differentially benefited counselors who showed inadequate performance, notably in the area of increasing their use of fundamental MI strategies, though not in other areas of MI adherence and competence. In contrast, counselors who reached adequate standards after only taking a web course did not uniquely improve their performance over time in the absence of further training. As designed in this pilot study, a stepwise approach differentially focused on those with poorer skills and simply monitored those counselors who already performed MI well enough in order to concentrate training resources and efforts on those who needed to implement MI better. A higher adherence and competence cut-off criterion could be used to “raise the bar” when the aim is to train counselors to higher levels of proficiency.

The training provided in this pilot study concentrated mostly on basic principles and techniques of MI in order to help counselors practice the approach adequately. These fundamental strategies (e.g., reflective listening to convey empathy) underpin the style in which MI is delivered and are considered the foundation from which other techniques in MI are used (Miller & Rollnick, 2002). The focus on fundamentals as a cornerstone of adequate MI performance is consistent with recommendations for a staged approach to teaching MI, wherein the emphasis on more directive strategies for eliciting clients’ motivations for change or for skillfully handling resistance typically comes later and represents more advanced skill sets (Miller & Moyers, 2006). The MI consistent adherence ratings were highest for fundamental strategies and formed the basis for most counselors achieving the adequate performance criterion. However, counselors who met this criterion early could have been provided with more advanced training opportunities rather than being “stepped” out of further training opportunities. This alternative approach might have resulted in significant advanced MI strategy adherence and competence performance improvements for this group. Matching counselor training needs to different training approaches has been largely unexamined in the counselor training literature.

This pilot study also found that baseline achievement of adequate MI performance standards moderated training outcomes. Only counselors who began stepwise MI training below criterion significantly improved their performance over time. Prior counselor training studies have not used counselors’ baseline levels of treatment adherence or competence as inclusion or exclusion criteria, as occurs in a parallel fashion in client treatment trials where clients have to demonstrate the targeted problem (e.g., substance use 28 days prior to randomization) to be eligible to participate in the pilot study. As designed, counselor training studies likely hamper their ability to demonstrate the effectiveness of different training strategies by including counselors who already have the targeted skills and have little room for additional improvement.

An interesting finding was that the group of counselors who required additional training to meet adequate MI standards was comprised of older certified alcohol and drug counselors who had been working in the field for many years. The addiction treatment workforce is aging (Kaplan, 2003) and has relied upon a nonprofessional 12-step approach often delivered by people in recovery (Guydish, 2003). Recommendations and mandates for the field to adopt evidence-based treatments (Reickmann, Kovas, Fussell &, Stettler, 2009) will need to take into account the workforce’s demographics and provide sufficient resources for the type and amount of training needed to help counselors achieve and sustain adequate practice standards. Larger randomized multi-site controlled trials, including protected training time and financial incentives for counselors (for completing each training step) and for clients (for allowing audiotaped sessions), are needed to first establish the efficacy of a stepwise training approach and then determine how counselor characteristics may moderate these effects.

Finally, the feasibility of using a criterion-based stepwise training approach deserves consideration. The time required for training provided in this pilot study was less than what is commonly recommended for learning MI training (e.g., 8-hour rather than a 2–3 day workshop; 1–2 supervisions over two months instead of biweekly meetings over several months), and all training was easily accessible (e.g., web course available 24 hours a day everyday) or delivered on-site in small time blocks (e.g., a series of four 2-hour weekly workshops). Nevertheless, 7 counselors discontinued participation because they believed they had insufficient time in their work day for the pilot study’s training activities and assessments. Large caseloads and busy schedules, combined with difficulty in obtaining client consent to audiotape sessions, were the largest barriers to participation noted by counselors. In addition, assessment of MI skills between training steps required having trained raters who could reliably and validly discern adequate from inadequate performance. While some efforts have been made nationally to prepare professionals for this purpose (Martino et al., 2010), accessing these resources and having performance evaluations conducted in a timely fashion may be difficult. Implementing a criterion-based stepwise training approach for teaching counselors behavioral treatments requires substantial organization, coordination, and commitment on the part of the trainers, evaluators, counselors, and sponsoring agencies.

This pilot study had several limitations: 1) small sample size; 2) no wait-list or attention control condition; 3) no randomization to training conditions (control vs. stepwise); this would require oversampling to the stepwise condition to provide a sufficient number of counselors who receive each training step; 4) no follow-up period to test the durability of training effects; 5) self-selected client sessions that might not reflect counselors’ use of MI more broadly within their programs or overestimate the actual effects obtained in this pilot study; 6) high participant attrition that might further over-estimate effects if counselors dropped out because they were unable to perform MI adequately or meet the training requirements; 6) no assessment of clients’ verbal statements within sessions (e.g., frequency of statements that favor or disfavor change), a good proxy for behavior change (Amrheim, Miller, Yahne, Palmer, & Fulcher, 2003) or their treatment outcomes; 7) the infrequency of counselor MI inconsistencies at all assessment points that disallowed understanding how a stepwise training approach might impact this area; and 8) no independent fidelity checks of the trainer’s conduct of the Web course, workshop, or supervision beyond the counselors’ satisfaction ratings.

Nonetheless, this pilot study provides preliminary evidence for the feasibility and effectiveness of a criterion-based stepwise approach for training counselors in behavioral treatments. The pilot study’s findings suggest that different counselors likely require different types and amounts of training in order to perform a behavioral treatment adequately. Future research should focus on determining which counselors should be targeted for skill improvement and what types of training should be provided to them.

Acknowledgments

The U.S. Veterans Administration New England Mental Illness Research Education and Clinical Center supported this pilot study. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the U.S. Veterans Administration. The authors are grateful to the program directors where the pilot study occurred (Judith Cooney, Laurie Harkness, David Pilkey, Ismene Petrakis) and the independent tape raters (Joanne Corvino, Francis Giannini, Rachel Hart, Karen Hunkele, Mark Lawless, Charles Pearson). The rating scales described in the pilot study are available from Dr. Martino.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Amrheim P, Miller WR, Yahne CE, Palmer M, Fulcher L. Client commitment language during motivational interviewing. Journal of Consulting and Clinical Psychology. 2003;71:862–78. doi: 10.1037/0022-006X.71.5.862. [DOI] [PubMed] [Google Scholar]
  2. Baer JS, Ball SA, Campbell BK, Miele GM, Schoener EP, Tracy K. Training and fidelity monitoring of behavioral interventions in multi-site addictions research: A review. Drug and Alcohol Dependence. 2007;87:107–118. doi: 10.1016/j.drugalcdep.2006.08.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Baer JS, Rosengren DB, Dunn CW, Wells EA, Ogle RL, Hartzler B. An evaluation of workshop training in motivational interviewing for addiction and mental health clinicians. Drug and Alcohol Dependence. 2004;73:99–106. doi: 10.1016/j.drugalcdep.2003.10.001. [DOI] [PubMed] [Google Scholar]
  4. Ball SA, Bachrach K, DeCarlo J, Farentinos C, Keen M, McSherry T, Polcin D, Snead N, Sockriter R, Wrigley P, Zammarelli MA, Carroll KM. Characteristics, beliefs, and practices of community clinicians trained to provide manual-guided therapy for substance abusers. Journal of Substance Abuse Treatment. 2002;23:309–318. doi: 10.1016/s0740-5472(02)00281-7. [DOI] [PubMed] [Google Scholar]
  5. Ball SA, Martino S, Nich C, Frankforter TL, Van Horn D, Crits-Christoph P, Woody GE, Obert JL, Farentinos C, Carroll KM. Site matters: Motivational enhancement therapy in community drug abuse clinics. Journal of Consulting and Clinical Psychology. 2007;75:556–567. doi: 10.1037/0022-006X.75.4.556. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Beidas RS, Kendall PC. Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology Science and Practice. 2010;17:1–30. doi: 10.1111/j.1468-2850.2009.01187.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bryk KT, Raudenbush SW. Hierarchical Linear Models: Applications and Data Analysis Methods. 2. Newbury Park, CA: Sage Publications; 2002. [Google Scholar]
  8. Carroll KM. New methods of treatment efficacy research: Bridging clinical research and clinical practice. Alcohol Health & Research World. 1997;21:352–358. [PMC free article] [PubMed] [Google Scholar]
  9. Carroll KM, Ball SA, Nich C, Martino S, Frankforter TL, Farentinos C, Kunkel L, Mikulich-Gilbertson S, Morgenstern J, Obert JL, Polcin D, Snead N, Woody GE. Motivational interviewing to improve treatment engagement and outcome in individuals seeking treatment for substance abuse: A multisite effectiveness study. Drug and Alcohol Dependence. 2006;81:301–312. doi: 10.1016/j.drugalcdep.2005.08.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Carroll KM, Martino S, Suarez-Morales L, Ball SA, Miller WR, Añez L, Paris M, Nich C, Frankforter TL, Matthews J, Farentinos C, Szapocznik J. Multisite Randomized Controlled Effectiveness Trial of Motivational Enhancement for Spanish-Speaking Substance Users. Journal of Consulting and Clinical Psychology. 2009;77:993–999. doi: 10.1037/a0016489. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Collins LM, Murphy SA, Bierman KL. A conceptual framework for adaptive preventive interventions. Prevention Science. 2004;5:185–96. doi: 10.1023/b:prev.0000037641.26017.00. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Cucciare MA, Weingardt KR, Villafranca S. Using blended learning to implement evidence-based psychotherapies. Clinical Psychology: Science and Practice. 2008;15:299–307. [Google Scholar]
  13. Falender CA, Shafranske EP. Competence in competency-based supervision: Construct and application. Professional Psychology: Research and Practice. 2007;38:232–240. [Google Scholar]
  14. Guydish J. Introduction: Dissemination from practice to research. In: Sorensen JL, Rawson RA, Guydish J, Zweben JE, editors. Drug abuse treatment through collaboration: Practice and research partnerships that work. Washington, DC: American Psychological Association; 2003. pp. 13–16. [Google Scholar]
  15. Kaplan L. Substance Abuse Treatment Workforce Environmental Scan. Rockville, MD: U.S. Department of Health and Human Services Substance Abuse and Mental Health Services Administration Center for Substance Abuse Treatment; 2003. from http://partnersforrecovery.samhsa.gov/docs/Environmental_Scan.pdf. [Google Scholar]
  16. Lundahl BW, Kunz C, Brownell C, Tollefson D, Burke B. Meta-analysis of motivational interviewing: Twenty Five years of empirical studies. Research on Social Work Practice. 2010;20:137–160. [Google Scholar]
  17. Martino S. Delivering evidence-based treatments: Strategies for counselor training. Addiction Science and Clinical Practice. in press. [PMC free article] [PubMed] [Google Scholar]
  18. Martino S, Ball SA, Gallon SL, Hall D, Garcia M, Ceperich S, Farentinos C, Hamilton J, Hausotter W. Motivational interviewing assessment: Supervisory tools for enhancing proficiency. Salem, OR: Northwest Frontier Addiction Technology Transfer Center, Oregon Health and Science University; 2006. [Google Scholar]
  19. Martino S, Ball SA, Nich C, Canning-Ball M, Rounsaville BJ, Carroll KM. Teaching community program clinicians motivational interviewing using expert and train-the-trainer strategies. Addiction. doi: 10.1111/j.1360-0443.2010.03135.x. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Martino S, Ball SA, Nich C, Frankforter TL, Carroll KM. Community program therapist adherence and competence in motivational enhancement therapy. Drug and Alcohol Dependence. 2008;96:37–48. doi: 10.1016/j.drugalcdep.2008.01.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Martino S, Brigham GS, Higgins C, Gallon S, Freese TE, Albright LM, Hulsey EG, Krom L, Storti SA, Perl H, Nugent CD, Pintello D, Condon TP. Partnerships and pathways of dissemination: The NIDA-SAMHSA Blending Initiative in the Clinical Trials Network. Journal of Substance Abuse Treatment. 2010;38:S31–S48. doi: 10.1016/j.jsat.2009.12.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Miller WR. Enhancing Motivation for Change in Substance Abuse Treatment TIP Series 35. Rockville, MD: U.S. Department of Health and Human Services; 1999. Publication No (SMA) 02-3693. [Google Scholar]
  23. Miller WR, Sorensen JL, Selzer JA, Brigham GS. Disseminating evidence-based practices in substance abuse treatment: A review with suggestions. Journal of Substance Abuse Treatment. 2006;31:25–39. doi: 10.1016/j.jsat.2006.03.005. [DOI] [PubMed] [Google Scholar]
  24. Miller WR, Moyers TB. Eight stages in learning motivational interviewing. Journal of Teaching in the Addictions. 2006;5:3–17. [Google Scholar]
  25. Miller WR, Rollnick S. Motivational Interviewing: Preparing People for Change. 2. New York: Guilford Press; 2002. [Google Scholar]
  26. Miller WR, Yahne CE, Moyers TE, Martinez J, Pirritano M. A randomized trial of methods to help therapists learn motivational interviewing. Journal of Consulting and Clinical Psychology. 2004;72:1050–1062. doi: 10.1037/0022-006X.72.6.1050. [DOI] [PubMed] [Google Scholar]
  27. Mitcheson L, Bhavsar K, McCambridge J. Randomized trial of training and supervision in motivational interviewing with adolescent drug treatment practitioners. Journal of Substance Abuse Treatment. 2009;37:73–78. doi: 10.1016/j.jsat.2008.11.001. [DOI] [PubMed] [Google Scholar]
  28. Norwinski J, Baker S, Carroll K. NIAAA Project MATCH Monograph Series, Vol. 1, DHHS Publication No. (ADM) 92-1893. Washington: Government Printing Office; 1992. Twelve-Step Facilitation Therapy Manual: A Clinical Research Guide for Therapists Treating Individuals with Alcohol Abuse and Dependence. [Google Scholar]
  29. Prochaska JO, DiClemente CC, Norcross JC. In search of how people change: Applications to addictive behaviors. American Psychologist. 1992;47:1102–1114. doi: 10.1037//0003-066x.47.9.1102. [DOI] [PubMed] [Google Scholar]
  30. Reickmann TR, Kovas AE, Fussell HE, Stettler NM. Implementation of evidence-based practices for treatment of alcohol and drug disorders: The role of the state authority. Journal of Behavioral Health Services Research. 2009;36:407–19. doi: 10.1007/s11414-008-9122-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Santa Ana EJ, Carroll KM, Añez L, Paris M, Ball SA, Nich C, Frankforter TL, Suarez-Morales L, Szapocznik J, Martino S. Evaluating motivational enhancement therapy adherence and competence among Spanish-speaking therapists. Drug and Alcohol Dependence. 2009;103:44–51. doi: 10.1016/j.drugalcdep.2009.03.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Shafer MS, Rhode R, Chong J. Using distance education to promote the transfer of motivational interviewing skills among behavioral health professionals. Journal of Substance Abuse Treatment. 2004;26:141–48. doi: 10.1016/S0740-5472(03)00167-3. [DOI] [PubMed] [Google Scholar]
  33. Sholomskas DE, Carroll KM. One small step for manuals: Computer-assisted training in twelve-step facilitation. Journal of Studies on Alcohol. 2006;67:939–945. doi: 10.15288/jsa.2006.67.939. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Sholomskas DE, Syracuse-Siewert G, Rounsaville BJ, Ball SA, Nuro KF, Carroll KM. We don’t train in vain: A dissemination trial of three strategies of training clinicians in cognitive behavioral therapy. Journal of Consulting and Clinical Psychology. 2005;73:106–115. doi: 10.1037/0022-006X.73.1.106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin. 1979;86:420–429. doi: 10.1037//0033-2909.86.2.420. [DOI] [PubMed] [Google Scholar]
  36. Walters, Matson, Baer, Ziedonis Effectiveness of workshop training for psychosocial addiction treatments: A systematic review. Journal of Substance Abuse Treatment. 2005;29:283–293. doi: 10.1016/j.jsat.2005.08.006. [DOI] [PubMed] [Google Scholar]

RESOURCES