Abstract
Motivational interviewing (MI) is an evidence based intervention with considerable support for promoting behavior change across a broad range of health and mental health issues. Despite its effectiveness, challenges associated with learning the approach may limit its full implementation in many clinical settings. The aim of the present study was to evaluate a supervised MI training practicum implemented within a doctoral internship/postdoctoral fellowship training Program. The goal of the practicum was to enable each trainee to achieve expert competence in MI. Participants were 29 psychology doctoral interns and 1 postdoctoral fellow who participated in the training as part of their internship or fellowship program. Training included an initial workshop followed by a supervised practicum during which progress towards an a priori established expert competence benchmark was tracked through the use of an established coding system. Results indicated that trainees were satisfied with the supervision received. Three trainees did not achieve the a priori benchmark due to schedule conflicts. The 27 trainees who achieved the benchmark required between 4 and 20 supervision sessions to do so (mean = 9.22, SD = 3.77). With the exception of reflective listening skill, prior training, baseline skill, and self-reported motivation were not associated with number of supervision sessions required to achieve the benchmark. Implications for training and dissemination of MI in clinical settings are discussed.
Keywords: Motivational Interviewing, Supervision, Dissemination, Clinical Psychology, Education and Training
Motivational Interviewing (MI) is a therapeutic approach with demonstrated efficacy in improving outcomes related to substance use, health behavior, and probation and parole (Lundahl, Tollefson, Kunz, Brownell, & Burke, 2010), and emerging support for its efficacy in addressing other mental health problems (Westra, Aviram, & Doell, 2011). As such, MI may improve the effectiveness of psychologists across a broad range of treatment settings and with varying presenting problems. However, Miller and Rollnick (2009) have noted that MI is not easy to learn. Recent meta-analyses of MI training studies indicate that 12–16 hours of workshop training results in improvements in MI skill but does not result in proficiency or enduring skill increases for most clinicians unless they also receive coaching or supervision (de Roten, Zimmerman, Ortega, & Despland, 2013; Schwalbe, Oh and Zweben (2014). However the amount of supervision necessary for most clinicians to achieve proficiency benchmarks remains uncertain (Smith et al., 2012).
Organizing MI training and supervision around a desired outcome, rather than a predetermined training package or sequence is consistent with a competency-based model in professional psychology training. Within this model, trainees are expected to achieve competency benchmarks as determined by adequate and appropriate assessment of competencies (Fouad et al., 2009; Kenkel, 2009; Rodolfa et al., 2005). Consistent with this model, a goal of the current education project was to implement and evaluate an individualized, supervised practicum in MI. The practicum provided each intern and fellow with the necessary amount of training to achieve expert competence in MI. Three key questions were of interest in the evaluation of the supervision practicum: 1) How satisfied are trainees with this model of supervision? 2) How many sessions of supervision would be necessary to enable trainees to achieve a pre-determined threshold of competence in MI (Moyers, Martin, Manuel, Miller, & Ernst, 2010)? 3) Which, if any, baseline characteristics would be associated with the dose of supervision required by each intern or fellow?
Method
Participants
Between 2010 and 2013, 29 doctoral interns (13 men, 16 women) and 1 man completing a postdoctoral fellowship at [redacted] participated in a MI curriculum as part of their clinical training. Informed consent was obtained from trainees to have information collected for training and program evaluation purposes used for educational research purposes. No trainees declined consent. This project was designated as exempt human subjects research by the [redacted] Institutional Review Board (IRB) and VAMC involvement as a practicum site was determined not to meet the definition of human subjects research by the VAMC IRB.
Measures
Background characteristics and prior training
Trainees provided information on select demographic characteristics, specialty areas (e.g., child, adolescent, and adult), prior training in MI, and years of experience as a therapist or counselor.
Baseline skill
Trainees completed the Video Assessment of Simulated Encounters-Revised (VASE-R) prior to initiating the MI training curriculum. During the VASE-R, an individual responds to 18 timed prompts inserted into three video-based clinical vignettes with actors portraying difficult substance abuse clients (Rosengren et al., 2005). Each prompt assesses a particular MI skill. Responses are scored on a 3 point scale (0 = MI-inconsistent to 2 = MI-consistent). VASE-R total scores range from 0 to 36, and subscale scores have varying ranges: reflective listening (0–8), responding to resistance (0–10), summarizing (0–6), change talk (0–6), and developing discrepancy (0–6). Higher scores indicate greater MI skill. Rosengren, Hartzler, Baer, Wells, and Dunn (2008) identified VASE-R score benchmarks for qualifying trainees as (a) untrained, (b) beginning proficiency, and (c) expert proficiency.
Baseline attitudes
At the outset of the MI curriculum, trainees completed two copies of the Change Questionnaire Version 1.2 (Miller, Moyers & Amrhein, 2005). This 12-item instrument is designed to assess motivation to change a particular behavior along several dimensions. Respondents are asked to rate on an 11 point scale, ranging from “0 = definitely not” to “10 = definitely” the degree to which they agree with each of 12 statements about the change under consideration. Respondents were asked about two specific changes: 1) to learn Motivational Interviewing; and 2) to integrate Motivational Interviewing into daily clinical practice. Chronbach’s alphas for the two versions were α = .88 and α = .96, respectively.
Trainee Perception of Supervision
A 12-item measure was developed to assess intern perceptions of and satisfaction with the supervision received in MI (see Table 1). Respondents use a 10-point scale ranging from 1 = “not at all” to 10 = “very much” to indicate how much they agree with each statement. This measure, was collected anonymously in the final months of the training year. Chronbach’s alpha in the current sample was.96.
Table 1.
Item | n | Mean | SD |
---|---|---|---|
My supervisor had the necessary skills in both motivational interviewing and supervision to help me improve as a motivational interviewer. | 28 | 9.64 | 0.99 |
I am a better motivational interviewer because of the supervision I received. | 28 | 9.32 | 1.42 |
My supervisor provided fair and objective feedback on my motivational interviewing skills. | 28 | 9.50 | 0.96 |
My supervisor helped me identify my strengths and weaknesses as a motivational interviewer. | 28 | 9.14 | 1.41 |
My supervisor helped me improve my motivational interviewing skills while still showing respect for other counseling approaches I use. | 28 | 9.25 | 1.29 |
My supervisor helped me learn to incorporate motivational interviewing with other counseling approaches I use. | 28 | 7.75 | 2.49 |
My supervisor supported my self-efficacy as a therapist. | 26 | 9.19 | 1.44 |
My supervisor asked me for my impression of how a session went, before providing me with feedback. | 28 | 9.96 | 0.19 |
My supervisor was responsive to concerns or problems I raised during our supervision sessions. | 28 | 9.50 | 1.11 |
My supervisor made him or herself and other resources available to me. | 28 | 9.36 | 1.31 |
Overall I was satisfied with the supervision I received in Motivational interviewing. | 28 | 9.50 | 1.26 |
I would recommend this supervisor to a friend. | 28 | 9.64 | 1.06 |
Supervision Dose
The a priori training benchmark established for the MI curriculum was defined as two consecutive 10-minute work samples coded at or above the expert competence level on the Motivational Interviewing Treatment Integrity Code 3.0 (MITI 3.0; Moyers et al., 2010) by at least two supervisors. The number of supervision sessions a trainee required to achieve the benchmark was documented by supervisors. Patients who received MI were not consented to participate in educational research, so MITI codes were not used as data.
Procedure
Workshop
Consistent with typical MI training models (de Roten, Zimmerman, Ortega, & Despland, 2013; Miller, Yahne, Moyers, Martinez, & Pirritano, 2004; Smith et al., 2012), a 15-hour introductory MI workshop was presented by the first author to all trainees prior to their participation in the supervised practicum. The workshop integrated didactic presentations, video and live demonstrations, and experiential exercises to teach basic MI concepts and skills. The workshop was organized to reflect the sequence of learning MI outlined by Miller and Moyers (2006), and content was informed by a survey of MI trainers about recommended best practices (Schumacher et al. 2012). During the workshop, trainees recorded a role-played MI session and received subsequent feedback about their performance.
Supervision
Following the MI workshop, interns completed an MI practicum in the adult substance abuse treatment programs affiliated with the training program. The vast majority of MI practicum sessions were conducted at those placements, and all interns completed several MI sessions within a substance abuse treatment program. Interns could also choose to complete a small number of supervised MI sessions in other treatment programs. Supervisors were recognized trainers in MI. At the time of project completion, the first author had been a member of the Motivational Interviewing Network of Trainers for 8 years. The second and fifth authors were National MI trainers in the Veterans Health Administration.
Work Samples
To generate a work sample for supervision, an intern would conduct a 30–60 minute motivational interview with a patient receiving services at his or her practicum site. Patients provided consent to have their session audio-recorded for supervision purposes using the informed consent procedures in place at each practicum site.
Supervision Preparation
Interns were instructed that following each MI session, they should code 10–20 minutes of the session using the MITI 3.0 (Moyers et al., 2010) and submit their coding sheet and the sample to their supervisor. Supervisors also had the option of instructing interns to fill out a brief 3-item, open-response form indicating: 1) what they thought went well in the session; 2) what they thought did not go well in the session; and 3) whether or not they would like the supervisor to listen to a specific portion of the session. Supervisors coded an intern’s work sample with the MITI 3.0 either prior to or during supervision sessions.
Supervision Sessions
Supervision sessions generally followed the format suggested by Martino et al. (2006). The supervisor first elicited and reflected the intern’s perception of the session and then asked permission to either review his or her feedback based on the MITI 3.0 with the intern or to listen and code the session collaboratively with the intern. After an intern’s strengths and weaknesses in the implementation of MI during a particular work sample were identified, the supervisor and intern worked collaboratively to identify goals for improvement.
Meta-Supervision
Although the first and second author provided primary supervision to 27 out of the 30 participants, a total of 3 faculty members served as MI supervisors [redacted] and met for weekly meta-supervision. During meta-supervision, supervisors would: 1) discuss any problems or challenges in providing supervision or applying the MITI 3.0 (Moyers et al., 2010) to work samples; 2) review the progress each intern or fellow was making in the practice of MI; 3) listen to and consensus code recent work samples using the MITI 3.0 to determine whether a priori benchmark had been met; and 4) discuss MI training goals for each trainee.
Results
Participant Characteristics
Participants most frequently indicated that their age fell into the category of 18 and 30 (n = 24; 80%), their race was White (n = 25; 83%), and their ethnicity was Not Hispanic or Latino (n = 27; 90%). Half of participants indicated their clinical specialty was adult only and 27% indicated they specialized in children/adolescents only (n = 8; 27%). Participants reported an average of 4.3 (SD = 1.42) years of experience as a therapist or counselor. Slightly more than half reported they had never received formal training in MI (n = 16), 77% (n = 23) indicated they had never watched an MI training video, and 70% (n = 21) indicated they had at least skimmed Motivational interviewing: Preparing people for change (Miller & Rollnick, 2002).
As shown in Table 2, on the Change Questionnaire Version 1.2, the mean item score on this measure for the target of learning MI was 8.6 (SD = 0.91) and the mean item score for the target of implementing MI in their daily practice was 7.93 (SD = 1.54). An item score of 10 represents “definite” agreement with a statement about readiness to change. Total score for the VASE-R, as well as the Responding to Resistance, Summarizing, and Eliciting Change Talk subscale scores fell within 1-point of the beginning proficiency benchmark established by Rosengren et al. (2008). VASE-R Reflective Listing and Developing Discrepancy subscale scores exceeded the beginning proficiency benchmark.
Table 2.
Pearson Correlations | |||||||||
---|---|---|---|---|---|---|---|---|---|
Variable (Beginning/Expert VASE-R Benchmark) | Mean (SD) | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
1. No. Supervision Sessions | 9.22 (3.77) | .05 | .04 | −.10 | −.43* | .05 | −.14 | .03 | .12 |
2. CQ - Motivation to Learn MI | 8.60 (0.91) | -- | .72** | .23 | .08 | .22 | .16 | .08 | .13 |
3. CQ - Motivation to Implement MI | 7.93 (1.54) | -- | -- | .25 | .22 | .11 | .13 | .36 | −.01 |
4. VASE-R Total Score (26/31) | 25.59 (4.74)a | -- | -- | -- | .65** | .87** | .63** | .56** | .26 |
5. VASE-R Reflective Listening (6/7) | 6.67 (1.39)b | -- | -- | -- | -- | .45* | .49** | .32 | −.30 |
6. VASE-R Respond. to Resistance (8/9) | 7.15 (2.16)a | -- | -- | -- | -- | -- | .45* | .30 | .23 |
7. VASE-R Summarizing (3/5) | 2.93 (1.38)a | -- | -- | -- | -- | -- | -- | .12 | −.17 |
8. VASE-R Elicit. Change Talk (4/5) | 3.93 (1.33)a | -- | -- | -- | -- | -- | -- | -- | .04 |
9. VASE-R Dev. Discrepancy (4/5) | 4.93 (1.36)b | -- | -- | -- | -- | -- | -- | -- | -- |
Note. CQ = Change Questionnaire Version 1.2. Mean item score for the CQ is presented. VASE-R = Video Assessment of Simulated Encounters Revised. Item summary scores for the VASE-R are presented.
within one point of VASE-R beginning proficiency benchmark,
exceeds VASE-R beginning proficiency benchmark.
p < .05,
p < .01.
Satisfaction
Intern and fellow satisfaction questionnaires were submitted by 28 participants. As shown in Table 1, overall trainees were very satisfied with the supervision they received, and reported a mean satisfaction item rating of 9.31 (SD = 1.09) on a 10 point scale.
Supervision Dose
Three participants were excluded from these analyses. One participant had served as a therapist on one of the authors’ clinical trials prior to the start of the internship year. Two participants never achieved the a priori training benchmark due to scheduling challenges: one completed a total of 14 sessions and achieved one sample at expert competence and the other completed a total of 10 sessions and did not achieve expert competence on any sample. Of the remaining 27 participants, the number of sessions required to achieve the benchmark ranged from 4 to 20 (Mean = 9.22; SD = 3.77).
Factors Associated with Supervision Dose
As shown in the correlation matrix presented in Table 2, among the 27 trainees who achieved the benchmark, scores on the Change Questionnaire Version 1.2 were not significantly correlated with the necessary supervision dose. With the exception of the Reflective Listening subscale, scores on the VASE-R also failed to demonstrate significant correlations with this outcome. The Reflective Listening subscale score was significantly negatively correlated with supervision dose (r = −.47, p = .03).
A series of three independent samples t-tests were conducted to determine whether trainees who had prior formal training in MI, had watched one or more MI videos, or had read some of all of the Miller and Rollnick (2002) MI text required a significantly different number of sessions than peers who had not had these prior experiences. Comparison of the mean number of supervision sessions required by those with prior formal training (mean = 8.00, SD = 2.65) and those without (mean = 10.36, SD = 1.17), revealed no significant difference between the groups t (25) = 1.68, p = .11, however, was associated with a large effect size (d = 1.15). Comparison of the mean number of supervision sessions required by those who had read the MI book (mean = 8.74, SD = 3.09) to those who had not (mean = 10.38, SD = 5.10), revealed no significant difference between the groups t (25) = 1.03, p = .31, and was associated with a small effect size (d = .39). Comparison of the mean number of supervision sessions required by those who had watched at least one MI training video (mean = 9.33, SD = 3.20) to those who had not (mean = 9.19, SD = 3.98), also revealed no significant difference between the groups t (25) = 0.08, p = .94, d = .04.
Discussion
The results of this project are consistent with the competency model of training (e.g., Fouad et al., 2009), which promotes an individualized approach to helping trainees achieve benchmarks. The number of sessions necessary to enable an intern to meet the expert competence benchmark ranged from 4–20. If a set number of feedback-based supervision sessions had been used instead of the individualized, outcome-based approach, the majority of trainees would not have reached the expert competency benchmark. In fact only 4 (14.8%) would have been successful within 5 post-workshop supervision sessions, a number used in MI supervision research (e.g., Smith et al., 2012). This hypothetical outcome would have been comparable to previous well-controlled studies, indicating that most learners do not reach the expert competency benchmark with five feedback and coaching sessions.
In contrast to some prior work in MI training suggesting that clinician motivation to learn MI is an important predictor of training outcomes (e.g., Miller et al., 2004), self-reported motivation was not associated with the number of sessions required to achieve the competency benchmark in the current project. In interpreting this finding it is important to note that the current sample was a highly motivated sample overall. With the exception of reflective listening, baseline level of skill in MI as measured with the VASE-R (Rosengren et al., 2005; 2008) was also not significantly associated with the number of sessions of supervision necessary to enable an intern to achieve the competency benchmark. Scores on the reflective listening subscale of the VASE-R had a significant negative correlation of medium to large magnitude with this training outcome. This is consistent with prior findings that baseline empathy and reflective listening predict training outcomes (Smith et al., 2012). The association between prior formal training in MI and number of supervision sessions required failed to reach statistical significance (i.e., p = .11) but was associated with a large effect size (d = 1.15). Given the large effect size, this finding can be tentatively interpreted as being consistent with research indicated that baseline MI skill positively predicts of MI training outcomes (Carpenter et al., 2012).
Limitations and Future Directions
The current educational research project was characterized by several limitations including: a small sample size, anonymous collection of intern perception of training data that prevented linakage to other data, no control group, no comparison of supervisor MITI 3.0 codes at the item-level to establish inter-rater reliability, and no rigorous monitoring of supervision fidelity. Another limitation related to the measures used to evaluate supervision outcomes in the current project is that they were primarily focused on intervention competence. The supervised MI practicum provides a rich opportunity for trainees to develop and demonstrate numerous competencies relevant to the practice of psychology such as cultural competence and self-reflective practice (e.g., Rodolfa et al., 2005). Although these aspects of performance were evaluated by supervisors, these evaluations were not linked specifically to the MI practicum.
Findings from the present study, as well as prior MI training research (Smith et al., 2012), suggest that MI training protocols may benefit from a more variable training process that titrate the amount of supervision to the needs of the trainee. Future research is necessary to further explore the dose of supervision required to achieve expert competency and how trainees acquire MI skill over the course of supervision, as well as the amount of supervisor and trainee time and financial costs associated with such supervision.
Acknowledgments
This work was funded by a research education grant from the National Institute on Drug Abuse (R25 DA026637, PD: Schumacher)
References
- de Roten Y, Zimmermann G, Ortega D, Despland J. Meta-analysis of the effects of MI training on clinicians' behavior. Journal of Substance Abuse Treatment. 2013;45:155–162. doi: 10.1016/j.jsat.2013.02.006. [DOI] [PubMed] [Google Scholar]
- Carpenter KM, Cheng WY, Smith JL, Brooks AC, Amrhein PC, Wain RM, Nunes EV. “Old dogs” and new skills: How clinician characteristics relate to motivational interviewing skills before, during and after training. Journal of Consulting and Clinical Psychology. 2012;80:560–573. doi: 10.1037/a0028362. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fouad NA, Grus CL, Hatcher RL, Kaslow NJ, Hutchings PS, Madson MB, Collins FL, Jr, Crossman RE. Competency benchmarks: A model for understanding and measuring competence in professional psychology across training levels. Training and Education in Professional Psychology. 2009;3:S5–S26. doi: 10.1037/a0015832. [DOI] [Google Scholar]
- Kenkel MB. Adopting a competency model for professional psychology: Essential elements and resources. Training and Education in Professional Psychology. 2009;3:S59–S62. doi: 10.1037/a0017037. [DOI] [Google Scholar]
- Lundahl BW, Tollefson D, Kunz C, Brownell C, Burke B. Meta-analysis of Motivational interviewing: Twenty five years of research. Research on Social Work Practice. 2010;20:137–160. doi: 10.1177/1049731509347850. [DOI] [Google Scholar]
- Martino S, Carroll KM, Nich C, Rounsaville BJ. A randomized controlled pilot study of motivational interviewing for patients with psychotic and drug use disorders. Addiction. 2006;101:1479–192. doi: 10.1111/j.1360-0443.2006.01554.x. [DOI] [PubMed] [Google Scholar]
- McCutcheon SR. Competency benchmarks: Implications for internship training. Teaching and Education in Professional Psychology. 2009;3:S50–S53. doi: 10.1037/a0016966. [DOI] [Google Scholar]
- Miller WR, Mount KA. A small study of training in motivational interviewing: Does one workshop change clinician and client behavior. Behavioural and Cognitive Psychotherapy. 2001;29:457–471. doi: 10.1017/S1352465801004064. [DOI] [Google Scholar]
- Miller WR, Moyers TB. Eight stages in learning motivational interviewing. Journal of Teaching in the Addictions. 2006;5:3–17. doi: 10.1300/J188v05n01_02. [DOI] [Google Scholar]
- Miller WR, Moyers TB, Amrhein P. Change Questionnaire Version 1.2. Department of Psychology, University of New Mexico; Albuquerque: 2005. Unpublished measure. [Google Scholar]
- Miller WR, Rollnick S. Motivational interviewing: Preparing people for change. 2. New York: Guilford; 2002. [Google Scholar]
- Miller WR, Rollnick S. Motivational interviewing: Helping people change. 3. New York: Guilford; 2013. [Google Scholar]
- Miller WR, Rollnick S. Ten things motivational interviewing is not. Behavioural and Cognitive Psychotherapy. 2009;37:129–140. doi: 10.1017/S1352465809005128. [DOI] [PubMed] [Google Scholar]
- Miller WR, Yahne CE, Moyers TB, Martinez J, Pirritano M. A randomized trial of methods to help clinicians learn motivational interviewing. Journal of Consulting and Clinical Psychology. 2004;72:1050–1062. doi: 10.1037/0022-006X.72.6.1050. [DOI] [PubMed] [Google Scholar]
- Moyers TB, Martin T, Manuel JK, Miller WR, Ernst D. Revised Global Scales: Motivational Interviewing Treatment Integrity Code 3.0 (MITI 3.1.1) Center on Alcoholism, Substance Abuse, and Addictions; The University of New Mexico: 2010. Unpublished manual. [Google Scholar]
- Rodolfa ER, Bent RJ, Eisman E, Nelson PD, Rehm L, Ritchie P. A Cube model for competency development: Implications for psychology educators and regulators. Professional Psychology: Research and Practice. 2005;36:347–354. doi: 10.1037/0735-7028.36.4.347. [DOI] [Google Scholar]
- Rosengren DB, Baer JS, Hartzler B, Dunn CW, Wells EA, Ogle R. Video Assessment of Simulated Encounters (VASE-R): Administration and scoring manual. Alcohol & Drug Abuse Institute; University of Washington: 2005. [Google Scholar]
- Rosengren DB, Hartzler B, Baer JS, Wells EA, Dunn CW. j Drug and Alcohol Dependence. 2008;97(1–2):130–138. doi: 10.1016/j.drugalcdep.2008.03.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schumacher JA, Coffey SF, Walitzer KS, Burke RS, Williams DC, Norquist G, Elkin TD. Guidance for new motivational interviewing trainers when training addiction professionals: Findings from a survey of experienced trainers. Motivational Interviewing: Training, Research, Implementation, and Practice. 2012;1:7–15. doi: 10.5195/mitrip.2012.7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schumacher JA, Madson MB, Nilsen P. Barriers to learning motivational interviewing (MI): A survey of MI Trainers. Journal of Addictions and Offender Counseling. 2014;35:81–96. doi: 10.1002/j.2161-1874.2014.00028.x. [DOI] [Google Scholar]
- Schwalbe CS, Oh HY, Zweben A. Sustaining motivational interviewing: A meta analysis of training studies. Addiction. 2014;109:1287–1294. doi: 10.1111/add.12558. [DOI] [PubMed] [Google Scholar]
- Smith JL, Carpenter KM, Amrhein PC, Brooks AC, Levin D, Schreiber EA, Travaglini LA, Hu MC, Nunes EV. Training substance abuse clinicians in motivational interviewing using live supervision via teleconferencing. Journal of Consulting and Clinical Psychology. 2012;80:450–464. doi: 10.1037/a0028176. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Westra HA, Aviram A, Doell FK. Extending motivational interviewing to the treatment of major mental health problems: Current directions and evidence. Canadian Journal of Psychiatry. 2011;56:643–650. doi: 10.1177/070674371105601102. [DOI] [PubMed] [Google Scholar]