Assessment of a Motivational Interviewing Curriculum for Year 3 Medical Students Using a Standardized Patient Case

Abstract

Objective

We sought to evaluate a year 3 motivational interviewing (MI) curriculum using a standardized patient case.

Methods

The 2-hour small group MI curriculum included a didactic presentation followed by interactive role plays. During a clinical skills assessment at the end of year 3 the MI skills of 80 students who had participated in the curriculum were compared with those of 19 students who had not participated.

Results

The standardized patient reliably rated the students on their performance of 8 items. Students who had participated in the MI curriculum were significantly more proficient than nonparticipating students in the performance of 2 strategic MI skills, importance and confidence rulers (ps < .006). The groups did not differ in their use of patient-centered counseling skills or collaborative change planning commonly used in MI.

Conclusions

Third year medical students can learn to use MI skills that specifically aim to enhance patients’ motivations for change.

Practice Implications

Medical schools should consider providing students with MI training and MI skill assessments using standardized patient cases to help students prepare to counsel patients for behavior change.

1. Introduction

Motivational Interviewing [1] is an evidence-based treatment for addictive and other problematic behaviors [2]. The treatment blends patient-centered counseling skills (open questions, reflections) with strategies or techniques that directly elicit a patient’s motivations for change and commitment to a behavior change plan [3]. Given the importance of human behavior (e.g., chemical dependency, diet, exercise, risk taking, non-adherence) in the pathogenesis and treatment of numerous chronic medical disorders such as atherosclerosis, cancer, diabetes, hypertension, HIV-AIDS, and renal and hepatic insufficiency, some medical schools have begun teaching MI to students [4–6].

We previously described a brief curriculum for teaching third year medical students how to use MI to communicate with patients about unhealthy behaviors in the context of the medical interview [5]. The curriculum is based on descriptions about how to use MI in medical settings [7–9]. Evaluation of this curriculum showed that students significantly increased their frequency and depth of reflections and reduced behaviors inconsistent with MI in their responses to textural vignettes. They also gained knowledge about MI, interest in the approach, confidence in their MI abilities, and commitment to use MI in medical practice [5]. Other medical student MI curriculum evaluations have demonstrated similar improvements in knowledge [4,6] and confidence as well as increased use of MI skills in response to videotaped clinical vignettes [4].

This study extends our previous evaluation by examining the impact of the curriculum upon students’ ability to use MI in a standardized patient medical interview. We chose to perform the assessment using a standardized patient because 1) this method has the advantage of providing a consistent clinical encounter across students, and 2) the use of standardized patients is a widely used and reliable clinical skills assessment approach in medical education [10–12] and for evaluating MI skills [13]. Moreover, if trained to reliably evaluate the students’ use of specific MI skills using a performance rating scale, standardized patients could provide an efficient method for assessing students’ MI proficiency rather than incorporating more labor intensive, complex, and costly independent rater systems [14–16]. We hypothesized those students who received the curriculum, in comparison to those who did not participate in it, would demonstrate better use of MI skills as determined by the standardized patient’s ratings.

2. Methods

2.1 Participants

The subjects were 99 Yale University medical students who participated in a clinical skills assessment in May through June 2009. No students were excluded from participation in the curriculum or the skills assessment.

2.2 Procedures

Yale medical students attended a 2-hour MI curriculum as part of their psychiatry clerkship. Clinical faculty (psychologists, psychiatrists, mental health counselors) who had expertise in MI taught the curriculum to small groups of 3–6 students. They first introduced students to a 3-step MI performance model (Table 1). Following this didactic component, the faculty role played patients with addictive and other behavioral problems for students to interview. As students conducted the interview, the faculty gave the students performance feedback and coached them how to use MI more proficiently. Emphasis was placed on improving their use of patient-centered counseling skills (e.g., open questions, reflective listening) and strategies to elicit the patients’ statements that favor change [5]. In particular, students were taught how to use a motivational ruler technique to increase the importance and confidence the patients attached to changing their behavior [7].

Table 1.

Three steps in using motivational interviewing in a medical interview

Facilitate the patient’s story while avoiding MI inconsistent actions Open ended questions Reflective listening Affirmations Empathy Facilitate change talk using importance and confidence rulers Make a change plan with the patient, giving advice only when solicited or with permission

All students were evaluated in the clinical skills assessment at the end of their third year. The clinical skills assessment consisted of 8 problem-focused standardized patient cases designed to test a variety of clinical skills. For one of these cases we trained a standardized patient to portray a person remanded by the court to have a medical evaluation following an arrest for driving while intoxicated. Students were provided with a brief description of the case and then conducted a 10-minute interview in which they were instructed to motivate the patient to address her alcohol use problem, including negotiating a plan for change if the patient agreed to reduce or stop drinking. At the conclusion of the interview, the standardized patient assessed the students’ MI skills. All interviews were videotaped. Because some students had not yet taken the psychiatry clerkship prior to the clinical skills assessment and others had taken psychiatry before the workshop existed, we were able to compare the MI skills of students who had and had not participated in the curriculum.

2.3 MI skill assessement

We developed a rating scale with 8 items that corresponded to the skills taught in the MI curriculum. The items are listed in Table 2. Unlike existing MI performance rating scales in which independent raters use exact frequency counts to determine the practitioners’ adherence, the standardized patient assigned a rating on a scale of 1–5 guided by anchors for each item. The anchors described different levels of proficiency in using each strategy, with lower scores indicative of the use of a strategy without specific attempts to enhance motivation (e.g., asked a neutral open question as in, “What do you think about that?”), and higher scores reflecting the strategic elicitation of motivation for change when using the strategy (e.g., asking an open questions that draws out patient motivation as in, “How would improving your diet benefit your health? The standardized patient was trained in the use of the scale by first reviewing the items and then practicing rating them at the end of role plays with FH; these role plays targeted a range of proficiency across the items. Rating inaccuracies were discussed and resolved until the standardized patient was able to reach agreement with FH on all items in one entire interview. The standardized patient completed the rating scale for each student immediately following the interview. It took 5 minutes to complete.

Table 2.

Standardized patient motivational interviewing performance rating scale items

Did the student use open-ended questions? Did the student use reflections? Did the student affirm your efforts to change? Did the student respond empathically to your emotions? Did the student avoid using unsolicited advice, direct confrontation, or authority? Did the student use the ruler technique to assess how important changing is for you? Did the student use the ruler technique to assess your confidence in being able to change? Did the student collaboratively discuss with you a specific plan for making a change?

Note: Each item is rated on a 5-point scale with 1 = did not demonstrate the skill, 2 = made a poor attempt at the skill, 3 = perform the skill without any strategy for eliciting motivation for change, 4 = performed the skill at least once involving strategy for eliciting motivation for change, 5 = performed the item 2 or more times involving strategy for eliciting motivation for change. Ratings for item 5 are reverse scored and only based on frequency (e.g., 1 = occurred > 3 times, to 5 = never occurred).

2.4 Statistical procedures and analysis

Videotapes of 6 randomly selected student interviews of the standardized patient (balanced for those who did and did not received the curriculum) were independently reviewed by three expert raters (FH, AF, SM) after the standardized patient had already rated the interviews using the scale. Intraclass correlation coefficients (ICC) [17] using a two-way random effects model (2.1) were used to establish the reliability of the standardized patient with the experts. Independent t-tests using a Bonferroni-corrected alpha of .006 (.05/8) were used to analyze differences in the eight item ratings between students who had and had not participated in the MI curriculum. Pearson correlations examined the relationship of the time since students received MI training (in months) and the MI scale item ratings.

3. Results

3.1 Description of students

Fifty-two percent of students were men and 48% were women. The largest ethnic group was Caucasian (54%) followed by Asian (18%), African American (10%), Indian/ Pakistani (10%), Latino (5%), Native American (2%) and other (1%). About half the students were expected to graduate in May 2010 (54%) with the remainder graduating in 2011 (43%) or later (3%).

3.2 MI curriculum participation

Eighty students (81%) had participated in the MI curriculum, an average of 7 months before taking the clinical skills assessment (SD = 3.7; range = 3–18). Nineteen students did not receive the MI curriculum. There were no significant gender, ethnic or year of graduation differences between students who attended the MI curriculum and those who did not.

3.3 Scale Reliability

ICCs (see Table 3) indicated good to excellent reliability for 7 of 8 scale items [18], with the exception of open questions. Closer examination of the ratings for open questions revealed that the poor reliability (ICC = .22) was a matter of restricted range in the values (i.e., all students received ratings of 4 or 5 from all raters) rather than due to a lack of rater agreement.

Table 3.

Ratings by the standardized patient of students’ MI skills

		Received MI Curriculum		Did Not Receive MI Curriculum
Scale Items	ICC	M	SD	M	SD	t-statistic	p-value
Open-ended questions	.22	3.95	.90	4.47	.772	−2.34	.021
Reflections	.75	3.34	1.25	4.47	.772	−.94	.347
Affirmations	.60	3.00	1.72	3.05	1.78	−.12	.905
Empathy	.79	2.94	1.57	3.26	1.45	−.82	.416
Avoid MI inconsistent actions	.65	3.33	1.67	3.32	.82	.03
Importance ruler	.77	3.61	1.70	2.37	1.52	2.91	.004*
Confidence ruler	.79	2.91	1.74	1.53	1.17	3.28	.001*
Change Plan		3.54	1.12	3.89	1.05	−1.26	.211

Note: Eighty medical students received the MI curriculum and 19 did not. Independent t-tests using a Bonferroni-corrected alpha of .006 (.05/8) were used to analyze rating differences between students who had and had not received the MI curriculum.

3.4 MI proficiency

Across groups, students similarly made use of all the patient-centered counseling skills (open questions, reflections, affirmations, empathy), demonstrating these skills at least once, though not with specific strategy for supporting behavior change. They used MI inconsistent actions once or twice in the transaction, typically instances of unsolicited advice (see Table 3). Students who attended the curriculum were significantly more proficient in the use of both the importance and confidence rulers when compared with students who had not attended it (ps < .006). MI-trained students were more likely to use these ruler techniques and to do so in a manner that attempted to elicit the patient’s motivations for change. Analyses of gender, ethnicity, and year of graduation with group yielded no significant interactions, nor did any of these factors moderate performance. More months since receiving the MI curriculum was related to less proficient use of the confidence ruler strategy (r = −.26, p = .02).

4. Discussion and conclusion

4.1 Discussion

This study demonstrated that medical students who participated in a 2-hour curriculum in MI, compared to similar students who had not had this training, were significantly more proficient in their use of the importance and confidence ruler technique, though no different in their use of patient-centered counseling skills, avoidance of MI inconsistent actions, or collaboratively discussing plans for changing behavior. The study also showed that a standardized patient can reliably rate the medical students’ use of specific MI skills when trained how to use a simple rating scale.

Because of time constraints in the 3^rd year schedule our curriculum needed to be delivered in a single 2-hour session during the psychiatry clerkship. We therefore designed an experiential component in which students could be observed and receive immediate feedback to maximize skill development. It is notable that a training effect was achieved and sustained for an average of 7 months. Students who did and did not attend the MI curriculum were equally proficient in the use of 6 techniques assessed by the standardized patient, including open ended questions, reflective listening, affirmations, empathy, avoiding giving unsolicited advice, and planning for change. This finding might be explained by the required year 1 and 2 patient-centered interview skills curriculum in which students have multiple opportunities to practice facilitative interview techniques with standardized and real patients [19]. The results suggest that differences in proficiency in the use of rulers are unlikely to be due to differences in students’ intrinsic or acquired ability to perform a medical interview. Future efforts to teach students MI might involve training medical school faculty in MI and in how to integrate its use into their curricula to reinforce and extend the training effects of the brief MI curriculum.

This study’s results add to the literature about the effectiveness of curricula to train medical students in MI. Prior studies either did not assess demonstrated use of MI skills in a standardized patient encounter [4–5] or only did so at the end of the course rather than at a follow-up point several months later [6]. The standardized patient’s ability to discriminate between the student groups in their proficiency in using the ruler techniques on average 7 months after receiving the MI curriculum suggests that brief, targeted, and highly experiential courses with small student to faculty ratios may have enduring training effects.

This study also suggests that standardized patients may be able to reliably evaluate medical students’ MI performance when carefully trained in the use of a post-encounter rating scale such as the one we developed for this study. Our decision to approach student evaluation in this manner was driven by feasibility. The gold standard for evaluating MI performance is the use of adherence and competence rating scales used by independent raters [14–16]. These scales require several days of training for them to be used reliably and, thus, are expensive to employ. Moreover, unless raters are immediately available to score the encounter, feedback derived from them is delayed. While our scale requires further psychometric development to replicate its reliability and establish its validity (e.g., compare standardized patient-rated encounters to independently rated ones), our study indicates standardized patient evaluation of MI practice is a promising approach.

Our study has several limitations: 1) students were not randomized to training conditions; 2) baseline levels of MI skills were not assessed; 3) the follow-up time period was not standardized; 4) only one standardized patient encounter was used to evaluate the students’ MI skills; and 5) the students’ use of MI skills in actual patient encounters was not determined. Future randomized controlled training trials are needed to more robustly test the effectiveness of MI training curriculums in medical schools as well as to determine the cost-effectiveness of such efforts. Nonetheless, at a time when budgetary constraints and increasing clinical workloads have led medical school faculty to seek efficient teaching and assessment methods, this study suggests that a brief 2-hour curriculum with standardized patient assessment may provide medical schools with a feasible approach for teaching students MI.

4.2 Conclusion

Participation in a brief MI curriculum resulted in a sustained improvement in students’ ability to counsel patients for behavior change as assessed in a standardized patient case.

4.3 Practice implication

Medical schools should consider providing MI curricula and standardized patient assessments of MI performance to help future physicians develop skills for counseling patients for behavior change. Our study suggests that medical school may be an opportune time to begin training in MI.