Improving Medical Students’ Success in Promoting Health Behavior Change: A Curriculum Evaluation

Abstract

Introduction

Effective behavior change counseling is an important component of the optimal care of patients, yet only a minority of medical schools currently include such training in their curriculum.

Aim

To design and evaluate a formal curriculum to teach medical students the principles of motivational interviewing (MI) that will improve knowledge, skills, and confidence in the area of counseling patients for health behavior change.

Participants

Fifty-three 3rd year medical students at the University of California, San Diego.

Program Description

A 4-week curriculum consisting of four 2-h sessions, in a small group format (8–12 students). Educational strategies included a combination of short didactics, video demonstrations, small group role plays, and interactive exercises.

Program Evaluation

Students completed identical pre- and post-assessments, consisting of a questionnaire measuring confidence and knowledge, and a performance assessment using the Video Assessment of Simulated Encounters-Revised (VASE-R) tool. Knowledge improved significantly (pre-mean: 7.04, post-mean: 11.54; P < 0.001), as did skill development (pre-mean: 7.02, post-mean: 9.47; P < 0.001). Student satisfaction with behavior change counseling training improved from 3.6 to 8.1 (P < 0.001). Students were significantly more confident (P < 0.001) in their abilities to assess a patient’s readiness for change and counsel the patient on behavior change after the course.

Discussion

Participation in a focused curriculum on the use of motivational interviewing techniques significantly improved 3rd year medical students’ knowledge, confidence, and skills in the area of behavior change counseling. These gains may help students succeed in promoting good health habits in their future patients.

INTRODUCTION

Behavior change is an important outcome in disease prevention efforts, since many personal habits, such as smoking and excessive alcohol consumption, put patients at risk of illness.¹ When physicians communicate effectively, patients are more likely to adhere to treatment and follow advice.² In spite of this, only a minority of medical schools currently include behavior change counseling training in their curricula, and those that do often lack structure and consistency.³ Previous efforts to evaluate behavior change counseling curricula have been aimed at determining immediate gains in knowledge or confidence, but not necessarily in skill development.^4–6 The need exists to establish the effectiveness of a course that impacts knowledge, confidence, and skill development within an instructionally sound framework.

We describe the design and evaluation of a behavior change counseling curriculum for 3rd year medical students at the University of California, San Diego (UCSD). We aim to demonstrate that a course built upon sound instructional design principles can effectively improve the knowledge, confidence, and skill level of medical students in the area of behavior change counseling.

PROGRAM DESCRIPTION

Curriculum Background

We chose to build our new course around Motivational Interviewing (MI), which is an evidence-based method recommended by clinical guidelines to help people engage in behavior change.⁷ MI utilizes a patient-centered counseling style in which doctors reduce resistance, develop discrepancy, analyze and resolve ambivalence, and support patient self-efficacy.⁸ The goal of MI is to help patients increase their intrinsic motivation for change, thereby strengthening their commitment to a plan for altering unhealthy behavior. Originally developed to help people with alcohol problems, MI has been employed in assisting patients with a variety of health issues, including tobacco cessation, drug use, weight loss, and medication compliance.^9–11

We adapted the core MI principles developed by Miller and Rollnick⁸ by adding mutual agenda setting and individualized health-related feedback components. We believe this better suits general medical practice. The framework called “the seven steps of medical MI” is presented in Table 1.

Table 1.

The Seven Steps of Medical MI

1. Mutual agenda setting

2. Decision balance (The good vs. the not-so-good aspects of behavior)

3. Interest, readiness, and confidence rulers (On a 1–10 scale, how interested are you to change any aspect of your behavior? How ready are you to make that change right now, on a scale from 1–10? How confident are you that you could make that change, on a scale from 1–10?)

4. Individualized feedback (Sharing patients’ own health data)

5. Summary

6. Key question (Ask patient “Where do we go from here?”)

7. Negotiate change plan (In partnership with patient)

Learning Objectives

Learning objectives for the curriculum (Table 2) emphasized skill development and knowledge acquisition.

Table 2.

Learning Objectives for Behavior Change Counseling Curriculum

▪ Third year medical students will be able to:

▪ Differentiate between open-ended and closed-ended questions

▪ Correctly identify a patient’s readiness to change behavior

▪ Correctly distinguish the appropriate counseling strategy for a patient in each of Prochaska’s Six Stages of Change

▪ Define and apply the five key principles of MI and the OARS (open-ended questions, affirmations, reflections, and summaries) techniques

▪ Demonstrate reflective listening techniques, using both simple and strategic reflections, and elicit “change talk” in a patient needing to undergo a health behavior change

▪ Utilize the seven steps of Medical MI in a patient interview, including setting an agenda, measuring willingness and confidence to change behavior, and providing individualized feedback

▪ Rate behavior change counseling as an important and effective medical intervention by physicians

Learner Population

Third year medical students participated in the course as part of the ambulatory care block during their 12-week core medicine clerkship. All participants gave written informed consent. The UCSD Institutional Review Board approved the study.

Educational Content and Methods

The 4-week curriculum consisted of four 2-h sessions with 8 to 12 students in a group. To provide a multi-disciplinary perspective, a primary care physician, a social worker, and a graduate student educator faciliated the course. The educational strategies and active learning exercises employed during each session are described in Table 3.

Table 3.

Session Descriptions

Session 1	Pre-assessment:
	▪ Measured knowledge, skill performance, and confidence in counseling patients through behavior change
	Content:
	▪ Background and basic techniques of MI
	▪ Stages of behavior change
	Interactivity:
	▪ Students received various patient statements and were asked to determine the patient’s stage of change and respond with an open-ended question and a strategic reflection
	Homework assignment:
	▪ Students were to practice the newly learned MI techniques with patients each week and video record themselves during an interview with one of their clinic patients, to be shared with the class during the final session
Session 2	Content:
	▪ Advanced principles of MI
	▪ Step-by-step instructions for interviewing in a medical setting
	Interactivity:
	▪ Students were shown a video of a physician interviewing a patient using MI, followed by group discussion
	▪ Round-robin group role play to practice the seven steps of medical MI
Session 3	Content:
	▪ Describe strategies to help pre-contemplative patients begin the change process
	Interactivity:
	▪ Group brainstorming activity to explore common traits of pre-contemplative patients
	▪ Role play designed to demonstrate how physicians can “create” difficult patients through ineffective communication
	▪ Role play vignettes with “resistant” patients
	▪ Instructors observed the exercises and provided both individual and group feedback
Session 4	Interactivity:
	▪ Presentation of students’ video-recorded patient interviews
	▪ Peer review and group discussion
	▪ Post-assessment
	▪ Students were able to evaluate their progress by comparing their pre/post assessments
	Course evaluation

PROGRAM EVALUATION

Study Population

Fifty-three 3rd year medical students participated in the study between March and August 2006. Sixty percent of students (n = 21) took the course during the first half of their 3rd year of medical school, while the remainder (n = 32) took the course during the latter half.

Evaluation Methods

Pre- and post-assessments targeted students’ knowledge of MI techniques (14-item quiz) and confidence in using these skills (measured as pre, post and retrospective pre). A portion of the Video Assessment of Simulated Encounters-Revised (VASE-R) was used to assess skill development.¹² The VASE-R is a group-administered method for measuring MI skills, consisting of three videotaped vignettes of actors portraying substance abusers. Each vignette is followed by six questions asking students to generate written responses consistent with MI principles. The VASE-R assesses five MI “microskills”: reflective listening, responding to resistance, summarizing, eliciting change talk, and developing discrepancy. The first case from the VASE-R was administered following the instruction protocol, using the video material provided by Rosengren and colleagues. Results from each student were analyzed individually by both authors, following the VASE-R scoring manual instructions. Each of the six responses received a score of 0, 1, or 2, based on specific scoring rules. In the event that a score differed between raters, an average score was calculated.

At the end of the course, students wrote Commitment to Change (CTC) statements that identified specific, measurable changes they would employ in their interactions with patients as a result of the course.¹³ Three months later, students were e-mailed an online survey to determine how fully those commitments were implemented and what, if any, barriers existed. Follow-up e-mails were sent to increase response rate.

Quantitative data derived from the online survey, pre/post assessments, and course evaluations were analyzed both descriptively (measures of central tendency, frequency distributions) and inferentially (paired t-tests, Cohen’s d effect size). In addition, students who took the course during their first half of their 3rd year were compared to those taking the course during the latter half of the year to determine if any differences existed (Mann Whitney).

Evaluation Results

Fifty of 53 students (93%) rated the instruction of the course as strongly positive (a 5 or 6 on a 6-point scale, 6 being the highest quality), and 49 of 53 (91%) thought the course was a valuable experience (rated a 5 or 6 on a 6-point scale, 6 being the most valuable). On the pre/post assessment, students rated their level of satisfaction with their behavior change counseling training to date on a 1 to 10 scale, with 10 being the highest. Training was rated as 3.6 before the course and 8.1 after its completion (P < 0.001).

There was an improvement in MI-related knowledge (pre-mean: 7.04, post-mean: 11.54; P < 0.001). Students were significantly more confident (P < 0.001) in their abilities to assess a patient’s readiness for change and counsel the patient through a behavior change after the course (Table 4). Students consistently rated their confidence prior to the course higher in the pre-assessment compared with the retrospective pre-course evaluation.

Table 4.

Level of Student Confidence Before and After Behavior Change Counseling Course

Question	Pre-course	Retrospective	Post-course	P-value
How confident are you in your skills to assess a patient’s readiness for change?	5.70	4.48	8.06	<0.001
How confident are you in your skills to counsel a patient to change a health-related behavior?	4.98	4.04	7.92	<0.001

*Only those who completed both the pre/post assessment (50 of 53, 94%) were analyzed.

†Students rated confidence on a scale from 1 to 10, 10 being the highest.

‡P-value for difference between pre-course and retrospective, and pre- and post-course

§”Retrospective” refers to assessment of confidence BEFORE the course, but measured AFTER the course.

On the video portion of the pre/post assessments (VASE-R), students’ application of MI skills improved significantly (pre-mean: 7.02, post-mean: 9.47; P < 0.001). Effect size, the measure of the strength of the relationship between the scores of the pre- and post-assessments, is shown in Table 5. The results can be considered moderate (0.5 – 0.7) to strong (greater than 0.7) on most skills.

Table 5.

Application of MI Concepts Before and After Behavior Change Counseling Course as Measured by the VASE-R

Assessed concept	Pre-assessment mean	Post-assessment mean	Effect size (Cohen’s d) using σpooled	P-value
Reflective listening I	1.22	1.69	1.01	<0.001
Reflective listening II	1.27	1.51	0.53	0.006
Responding to resistance	0.80	1.22	0.59	0.004
Summarizing	0.98	1.55	0.89	<0.001
Eliciting change talk	1.02	1.69	1.25	<0.001
Developing discrepancy	1.73	1.80	0.14	0.322
Total exam	7.02	9.47	1.54	<0.001

†Responses were scored from 0 to 2, with 2 being the highest, for how well the concept was applied

‡Only those who completed both the pre/post assessment (50 of 53, 94%) were analyzed

No significant differences existed between the students who took the course during their first half of their 3rd year as compared to those who took the course during the latter half of their 3rd year, as noted by comparing the total pre- and post-assessment means of each group (P > 0.05).

Of the students who responded to the online survey (24 of 53, 45%), most of the CTC statements (58 of 72, 81%) were partially or fully implemented. Of those students who wrote CTC statements that were not implemented at all (10 of 24, 42%), six were unable to find a suitable patient to employ their skills. Five indicated they did not have enough time to implement that change.

DISCUSSION

We developed a curriculum that has been successfully incorporated into the 3rd year core medicine clerkship and has resulted in participant satisfaction and improved knowledge, confidence, and skill development in the area of health behavior change counseling.

After completing the course, students declared they considered MI to be an effective way to change patient behavior, and they reported they now used MI strategies when interviewing patients. The video-recorded student-patient interactions presented on the final day of the course consistently showed students successfully implementing MI skills with real patients in a wide variety of contexts, from traditional health behavior topics, such as tobacco and alcohol cessation, to more evolved uses, such as medication adherence and weight loss interventions.

The curriculum significantly improved students’ interviewing skills as measured by the VASE-R, a novel and previously validated measure. Others have found that the VASE-R was an effective measure for evaluating MI skills in a variety of contexts and provided an enriched stimuli and degree of interpersonal engagement over some existing methods (e.g., questionnaires).¹² Similar moderate to large effect sizes have been reported in studies of communication skills teaching for medical students using performance-based assessments.¹⁴ While we did not measure actual practice performance, we are hopeful that students’ enhanced confidence and skill mastery in this area will translate into more effective relationships with their future patients resulting in better quality of care and ultimately improved health outcomes.

Medical schools searching for effective ways to enhance doctor-patient communication training may benefit from adopting a similar program. Costs are minimal. The main limiting factor is likely to be the identification of faculty with both the time and adequate training in MI techniques.

Our study has limitations. Due to class time constraints, only a portion of the VASE-R assessment tool was employed, potentially weakening its prior validation. If the budget had allowed, the use of standardized patient encounters to provide practice opportunities and formative assessment of the students’ skills would likely have enriched the learning from video and role-play exercises. Actual patients could also be a valuable source of feedback. Only 45% of students responded to the online follow-up survey, which could have introduced selection bias as students who implemented their MI skills may have been more likely to respond. In future work we hope to evaluate long-term retention of MI knowledge and skills by incorporating a behavior change standardized patient case in the required Clinical Performance Exam, a comprehensive Objective Structured Clinical Examination, assessment of clinical competence administered at the end of the students’ 3rd year of medical school. Future studies should ultimately focus on patient outcomes as a measure of true effectiveness of such a program.