Motivational Interviewing Workshop in a Virtual World: Learning as Avatars

Abstract

Background

Limited research has been done to understand outcomes of continuing medical education offered in three-dimensional, immersive virtual worlds.

Objectives

We studied a case of a virtual world workshop on motivational interviewing (MI) applied to smoking cessation counseling and its educational impact.

Methods

To facilitate content development and evaluation, we specified desired MI competencies. The workshop consisted of three sessions, which included lectures, practice with standardized patients, and chat interactions. Data were collected from 13 primary care physicians and residents through workshop observation, and pre- and three-month post-workshop telephone/Skype interviews and interactions with standardized patients. Interactions with standardized patients were assessed by an expert using a validated MI tool, and by standardized patients using a tool developed for this study. For 11 participants who attended two or three sessions, we conducted paired-samples t-tests comparing mean differences between the competency scores pre- and post-event.

Results

Expert assessment showed significant improvement on six of seven MI competencies (p< .05). All participants reported learning new knowledge and skills, and nine described incorporating new learning into their clinical practice. Practicing MI with standardized patients and/or observing others' practice appeared to be the most helpful workshop component.

Conclusions

The evaluated workshop had positive impact on participants' competencies and practice as related to MI applied to smoking cessation counseling. Our findings support further exploration of three-dimensional virtual worlds as learning environments for continuing medical education.

Introduction

Three-dimensional (3-D) virtual worlds are increasingly viewed by educators and researchers as learning environments with tremendous potential for the education of health professionals.^1-4 A virtual world, also known as a 3-D multiuser virtual environment, is a computer-based, simulated multimedia environment, which offers graphical representation of a physical space where real people use avatars, their digital self-representations, to interact with each other and objects.^1,5 An avatar may resemble the person's appearance or look different. Navigated by the person, it makes gestures, walks, runs, sits, jumps, and flies. When the person types, words appear in a chat box on participants' computer screens. When the person talks the avatar talks, too, and can be heard by other avatars positioned nearby. A virtual environment, which can be anything from a beach to a library, is usually expressed through color- and detail-rich graphics. Emphasizing its realistic nature, the anthropologist Boellstorff⁶ called it a place of human culture realized by a computer program through the Internet. More than 250 virtual worlds have emerged in the past decade.⁷

Sharing characteristics with other forms of distance education, virtual worlds allow learners to study at convenient locations, overcome geographic boundaries, and save on travel expenses. Moreover, immersion and social presence associated with virtual worlds make them particularly attractive to educators and learners. Immersion refers to a sensation of being enveloped by and interacting with the virtual surroundings,^8,9 while social presence is-a sense of being present with other avatars at the same time and at the same place despite geographic distance.⁹ Immersion and social presence make virtual worlds conducive to experimentation,¹⁰ simulation,¹¹ collaboration,¹² community-building,^13,14 and constructivist learning.^12,15

The well-documented challenges for teaching in a virtual environment are technical problems associated with hardware and software requirements, bandwidth, and firewalls,^3,15 a steep learning curve to acquire technical skills,¹⁵ and the potential for distraction and exposure to inappropriate content.¹⁵ Nevertheless, the majority of virtual world educational activity reports^3,16-20 show evidence of learner satisfaction with their in-world experience. Then again, limited research has been done to measure outcomes of virtual world educational programs beyond learner satisfaction.^{2, 20}

We conducted a pilot study to advance understanding of virtual world continuing medical education (CME) and its educational impact. The focus of this CME activity was to teach a set of motivational interviewing (MI) skills applied to smoking cessation counseling. MI is a collaborative, patient-centered counseling approach aimed at eliciting and strengthening the patient's intrinsic motivation to change his or her behavior.²¹ It is an effective approach to help tobacco users make a successful attempt to quit. A recent systematic review²² showed that MI versus brief advice or usual care resulted in a modest but significant increase in quitting. However, primary care clinicians lack MI skills and underutilize this approach.²³

Methods

We designed a pilot, mixed methods research study²⁴ to examine a case of a virtual world CME workshop on MI applied to smoking cessation Our goal was to explore the potential to teach communication skills to physician learners using a virtual world classroom and assess the educational outcomes, including changes in MI skill proficiency and use of MI in clinical practice. The workshop was offered to primary care clinicians and conducted in Second Life (SL). SL is a 3-D virtual world that was created by Linden Lab in 2003.²⁵ With approximately 27 million registered users,²⁶ it is among the most popular virtual worlds.

Multiple triangulation (i.e., data and methodological) was used to enhance the completeness and confirmation of study results.²⁷ It was accomplished by collecting quantitative and qualitative data through objective assessment of learner interactions with standardized patients,^28,29 workshop observation, and interviews with learners. This study was approved by the University of Wisconsin Health Sciences Minimal Risk Institutional Review Board and conducted in 2011.

Educational Intervention

The workshop addressed the 5 A's best practices for smoking cessation, Ask, Advise, Assess, Assist, and Arrange for Follow-up,³⁰ and MI techniques focusing on developing empathic partnership and eliciting change talk. As a result of this training, clinicians were expected to enhance their knowledge and skills as related to the 5 A's and develop their MI competencies as follows: (1) collaboratively discuss lifestyle choices with patients; (2) evoke patient thoughts and opinions on the advantages and disadvantages of health behavior; (3) respond with, and demonstrate, complete empathy and avoid correcting patient's opinions or “preaching”; (4) elicit the patient's motivation, concerns and values concerning the lifestyle choice; (5) respond to the patient's core concerns about lifestyle modification; (6) restate the patient's reasoning and discrepancies between behaviors and goals or values; and (7) roll with resistance. The desired impact on practice included application of MI competencies and integration of practice changes into routines.

Prior to the workshop, the enrolled participants were offered a one-hour webinar and technical support to help them set up an avatar and acquire basic SL skills. The workshop consisted of three two-hour weekly sessions in SL. Each session included a presentation, discussions, and practice exercises. Presentation slides were adapted from those used for real-world, face-to-face events.

Two presenters alternated talking in front of the audience. They used slides, MI role-modeling with two standardized patients, and visual learning special effects, such as cigarette butts raining from the sky to demonstrate the quantity smoked in one year by a pack-a-day smoker (Figure 1). Participants were engaged in discussion through the SL chat where they typed their questions, answered the presenter's questions, and shared comments.

Figure 1. Presentation in the Big Auditorium: Cigarette Butts Covering the Presenter to Demonstrate the Quantity Smoked in One Year by a Pack-a-day Smoker.

A 30-minute exercise during the first session involved participants writing what to say to a patient and presenters providing verbal feedback. For the 75-minute role-play during the second and third sessions, participants were divided into groups of 6-9 and transported to separate rooms (Figure 2). Each participant practiced MI skills by talking with the standardized patient for 10 minutes, receiving feedback from the presenter (verbal and through the chat) and other participants (through the chat), and observing how others interacted with the patient (Figure 3). The participant-patient interactions were video-recorded and posted to a blog site. For homework, participants were asked to review these video-recordings and comment on their own video and those of two other participants.

Figure 2.

Small Group Practice Exercise: the Participant Interviewing the Standardized Patient (in the Front of the Room), the Presenter (Closer to the Board) and Other Participants Sitting in Chairs, and Project Staff Members Timing the Exercise and Providing Technical Support (in the Back of the Room).

Figure 3. Small Group Exercise: Participant Practicing Motivational Interviewing with the Standardized Patient; Chat Box Showing Participants' Feedback (Avatars' Names Erased).

Two program staff members provided technical support during the sessions. Participants could earn AMA PRA Category 1 ™ or AAFP Prescribed credits. There was no charge for participation.

Study Participant Recruitment

It was expected that up to 50 primary care clinicians would be recruited to participate in the workshop through websites of the partner organizations, announcements to health-related groups in SL, flyers at the live educational meetings, and emails to former CME participants. We emailed an invitation to participate in the study to all enrolled workshop participants to recruit 12 individuals who were first to agree.

Data Collection and Analysis

Pre-event data collection occurred 1-14 days before the first session via Skype, telephone, or both, with a study participant, a researcher, and a standardized patient. First, the researcher asked the participant to state her or his learning goals, describe practice relative to smoking cessation, and self-assess, using a scale from 1=the lowest to 4=the highest, knowledge of the 5 A's best practices, MI knowledge, and MI skills. The interview lasted approximately 15 minutes. Then, the participant was given a clinical scenario and asked to conduct a 10-minute smoking cessation counseling session with the standardized patient.

Post-event interviews took place three months after the workshop. Participants were given 10 minutes to interact with the same standardized patient and scenario as during the pre-event. Participants then underwent a 20-minute interview to respond to the same self-assessment questions as at the pre-event, and describe any changes in their knowledge/skills/practice.

All interview/standardized patient sessions were audio recorded and transcribed. We used a validated tool, Motivational Interviewing Skills for Health Care Encounters (MISHCE),³¹ to evaluate 15 specific MI behaviors, such as “Elicits/addresses patient's desired health outcomes/goals” and “Develops discrepancy”. A trained member of the research team scored the audio recordings using the MISHCE tool. Each behavior was rated as 0=Deficient, 1=Developing, or 2=Accomplished. Also, immediately after each session, the standardized patients completed an assessment form consisting of nine statements developed by the project team (e.g., “The physician asked me to talk about why I make the choices I make about my health”). Each statement was evaluated as 0=Disagree, 1=Agree Somewhat, or 2=Agree. The research team, with an input from the researcher who validated MISHCE,³¹ mapped MISHCE behaviors and statements in the patient assessment form to seven MI competencies. Then, MISHCE scores and patient scores were converted to the scores for the MI competencies. Paired-samples t-tests comparing mean differences between the competency scores pre- and post-event were conducted for both the patient assessment and the expert assessment.

Qualitative analysis involved review of interview transcripts, observation notes and recorded chat to reveal themes relevant to perceived outcomes, educational components that contributed to these outcomes, and factors influencing practice change. This analysis resulted in a case summary for each study participant, followed by cross-case analysis to identify commonalities and differences among the cases.

Results

Participants

It took three months to recruit 22 physicians, residents, and nurse practitioners to participate in the workshop. Thirteen of them agreed to participate in the study, including two family physicians, two general internists, and nine Family Medicine residents (from eight states). The mean number of years in practice for the four practicing physicians was 19.The majority of residents were finishing the second or third year of their residency. Eight were males; four had previous experience using SL; and six attended the pre-event webinar. Nine participated in all sessions, two participated in two sessions, one participated in one session, and one did not participate. Data from 11 participants who attended two or three sessions were included in the analyses.

Satisfaction With the Workshop

Based on interviews, participants were satisfied with the workshop. Four participants reflected on SL in particular. Positive comments described SL as “interesting”, “fun”, “exciting”, and “different”. One participant said: “It is kind of amazing that we had people from all over the country and from many places doing the classes [together].” Critiques of SL included the lack of facial expressions/difficulty to read body language, the technical hurdles, and adjusting to use of an avatar.

Based on observations, technical challenges were minimal. They included difficulty finding the auditorium, lost sound, poor internet connection, lack of avatar navigation skills, and difficulty accessing video-recorded role-plays. The number of technical issues was greatest in the first session and decreased in subsequent sessions.

Self-Assessment of Knowledge and Skills

During interviews, participants' self-assessments of 5 A's knowledge, MI knowledge, and ability to apply MI to smoking cessation identified perceived improvements (Figure 4). Two participants assessed their MI knowledge lower post-event than pre-event, because they realized there were MI-related facts and techniques of which they were not previously aware. All participants, including the above two, indicated learning something new, such as “how to be less direct and more engaging in the interview technique”, “not being judgmental”, “amplifying ambivalence”, and “rolling with resistance” (quotations from interviews).Several residents felt that they needed additional MI experience with their patients to further develop their skills.

Figure 4. Participants' Self-Assessment of Five A's Knowledge, MI Knowledge, and Ability to Use MI With Patients*.

MI-Motivational interviewing

* On a scale of -4, with 1=the lowest and 4=the highest

Objectively Measured Changes in Competencies

According to expert assessment of pre/post-event standardized patient encounters, nine participants demonstrated three or more improved competencies. Mean post-event scores for competencies did not exceed 0.833, on a scale with the minimum value 0 and the maximum value 2.000. However, analysis revealed significant improvement (p<0.05) on six of seven competencies. When adjusted for multiple comparisons (p<0.007, Bonferroni correction), two competencies remained significantly improved: demonstrate empathy/avoid “preaching”, and respond to the patient's core concerns about lifestyle modification (Table 1).

Table 1. Assessment of Standardized Patient Encounters by Expert: Post-Event Scores Compared with Pre-Event Scores.

Motivational Interviewing Competency	Pre-Event		Three Months Post-Event		Mean Difference
	Mean Score	Standard Deviation	Mean Score	Standard Deviation	Post-Score Minus Pre-Score	Standard Deviation	Significance, 2-tailed (p value)
1. Collaboratively discuss lifestyle choices with patients.	.275	.221	.686	.376	.410	.450	.013*
2. Evoke patient thoughts and opinions on the advantages and disadvantages of health behavior.	.369	.280	.783	.299	.414	.475	.022*
3. Respond with, and demonstrate, complete empathy and avoid correcting patient's opinions or “preaching”.	.213	.194	.668	.327	.455	.404	.004**
4. Elicit the patient's motivation, concerns and values concerning the lifestyle choice.	.313	.247	.758	.425	.444	.567	.027*
5. Respond to the patient's core concerns about lifestyle modification.	.286	.200	.751	.310	.465	.396	.003**
6. Restate the patient's reasoning and discrepancies between behaviors and goals or values.	.244	.221	.833	.418	.589	.558	.009*
7. Roll with resistance.	.604	.305	.808	.429	.203	.514	.300

^*Significant improvement (p< 0.05).

^**Significant improvement using Bonferroni correction (p< 0.007).

Assessment by standardized patient documented that all but one participant demonstrated two or more improved MI competencies at the post-event. On a scale with the minimum value 0 and the maximum value 2.000, mean scores for competencies varied from 0.636 to 1.001 pre-event, and from 1.091 to 1.304 post-event. Pre/post mean score differences showed improvement on all seven competencies, but none was statistically significant.

Changes in Clinical Behavior

Based on interviews, nine of eleven participants reported applying newly learned MI strategies to help their patients quit smoking and make other desired health-related behavior changes (e.g., in respect to weight loss). Physicians noted becoming more efficient in motivating smokers to quit. Residents reported using MI with some of their patients, but they acknowledged the need to gain more experience with MI.

For example, a general internist noted that, before the workshop, he relied on “hammering on the health benefits”, when addressing smoking with his patients. After the workshop, he noticed himself “reasoning with the patient as to why they should stop.” Another example was a third-year resident who began to use what she had learned with her patients:

Third-Year Family Medicine Resident: “You're kind of thinking like here you are, a standardized patient, you know, that sounds interesting, but does that really, and we all said that to each other, like that doesn't really work in a real patient. But when I actually had a patient, and I started talking to them about it, and kind of doing the same sort of questioning and, you know, the motivational stuff, it was like, oh, wow, this actually works on our patients!”

Other participants also discussed palpable practice changes, such as:

Family Physician: “I'm much more open in interviewing people now. I know exactly what to ask and when, you know, and I think I'm much more confident in that aspect.”

First-Year Family Medicine Resident: “[MI] has definitely helped me kind of look in the mirror at how I'm kind of encouraging people, and I think it's changed the way that I've done things, not at every visit, but it's definitely probably once a day changed the way I kind of go about talking to people.”

What Contributed to the Achieved Outcomes?

All but one participant attributed their changes in knowledge, skills and practice to their participation in the workshop. This one participant could not separate the educational impact of the MI workshop from the impact of a previously attended training.

The following educational components appeared to have the greatest influence on the outcomes: practicing MI with the standardized patients, receiving feedback from the instructor and peers, and observing how others interacted with the standardized patients. One participant highlighted the importance of interaction with instructors and peers throughout the sessions, including chat. A few participants reflected on the effective structure of the workshop.

Perceived facilitators of practice changes were seeing patients quit smoking, supportive colleagues and professional community, and availability of resources in the participant's hospital or clinic. Time constraints were reported as the major barrier for incorporating MI into clinical practice.

Discussion

This study examined a case of virtual world CME on MI for primary care clinicians and physicians-in-training. Although objective pre and three-month post assessment indicated that participants did not fully develop the MI competencies, it showed a promising progress in the desired direction. Furthermore, participants reported improvements in knowledge, skills, and some aspects of their clinical practice relative to smoking cessation and MI.

These findings were reasonable to expect, given that standardized patient simulations are known to facilitate changes in clinical practice and patient outcomes, in particular, when used to teach communication skills.³² Our findings are consistent with and help to advance prior research. Two virtual world CME studies that used role-play exercises documented positive outcomes, although one study³ did not assess behavior change, and another one²⁰ objectively measured competencies pre- and immediately post-event, but did not examine the sustainability of changes. Studies of face-to-face MI workshops showed various degrees of success in terms of acquisition and application of skills. However, the researchers^33,34,35 seem to be in agreement in that a workshop alone can help clinicians acquire some MI skills but is unlikely to result in enduring competence, and clinicians need ongoing support over a long period of time, such as feedback, coaching, and follow-up training, to continue skill building, retain competence, and incorporate MI in the routine practice.

Technology challenges and limitations, such as lack of facial expression, were notable but not dominant in our study. We found it interesting that participants seemed to focus on the content and their learning rather than technology when reflecting on their SL experience, which may be a positive sign of acceptance of virtual worlds as a CME environment.

We also argue that 3-D virtual worlds are more than a new technology and concur with researchers³⁶ who encourage re-consideration of what learning means in such spaces. Social presence is realized in a virtual world differently than in the real world. For example, people use their body unconsciously in real life, while virtual world users have to click a mouse or function buttons to create avatar movement, meaning they must remember and order their bodily reactions. How might this influence learning communication skills? It may be both a distraction and an advantage, because virtual world learners become more conscious of non-verbal communication and more intentional in their choices. Another relates to anonymity. When learners do not share their real names, does the anonymity of their avatar create a safer environment for greater risk-taking which can lead to greater learning gains?

As educators work with virtual learners, they may need to adapt their real-world curriculum and instruction to accommodate the limitations and leverage the opportunities of virtual worlds. Converting traditional slides and handouts into activities and demonstrations, and using animated visual effects, virtual world video-recording, local chat and other feasible in-world tools can enhance the instruction, but call for innovative thinking and new skills. Rethinking teaching approaches and acquisition of new skills require patience and time, but such investments will allow educators to take full advantage of virtual world affordances, achieve synergy between the environment and content, and enhance teaching-learning transactions.

Study limitations include a small convenience sample, absence of a comparison group, potential bias in expert assessment as the expert knew whether the standardized patient encounter was recorded pre- or post-event, use of a non-validated patient assessment form, and shortcomings of self-reports and recall. To mitigate limitations and increase the study trustworthiness, data were collected from multiple sources using multiple methods. Given the limitations, our findings should not be generalized to the broader population of primary care clinicians.

Our study supports further exploration of 3-D virtual worlds as learning environments for continuing education of family physicians and other primary care clinicians. Immersive and bridging space and time, virtual education can help busy clinicians improve their practice. As virtual words evolve and occupy a more central place in education,^36,37 it is important for family medicine teachers to engage with in-world learning environments in order to fully understand their affordances and shape their development to work well for the primary care domain.