Mitigating Misinformation Toolkit: A Medical Student Role-Play Curriculum on Communication Techniques to Facilitate Vaccine Misinformation Conversations

Abstract

Introduction

The COVID pandemic and affiliated infodemic led to widespread health misinformation, generating confusion and distrust. Physicians must identify and address misinformation, with attention to cultural/health literacy, equity, and autonomy. Most medical students receive training in core communication techniques but are rarely taught how to combat misinformation with patients and lack opportunities for practice in diverse settings.

Methods

We used mixed methods to evaluate the impact of a role-play-based training curriculum on 44 third- and fourth-year medical students’ comfort and confidence applying ask-respond-tell-seek solutions (ARTS) and motivational interviewing (MI) to discuss vaccine hesitancy, using COVID-19 as an example. There were three training iterations: prior to volunteering at a community health fair, during a medicine clinical rotation, and during a pediatrics rotation. Pre- and postsession questionnaires were administered. Likert-scale questions assessed comfort and confidence using ARTS and MI. Narrative responses focused on previous experiences with vaccine hesitancy, challenges faced, and session takeaways.

Results

Students’ comfort, confidence with ARTS/MI, and self-reported ability to discuss COVID-19 vaccinations improved as measured by pre- and postsession surveys (p < .05). Qualitatively, students reported increased confidence delivering recommendations in plain language and exploring patients’ thought processes behind choices.

Discussion

Reinforcement of core communications strategies in medical school can positively impact trainees’ ability and ease addressing misinformation. We recommend this 45-minute training session to effectively increase medical students’ comfort and confidence in discussing COVID-19 vaccines with patients. It can be adapted to any health professions school with an existing communications thread.

Educational Objectives

By the end of this activity, medical students will be able to:

• 1.Identify COVID vaccine misinformation that arises during discussions with patients and community members in practice through examples.
• 2.Identify and affirm that addressing COVID vaccine misinformation is a critical skill in practice.
• 3.Apply communications skills, including seeking collaborative solutions and motivational interviewing, to address vaccine misinformation with patients by engaging in structured practice through role-plays.
• 4.Report increased comfort addressing COVID vaccine misinformation in clinical practice and in the community-based setting.

Introduction

During the COVID-19 pandemic, Northwell Health, the largest private employer and health care provider in New York State¹ and the clinical home for the Donald and Barbara Zucker School of Medicine, stood at the 2020 epicenter of the U.S. pandemic and cared for more patients with COVID-19 than any other health system in the country.² Patients, health care providers, and other stakeholders to health care experienced almost immediate and profound knowledge deficits and confusion across several domains (e.g., disease pathogenesis, optimal diagnostic and therapeutic management, and effectiveness/safety of preventative public health measures like vaccination and masking), heightening societal levels of anxiety and mistrust. The current infodemic exposed a new era of public health vulnerability.³ An analysis at Northwell Health revealed that stakeholders expect health care providers to competently identify health misinformation and feel empowered to address it and that this should be done with attention to cultural/health literacy, equity, autonomy, beneficence, and professionalism, whether in private one-on-one provider-patient encounters or in physical or virtual public spaces.^3–6

Medical students are uniquely positioned to advocate for patient needs. Our students worked collaboratively on the front lines, caring for patients with COVID-19 in Queens, NY, one of the most diverse counties in the country.⁷ Patients trusted students and shared misgivings about vaccinations, including concerns regarding experimentation, cost, and fear of deportation. By improving medical students’ health communication skills around vaccine hesitancy and vaccine misinformation, we can leverage the key role medical students play in the community. As stewards of our community's health, we must understand the root causes of this crisis and devise novel curricular strategies to combat it. We have built upon existing educational infrastructure to create a theory-based model for addressing health misinformation and have developed novel resources to train our learners to apply this model to address health misinformation with patients.

Vaccine communication training is not currently a standard component of medical school curricula, and there are limited data regarding the prevalence of this education nationwide. Per the AAMC Curriculum Inventory tool, demonstrating interpersonal and communication skills that result in effective exchange of information is cited as a core competency in the Physician Competency Reference Set, but addressing misinformation, a higher-level skill, is not explicitly outlined as a core topic.⁸ Studies have revealed gaps in the ways students and providers feel equipped to address misinformation. In a 2022 study by Frisch, Chaudhary, Zhang, Parkas, and Forsyth, medical students reported a lack of confidence in addressing misinformation around vaccines: Only 9% of students (n = 21) corrected misinformation when someone in the community shared it, while 87% (n = 198) reported saying nothing and changing the subject.⁹ A study examining clinician perspectives on vaccine hesitancy revealed that most (76%) wanted additional training navigating these discussions.¹⁰

Prior implementations of teaching modules in medical school education sought to address these gaps through self-study modules and standardized patient encounters.^9,11 Earlier curricula have examined the importance of discussing vaccine hesitancy and addressing misinformation through a flipped classroom approach,¹² a case-based approach,¹³ and the CASE (collaborate, about me, science, explain/advise) method of communication.¹⁴ Only one of these curricula, however, considers the importance of motivational interviewing (MI) in its design.¹⁴

To improve student comfort and confidence addressing vaccine misinformation, foster more effective patient-physician relationships, engender trust, and promote patient-informed decision-making, we developed an active learning, role-play-based curriculum grounded in the three-function model of communication, the ask-respond-tell-seek solutions (ARTS) framework, and MI.⁶ Cole and Bird's three-function model is a well-established one for patient interviewing and serves as the basis for our school's communications skills curricula.⁶ The three functions, defined as build the relationship, assess and understand, and collaborative management, respectively, provide a framework for learning how to master the patient interview and navigate potentially challenging patient interactions.⁶ ARTS provides students with a structured pathway for helping patients formulate possible solutions while mitigating the influence of their personal biases and misconceptions. MI is an evidence-supported counseling style situated within an atmosphere of respectful patient-physician collaboration promoting constructive conversations exploring one's readiness, ambivalence, and resistance to behavior change.⁴ As our school's existing communication skills curricula integrate ARTS and MI within the collaborative management function of the three-function model, we leveraged our students’ knowledge of these best practices in designing this curriculum. Lastly, we sought to create a curriculum tailored to a student's role in both the community and clinical settings, something we observed from our review of the literature that had not previously been done.

Methods

Population

Third- and fourth-year medical students were included in our activity because students at this point in their medical education had all completed their preclerkship communications curriculum. The first iteration of our session served as a pilot, where third- and fourth-year students voluntarily signed up to participate. These students had also volunteered to work at a local community health fair day following this didactic as an opportunity for real-world practice. This project was approved by the Hofstra University IRB Committee (Ref#: 20220831-SOM-PEL-1, August 31, 2022).

Facilitators

Two fourth-year medical students who had completed their preclerkship and clerkship communications curriculum, as well as a master's in public health, along with two supervising core faculty members who oversaw the school's communications curriculum, served as facilitators.

Learning Environments

To support distance learning for students on clinical rotations, the curriculum was delivered in a synchronous digital milieu using a commercially available web-based videoconferencing platform (Zoom). Screen sharing, synchronous public chat messaging, and digital breakout rooms were utilized to encourage engagement and interaction as if in person.

Implementation and Summary of Content

There were three training iterations: prior to volunteering at a community health fair, during a medicine clinical rotation, and during a pediatrics rotation. Each educational session was 45 minutes and contained the same educational content. To easily integrate the session into the third-year curriculum, it was delivered as a didactic during scheduled teaching time for students on their third-year medicine and pediatrics rotations. At our institution, students on these rotations have mandatory didactics 4 days a week during their lunch break.

A facilitator guide (Appendix A) was developed to provide a step-by-step approach to implementing the session and explain how/when to use the appendices. We also created PowerPoint slides and speaker notes for the session (Appendix B). At the start of the Zoom session, all students downloaded files that were sent by the facilitators via the Zoom chat feature. These files included a handout with information on COVID-19 vaccine schedules and 12 common examples of COVID-19 misinformation with factual explanations addressing the statements (Appendix C), a role-play rubric (Appendix D), and an example role-play demonstrating how to incorporate ARTS and MI for facilitators and students to reference (Appendix E). Additionally, four role-plays were created, two for the medicine didactic (Appendix F) and two for the pediatrics didactic (Appendix G), which were included in the medicine and pediatrics sessions, respectively.

The session began with completion of a presurvey (Appendix H). The next 15 minutes were dedicated to teaching using the PowerPoint (Appendix B). We talked through a brief overview of the epidemiology of the COVID-19 pandemic, including prevalence, incidence, deaths, and vaccination rates. Emphasis was placed on the impact of the virus on marginalized groups who, based on available data, were both more significantly affected by COVID infection¹⁵ and less likely to be vaccinated.¹⁶ Currently available vaccines and vaccination schedules (appropriate for the time the curriculum was implemented) were reviewed. Students were asked to reflect on previous experiences they had had, if any, discussing vaccines with patients; they were invited to share these reflections with the group either by unmuting and speaking out loud or by sharing in the Zoom chat. Following the reflections, we continued using the PowerPoint (Appendix B) and transitioned to a 5-minute review of ARTS and MI.

Students were then separated randomly into pairs using the Zoom breakout room feature and engaged in a 15-minute role-play (Appendices D, E, and, depending on whether students were in medicine or pediatric clerkship, F or G, respectively). Students were instructed to take turns playing patient and physician, allowing each one an opportunity to practice ARTS and MI communication skills. Students were told to complete both role-plays (Appendix F or G) in the 15-minute period and were expected to keep track of time on their own. The role-plays contained a brief problem representation and basic instructions for each individual playing the physician or the patient, offering guidance for the conversation. Facilitators also reminded students to utilize Appendix C as a reference for information on vaccines and how to address common misconceptions and Appendix E for a review of ARTS, MI, and a sample role-play, if needed. Students were expected to reference the role-play rubric (Appendix D) when evaluating their conversation. At the end of 15 minutes, facilitators closed the breakout rooms, bringing students back to the main Zoom room.

The session was concluded with a 5-minute wrap-up including a debrief on the experience and offering space for reflection, offering chances to share successes, challenges, or takeaways. Students were again invited to share reflections by unmuting themselves or using the chat feature.

A key emphasis throughout the session was embracing a culture of appreciative inquiry, mutual respect, cultural humility, and respect for autonomy in approaching these conversations. It was also reinforced that, when using these skills in practice, an initial measure of effectiveness would be building rapport with patients; behavior change, while the ultimate goal, could take significantly more time.

Learner Assessment/Program Evaluation

To evaluate the effectiveness of our interactive curriculum, participants were surveyed at the start and end of each session using an internally developed questionnaire (Appendix H). The survey was provided to students via pasting a survey link into the Zoom chat feature. This was done at the beginning and end of the PowerPoint didactic session. One reminder email containing the postsurvey was sent to all students immediately following the session. The same pre- and postsession questionnaire was used across all three implementations. The survey contained multiple questions assessing students’ comfort and confidence using ARTS and MI both generally (for any purpose) and specific to conversations with patients about vaccines. A 4-point Likert-type scale (1 = strongly agree, 4 = strongly disagree) was employed for these questions. We included an open-ended prompt in our questionnaire inviting students to share narrative reflections of previous experiences they had had with vaccine hesitancy, what challenges they had faced in holding these conversations, and what takeaways they had following the session.

Data Analysis

Data were analyzed using IBM SPSS Statistics version 28.0. Descriptive statistics were presented as the mean for ordinal variables and as frequency and percentage for categorical variables for each survey-item response. Within-subjects analyses were performed using the Wilcoxon signed rank test for ordinal variables and the McNemar test for categorical variables (pre- vs. postsession).

Results

This curricular session was delivered three times. Across all three implementations, 44 third- and fourth-year medical students participated. During the first session, there were 11 students (combination of third- and fourth-year students) in attendance; 22 and 11 third-year students attended and completed the second and third sessions, respectively. All 11 participants from the first session completed pre- and postsession surveys (response rate of 100%). Of the 22 participants in the second session, 17 completed the presession survey (77% response rate); 15 participants completed the postsession survey (68% response rate). In total, we were able to compare 64% of participants’ pre- to postsurveys (14 of 22). Nine of the 11 participants from the third session completed presession surveys (82% response rate), and seven participants completed postsession surveys (64% response rate). We compared 64% of the participants’ pre- to postsurveys (seven of 11).

Table 1 shows all demographic information of the sample. The average age of participants was 26 years, with a range from 23 to 36 (SD = 2.6). The majority of students were White (n = 22, 60%) and identified as cis women (57%). Internal medicine was the most commonly identified specialty that students reported they were likely to apply to for residency (n = 11, 30%).

Table 1. Demographics of the Sample^a.

Half of the students (n = 18) reported having clinical experiences where they had to address COVID-19 vaccine-specific misinformation with patients prior to the interactive session. Most of these reported they had two to five clinical encounters (n = 9, 24%), followed by one to two (n = 6, 16%). Twenty-four students (65%) reported having clinical experiences in which they discussed obtaining the COVID-19 vaccine with patients prior to the interactive session. Thirteen students (35%) reported they had two to five of these clinical encounters, while 11% reported both one to two encounters and five to 10 encounters.

Results from the first session were compared and evaluated separately from the second and third sessions because the first group contained fourth-year students and was delivered 8 months earlier. Additionally, lessons learned from the first session led to slight differences in the delivery of the subsequent two interactive sessions.

Tables 2 and 3 show mean scores pre- and postsession. For the 4-point Likert-scale items, significant differences were observed between the pre- and postsession scores in comfort and confidence for nearly all items. One exception was noted: No significant differences were observed among the pre- and postsession scores of participants’ comfort engaging in MI from the second and third sessions. In the first interactive session, the pre- and postsession scores for comfort among participants demonstrated significance. A significant difference was observed in students’ self-reported medical knowledge to counsel a patient on COVID-19 vaccines for all groups. For categorical variables, students felt more comfortable addressing COVID-19 vaccine misinformation specifically about vaccine safety and effectiveness after the first interactive session, but this was not statistically significant. There were limited or no improvements in other categorical variables following the first interactive session. Following the second and third interactive sessions, students felt more comfortable addressing COVID-19 vaccine misinformation, but the only result to reach significance was seen specifically around addressing vaccine effectiveness.

Table 2. Mean Scores From the Pre- and Postsession Surveys.

Table 3. Participants’ Comfort Addressing COVID-19 Vaccine Misinformation.

Students reported in their narrative responses that they had increased confidence and ease with delivering COVID-19 recommendations in plain language and felt more comfortable exploring patients’ thought processes behind their choices. Additionally, students reported that they learned how to create a safe and supportive space to engage in conversations around vaccine hesitancy by asking permission, having patience, and meeting patients where they were at when it came to behavior change. Multiple students also commented that the session provided an opportunity to thoroughly review MI and ARTS.

Discussion

Our interactive communications training session was an effective and time-efficient tool to increase medical students’ self-reported knowledge, comfort, and confidence engaging in conversations about vaccine hesitancy with community members and patients using the specific communication techniques of ARTS and MI. These communication skills are the backbone of collaborative patient-physician relationships and are fundamental strategies for combating vaccine misinformation.⁶ With the rise in vaccine hesitancy, which was amplified in the setting of the COVID-19 pandemic, providing students with a space to review and practice these conversations is especially important. Additionally, it is notable that over 50% of participants reported having one or multiple experiences in clinical interactions where they encountered vaccine hesitancy. This suggests our study population was an appropriate target audience for this endeavor. Training serves to equip students with the tools needed to engage with patients more comfortably and confidently in clinical settings.

This session was specifically developed to leverage and build upon required communication curricula throughout medical schools. The Liaison Committee on Medical Education (LCME) defines relevant standards for accreditation that must be met by medical education programs leading to the MD degree.¹⁷ LCME element 7.8 states that medical schools must include “specific instruction in communication skills as they relate to communication with patients and their families, colleagues, and other health professionals.”¹⁷ The most recent LCME Annual Medical School Questionnaire Part II found that 146 of 147 schools include communications skills in their curricula during the preclerkship phase and that 141 of 147 schools include communications skills during the clerkship phase.¹⁸ The prevalence of existing communications training in medical school curricula creates a natural and feasible opportunity for schools to integrate vaccine hesitancy training into their education programs.

We recognize that not all schools utilize ARTS and MI specifically. To implement our instructional plan, schools can either (1) use ARTS, (2) align the concept of ARTS and the collaborative and participatory framework they already use, or (3) substitute a preferred collaborative and participatory framework into the associated role-plays. MI is an evidence-based approach for behavior change⁴ that may not be included in a medical school's communications curriculum. We created Appendix E for schools that do not currently use ARTS or MI in order to provide more information about these concepts and a script of language that reflects their incorporation. Appendix E provides a detailed example and can be adapted to a school's existing communications framework as needed.

In our sample population, most students reported feeling comfortable and confident at baseline in the use of ARTS and MI in discussions surrounding vaccine hesitancy, which speaks to their established foundational training in communications practice. Students demonstrated increased comfort and confidence across the three cohorts, with initial responses of agree and subsequent responses of strongly agree. This result suggests that these skills can be built upon and was likely due to the collective effort of skills-based practice and reinforcement of empathy skills. Supplementary sessions, such as this 45-minute interactive session, serve to promote sustainability through continued learning and deliberate practice built upon prior basic communication training.

An important theme that emerged was grounding this work in cultural humility and respect for patient autonomy as a mechanism to preserve and strengthen the patient-physician relationship. Qualitative reports highlighted this, particularly in the two latter implementations of the session. Numerous students commented on the importance of “meeting patients where they are at,” being consciously respectful of individual perspectives, and cultivating a nonjudgmental, welcoming space for discussion.

Throughout the implementation process, one important takeaway was the usability of delivering this content virtually. During development, we discussed whether a virtual setting would be effective or whether the session would best be delivered in person. Responses from all three sessions were overwhelmingly positive, and students were actively engaged and interactive throughout. Hosting this session virtually may have strengthened its delivery given the minimization of barriers to accessing it and reaching students located at different training sites. Furthermore, the chat feature on Zoom allowed students who felt more timid and may have been uncomfortable speaking aloud to share their thoughts. While the content was successfully delivered in 45 minutes, an hour may be a more ideal time allotment. Sending students the presurvey in advance may have helped reduce time barriers by decreasing the time needed to fill it out prior to beginning the session. However, having an extra 15 minutes would ease the time constraint, give students more time to complete the surveys, and allow for a bit more flexibility during the group discussions.

While this activity was considered a success overall, there are some limitations that are important to address. Despite three different implementations, the total sample size was small. Repeated delivery of this content with more students may be helpful to get a bigger picture of its effectiveness. Students were incentivized to complete the pre- and postsurveys with a raffle to win a $10 gift card, which could limit reproducibility and may have resulted in students completing the survey for the sake of completion rather than providing genuine answers. Additionally, surveys only evaluated students’ perceptions of content knowledge, comfort, and confidence. Students were not evaluated on real-world delivery and application of this content, nor was sustainability of these skills assessed. Future research should focus on students’ ability to apply their communication skills with patients as well as demonstrating whether the acquired knowledge from the session impacts their practice during future rotations or as a resident physician.

This session was implemented with students who had instruction during their 2 preclerkship years on communication techniques that included ARTS and MI. While we have provided instructions for how to adapt to a curriculum that does not include ARTS and MI, we have not tested that situation, and it is possible our results may not be generalizable to such a population. Furthermore, we utilized two fourth-year medical student facilitators to deliver the sessions, but not all institutions may have fourth-years who are equipped to assume this role. While the use of student facilitators is a strength of this implementation, it is not a requirement. Faculty experts who are well versed in this content and in communications techniques are more than sufficient to lead this session.

This 45-minute interactive role-play-based training session is an effective way to increase medical students’ comfort and confidence in discussing COVID-19 vaccines with patients. The session is best employed with medical students working in clinical environments who have completed basic training in communication skills. It can be easily adapted to any health professions school with an existing communications thread.

Future directions for this work include incorporating the voices and experience of trainees in other health care roles, such as nursing or physician assistant education, and adapting the curriculum to meet the needs of these respective roles. Furthermore, a future goal is to employ the curriculum longitudinally as a component of communications development for trainees in a variety of other specialties beyond medicine and pediatrics.