Seven ways to get a grip on preparing for and executing an inclusive virtual multiple mini interview

Abstract

COVID-19 restrictions have prompted many medical schools to shift to virtual interview methods for medical school applicant selection. While extensive reflection has been documented around both the process and benefits of transitioning to a virtual Multiple Mini Interview (V-MMI) format, less attention has been given to examining the unintended consequences of this adaptation on increasing representation from underrepresented groups. In this Black Ice article, we consider the equity implications of taking a virtual approach to conducting MMIs and present some practical tips to ensure medical schools are giving attention to and addressing equity issues that may affect applicant and assessor engagement and success. The following seven recommendations include actionable steps medical schools can take immediately to optimize the interview process. This guide can be adapted to residency matching services and other health professions education programs that utilize the MMI.

Introduction

To accommodate COVID-19 restrictions, many medical schools adapted the Multiple Mini Interview (MMI) to a virtual format.^1-6 Significant changes have been made to the MMI format and process to adhere to public health measures. These include, but are not limited to, changes to interview questions and the number of interview stations, changes to timing, preparation and scheduling, greater reliance on web-based technologies, and modification to the scoring system.^1,7,9 While the innovation and adaptability required to swiftly transition to virtual MMIs (V-MMIs) has been extensively documented,^1-3,5-6,8,9 this shift additionally affords opportunity to reflect upon the unique challenges and opportunities that transitioning to a virtual platform can expose.

What follows is guidance on how medical schools can optimally execute and maximize an inclusive V-MMI experience. These tips are an amalgamation of literature largely focused on admission processes, publicly available information on medical school websites, and lessons learned from user experience, based on the authors experiences studying equitable medical school recruitment and selection policies and processes. Our goal is to support medical schools challenged with how to tangibly address equity concerns resulting from the adaptation to a virtual version of the MMI. Here we focus primarily on advancing the demographic representativeness of medical applicants and MMI assessors. We recognize, however, that what will work for some medical schools, will not easily translate to others. We encourage readers to take inventory of the structures and processes within their unique setting that may inadvertently prevent medical applicants and MMI assessors from easily engaging in the V-MMI process (e.g., access to reliable internet or perhaps people are colorblind).

Optimizing the V-MMI experience

1. Consider the affordances and barriers in using an online platform

The MMI plays a distinct purpose in the medical school selection process. However, we cannot ignore the fact that the decision to facilitate MMIs virtually means a myriad of new ‘actors’ (e.g., videoconferencing systems, additional staffing-like IT staff) have quickly mobilized, forcing medical schools to critically reflect on the affordances and barriers of an online delivery model, particularly at a time of unprecedented uncertainty. We encourage medical schools to consider the affordances and barriers in using an online platform to assess applicants and to talk explicitly about how an individual’s race, class, language, and/or confidence with video conferencing tools grants some individuals privilege while it impedes others’ experiences.^1,10,11

Affordances

The V-MMI can afford some applicants and assessors a more accessible opportunity, especially those located out of province, in rural areas, or low socioeconomic groups.^2,4,6,8,9 By increasing the likelihood of more rural or geographically diverse applicants, programs improve their chances of admitting students who reflect the diversity of society.³ As for assessors, a V-MMI may be enticing as they can conveniently participate from the comfort of their home or neighbourhood without the additional time commitment and costs associated with travelling to campus.^4,6,8 The convenience of an asynchronous V-MMI may additionally boost recruitment success of assessors, as well as the fair assessment of applicants. Applicant responses could be recorded, and thus, can be interpreted by multiple assessors to reduce assessor bias.¹²

Identifying barriers & mitigation strategies

It is critical for medical schools to remain vigilant to the questions of: how does an accommodation to V-MMI influence the equitable participation of diverse groups? And, who consistently experience barriers to participating in the MMIs?

Despite the above-mentioned affordances, it is equally important to identify issues or limitations that may negatively influence an applicant or assessor’s V-MMI experience. For instance, some individuals may not have a reliable internet connection or access to the equipment required to participate in a V-MMI.^1,4,5,7 It is also important to remember that V-MMIs during a pandemic may elicit new stressors previously unconsidered in the traditional MMI format, such as needing to manage distractions associated with home environments or learning to use different technologies and platforms.^3,12 To overcome any number of circumstances that can affect applicant and assessor performance requires some creativity and commitment from institutions.

One potential solution is for institutions to create a technology equipment library where applicants and assessors can borrow the necessary equipment needed to participate in the virtual interview process. Another option is to provide local applicants and assessors with the option to book a private interview space equipped with reliable internet and access to technological support should they encounter technical difficulties.⁴ This will alleviate some anxiety and stress about interacting online and help aid in their well-being.

2. Consult with campus centres that have expertise on student access, inclusion, and accommodation support

As experienced stewards of inclusion, campus centres have a commitment to cultivate student programs, services, and skills that ensure each person receives the support they need to perform at their highest potential. Medical schools are encouraged to reach out to different groups on-campus such as the student accessibility centre for assistance on how to support and reduce barriers to participating in the V-MMI process. Consulting diverse campus centres allows for the implementation and delivery of an inclusive V-MMI and may even illuminate new accessibility considerations such as colorblindness and close captioning. For instance, allowing applicants’ interview responses to be dictated to a scribe (e.g., Voice to Text a Google Chrome extension) allows the equal and inclusive participation of hard of hearing or deaf assessors. An inclusive V-MMI experience requires input from individuals across different disciplines, who understand the differences in students’ experiences. The earlier these conversations can take place, the better.

Preparing applicants & assessors for the V-MMI experience

3. Provide guiding information on virtual process

In addition to the traditional MMI preparatory work, applicants and assessors must also familiarize themselves with the chosen virtual format (e.g., synchronous or asynchronous). Medical schools should acknowledge the invisible labour that applicants and assessors undergo to prepare for the MMI and provide supports to ensure they are sufficiently prepared for the chosen V-MMI format. Considerations to incorporate into the invitation email includes information on format, platform (e.g., Kira Talent, Teams, Zoom), and anticipated time commitment. Tailored resources should be attached based on the method and system. For instance, resources for an asynchronous V-MMI could include information regarding the rules and regulations for completing the interview and the time limit for submitting the recorded answers or describing the time period for assessors to complete their assessments. For a synchronous interview, medical schools should consider providing information on what to expect upon login (e.g., calendar invite includes 10 minutes within a waiting room prior to the start of the formal interview). Additionally, encourage assessors and applicants to clarify when they are taking a deliberate pause. Otherwise, the pause could be mistaken for a frozen or dropped call.

4. Offer applicants and assessors on-boarding for the virtual platform

Since the virtual interview format is a relatively new innovation in medical school selection processes, both applicants and assessors will need to go through a series of steps to install the platform, followed by an onboarding process in addition to reviewing the traditional MMI preparation materials. The onboarding process is recommended to be completed well in advance of the scheduled interview date to alleviate any technological challenges and allow time to arrange solutions. Assessors and applicants should practice within the platform to ensure they are comfortable in the virtual and physical space, have access to the required technology, and their devices support the interview platform. While this on-boarding process can take multiple forms, we encourage medical schools to provide a virtual on-boarding experience that mirrors the chosen V-MMI method. For instance, a synchronous on-boarding process might resemble the virtual recruitment events medical schools have adapted to during the COVID-19 pandemic, such as advanced registration for one of several virtual on-boarding events. Medical schools should encourage applicants and assessors to attend using the same equipment and internet connection they will use during the V-MMI and consider providing information on how individuals can test minimum technical requirements, or links to testing internet connection, to prepare for any technology glitches.

5. Equip applicants and assessor with the tools for success

Applicants and assessors should have access to a pen and paper throughout the V-MMI. Based on interviews with applicants, this tool significantly improves the experience and outcome of the V-MMI. It is much more effective to write down reference notes during the preparation period, than to try and take a “mental note”. The high stress environment can easily cause applicants to forget their train of thought or respond with what might be perceived as a disorganized reply. As for assessors, we recommend having access to tools (e.g., pen and paper or a separate device) to help mitigate distractions on their computer screens when engaging synchronously or asynchronously to have assessors undivided attention on the applicant. This is particularly important when the V-MMI is completed asynchronously, and assessors may re-watch videos. Having initial notes to return to can help assessors reflect upon their thought process and initial reactions.

In addition to the information assessors receive for the MMI, consider providing assessors with applicants contact information in case of video and audio disconnection.⁴ In addition, it is encouraged that applicants and assessors be provided with guidelines on placing the camera at eye level to deter from any distractions given odd positioning or angles (e.g., looking at an assessor’s chin) and the ideal placement of the microphone (if external to computer). At each point of communication, it is encouraged to provide information on technological support available to applicants and assessors.

Maximizing engagement opportunities after the V-MMI experience

6. Solicit feedback from assessors and applicants on the V-MMI experience

To sustain equitable and inclusive V-MMI experience in future years, medical schools may consider solicitating feedback from assessors and applicants to identify potential gaps in training and areas for improvement in upcoming admission cycles. When soliciting feedback, it is encouraged to consider assessors and applicants’ satisfaction with the platform and format, but also their perceptions of fairness and equitable access in participating in the V-MMI format. Again, medical schools are urged to collaborate with relevant student accessibility champions on their university campus on how best to approach the feedback process and promote inclusivity.

7. Establish virtual spaces that welcome a bit of levity and foster social connection

It is important for medical schools to foster social connections among applicants, current students, faculty, and leaders to identify features of the program, determine suitability and fit, and learn more about the institutional culture and proximity of resources. To do so, medical schools may consider producing a video tour of their classroom spaces, laboratories, and simulation suites, as well as footage of the surrounding neighborhood, and city.⁴ They could additionally consider including information on childcare options, cost of living, transit availability, access to on- and off-campus activities, and first-person insights into medical school from diverse perspectives and identities (e.g., current learners, faculty, leadership, and alumni). This video could be emailed to individuals at any stage of the interviewing process. Although, we encourage institutions to deliberately create welcoming virtual spaces to debut the video followed by facilitating an opportunity for applicants, faculty, and leaders to socialize and interact in meaningful ways online. There are several ways to foster social connections across the V-MMI experience from pre-discussion boards to hosting a trivia Zoom-party. In the time of COVID-19, individuals desire opportunities to meaningfully connect; medical schools need to consider creative ways to foster social connections among and between school leaders and future medical applicants.

Conclusion

The COVID-19 pandemic has prompted radical changes to the medical school application process, including how admission offices run their MMIs. While the pandemic has brought forth many challenges and uncertainties, it has equally reaffirmed just how nimble and innovative medical schools have been throughout this global crisis. It is difficult to predict whether the shift to online MMIs will be permanent; however, we believe many of these tips are relevant for both online and in-person contexts within and beyond medical education, such as residency matching services (e.g., CaRMS) and other health professions education programs.

Conflicts of Interest

None to declare.

Funding

SSHRC Insight Development Grant. File Number: 430-2020-01096