Implementing an online OSCE during the COVID‐19 pandemic

1. PROBLEM

Due to the COVID‐19 pandemic, most dental schools in the United States ceased all in‐person activities. ¹ Lessons learned from past outbreaks demonstrated the need for new applications of information technology to appropriately educate students. ²

Objective Structured Clinical Examinations (OSCEs) are assessments that measure clinical competence by physically rotating students through multiple stations. At each station, the student demonstrates competence through oral or technical examinations for a specified amount of time. ³ OSCEs are conducted at Harvard School of Dental Medicine (HSDM) throughout the predoctoral curriculum. During the pandemic, lectures and case presentations were able to be transitioned into e‐learning through platforms like Zoom (Zoom Video Communications, San Jose, CA). ⁴ However, because students must be physically rotated, it was challenging to convert OSCEs to an online form. There was an evident need to creatively restructure the OSCE to reduce curriculum disruptions.

2. SOLUTION

With careful planning, successful online OSCEs were administered. The Zoom platform was selected due to examiner and student familiarity and the “breakout room” ⁵ feature, which allows private minisessions (one‐on‐one) between host‐selected participants. One Zoom link with 21 pre‐assigned breakout rooms was created. There were 2 examiners for each of 10 disciplines, plus 1 “rest station” (Figure 1). Within a discipline, 1 examiner was designated as “Red” and the other as “Green.” Thirty‐four students were equally split into 2 sessions; within each session, students were assigned to Red or Green group. A clinical case was disseminated to each student session 1 hour before the start times. For standardization, examiner calibration sessions were conducted 1 month and 1 week prior. A pre‐OSCE run‐through was performed with examiners and students.

FIGURE 1.

OSCE faculty station order.

The Zoom host placed examiners in breakout rooms prior to student entry. Students were asked to sign in with their full names, leaving cameras and microphones continually on to promote academic integrity. Once all students assigned to a session had joined, they were placed into their first breakout rooms by the Zoom host. Six minutes were allotted per station for oral examinations based on the clinical case that students were given. When the allotted time had passed, students were moved to the next room of the appropriate color (e.g., Red Ortho to Red Peds). One‐minute warnings were broadcasted to each room before a move. Examiners graded students on Qualtrics (Qualtrics, Provo, UT) or on hard copy.

3. RESULTS

Feedback from students and examiners was obtained (Table 1). Notably, most students thought the online OSCE was similarly successful to traditional OSCEs, and all student respondents felt they were able to fully showcase their knowledge. Half of student respondents thought that even future OSCEs should be performed online. Examiners also felt the OSCEs ran well overall. Several examiners commented that pre‐OSCE measures like calibrations and run‐throughs were integral. There were only a few problems related to technical difficulties (Table 1). Recommendations from our experiences are listed in Table 2.

TABLE 1.

Student and evaluator/staff feedback for the online OSCE

Feedback	Students	Examiners/staff
Positive	I didn't have to do anything, I was nicely placed into the next room which was a nice change from walking around the clinic. I felt it took less time this way and had less nerves. I liked that I was able to stay in the same place without having to spend time shuffling around so that I could prepare for each section before the short time began. I liked the privacy and one‐on‐one. It didn't waste precious clinic time. Even though we had the internet at our fingertips (because we were on our computers) it was not usable [to look up answers] because it was such an intimate Zoom space.	I am still in awe about how well the OSCE worked out. I have very little to add except well done. Rehearsal was very helpful to learn what exact would happen during the OSCE. Went very smoothly. So impressed. Given the limitations of the tools available, there isn't much I'd have suggested be done differently.
Needs improvement	The only problem was that one of my sessions got cut short and I had a poor connection/freezing screen for one of my rotations. Quality of the images and sound was not always clear and leaving the faculty without any warning was disconcerting. I had a poor connection with one of my faculty for the whole 6 minutes, thus I had to type my responses in the chat and wasn't sure if my answers were properly received until I got confirmation via chat. I was confused and left the room. It took about 10‐15 minutes for me to be moved back to the correct room.	Hands‐on questions were not possible for virtual OSCE. Grading between students was little challenging. Sometimes had Wi‐Fi issues and could not grade using Qualtrics. There were some small unforeseen challenges (such as people leaving their breakout rooms rather than waiting to be moved) but nothing that caused any great challenges.

TABLE 2.

Recommendations for online OSCEs

Technical

There should be established communication tools between examiners (cell phones, emails) during the OSCE. Having supporting staff as co‐host is recommended so that they can take over as host in the event the original host loses Internet connection. With a greater number of students, it would be more appropriate to have several separate Zoom links with a corresponding number of hosts. This reduces the amount of breakout room‐moving 1 host must do, therefore reducing technical errors.

Workflow

The rehearsal is an important tool for examiners to learn what to expect during the OSCE. Pre‐OSCE calibration between faculty is necessary. Examiners should write OSCE questions with the understanding there may sometimes be <6 minutes for questioning. The case should be screen‐shared on Zoom at all times by examiners to maximize the amount of time with the student.

Considering the overall success of the online OSCE and positive feedback from students and examiners/staff, there may be value in moving traditionally in‐person assessments online, even post‐pandemic.

Kakadia R, Chen E, Ohyama H. Implementing an online OSCE during the COVID‐19 pandemic. J Dent Educ. 2021;85(Suppl. 1):1006–1008. 10.1002/jdd.12323