Remote Assessment of Clinical Skills During COVID-19: A Virtual, High-Stakes, Summative Pediatric Objective Structured Clinical Examination

Problem

Due to requirements for social distancing to protect the health of students, faculty, and standardized patients (SP) during the coronavirus pandemic, the pediatric clerkship Objective Structured Clinical Examination (OSCE) could not safely take place within our simulation center.

Approach

We adapted a previously live-only OSCE to be delivered virtually via teleconferencing software, a “teleOSCE”. TeleOSCE platforms have previously been described for use in formative assessment of students in rural locales as well as in assessment of basic telemedicine skills.^{1 , 2} More recently, educators from Singapore executed a live OSCE while respecting social distancing rules due to coronavirus.³ Our teleOSCE is unique in its use of an entirely virtual platform for a high-stakes summative assessment of clinical skills, while still maintaining locally established SP checklists, communication scoring tools, and faculty observation rubrics. This allowed us to preserve a core assessment that is given to all clerkship students at the completion of their pediatric rotation.

Zoom software (Zoom Video Communications, Inc, San Jose, Calif) was selected due to its simplicity, audio/video quality, breakout room features, and affordability. All students, faculty, SPs, and administrators participated from the privacy of their respective locations. Following a thorough test run with nonclerkship student volunteers and to ensure feasibility, we settled on testing 7 students per half day over 3½ days (n = 49) whereas our standard, in-person OSCE can reliably assess up to 16 students per half day session.

Following a brief orientation in the “hallway” (ie, main teleconference room), students were given 3 minutes to read the “doorway folder” (ie, pre-encounter instructions on the web-based simulation learning management system CAELearningSpace [CAE, Sarasota, Fla]). Then students were moved into the “exam room” (ie, Zoom breakout room) where the SP, a faculty observer, and an administrator were prepositioned for the start of the encounter. Students had 22-minutes to complete each patient encounter, after which they left the “exam room.” Back within LearningSpace, students had 13 minutes to complete a postencounter note while faculty and SPs simultaneously completed respective assessments of the student. Upon completion of the postencounter note, students were then returned to the “hallway” to read their subsequent “doorway folder” and proceed into the next “exam room,” ultimately working through 4 pediatric cases.

Outcomes to Date

Statistical analysis suggests the teleOSCE is a comparable assessment to the live OSCE. TeleOSCE participants (N = 49) were compared to live OSCE participants (N = 173) from the previous 3 years. There was no difference in mean score (mean difference −1.1; 95% confidence interval −2.8 to 0.7; P = .2) or failure rate (rate difference 2%; 95% confidence interval −0.7% to 10.7%; P = .06) between the groups.

Next Steps/Planned Curricular Adaptations

The demonstrated effectiveness of this novel teleOSCE makes it a feasible tool for both undergraduate and graduate medical education programs adjusting to the current COVID-19 restrictions. Even beyond the restrictions, this model will allow institutions with remote learners to perform summative assessments while saving on travel time and costs. Additionally, the teleOSCE is easily adjusted to accommodate telemedicine assessments (both formative and high stakes), which are quickly becoming essential in medical education to reflect the newly increased importance of telemedicine skills in modern clinical care.