Commentary: Zoom into the future with the virtual mock oral examination

Leora B. Balsam, MD

Central Message.

Video-conferencing can connect trainees and examiners for mock oral examinations that simulate the ABTS certifying examination.

See Article page 128.

The American Board of Thoracic Surgery (ABTS) certifying examination is an oral examination that evaluates clinical problem solving, surgical judgment, and behavior. Passing the examination is a necessary step in the ABTS board-certification process, with pass rates ranging from 72% to 84% over the last 5 years.¹ The proportion of first-time passage on the examination is also a measure used by the American Council of Graduate Medical Education to assess cardiothoracic training programs, whereby poor performance may lead to loss of accreditation.

Many training programs have implemented mock oral examinations to simulate the ABTS certifying examination. There is considerable variability in the structure of these examinations, but the basic premise remains the same: a question bank, grading structure, examiners, and examinees are needed. Major barriers to implementation include access to suitable scenarios and examiners.² Regional mock oral examinations are offered in some cities and geographies, which increases the pool of examiners and allows residents to be paired with examiners from other institutions. These collaborative mock oral examinations better simulate the experience of the ABTS certifying examination, where the candidate is examined by unfamiliar examiners.

In the field of surgery, some, but not all, studies have found a positive correlation between trainee performance on mock oral examinations and first-time passage of the American Board of Surgery certifying examination.3, 4, 5 When surveyed, most residents find the mock oral experience to be “useful,” even if it doesn't directly translate into better performance on the certifying examination. In addition, program directors may use it as a measure for identifying areas of weakness in the individual trainee and the program as a whole.

In this issue of the Journal, Fiedler and colleagues⁶ describe the development of a multi-institutional mock oral board examination on a video-conferencing platform (Zoom). Residents and faculty from 6 cardiothoracic surgery training programs participated, with residents being examined by a pair of faculty members in a virtual thoracic room and a virtual cardiac room. In post-examination surveys, the feedback from both trainees and examiners was overwhelmingly positive.

Out of necessity, the coronavirus disease 2019 (COVID-19) pandemic has brought virtual meeting platforms to the forefront in our professional lives. With its potential to bring together examiners and examinees from different geographies in real-time, video-conferencing offers a reasonable simulation of the actual certifying examination. It is uncertain when and how the 2020 ABTS certifying examinations will be performed, as the need for social distancing and travel restrictions are significant barriers. We are at an interesting inflection point where the ABTS has the opportunity to re-evaluate the examination structure and its value.

The study by Fiedler and colleagues⁶ is exploratory and does not attempt to show a relationship between the virtual mock oral examination and performance on the actual certifying examination. As a simulation, a notable weakness was that the examiners were not all ABTS-certified (some were board-eligible); therefore, their ability to administer the examination, judge performance, and gauge the “usefulness” of the experience is uncertain. Finding qualified examiners is a common theme when discussing barriers to implementing mock oral boards.² However, the virtual platform does create the opportunity to access a wider pool of faculty across different institutions and geographies, which may allow for a better simulated examination in the future.

The ABTS may take note of this pilot as it considers how to move forward with the certifying examination amidst our “new normal” in the COVID-19 era. Video-conferencing may in fact be a better strategy for the examination both during and postpandemic, allowing for greater flexibility in scheduling and significant cost reduction.