“Feedback is a gift. Ideas are the currency of our next success.”
—Les Wallace and Jim Trinka1
In 2018, the American Board of Anesthesiology (ABA) introduced a new Objective Structured Clinical Examination (OSCE).2 This examination, plus the longstanding Standardized Oral Examination (SOE), became the APPLIED Examination, the final step of its certification process. That same year, the Anesthesiology Residency Review Committee (RRC) of the Accreditation Council of Graduate Medical Education (ACGME) updated the Program Requirements for Anesthesiology, raising the bar of residency programs’ accountability for trainees’ post-graduate success. Previously, 70% of a program’s trainees needed to achieve board certification within 5 years of graduation. The new requirement specified that at least 70% of eligible graduates pass the written (ADVANCED) examination on first attempt. A 2020 update stipulated that programs’ first pass attempt rate on both ADVANCED and APPLIED examinations be either >80% or higher than the bottom fifth percentile of residency programs in the specialty.3
This new requirement highlighted a longstanding disconnect between anesthesia residency programs, the RRC, and the ABA. At the time, the ABA’s Record of Training Information Database (RTID) provided only limited information about graduate performance. Programs could view individual trainee’s successes and failures on the BASIC and ADVANCED examinations as well as the percentage of graduates who had completed the board certification process. However, programs were not privy to results of the APPLIED examination. Programs had to obtain this information directly from their former trainees or by repeatedly searching the “Check Certification” portal on the ABA website.
In the ensuing years, the ABA leadership received copious feedback about this issue both directly and during question-and-answer sessions at multiple national meetings. In response, the ABA has recently introduced significant upgrades to the RTID system. The ABA now reports first-time exam pass rates for both the ADVANCED and APPLIED examinations. This information gives programs a better idea of how they are complying with the ACGME’s 80% requirement. This change represents an important first step, and we appreciate the ABA’s responsiveness to our feedback. However, we believe more needs to be done.
Currently, the RTID provides ADVANCED and APPLIED examination results for individual trainees. But the RTID database does not report pass/fail results for the individual components (SOE and OSCE) of the APPLIED examination, nor does it indicate whether a candidate attempted the APPLIED at all.
Why do programs need this information? Learning includes failure, but failure can yield a crisis of confidence, shame, and isolation. Mentorship in these moments may have a lasting impact on a young physician’s professional development, mental health, and overall well-being. Anesthesia residency program directors want to be that lifeline for graduates who struggle with the APPLIED examination, but we lack a direct feedback loop to trigger outreach. Our mentoring relationships with trainees do not end abruptly after we sign their case logs and hand them a diploma. With the ACGME’s encouragement, we assist, mentor, and advise graduates for years (and often decades) after they leave our institutions.
Privacy concerns thwarted previous efforts to release APPLIED examination results to programs, as graduates are no longer part of a training program when they take the exam. However, candidates also take the ADVANCED examination after graduation from training, but the ABA has been releasing these individual results for years. Since the ABA has found a way to release aggregate APPLIED examination results, it is unclear to us what the remaining barriers are to releasing the results of these separate components for individual candidates.
In summary, we are thankful for the ABA’s recent upgrades to the RTID. The ABA has shown that it has the tools and the platform needed to effectively deliver needed feedback to programs about trainee’s examination performance. We ask that the ABA take the next step and share individual residents’ performance on both the SOE and OSCE components of the APPLIED examination with the trainees’ respective residency programs. This data will help training programs identify and address deficits within their educational environment. More importantly, this information will help anesthesia residency programs reconnect with and assist graduates who encounter setbacks in their pathway to board certification.
Funding Statement
Funding and financial disclosures: None
Footnotes
Conflicts of interest: None
References
- 1.Wallace L, Trinka J. Bloomington, IN: iUniverse; 2007. A Legacy of 21st Century Leadership: A Guide for Creating a Climate of Leadership Throughout Your Organization. [Google Scholar]
- 2.Warner DO, Lien CA, Wang T, et al. First-year results of the American Board of Anesthesiology’s Objective Structured Clinical Examination for Initial Certification. Anesth Analg . 2020;131(5):1412–8. doi: 10.1213/ANE.0000000000005086. [DOI] [PubMed] [Google Scholar]
- 3.The Accreditation Council for Graduate Medical Education Anesthesiology program requirements and FAQs. https://www.acgme.org/specialties/anesthesiology/program-requirements-and-faqsand-applications/ Accessed April 25, 2023.