What Can a Pandemic Teach Us About Competency‐based Medical Education?

The COVID‐19 pandemic is disrupting educational and clinical environments, and in some regions the workforce may not be adequate to respond to the needs of the community. This, therefore, presents an opportunity for the medical education community to reconsider time‐based training and embrace a competency‐based progression to accelerate entry into the workforce. This commentary discusses undergraduate and graduate medical education response to workforce pressures of COVID‐19. On the one hand, some medical schools are moving toward competency‐based (early) graduation from medical school. On the other hand, residency programs have generally held to time‐based completion of training. In the context of this clinical and educational disruption, there are two challenges to CBME progression of trainees. The first challenge is whether there is trust in competency‐based assessment to permit time‐independent progression. The second involves a number of logistic issues to competency‐based progression.

COMMENTARY

For over a decade the medical education community has explored competency‐based physician training. ¹ Competency‐based medical education (CBME), organized around predefined abilities and outcomes, is intended to improve patient care through ensuring competent performance. CBME gained significant traction in the United States with the launch of the Accreditation Council on Graduate Medical Education (ACGME) Outcome Project in 1998. ² Each specialty has subsequently determined subcompetencies and developmental milestones for their residents. In medical schools there has been similar attention to outcomes‐driven models. While the Liaison Committee on Medical Education (LCME) has not mandated specific competencies, the standards require that each school set the outcomes and ensure that every graduate achieves them. ³

Typically, both undergraduate and graduate medical education mandate that trainees achieve competencies while adhering to a time‐based structure determined by the relevant accreditation organization (LCME or ACGME). For the Doctor of Medicine (MD) degree, the LCME mandates a minimum of 130 weeks and most schools require 4 years of training. For residency the duration of each specialty’s training is determined by the program requirements (ACGME) and the specialty board. For example, the duration of emergency medicine’s (EM) postgraduate training program is set by the American Board of Emergency Medicine (ABEM), requiring 46 weeks per year of training for either 3 or 4 years.

The COVID‐19 pandemic has resulted in numerous impacts on health care delivery and medical education training. In a time of greater accountability of the profession, medical educators must ensure that every graduate is prepared for practice with the needed knowledge, skills, and attitudes. In this article we argue that the pandemic, and perhaps more importantly, the medical education community’s response to the pandemic, has resulted in an additional unanticipated opportunity for the CBME movement. Namely, the pandemic has pulled back the curtain on our inherent trust and distrust in the CBME construct. We highlight how the CBME model could play to our advantage in responding to educational and workforce concerns during the current pandemic, but how the actions of the medical education community may point toward a lack of confidence in this model.

THEORY OF CBME

Training of physicians is a continuum with key transitions from medical school to residency to independent practice. Following the CBME model, these transitions should be based on achievement of competency. However, in 2010, Hodges ⁴ highlighted that the traditional model of medical education is “a time‐based (or ‘tea‐steeping’) model, in which the student ‘steeps’ in an educational program for a historically determined fixed time period to become a successful practitioner.” In contrast, the CBME model focuses on the functional capabilities of the trainee at the time of transition (the graduating medical student, resident, or independently practicing physician). ⁴ While there is some progress toward CBME, mostly through the clearer definitions of what is competent at each level of training and in each specialty, medical education for the most part remains time‐based.

HOW CBME COULD APPLY TO THE COVID‐19 PANDEMIC

It is during times of severe disruption, such as the current COVID‐19 pandemic, that there is pressure on the medical education system to create necessary change to address society’s pressing needs. This was the case during previous times of national need, such as World War II and the Vietnam War, when many medical schools shortened their curriculum to 3 years. ⁵ We are observing a similar situation in the present climate. In some areas, the actual and anticipated needs for patient care have overwhelmed the health care workforce. In response, there is a critical need to surge the workforce. This, therefore, presents an opportunity for the medical education community to reconsider the dogma of time‐based training and embrace a true CBME model. If learners are deemed competent for residency training or independent clinical practice, there would be obvious advantages to society if we could allow them to enter the workforce at an earlier time.

RESPONSE OF THE MEDICAL EDUCATION COMMUNITY

The COVID‐19 pandemic has created an opportunity for medical education leadership to consider the potential for learners at various training stages to make the transition into the next level of training or practice based on their competence. However, the various professional, accrediting, and certifying organizations’ response have been diverse. The responses highlight disparity in how CBME has been implemented, underscoring bureaucratic and logistic issues that prevent a true competency‐based model.

TRANSITION TO RESIDENCY

Early in the pandemic, the LCME advised ⁶ that “the medical school should review its educational program objectives (EPOs), the learning objectives of its required courses and clerkships, and required clinical experiences and skills. If students have met these requirements and been assessed on these required learning objectives, they may be eligible for early graduation. The school should confirm the eligibility of each student with its Student Advancement and Promotion Committee.” In other words, if the students met the competencies required by the school for graduation, they could be allowed to graduate even in advance of the traditional 4‐year timeline.

In response to the need for health care providers and with endorsement from the LCME, medical schools, especially around New York, announced that they would graduate students early. Schools reviewed their students’ academic performance and offered students the opportunity to graduate early if they were deemed competent. Medical school leaders were confident in their graduates’ readiness for residency. Although the window allowing a compressed timeline for training was brief (only about 2 months), this represented a paradigm shift that allowed senior medical students who met the preexisting competency outcomes to graduate, even though they had not completed the full duration of the curriculum. As a consequence, medical students have been able to enter the workforce either as interns or “junior physicians” in supervised settings. ⁷ The outcome of this experiment has yet to be determined.

This approach represented a major step forward for medical schools to shift from time‐based to competency‐based training. Unfortunately, the ACGME voiced significant concerns including inadequate orientation to residency, limited supervision, binding match commitments, uncertainty if the additional months would count toward the time‐based residency duration, and funding. Thus, in some health systems the newly graduated physicians entered the workforce but not as interns in their specific residency program. In other settings, some of them were able to start their specific residency program early.

TRANSITION TO INDEPENDENT PRACTICE

The same urgency to increase the workforce to meet the regional health care needs of caring for the surge of patients during this pandemic put pressure on residency programs to graduate trainees early and certify their ability to practice independently. The American Board of Medical Specialties (ABMS), responsible for physician certification, and the ACGME which accredits residency and fellowship training programs, hold a “commitment to the public to ensure that physicians practice medicine safely and efficaciously.” While they acknowledged the programs’ expertise and authority to determine readiness for unsupervised practice they noted that this “authority and judgement are especially important during times of crises when traditional time‐ and volume‐based educational standards may be challenged.” However, in contrast to the LCME, there was no pathway offered for transition from time‐dependent to competency‐based graduation from residency. Instead, leadership organizations including the ACGME, and in the case of EM, the Residency Review Committee and ABEM did not take steps to permit early completion of residency through competency‐based progression. ABEM mandates 46 weeks of training ⁸ per year and a COVID‐related guidance noted that program directors should allow residents to be quarantined for a short period of time without extension of training and allowed for exceptions to the 46 week per year on a case‐by‐case basis. The focus was on time in training and less so competency‐based progression.

The pandemic has severely disrupted training with closed clinical settings, severely decreased patient and surgical volumes. It is possible that some specialties may need to determine different means of determining ready‐for‐independent practice. For example, if a specialty requires a set number of cases or months in clinic, leadership may need to return to competency or entrustment models for program completion.

WHAT THESE CHALLENGES SHARE ABOUT OUR TRUST IN CBME

This disparity in approaches across professional organizations and accrediting and certification bodies is instructive. Medical educators have not fully embraced CBME and they do not trust in trainee progression using a competency‐based model. We think that there are several reasons that there was not a widespread movement to transition residents early to independent practice, which point toward two inherent gaps in the CBME model in practice.

The first gap is assessment. The ACGME and ABMS noted in reference to COVID that

PDs [program directors] and CCCs [clinical competency committees] have the ability to assess trainee readiness for unsupervised practice in a variety of different ways, including utilization of Milestone data, Entrustable Professional Activities (EPAs), and review of data from a variety of assessment methods (in‐training examinations, clinical evaluations, case/procedural logs, multi‐source feedback, direct observation in real or simulated situations, etc.). While the types of competency assessments may vary across specialty, program, and institution, the ABMS and ACGME do expect programs to use rigorous, valid, and reliable combinations of assessments that are appropriate within each specialty. ⁹

Despite the fact that EM residency programs report over 300,000 milestone ratings each year to the ACGME (23 milestones twice per year for each of more than 7,000 residents), it is clear that there is a perceived lack of validity, reliability, or trustworthiness in that process. ¹⁰ , ¹¹ , ¹² , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ It is not clear that residency programs, if given the green light, would have had sufficient evidence or feel comfortable to make the high‐stakes summative competency decision to allow residents to graduate and enter independent practice.

The second gap is the logistic challenge posed by CBME. Time‐dependent progression is practical and predictable. Program directors know when residents will graduate and they can, for the most part, plan for the ED workforce to care for patients. While there are minor disruptions due to leaves of absence for medical reasons or pregnancy, program directors can depend on a set number of work‐months to run the ED. If residency programs utilized competency‐based progression, ED staffing challenges would require modification. In particular, it would mean the loss of advanced residents who are relied on for teaching, efficient disposition of patients, and care of the more critically ill patients. Further, the funding for residents is fixed in a time‐based model so movement to competency‐based progression would require a revision in the current funding model.

WHAT THESE GAPS MEAN FOR THE FUTURE OF CBME

Dr. Whelan of the Association of American Medical Colleges noted that “the COVID‐19 pandemic is dominating our educational and clinical environments and is now the biggest disruptor.” ¹⁷ When given the opportunity and necessity to transition to a CBME model of training to rapidly increase the workforce, for the most part, the medical education collective did not seize the moment. This illustrates a clear challenge to realization of CBME and is a call to action.

While substantial time has been spent investing in competency‐based paradigms such as milestones, when given the opportunity during the disruption, the field was not ready to act. Dr. Nasca, from the ACGME, noted that “the concepts of competency‐based medical education that have been introduced over the past 7 years position us favorably to deal with individual decisions that program directors will face as we emerge from the first phase of this crisis.” ¹⁸ So time may come. First and foremost, EM needs to invest time, energy, resources, intellectual capital in designing, testing, and implementing systematic programs of assessment to ensure competent transition to unsupervised practice. ¹⁹ , ²⁰ Educators need to trust the programmatic assessment data to be able to measure competency. There are ongoing efforts—the ACGME is revising EM milestones ²¹ and there is a project looking at EM Entrustable Professional Activities. ²² These will provide the opportunity to reengage with methods of assessment, building on the lessons learned and the increasing expertise of the educators in EM and beyond. There will need to be careful research to collect validity evidence for instruments used in assessment as well as summative judgments rendered. Importantly, exploring more distant outcomes such as patient outcomes research will be needed. It is only through good assessments that we will achieve the outcome we desire—trustworthy assessments to ensure competency to practice and ensure the safety of patients and society.

The pragmatic barriers are more difficult. They require working with Centers for Medicare & Medicaid Services (for funding), ACGME (for accreditation), and programs to manage the variable transition points. It may be helpful to work within the specialty, but ultimately it will require collaboration across organizations such as the Coalition for Physician Accountability, which is 12 physician organizations with the mission “to advance healthcare and promote professional accountability by improving the quality, efficiency, and continuity of the education, training, and assessment of physicians.” ²³ By working together, seamless competency‐based transitions can be actualized. It will be important to provide a feedback loop, to ensure that the graduates are indeed competent and ready for practice.

In summary, we hope that EM educators take this opportunity to lead the field first in rigorous assessments supported by validity evidence. The next step is to build programs of assessment that lead to trustworthy summative decisions to advance progress of residents in residency and into independent practice. Meanwhile EM and other educational leaders will need to work with organizations such as the ACGME, ABMS, as well as funding sources (Centers for Medicare & Medicaid Services) to address the logistic and funding issues. Through addressing these issues, we can further CBME in to a practical reality.

AEM Education and Training 2020;4:316–320