Introduction to the Milestones 2.0: Assessment, Implementation, and Clinical Competency Committees Supplement

Forty years have passed since the metamorphosis of the Accreditation Council for Graduate Medical Education (ACGME) in 1981 from its prior entity, the Liaison Committee on Graduate Medical Education (GME). Over the past 2 decades, the ACGME has transitioned from an emphasis on structure and educational processes to a focus on defining and measuring educational outcomes.¹ The transformation to competency-based medical education (CBME) was bolstered in 2013 with the Next Accreditation System,² which created the new tools of Milestones and clinical competency committees (CCCs). ACGME International (ACGME-I) was developed in 2009 in response to requests from other countries to transform their GME structure and improve educational outcomes. A decade later, ACGME-I accredits 19 sponsoring Institutions, 108 residencies, and 55 fellowships across 11 countries. CCCs and Milestones were introduced early in the international arena, and ACGME-I provided some of the earliest proofs of concept for both Milestones and CCCs.³

Milestones are narrative descriptors of a resident or fellow's developmental progression along an expected trajectory within each competency domain, typically from novice to aspirational levels. Although Milestones are not, themselves, “assessments,” they are valuable for framing feedback to GME learners. Milestones are not comprehensive as regards to “all” a physician needs to be, know, and do, but serve as building blocks upon which relevant curriculum and evaluation methods can be developed. The inaugural Milestones were developed independently by each specialty resulting in more than 150 Milestone sets.⁴ This siloed approach led to considerable variability in the number of Milestones and how each domain was characterized. Although this was necessary for some specialty-specific patient care and medical knowledge Milestones, many believed that standardization of Milestones across other competency domains would be highly desirable. Greater consistency would enable educators to collaborate as a learning community, primarily through the development and dissemination of common assessment tools and sharing of lessons learned. Further, professional development of faculty, residents, and coordinators could be enriched through diverse collaborative partnerships. Dr. Laura Edgar of the ACGME led the Milestones 2.0 initiative with the goal of “harmonizing” the Milestones.⁵

CCCs are faculty committees appointed within each GME program to regularly synthesize the program's available assessment data on resident performance. Committee members develop a shared mental model of how the Milestones are taught and assessed within their individual program. CCCs use the available assessment data to evaluate each GME learner's attainment of the Milestones and provide their recommendations on resident performance to the program director. The CCC Milestones synthesis is as much a measure about the program's performance as it is about the performance of the individual resident or fellow. Importantly, it provides formative feedback for the program and identifies areas for improvement in curriculum assessment practices and the learning environment. CCC findings can be of immense value to programs' program evaluation committees as they engage in an annual opportunity for continuous quality educational improvement. Similarly, sponsoring institutions can use Milestone data from each GME program to provide a snapshot of the program and overall institutional educational effectiveness.

In developing Milestones 2.0, the ACGME has partnered extensively with the GME community. The ACGME worked with institutional leaders, program directors, faculty, residents/fellows, coordinators, and administrators to co-create Milestones awareness and understanding. In this effort the ACGME has utilized a variety of strategies, including live workshops at its Annual Educational Conferences, webinars, and a series of written materials available at the ACGME website. These materials include the Assessment Guidebook, the Clinical Competency Committee Guidebook (3rd ed), the Milestones Guidebook (2nd ed), The Milestones Guidebook for Residents and Fellows (2nd ed), and the Milestones Implementation Guidebook. The ACGME, end-users, and ACGME-GME community partnerships have contributed considerable scholarship to the field. A recent Google Scholar search generated over 1800 references to CCCs and over 2700 to GME Milestones. The ACGME's Journal of Graduate Medical Education (JGME), along with other medical education journals, has regularly published contributions to this field of study.

The purpose of this JGME supplement is to provide high-quality, concise, up-to-date resources for the GME community. It features a review, a special article, an original research study, and 17 perspectives articles; a related Rip Out article and editorial are included in the April issue of JGME. Nearly 60 authors have contributed, including fellows, educators, researchers, coordinators, the public, and other physician faculty. Some of these resources provide important context and perspectives.^6–8 Other content is newly developed: the importance of co-production by GME learners and faculty,⁹ the perspectives of the Review Committees' public members,¹⁰ and a description of the validity framework that provides the theoretical basis underpinning the use of Milestones and professional judgment.¹¹ Some articles provide “just-in-time” learning: describing how to start a CCC,¹² creating shared mental models among CCC members,¹³ using best practices on effective group function to optimize CCC meetings,¹⁴ distinguishing purposes for which to use and “don't use” Milestones,¹⁵ and intentionally integrating Milestones into feedback and coaching.¹⁶ When available, evidence from the literature has been analyzed and summarized to disseminate innovative practices and to answer questions frequently raised by colleagues–both new and seasoned. Some articles, such as the descriptions from the workgroups on Milestones 2.0,^17–20 help explain “how we got here” and place new Milestones wording and concepts into better perspective. Others, such as the legal article,²¹ are designed to help the community proactively avoid harm. Several opportunities for faculty professional development are highlighted at the level of individual programs, institutions, and faculty.^22–24 The utility of Milestones for program learning and improvement is explored as well.^25,26 Finally, generalizable findings from work across programs and specialties is presented.²⁷

We acknowledge that this body of work, though intended to be comprehensive, has limitations. The residents who trained in the 7 early adopter specialties in which Milestones were reported in 2013–2014 are only a few years post-training and many are still in GME programs. There has not yet been adequate time to understand how experiences with Milestones and CCCs have shaped graduates' professional identity formation, affected their skills or attainment of competence, or affected their care of patients and populations. During this time period, 13 months (and counting), the COVID-19 pandemic has resulted in alterations of patient volumes and clinical activities. Furthermore, the ACGME recently completed the successful transition to a single accreditation system; we acknowledge that we have much to learn from our osteopathic colleagues through their implementation of Milestones and CCCs. We increasingly recognize the consequences of bias in medical education. Since both conscious and unconscious bias can distort assessment, adversely impact CCC judgment, and threaten the value of Milestones interpretation for the resident, the program, and the broader GME community, we look forward to future work to inform our responses in how to mitigate it.

Finally, the GME community is passionate and hardwired for continual learning and improvement. We have every confidence that we will discover better ways to assess learning, learners, and the learning environment: our patients and communities depend on it.

Our hope is that you find at least one article in this supplement that piques your curiosity, clears up a point of confusion, motivates you to share your own best practice, or inspires you to advance outcomes-based medical education scholarship to bring better answers to us all.