Milestones 2.0: A Step Forward

Development of the Initial Milestones

A key component of the Next Accreditation System are the Educational Milestones for all accredited residency and fellowship programs. Use of the milestones in trainee assessment began in 2013, as a key element of the Accreditation Council for Graduate Medical Education's (ACGME's) Next Accreditation System.¹ Milestones are used in resident and fellow outcomes-based assessment based on the 6 general competencies: medical knowledge (MK), patient care (PC), interpersonal and communication skills (ICS), practice-based learning and improvement (PBLI), professionalism (PROF), and systems-based practice (SBP). The 6 competencies were introduced by the ACGME and the American Board of Medical Specialties (ABMS) in 1999.

While the competencies were crafted with the intent to provide a shared model of professional development and advance assessment of physicians in training, their implementation faced challenges, as programs, faculty, and trainees lacked an understanding of the meaning of the competencies in the context of their specialty. In response, content experts from each specialty created subcompetencies, using narrative milestones to provide more specific and developmental descriptions of the general competencies. This included representatives from the ACGME Residency Review Committees, ABMS certification boards, program director groups, specialty societies, and residents and fellows. These volunteers collectively gave more than 5000 hours of their time to develop the specialty-specific milestones. These milestones were intended “to create a logical trajectory of professional development in essential elements of competency” and provided a “measurable framework of specialty-specific outcomes.”¹ When the milestones were first developed, each specialty had the flexibility to identify subcompetencies and write associated developmental milestones. Specialties reviewed program requirements, certification examination outlines and blueprints, curricula, national competency statements, the literature, and results from national consensus-building exercises to guide their process.²

Limitations of the Initial Milestones

The resulting specialty-specific milestones demonstrate substantial variability both with respect to content and how the developmental progression is operationalized across milestone levels.³ In 2013, the ACGME recognized the implementation of the milestones would be an iterative process, and that a periodic review and revision of the milestones would be beneficial, as experience and research evidence accrued. The ACGME made a commitment to begin this task within 3 to 5 years of Phase 1 implementation, as a part of a continuous quality improvement process. Over the course of the first 3 years of implementation, ACGME Milestone staff attended more than 200 program director meetings, focus groups, and visits to accredited programs and sponsoring institutions to gather feedback about the initial experience with milestones. These meetings helped to outline the revision process for Milestones 2.0.

Additional concerns were heard through various channels such as focus groups, interviews, and at the 2nd ACGME Milestones Summit in December 2016, where representatives from all ACGME core specialties were present. Stakeholders expressed dissatisfaction with the inconsistencies in the milestones and subcompetencies.⁴ In fact, these differences were noted to have “complicated efforts to share assessment tools across programs, and provide comprehensive faculty development across specialties.”⁴ These differences may hamper, rather than encourage, collaboration in assessment and faculty development activities across specialties.

Development of the 2.0 Milestones

In 2016, the ACGME Department of Milestone Development and Evaluation formalized the plan for the review and revision process. This work began with a review of data collected from the field, the data submitted for the twice-yearly milestone reporting, and learning from quantitative and qualitative research. To date, 40 national and multi-institutional studies about the milestones have been completed, and a bibliography of milestone research is available on the ACGME website.⁵

The quantitative research demonstrated validity evidence for key components of the milestones in several specialties.⁵ Qualitative research has focused on program directors' and trainees' experience with the milestones, Clinical Competency Committee processes, the value of milestone-based feedback, and milestone design. A thematic analysis of the milestones for the non-PC and non-MK domains, completed in preparation for the revision process, demonstrated wide variation among specialties. The analysis showed that across the 26 core specialties and the transitional year there were more than 230 different ways of describing PROF, 171 for PBLI, 176 for ICS, and 122 for SBP.⁴

Lessons from early research on the milestones has informed the milestone revision process. However, it is important to recognize that much has remained the same. Changes can be grouped into 4 categories: (1) selecting the milestone development groups; (2) reducing milestone complexity; (3) enhancing community engagement, including participation by public members; and (4) providing additional tools and resources for programs and sponsoring institutions.

Selecting the Milestone 2.0 Development Groups

There were many concerns regarding the differences in the non-PC and non-MK milestone content across specialties, prompting the ACGME to convene 4 groups to develop cross-specialty “harmonized” milestones for ICS, PBLI, PROF, and SBP. The 4 groups consisted of content experts, directors, interprofessional team members, and other faculty. Each group developed 2 to 3 subcompetencies that were applicable to all specialties and subspecialties. Stakeholders across the specialties will be asked to edit the language, as needed. In 2017, the harmonized milestones were made available for public review and comment.

The specialty-specific development groups for Milestones 2.0 include representatives from key stakeholder groups involved in initial milestone development. In addition, the groups include representatives from the osteopathic community and public members. The public members serve a vital role in ensuring that the Milestones 2.0 developers consider the quality and safety of patient care, as well as provide essential input into the non-PC, non-MK competencies. Additional volunteers represent 2 other stakeholders. Specialties that have direct-entry from medical school (eg, internal medicine, pediatrics, surgery) now include a representative from the Association of American Medical Colleges and the American Association of Colleges of Osteopathic Medicine to ensure that the link between undergraduate and graduate medical education is considered in milestone development. Finally, each specialty will hold a “Call for Volunteers” that is open to all members of the community, with the plan to select 5 individuals from those who apply. Of the specialties that have used this process to date, most of the volunteers have been junior and midcareer faculty, representing diverse program sizes, types, and locations.

Box Subcompetencies for the Harmonized Milestones.

Interpersonal and Communication Skills (ICS)

• Patient- and Family-Centered Communication (ICS-1)

• Interprofessional and Team Communication (ICS-2)

• Communication Within Healthcare Systems (ICS-3)

Practice-Based Learning and Improvement (PBLI)

• Evidence-Based and Informed Practice (PBLI-1)

• Reflective Practice and Commitment to Personal Growth (PBLI-2)

Professionalism (PROF)

• Professional Behavior and Ethical Principles (PROF-1)

• Accountability/Conscientiousness (PROF-2)

• Self-Awareness and Help-Seeking (PROF-3)

Systems-Based Practice (SBP)

• Patient Safety and Quality Improvement (SPB-1)

• System Navigation for Patient-Centered Care (SBP-2)

• The Physician's Role in Healthcare Systems (SBP-3)

Reducing Milestone Complexity

Another area in need of change was the language used to describe the milestone developmental levels, with complaints about the length of individual milestones (the number of words or number of subcompetencies) and the complexity of the language. During the revision process, it will be critical to examine the language of each milestone, to ensure it is clear and easy to understand (ie, removal of education jargon).

The harmonized subcompetencies for ICS, PBLI, PROF, and SBP are shown in the Box. Detailed information on the approaches each group took to develop them, are presented in 4 summaries published as online supplemental material.

Enhancing Community Engagement in Refining the 2.0 Milestones

Engaging with the specialty communities is vital to the refinement of the milestones. The first level of engagement is the Call for Volunteers. We will continue open communication with program directors by surveying them as part of the review and revision process, to examine their opinion of the current PC and MK milestones. Following development of the revised milestones, program directors, faculty, residents, and others will have the opportunity to review and comment on the draft content. We are able to engage with the patient community through the inclusion of public members on the development groups, maximizing the importance of patient perspective in quality, safety, communication, and patient-centeredness. These members have already demonstrated the ability to strongly advocate for the needs of the patient.

Providing Additional Tools and Resources

To ensure that programs have the resources available to aid in implementation of the revised milestones, a Supplemental Guide will be created for each specialty to provide insights into the intent of the subcompetencies, with examples for each level, sample assessment methods, and other available resources. Additionally, an Implementation Guidebook that highlights planning, change management, and continuous quality improvement will be available in the summer of 2018. Review and revision of the milestones have begun, and there are 15 specialties in various stages of development, with more starting soon. All specialties and subspecialties will have initiated the process by the end of 2020.

Much has been learned about the milestones over the past 5 years. The framework will remain the same, but version 2.0 will provide critical improvements. As the milestone revision process moves forward, we invite the specialty communities to be active partners, to ensure that the milestones are an effective tool for the assessment of residents and fellows.