Trainee Perspectives on the Writing and Implementation of Milestones 2.0

Given the importance of involving trainees in program development and assessment, the Accreditation Council for Graduate Medical Education (ACGME) sought to include residents and fellows in the development of Milestones 2.0. Herein, we provide the trainee perspective on the participation, writing, and implementation of Milestones 2.0.

Formation of Milestones 2.0

In developing Milestones 2.0, the ACGME viewed trainees as a key stakeholder group for the Milestones and critical to include in the development process.¹ Thus, in contrast with Milestones 1.0, which did not include residents or fellows, at least one trainee was asked to join each specialty's Milestones 2.0 work group and participate in all meetings for the discussion, review, and writing. Trainees were included through a variety of pathways, ranging from an open call for applications to appointment by the specialty residency review committee. As trainees, we had the unique experience of being involved with our specialty's Milestoness 2.0 work group (Box).

Box Milestones 2.0 Work Group: In the Room Where It Happened

Quotes from the authors on their individual experiences in the Milestones 2.0 work group:

• “After getting acquainted with the process, development, and expected implementation, the view of a training program completely shifted for me toward the view of progressive growth and continual education. Most importantly, I was treated as an equal, which is unlike many other trainee positions in societies or committees.”

• “The resident members in my work group had the opportunity to advocate for trainees within the Milestones. We felt our perspectives informed a number of important issues related to the Milestones including the lived experiences of trainees, bias in assessment, and growth mindset.”

• “Having been a member of 2 stakeholder groups as a junior program director and a recent fellow, I felt very comfortable expressing my own opinion and convincing senior program leaders about the importance of trainees and junior physicians in this Milestones 2.0 writing.”

• “I appreciated being able to voice my experiences with current Milestones and ways in which they could be better utilized/incorporated into the training process. It was clear that incorporating the trainee viewpoint and experience into Milestones 2.0 was a priority and I appreciated that my input was valued.”

Reflections on the Process

Involving trainees in the Milestones process allowed learners to have a seat at the table when writing Milestones 2.0. Representatives from accrediting bodies, specialty societies, institutional leaders, program directors, and public members have important, but different, viewpoints than a learner. First, trainees could provide feedback on the language used for the Milestones, ensuring that it is user-friendly and understandable without preexisting knowledge of medical education terminology. Second, trainees offered reflections on how the Milestones may influence their peers. Some examples included assessing the Milestones for whether or not they took into account the lived experiences of residents and fellows, integrating a growth mindset into the phrasing of subcompetencies, and attempting to address forms of racial or gender bias in assessment(s) resulting from the Milestones. A key place to address learner experience, growth mindset, and bias in assessment was in the Milestones 2.0 supplemental guides, which offered descriptions of learner examples, resources for assessment, and additional references. Third, as current or recent trainees, we were able to verify that the shared mental model of increasing Milestones levels matched our experiences and understanding of progression during training. Lastly, we sought to provide input in the writing process so that Milestones 2.0 could be best used for self-assessment, coaching, and growth.

Lessons Learned From Implementing Milestones 1.0

There was great variability in how programs oriented learners to the Milestones framework. Often, trainees do not have a strong grasp on the 6 core competencies, let alone the subcompetencies or Milestones. While not every learner needs to be an expert in competency-based medical education (CBME), programs are encouraged to have greater transparency in the assessment process through educational initiatives. This would include education on how assessments are performed, how the clinical competency committee interprets these assessments for Milestones evaluation, and how these scores are transmitted centrally to the ACGME. This ensures trainees understand the standards by which graduation, promotion, remediation, and termination decisions are made. Too often it is only in cases of remediation that a more intensive look at the Milestones occurs.

From our own experiences as trainees, we suggest education on the Milestones should also involve teaching a growth mindset. Indeed, undergraduate medical education often emphasizes summative assessments such as final grades or high-stakes certification examinations in a way that can lead to a fixed mindset. This fixed mindset means that learners assessed as Level 1 may feel they have failed, hindering their ability to self-assess and work toward improvement. Trainees may feel an expectation to score a 4 out of 5 on a Likert scale from the beginning of their training, but assessments based on Milestones are instead designed to be more descriptive and promote growth. In addition, those approaching graduation or in independent practice may find that Level 5 skills are achievable in their early post-training career; thus, the Milestones can also be used for ongoing development. The Milestones framework may offer a growth mindset approach that, when paired with frequent coaching, can promote lifelong learning and progression toward mastery.

We suggest more robust faculty development to train educators, coaches, and advisors on the use of Milestones. One example of this was the phenomenon of straight-line scoring, which makes summative Milestones reporting less useful for trainees. Straight-line scoring is the practice in which a trainee is rated the same score throughout, often in-line with their respective postgraduate year. Literature has suggested this practice is common and leads to lost assessment opportunities to provide trainees feedback on their abilities.^2,3

Following the initial creation of the Milestones, programs large and small were tasked with writing new assessment tools resulting in substantial changes to resident and fellow evaluations. Unfortunately, residents and fellows were at times left out of the process. Just as the Milestones 2.0 writing process was enriched by the learner perspective, we strongly believe that incorporation of the trainee stakeholder in the creation of new assessment tools is critical to their success.

Suggestions for Implementing Milestones 2.0 for Residency and Fellowship Programs

Successful implementation of Milestones 2.0 for trainees requires greater education on their purpose as well as involving learners in the development of Milestones-based assessments. We suggest that institutional graduate medical education offices and individual training programs provide instruction during orientation regarding the Milestones. Ensuring early trainee engagement will add value to the process of implementing principles of CBME. Suggested educational materials, including a handbook, slide set, and one-page summary, were put together by authors and made available by the ACGME online.⁴ Revisiting these resources and particularly prior to semiannual reviews may assist in making Milestones-based systems more useful. Indeed, we recommend trainees visit the Milestones twice a year to use for self-assessment and self-reflection.

The Milestones have had a real impact on every stakeholder; however, regarding informed decisions for graduation, promotion, remediation, and termination, the learner is at the center of such consequences. We hope that inclusion of trainee stakeholders in the writing and implementation processes will lead to the success and acceptance of Milestones 2.0 across graduate medical education.