Use and Potential Misuse of Milestones

The Challenge

By 2001, the Accreditation Council for Graduate Medical Education (ACGME) 6 core competencies were established, which introduced graduate medical education's (GME's) transition to competency-based medical education (CBME).¹ Competencies are teachable attributes that residents and fellows must develop in order to carry out professional roles. Milestones describe the developmental pathway for achieving each competency in more than 150 GME specialtes and subspecialties. Yet confusion remains about their optimal use as well as misuse, which may produce adverse consequences for all stakeholders (eg, residents, faculty, programs, sponsoring institutions [SIs]).

Rip Out Action Items

1. Review current specialty-specific Milestones.

2. Understand how they are operationalized in the program's curricula and assessments.

3. Analyze opportunities for revised curricula/assessment tools to f ill any program gaps.

4. Implement and periodically review policies and procedures so that Milestones are never used for functions for which they are not intended, such as rotation or experience assessments or resident/fellow summative assessments.

What Is Known

Milestones are narrative statements that describe skills, knowledge, and behaviors for performance in each of the 6 core competency domains and are arranged sequentially from attributes of novices to attributes that are aspirational.² Milestone-based assessments enable faculty to provide GME learners specific formative feedback and improvement goals through the defined progression of levels toward expertise.³ Aggregated Milestone data from within and across programs can facilitate program, SI, and review committee (RC) learning and improvement.

Milestones themselves are not assessments. They are constructs used by the GME community to build assessments. As constructs, Milestone assessments are open to interpretation and may be affected by context and other factors such as the skill and biases of assessors. These features can be mitigated during formative feedback sessions, but they make Milestone-based assessments inappropriate for high-stakes summative assessments of learning or competence. Thus, Milestone assessments should never be shared with certifying, credentialing or licensing boards or agencies, or potential employers, or provided as evidence of competence for legal purposes.⁴ When used for these unintended purposes, faculty may be reluctant to give honest assessments and clinical competency committees (CCCs) may be tempted to inflate Milestone judgements. Milestone security can facilitate the integrity of the assessment processes.

How You Can Start TODAY

1. Orient the program director, teaching faculty, residents/fellows, and program coordinator/administrator(s) to the current specialty-specific Milestones, how they are operationalized through the program's curricula and assessment tools, and specific (mis)uses.

2. Implement an annual CCC orientation program that defines the accepted and potential misuses of Milestones and Milestone-based assessments, to include emphasizing what they should not be used for:

   • “Cut and paste” end-of-rotation assessments

   • Sole summative remediation or advancement decisions

   • Assessment for a credentialing entity (eg, to seek hospital privileges)

   • Assessment for a potential employer, certifying specialty, or subspecialty board

   • Malpractice litigation (inferring competence of a resident/fellow involved in a malpractice suit)

3. Review program and SI policies that ensure the security of individual Milestone assessments.

What You Can Do LONG TERM

1. Encourage direct feedback: Milestone-based assessments are formative, not a summative “grade.” Use Milestones to initiate interactive feedback/coaching conversations on goals. Cultivate ongoing faculty and learner development, in a culture of supportive assessment and feedback.^2,5

2. Utilize the learning trajectory: Use other data sources with serial Milestone assessments to codevelop individualized learning plans, and promote learner self-confidence and self-efficacy. Use longitudinal specialty Milestone data to estimate the likelihood a Milestone rating will fall below an acceptable level at program completion. This learning analytic approach can predict residents with concerning performance earlier.²

3. Incorporate Milestones into program evaluation and share lessons learned: Incorporate aggregate and serial program Milestone data with national benchmarks in your program evaluation plan, to monitor overall curriculum and the effect of curricular changes. Share lessons learned with other programs.

4. Ensure ongoing security: Milestone assessments are aggregated when reviewed by the ACGME and their RCs. The ACGME uses aggregate specialty or program Milestones, as the “unit of analysis” in its reports. This prevents the misuse of Milestone assessment data as indicators of overall competence of an individual resident or fellow, and further ensures the security of individual Milestone data.

Table.

Comparing What Milestones “Are” and “Are Not”

What Milestones Are3	What Milestones Are Not3
Description of core competencies of a discipline or specialty	The totality of a discipline
Roadmaps for professional development	Inclusive of all essentials for autonomous medical practice
Guide to identify gaps in the curriculum	A complete representation of knowledge, skills, and behaviors that a specialty or subspecialty requires
Constructs to support a comprehensive assessment program including focused feedback	A free-standing “assessment tool” for a rotation or experience
Tools to drive improvement (learners, programs, RCs)	Tools to accredit a program
Framework for CCCs to synthesize performance data and make judgments about learner performance	High-stakes summative assessments to determine trainee progression
Constructs to help to identify trainees for context-specific, formative interventions	An exhaustive list of attributes sufficient for high-stakes summative decisions

Abbreviations: RC, review committee; CCC, clinical competency committee.