Introducing the Neurology Milestones

Introduction

In the Next Accreditation System (NAS) of the Accreditation Council for Graduate Medical Education (ACGME), 1 measure of the educational effectiveness of neurology residency programs will be residents' attainment of specific educational Milestones during their training. These Neurology Milestones detail neurology-specific and general clinical competency domains that are relevant to neurological training and the progression toward competence in the independent practice of neurology.

Milestone Development History

In 2011, the Neurology Residency Review Committee (RRC) of the ACGME and the American Board of Psychiatry and Neurology (ABPN) began the task of developing the Milestones for neurology at the request of the ACGME. The ACGME Milestone development process included the establishment of a Neurology Milestone Working Group, charged with creating neurology-specific educational Milestones, and a smaller Neurology Milestone Advisory Group, charged to provide feedback on the Milestones and to assist in their dissemination (box 1). The composition of the working group included representation from the ACGME, with members appointed by the Neurology RRC (then under the chairmanship of John Engstrom, MD); the ABPN, with members appointed by Larry Faulkner, MD, its president and CEO; and the Consortium of Neurology Program Directors (CNPD) of the American Academy of Neurology (AAN), with members who had volunteered for this role. The Milestone Advisory Group included appointees from the ACGME and the ABPN, and both the working group and the advisory group included a resident or fellow member. The members of the working group represented a variety of the major subspecialties of neurology (including general neurology), as well as geographic diversity and different types of teaching institutions. Steven L. Lewis, MD, then the incoming RRC vice-chair, was appointed to chair the Neurology Milestone Working Group and the Advisory Group.

Box 1 The Neurology Milestone Working and Advisory Groups

Neurology Milestone Working Group

Steven L. Lewis, MD, Rush University Medical Center, Chair

Colum Amory, MD, MPH, Albany Medical College

Chaouki Khoury, MD, Baylor Health Care System

Shannon M. Kilgore, MD, VA Palo Alto/Stanford University School of Medicine

Octavia Kincaid, MD, University of Illinois at Chicago Medical Center

Tracey Milligan, MD, Brigham and Women's Hospital

Amar Dhand, MD, DPhil, University of California, San Francisco

Jonathan P. Hosey, MD, Geisinger Health System

Ralph F. Józefowicz, MD, University of Rochester School of Medicine

Joseph Kass, MD, Baylor College of Medicine

Noor Pirzada, MD, University of Toledo College of Medicine

Sonja Potrebic, MD, PhD, Kaiser Permanente Los Angeles Medical Center

Patrick Reynolds, MD, Wake Forest College of Medicine

David Spencer, MD, Oregon Health & Science University

Laura Edgar, EdD, CAE, Accreditation Council for Graduate Medical Education (ACGME)

Louise King, MS, ACGME

Neurology Milestone Advisory Group

Steven L. Lewis, MD, Rush University Medical Center, Chair

Patricia Crumrine, MD, Children's Hospital of Pittsburgh

Larry Faulkner, MD, American Board of Psychiatry & Neurology

Janice M. Massey, MD, Duke University School of Medicine

Meredith Runke, MD, Indiana University School of Medicine

John W. Engstrom, MD, University of California, San Francisco

Timothy Brigham, MDiv, PhD, ACGME

The first meeting of the working group was held in October 2011. The Milestone development process started with a blank slate, given the absence of a pre-existing nationally developed neurology residency clinical curriculum (beyond basic program specifications for minimum durations of inpatient and outpatient training). Additionally, there was an absence of previously defined expectations for clinical or procedural expertise of graduates of neurology residency training programs. The priority of the first meeting of the working group was therefore to define a practical, relevant, and manageable (but nontrivial) set of Milestones for the domains of patient care, medical knowledge, and procedural competence appropriate to neurology training. The second priority was to determine clear levels of competency development within these domains, with the progression ultimately leading to competence to independently practice neurology. The working group members collaborated to provide expert consensus to the development of the Neurology Milestones.

The second meeting of the working group took place February 2012 and allowed the group to work with the framework it had established, and to add several Patient Care and Medical Knowledge Milestones. In addition, the group began to adapt Milestones from the 4 remaining ACGME competencies (Interpersonal and Communication Skills, Professionalism, Practice-Based Learning and Improvement, and Systems-Based Practice) as relevant to training and practice in neurology.

The draft that resulted from these meetings was presented at the CNPD course and business meeting at the April 2012 AAN Annual Meeting for feedback from the CNPD members.¹ Subsequently, members of the working group and other CNPD members participated in a pilot of Milestone assessment of their residents. While the specific Milestones were generally considered favorably, 1 concern was the amount of faculty time needed to complete each resident's Milestone assessment.

The advisory group met in October 2012 to review the Milestones and the results of the pilot prior to the final meeting of the working group. The advisory group recommended removing ambiguous language from some Milestone domains, and creating parallel descriptions across domains as much as possible. The advisory group also underscored the importance of communicating to program directors and the public that the ultimate purpose of the Milestones was not to evaluate all competencies required of graduating neurologists, but rather to document a crucial subset (ie, a sampling) of those competencies. Finally, the advisory group emphasized the importance of educating the faculty to correctly assess the Milestones.

The working group's final meeting took place in November 2012, and entailed discussion of the advisory group's suggestions, results of the Milestone pilot, feedback from the CNPD, and additional comments from the AAN. This input was used to further refine and simplify the Milestones as well as to clarify language to make the implementation as seamless as possible. Several additional Milestones that the group agreed were important were also incorporated.

As a result of feedback from the pilot project, the working group determined that assessing all of the Milestone domains for each resident every 6 months during residency would be impractical, in part because of significant variation among residencies in the specific clinical rotations and the postgraduate year in which they occur. Although the group agreed that a subset of domains (eg, the Global Domains and the General Competencies) should be assessed for each resident at the prescribed ACGME-mandated frequency (ie, every 6 months), the group felt that some of the Disease Category and Technical/Procedural domains could be assessed less frequently. Hence, the group recommended incorporation of a “Not yet rotated” check box, indicating that a given resident did not yet have enough experience for a meaningful assessment in certain domains.

General Features of the Neurology Milestones

The neurology-specific Patient Care and Medical Knowledge Milestone domains (box 2) include 5 “global” domains (straddling both Patient Care and Medical Knowledge), which the working group felt represented the underpinning of the practice of neurology, 11 “subspecialty” (Patient Care) domains comprising major disease categories, and 4 “technical/procedural” (Patient Care) domains critical to the practice of clinical neurology. The Milestone domains for Professionalism, Interpersonal and Communication Skills, Practice-Based Learning and Improvement, and Systems-Based Practice, which the working group adapted from Milestones developed by an ACGME expert panel, are listed in box 3.

Box 2 The Neurology Milestones: The Patient Care and Medical Knowledge Domains (the related Accreditation Council for Graduate Medical Education general competency is in parentheses)^a

Global Domains

History (Patient Care)

Neurologic Examination (Patient Care)

Management/Treatment (Patient Care)

Formulation (Medical Knowledge)

Diagnostic Investigation (Medical Knowledge)

Disease-Type (“Subspecialty”) Domains

Movement Disorders (Patient Care)

Neuromuscular Disorders (Patient Care)

Cerebrovascular Disorders (Patient Care)

Cognitive/Behavioral Disorders (Patient Care)

Demyelinating Disorders (Patient Care)

Epilepsy (Patient Care)

Headache Syndromes (Patient Care)

Neurologic Manifestations of Systemic Disease (Patient Care)

Child Neurology for the Adult Neurologist (Patient Care)

Neuro-Oncology (Patient Care)

Psychiatry for the Adult Neurologist (Patient Care)

Technical/Procedural Domains

Neuroimaging (Patient Care)

Electroencephalography (Patient Care)

Nerve Conduction Studies/Electromyography (Patient Care)

Lumbar Puncture (Patient Care)

Box 3 List of the Neurology Milestone Domains Within Each of the General Competencies of Professionalism, Interpersonal and Communication Skills, Practice-Based Learning and Improvement, and Systems-Based Practice

Professionalism

• Compassion, integrity, accountability, and respect for self and others

• Knowledge about, respect for, and adherence to the ethical principles relevant to the practice of medicine, remembering in particular that responsiveness to patients that supersedes self-interest is an essential aspect of medical practice

Interpersonal and Communication Skills

• Relationship development, teamwork, and managing conflict

• Information sharing, gathering, and technology

Practice-Based Learning and Improvement

• Self-directed learning

• Locate, appraise, and assimilate evidence from scientific studies related to the patient's health problems

Systems-Based Practice

• Systems thinking, including cost- and risk-effective practice

• Work in interprofessional teams to enhance patient safety

Establishing Milestone Validity, Utility, and Practicality

The Neurology Milestones were developed by expert consensus, with the intent to allow programs to assess individual resident development during neurologic training, and for the RRC to assess the program's ability to provide the appropriate clinical and educational milieu for their residents to achieve these competencies. Future studies to evaluate validity of these measures will be important, and will likely include tests of interrater reliability, assessment of correlation of certain Milestones with external measures such as in-service training and board examination performance, and even prospective or retrospective data on the practical use of these measures for trainees and programs.

Envisioned Practical Use in Evaluating Residents

Milestone assessment for neurology residents will begin in July 2014; the first 6-month assessment for most residents will therefore be completed around December 2014. Although the Neurology Milestones are not necessarily tied to specific rotations, it is likely that programs will find it helpful to revise or create addenda to their rotation assessment forms to assess the Milestones most relevant to those experiences. It will also be helpful for neurology programs and the CNPD to develop and share standardized assessment tools that will assist in allowing an accurate determination of an individual resident's Milestone attainment to better inform the deliberations of competency committees.

We expect that program directors will use Milestone data to help residents develop a personalized learning plan, adjust rotation schedules and elective experiences, and implement a remediation plan as needed. In addition, Milestone attainment may be used by the program director as support for the documentation of a resident's clinical competency to enter the unsupervised practice of neurology. Finally, the program may use its aggregate Milestone data for self-assessment and improvement of the residency program.

Recommendations for Competency Committee Composition and Functioning

The ACGME specifications for Clinical Competency Committees (CCCs) require at least 3 faculty members. The role of the CCC in advising the program director on resident progress calls for these faculty members to have clinical involvement with the residents and have knowledge of their performance. At present, the Neurology RRC and the Milestone Working and Advisory groups do not have additional requirements for the composition, leadership, or functioning of neurology programs' CCCs.

Conclusion

The Neurology Milestones will be used by programs in the NAS to help assess their residents' growth and attainment of neurologic competency, and by the ACGME as 1 of the data elements for the annual assessment of residency programs. The feasibility, practicality, and validity of implementation of the Neurology Milestones are still to be determined. At the very least, explicitly defining these specific competencies and their developmental steps is expected to better inform residents of some of the expectations of their training, and subsequently improve neurology training and neurologic patient care.

Finally, the Milestone development process for child neurology, chaired by Patricia Crumrine, MD, the immediate past-chair of the Neurology RRC, is currently underway. Milestone development for the ACGME-accredited subspecialties of neurology is in the planning stage.