Introduction
In the Accreditation Council for Graduate Medical Education (ACGME) Next Accreditation System (NAS), evaluation of performance on educational Milestones will be used for resident assessment and feedback and will also serve as an indicator of the educational effectiveness of family medicine residency programs.
The Family Medicine Milestone Committee developed the Milestones for family medicine with the goal of characterizing the breadth, depth, and integrative functions of the specialty in an understandable and manageable context for both residents and faculty. In developing the Milestones, the committee attempted to minimize duplication across competencies and sought to highlight family physicians' relationships with their patients by integrating the biopsychosocial model, population, and community health and by working in health care teams. The committee also felt it was important to emphasize the fact that family physicians must have the skills to deal with the complexity of patients and the health care system, as well as demonstrate the thinking skills critical to managing both the complexity and often the uncertainty over the course of an illness and over a patient's lifespan, particularly for chronic disease.
Milestone Committee Composition
The Family Medicine Milestone Committee was made up of 14 individuals selected from the members of the Review Committee for Family Medicine, the American Board of Family Medicine (ABFM), the Association of Family Medicine Residency Directors, and the American Academy of Family Physicians (box 1). The committee included 2 resident representatives.
Box 1 Family Medicine Milestone Committee
Suzanne Allen, MD, MPH, University of Washington School of Medicine, Chair
Tanya Anim, MD, Halifax Family Medicine Residency Program
Eileen Anthony, MJ, Accreditation Council for Graduate Medical Education (ACGME)
David Araujo, MD, Ventura Family Medicine Residency
Diane Beebe, MD, The University of Mississippi Medical Center
Julie Dostal, MD, Lehigh Valley Hospital Network
Tricia Elliott, MD, University of Texas Medical Branch at Galveston
Larry A. Green, MD, University of Colorado School of Medicine
Amy L. McGaha, MD, Creighton University School of Medicine
Richard Neill, MD, Perelman School of Medicine at the University of Pennsylvania
Steve Nestler, MD, ACGME Consultant
Perry A. Pugno, MD, MPH, American Academy of Family Physicians
Martin Quan, MD, David Geffen School of Medicine at UCLA
Adam J. Roise, MD, MPH, Northeast Iowa Medical Education Foundation
Allen F. Shaughnessy, PharmD, MMedEd, Tufts University at Cambridge Health Alliance Program
Penelope Tippy, MD, Southern Illinois University Carbondale Family Medicine Residency
Family Medicine Milestone Development
The Family Medicine Milestones were developed to be a living document. The Milestones describe how a resident moves through the developmental process from entrance into residency to becoming a competent family physician ready to enter independent practice. The Milestones provide a standard framework with which to assess residents' progress through training and, as such, can provide guidance to programs in developing flexible curricula and evaluation tools specific to their educational settings.
The Milestone committee reviewed the Milestones from Phase 1 specialties as well as the current and proposed program requirements for family medicine to develop the first draft. The committee also reviewed Milestones for the competencies of interpersonal and communication skills, professionalism, practice-based learning and improvement, and systems-based practice developed by an ACGME expert panel; historical and recent reports concerning practice and education in family medicine, including the original family practice residency requirements, the well-known Willard, Millis (Report of the Citizens Commission on Graduate Medical Education, commissioned by the American Medical Association, 1966), and Folsom (1967) reports; the Future of Family Medicine report; and recent publications from the literature of the Phase 1 specialties and family medicine education. The documents and reports reviewed are shown in box 2.
Box 2 References Used in the Development of the Family Medicine Milestones
Sullivan G, Simpson D, Cooney T, Beresin E. A milestone in the milestones movement: the JGME milestones supplement. J Grad Med Educ. 2013(suppl):1–4.
Hicks PJ, Englander R, Schumacher DJ, Burke A, Benson BJ, Guralnick S, et al. Pediatrics milestone project: next steps toward meaningful outcomes assessment. J Grad Med Educ. 2010;2(4):577–584.
Lesko S, Hughes L, Fitch W, Pauwels J. Ten-year trends in family medicine residency productivity and staffing. Fam Med. 2012;44(2):83–89.
Caraccio C, Burke AE. Beyond competencies and milestones: adding meaning through context. J Grad Med Educ. 2010;2(3):419–422.
American Medical Association. Essentials of approved residencies: special requirements for residency training in family practice. Chicago, IL: American Medical Association; 1969:318–320.
Bryan JE for Family Health Foundation of America. The Role of the Family Physician in America's Developing Medical Care Program: a Report and Commentary. St. Louis, MO: Warren H. Green Inc; 1968.
McWhinney IR. The primary physician in a comprehensive health service. Further reflections after a visit to the United States. Lancet. 1967;1(7481):91–96.
McWhinney IR. William Pickles Lecture 1996: the importance of being different. Br J Gen Pract. 1996;46(408):433–436.
Martin JC, Avant RF, Bowman MA, Bucholtz JR, Dickinson JR, Evans KL, et al. The future of family medicine: a collaborative project of the family medicine community. Ann Fam Med. 2004;(suppl 2):3–32.
Council on Medical Education and Hospitals of the American Medical Association, Citizens Commission on Graduate Medical Education. The Graduate Education of Physicians: The report of the Citizens Commission on Graduate Medical Education (Millis Report). Chicago, IL: American Medical Association; 1966.
American Medical Association, Ad Hoc Committee on Education for Family Practice. Meeting the Challenge of Family Practice (the Willard report). Chicago, IL: American Medical Association; 1966.
National Commission on Community Health Services. Health Is a Community Affair (the Folsom Report). Cambridge MA: Harvard University Press; 1966.
Folsom Group. Communities of solution: the Folsom Report revisited. Ann Fam Med. 2012;10(3):250–260.
Green LA, Jones SM, Fetter G Jr, Pugno P. Preparing the personal physician for practice: changing family medicine residency training to enable new model practice. Acad Med. 2007;82(12):1220–1227.
Graham R, Roberts RG, Ostergaard DJ, Kahn NB Jr, Pugno PA, Green LA. Family practice in the United States: a status report. JAMA. 2002;288(9):1097–1101.
Starfield B, Shi L, Macinko J. Contribution of primary care to health systems and health. Milbank Q. 2005;83(3):457–502.
The committee solicited review and feedback from members of the family medicine education community when the first draft of the Milestones was posted on the ACGME website. Detailed and specific feedback was received from the American Board of Family Medicine (ABFM), American Academy of Family Physicians, the Council of Academic Family Medicine, the Association of Family Medicine Residency Directors, and the Society of Teachers of Family Medicine. Individual and program feedback was encouraged, received, and reviewed. Multiple presentations were delivered to large groups of family medicine educators who were encouraged to provide feedback.
Testing the Family Medicine Milestones
The committee conducted both alpha and beta testing of the Milestones. Twelve residency programs participated in alpha testing and 31 programs in beta testing, and they all provided extensive feedback to the committee. The feedback was used for revising and improving the draft Milestones.
Participants in the alpha and beta testing of the Milestones were asked to use the Milestones as they will be used in the NAS. Programs created Clinical Competency Committees (CCCs) that tested the Milestones by using the evaluations for each resident from the previous year (including rotation evaluations, 360-degree evaluations, patient surveys, in-training examination scores, and others) as benchmarks. Participants in the pilot tests were asked to report how many faculty participated in the CCC, the number of residents reviewed, the range of time required to review the residents and the average amount of time spent reviewing each resident, any changes planned in the curriculum of the program because of what they learned from the Milestones, and any changes planned in their evaluation methods because of what they learned from the Milestones.
Milestone Organization
In the final version, there were 22 Milestone sets for specific subcompetencies in family medicine. Each Milestone set has 5 levels of competency development, which are described to guide residency programs in their assessment of individual residents. While the committee initially constructed a format that linked competency attainment to specific levels of training, there was agreement that this model would restrict programs and residents from achieving competency at various periods of time. The committee gave a great deal of thought to the skills and knowledge that were expected of entering residents and the knowledge and skills of family physicians ready to enter practice. A level of 4 is the level of attainment expected for entry into unsupervised practice, while a level of 5 is an aspirational level of performance that would likely not be achieved by all residents.
Representative of the major categories of patients for which family physicians provide care are 5 subcompetencies for patient care that reflect acute, chronic, complex, and preventive care, as well as procedures that family physicians perform. There are 2 subcompetencies for medical knowledge that reflect the breadth and depth of knowledge and the critical thinking skills required to be a family physician. The Milestone sets for professionalism, interpersonal and communication skills, practice-based learning and improvement, and systems-based practice were adapted from the expert panel's Milestone descriptions.
Milestone Validation and Usability Testing
The alpha and beta pilot tests provided information to the Milestone group about the usability of the Milestones and the programs' readiness to implement Milestones. The initial feedback from both alpha and beta testing validated the efficacy of the draft Milestones in reliably identifying low- and high-performing residents. Most of the programs that pilot tested the Milestones found them to be valuable in documenting resident competency to enter unsupervised practice. More than 80% of the programs that beta tested the Milestones reported they plan to modify their curriculum in some way based on their experience with the Milestones. Similarly, approximately 90% of the programs that participated in the beta testing plan to update their evaluation tools or add new evaluation tools based on their experience with the Milestones.
Use of Milestones in Resident Assessment and Maintenance of Certification
By describing residents' progression from entry into residency through graduation by using observable measures, the Family Medicine Milestones and the assessments by the CCC, using the Milestones, will enable family medicine programs to provide meaningful, formative feedback to residents on their progress toward competence in the specialty and will help identify residents with delayed progress who require added education and, if needed, remediation. At the program level, the Milestones will allow an assessment of the curriculum and will highlight both strengths and opportunities for improvement.
The ABFM has worked closely with the ACGME in development of the Family Medicine Milestones. The primary reason for doing so was to make certain that the Milestones could be reasonably integrated into the ABFM's Maintenance of Certification Program for Family Physicians (MC-FP). The ABFM anticipates that most graduating residents will not have achieved all of the Level 5 Milestones at the completion of training. The ABFM envisions the newly certified family physician's progress in achieving Level 5 status for each of the Milestones will be monitored during the initial MC-FP cycle. This information will provide useful feedback for the family physician in helping shape and formulate their continuing professional development while also closing the feedback loop to program directors with respect to the outcomes of their individual graduates.
Conclusions
Use of the Family Medicine Milestones in the assessment of residents and in providing data to the ACGME on educational program effectiveness will generate valuable information on the efficacy, validity, and utility of the Milestones. The Family Medicine Milestones will be updated as the Milestones' efficacy, validity, and utility are determined. These Milestones represent the beginning of a new approach to assessing family medicine residents and will usher in another period of dynamic development to improve residents' preparation to be outstanding family physicians.