What the ACGME’s Next Accreditation System Means to You

Evaluating whether a resident is qualified to graduate from an accredited orthopaedic surgery residency program remains a major educational challenge. To this point, residency programs have lacked robust approaches to measure resident performance.

In response, the Accreditation Council on Graduate Medical Education (ACGME) is introducing the Next Accreditation System (NAS) for better documentation of residency programs and resident performance [5, 8]. As part of this introduction, all specialties will incorporate the six general competencies (medical knowledge, systems-based practice, patient care, communication and interpersonal skills, professionalism, and practice-based learning and improvement) into “milestones” to gauge resident performance at various levels of training. This system is designed to evaluate residents, assess programs, and assure the public that graduates of accredited residencies are safe, competent, practicing orthopaedic surgeons.

The goal of this article is to outline the NAS, describe how it will affect teaching programs, and define what faculty members need to know before using the system.

Background and Description of Milestones

In 2011, a working group of eight orthopaedic surgeons—one from each subspecialty—developed reporting tools for orthopaedic resident performance [8]. The classifications ranged from beginning resident (Level 1) to the more advanced graduating resident (Level 5) (Fig. 1). The group of orthopaedic surgeons developed a list of sixteen milestone topics representing a cross-section of orthopaedic surgery, patient care evaluation, and orthopaedic knowledge (Table 1).

Fig. 1.

The ACGME requested competency-based reporting tools based on five levels of proficiency, ranging from novice resident (Level 1) to advanced graduating resident (Level 5). Reprinted with permission from the Accreditation Council for Graduate Medical Education and American Board of Orthopaedic Surgery

Table 1.

Milestone topics for patient care and medical (orthopaedic) knowledge

Milestone topics
ACL
Ankle arthritis
Ankle fracture
Carpal tunnel
Degenerative spinal conditions
Diabetic foot
Diaphyseal femur and tibia fracture
Distal radius fracture
Adult elbow fracture
Hip and knee osteoarthritis
Hip fracture
Metastatic bone lesion
Meniscal tear
Pediatric septic hip
Pediatric supracondylar elbow fracture
Rotator cuff injury

Reprinted with permission from the Accreditation Council for Graduate Medical Education and American Board of Orthopaedic Surgery.

What Will Change?

The ACGME mandated that residencies form a clinical competency committee to determine the competency of each resident using the orthopaedic surgery milestones [8]. The clinical competency committee should be composed of specialties whose expertise includes the milestones topics. The committee will use the milestones criteria to provide consistent resident performance data to the ACGME. Every 6 months, residency programs will upload deidentified resident data on resident performance to the ACGME website.

A program does not have to change their assessment tools or methods currently in place, but the clinical competency committee must translate assessments into the milestones criteria for every resident. For example, most programs use a standardized end-of-rotation evaluation of residents by faculty members. These forms are usually competency-based. The clinical competency committee will apply such information to the milestones assessment for a resident.

Schedule

Orthopaedic surgery is one of seven specialties—diagnostic radiology, emergency medicine, internal medicine, neurological surgery, pediatrics, and urology being the other six—in phase 1 of implementing the NAS [1]. Phase 1 specialties began using milestones criteria in July of this year with an initial upload of the first 6 months of data in December. Other programs, such as general surgery, will begin using the milestones in July 2014.

Residency programs will collect data that will be reported to the ACGME as milestones every 6 months for all residents in the program. The first reporting will begin in December. Data collected from July through June 2014 will be reviewed with reassessment of the milestones in the fall of 2014.

Limitations of the Next Accreditation System

Current resident assessment tools may be inadequate to assess resident performance using the milestones criteria [2–4]. The ACGME listed operative procedures a graduate should be capable of performing (Table 2), such as a primary total hip and total knee replacement [8]. It remains unclear whether large-scale data collection supports the use of either the selected topics or the behavior anchors established for the milestones [8].

Table 2.

Operative procedures listed in the ACGME milestones

Injury	Procedure(s)	Level
ACL reconstruction	Diagnostic arthroscopy, notchplasty, graft harvest	2
ACL reconstruction	ACL tunnel placement, graft passage, fixation	4
Ankle/mid-foot/hindfoot reconstruction	Surgically treating simple complications (incision and drainage)	4
Ankle fracture	Surgical reduction and fixation of a simple fracture	3
Ankle fracture	Surgical reduction and fixation of moderately complex fracture-trimalleolar or simple plafond	4
Carpal tunnel syndrome	Carpal tunnel release, either open or endoscopic.	4
Degenerative spine	Anterior, posterior, and posterior lumbar exposure and assist with hardware placement	3
Degenerative spine	Completion of decorticating for posterior lateral fusion, placing grafts.	4
Diabetic foot	One basic surgical approach to the diabetic foot-medial or lateral	2
Diabetic foot	Multiple approaches to the diabetic foot-plantar, medial, lateral	4
Distal radius fracture	Closed reduction and casting of distal radius fracture	2
Distal radius fracture	Operative exposure and surgical stabilization of uncomplicated extraarticular fracture	3
Distal radius fracture	Operative exposure and surgical stabilization of moderate interarticular fracture	4
Elbow fracture	Basic surgical approach	2
Elbow fracture	Surgical reduction and fixation of simple elbow fracture	3
Elbow fracture	Surgical reduction of moderately complex elbow fracture	4
Femoral or tibial shaft fractures	Surgical repair of simple tibia or femur fracture	3
Femoral or tibial shaft fractures	Surgical repair of moderately complex tibia or femur fracture	4
Hip fracture	Surgical repairs for complex fractures	4
Meniscal injury	Diagnostic knee arthroscopy and meniscal shaving	3
Meniscal injury	Meniscal repair (inside out or outside in technique)	4
Pediatric septic hip	Hip and drainage	4
Rotator cuff disease	Basic surgical approaches to the shoulder, including arthroscopy portals	2
Rotator cuff disease	Arthroscopic evaluation of the shoulder, biceps tendon pieces, subacromial decompression	3
Rotator cuff disease	Rotator cuff repair	4
Pediatric supercondylar fracture	Closed reduction and pinning of supercontinent fracture	4
Tumor	One basic surgical approach to a distructive lesion	2
Tumor	Prophylactic surgical stabilization (nails, plates, screws)	4
Tumor	Internal fixation on pending or actual pathologic fractures	4

Reprinted with permission from the Accreditation Council for Graduate Medical Education and American Board of Orthopaedic Surgery.

Although the goal of determining operative competency seems appropriate, the use of operative assessment tools to document resident performance requires review. The use of a global assessment tool at the end of rotation will not provide the same clarity as a specific instrument used at the time a procedure is performed. Ideally, a separate tool for operative evaluation of resident performance should also be developed for procedures listed in the milestones [6, 7].

The use of milestones requires additional paperwork and a new committee within each department that supports a residency. This requirement is an additional administrative burden with added cost. No additional financial support to residencies to facilitate implementation of the Next Accreditation System is expected to be forthcoming.

What Do I Need to Do?

Orthopaedic educators should be familiar with the milestones criteria in order to provide the best assessment for their residents and residency program. Residency programs are required to establish a clinical competency committee with oversight power of both resident assessment and evaluation using the milestones criteria. This group must achieve consensus regarding resident data for upload to the ACGME.

Those who teach residents should be familiar with the topics that are germane to their subspecialty area. For academic programs that include subspecialty rotations, representatives from each rotation should be present, as well as provide evaluation data of residents from each of their respective rotations. For example, residents rotating on the hand service would be evaluated in the generic areas of systems based practice, communication and interpersonal skills, practice based learning and improvement, and professionalism. They would also be evaluated in topics generally found in hand surgery (carpal tunnel syndrome and distal radius fractures) for patient care and medical knowledge.

Residency programs should review the milestone criteria, including how the criteria are used and how the data is collected. These tasks may be accomplished through routine faculty development sessions. Although assessment methods currently in place for a residency program may be used, the development of other assessment tools (such as operative assessment) is likely.

In its nascent form, the milestones criteria seem to provide better and more consistent resident assessment, which may help programs with promotion decisions, particularly for the marginal resident. However, like all new programs, a period of trial and review will likely be necessary in order for the milestones to become a reliable system.