New Paradigms in Post-Graduate Surgical Education

Abstract

With a growing focus on patient safety and trainee education, the Accreditation Council for Graduate Medical Education implemented changes including work hour restrictions, focused clinical competencies, and the Next Accreditation System (NAS). The NAS poses initial challenges on residencies with the implementation of surgical simulation programs and defining resident competency. It is the hope that innovative training methods will allow for improved advancement of knowledge and surgical skills given the current direction of post-graduate surgical training.

Introduction

Medical education and surgical training has changed drastically throughout the past century. William Stewart Halsted, the “father” of American surgical residency system developed a residency program at John’s Hopkins in 1889 with the following principles: knowledge of basic sciences, research and graduated patient responsibility.^1,2 These principles are still the focus of modern day resident education. Halsted’s rigorous program had no definite length of training in which assistant residents underwent competition to advance. Halsted envisioned “a system...which will produce not only surgeons but surgeons of the highest type, men [and women] who will stimulate the first youths of our country to study surgery and to devote their energies and their lives to raising the standard of surgical science.”¹

In contrast, Edward D. Churchill of Massachusetts General Hospital greatly opposed Halsted’s surgical training model due to the indefinite residency time. Churchill believed surgical training should be based on instructed knowledge and developed a five-year residency program in 1931-the rectangular model. He believed the competitive nature of the pyramidal model created too much pressure on the resident which distracted from their education.²

During this same time, the Clinical Congress of Surgeons, now the American College of Surgeons (ACS), formulated minimum requirements for surgical education in 1912 after the Flexner report exposed overproduction of physicians and inadequacy of medical training.^1,3 In 1937 the American Board of Surgery implemented the first examination to evaluate resident knowledge. Churchill’s rectangular model gained popularity, while Halsted’s principle of graduated patient responsibility was maintained to become what we now know as residency.²

Since Medicare and Medicaid were implemented in 1964, new challenges to resident education have arisen. Attending surgeon presence was required for all procedural billing. Prior to Medicare, residents were often responsible for all aspects of care, including surgery as they received graduated responsibility. This system of patient care, a Halsted key principle, is today not as pronounced due to current social, ethical, political and economic climates.² In order to accomplish adequate residency training in the 21^st century, when knowledge is vast and ever growing, standardized resident evaluation beyond test questions is needed.¹ Therefore the ACGME implemented the next accreditation system (NAS) in an attempt to institute standardized outcomes-based residency education with the goal of graduating competent residents.

The Next GME Accreditation System

The Accreditation Council for Graduate Medical Education (ACGME) was established in 1981 to address two major issues, the variability of resident education quality and formal subspecialty education. Today the goal of the NAS is to better prepare physicians to practice in the 21^st century, accelerate ACGME accreditation based on educational outcomes, and reduce burden associated with site visits.⁴ In addition to NAS, the Clinical Learning Environment Review (CLER) was developed in 2012 to address patient safety, resident and fellow supervision, health care quality, transition of care, duty hours and fatigue management, and institution professionalism. All of these changes represent a pivotal shift from process-based learning to an outcomes-based education.⁵

The NAS requires programs to submit resident milestone data every 6 months, which is then synchronized with residents’ semi-annual evaluations.⁴ These milestones are individualized to each specialty in order to fulfill the six core competencies: medical knowledge, patient care, professionalism, interpersonal communication, practice-based learning with demonstration of personal improvement, and systems based practice with system improvement.⁶ The Residency Review Committee (RCC) for orthopaedics chose to divide medical knowledge and surgical competency into milestones based on core procedures. The RCC chose to base these milestones on 16 procedures thought to be representative of fundamental knowledge required for a successful orthopaedic career—a “biopsy” of the residents medical knowledge and patient care skills. In total, orthopaedics has 750 milestones that are reported to the ACGME every 6 months on each resident—the most of any specialty.⁷

By July 2014 all specialties entered the NAS. The hope was that data driven annual review of performance, rather than episodic site visits would help identify low-performing residency programs more quickly and the appropriate program support could be given before major issues arise. The NAS also allows the ACGME to extend site visits from 5 to 10 years for high functioning programs. In addition, with the frequency of data collections, a higher rate of turnover is possible for programs on probationary status. Early data from the NAS demonstrates that the number of accredited residency programs has gone from 9472, prior to the NAS, to 9793 in 2014–2015, after implementation of the NAS. Only 34 of the 328 programs on “Probationary accreditation” or “Continued accreditation with warning” did not progress to full accreditation. There were also 189 programs newly identified as needing a warning or probationary status.⁴ This early data is evidence of the effectiveness of the ACGME’s NAS.

Development of Objective Competency Measures

While the driving force behind the ACGME NAS is the competence of graduating residents, the ACGME does not clearly define competence. This is largely because clinical competence is an individual trait and the ACGME certifies programs, not individual residents. Therefore the milestones provide a framework for the knowledge and skills residents should achieve combined with a measurable tool to identify areas of improvement. These milestones assess what residents are doing in the clinical workplace, rather than how they perform on written and simulated examinations. With regards to surgical specialties, these skill evaluations are used to assess how well programs are meeting ACGME standards to provide adequate operative experience with sufficient exposure.

Competence, by definition, is subjective and dependent on the individual. Milestones are part of an attempt to objectively evaluate something that is inherently subjective. This may be perplexing within surgical specialties because the ACGME Review Committee for surgery states that “Performance of this minimum number of cases by a resident must not be interpreted as equivalent to competence achievement.” Therefore, it is still solely up to the individual program to appropriately identify residents that are not adequately meeting milestones as expected and create a plan for addressing deficiencies. In addition, competency is not a stable trait. Some who competently perform a procedure one day may not the next day or a decade later. Potts elegantly describes competency as something that is “usually easily recognized when it is observed and even more easily recognized in its absence… Competency is an ill-defined but observable task-specific and unstable condition of individuals.”⁶

There are multiple barriers to true competency-based training described by Potts et al which are relevant to the challenges programs may face with the NAS. There is a need to educate faculty at the institution and all of the programs’ associated hospitals on teaching, feedback and evaluation of competency. This requires both structural and educational changes which make a solely competency-based educational system difficult to implement under the current NAS.⁶ For high performing programs, there may be opportunities to apply for experimental competency based curriculums. Even within the parameters of the NAS, residency programs are instituting novel training systems that embody this trend towards standardized competency-based evaluation.

Surgical Skills Training

Although there is currently no optimal method to measure surgical competency, the move of resident education from an apprenticeship model to a competency-based type has led to the implement of surgical “boot camps” across the nation.^9,10 With 80-hours per week duty hour restrictions, the variability of exposure to surgical procedures, and heightened concern of medico-legal consequences, training programs are attempting to give residents a solid set of surgical skills upon which to build. Current resident physicians are now expected to reach the same level of competency as their predecessors over a shorter amount of clinical hours. Therefore alternative methods for surgical training are becoming more appealing. Synthetic, animal, and cadaver models have been commonly utilized in surgical simulation training. While this training has been used for decades, these exercises have only recently been quantified by standardization and the development of goals and objectives for use in resident education.¹¹

Surgical simulation has been influenced by long-standing models of technical education in other fields such as aviation, where simulation exercises encompassing various scenarios are a regular practice.¹¹ These exercises allow learning in a low-stress environment, have no patient risk, and give immediate feedback. In arthroscopic surgery, evidence indicates that virtual simulation decreases the learning curve for junior residents while allowing adequate transfer for skills to the operative room.¹² The decreased learning curve could lead to decreased operative time and possibly decreased patient complications. These results are promising for the orthopaedic field and have mirrored the laparoscopic studies in general surgery training programs. In 2009 the ACS became the first surgical field to require residents to pass a technical proficiency program in order to obtain board certification.¹³

At this time the American Board of Orthopaedic Surgery does not currently have a technical skills examination required for board certification, but it appears the field is moving in that direction. In July 2013 the ABOS and RRC introduced 17 surgical skills modules to be implemented in all orthopaedic surgery programs during the PGY-1 year.¹⁵ This was an important step in orthopaedic training towards standardizing the evaluation of surgical skills. Many programs have implemented 1 month of basic skills training^9,10 whereas at our residency program, PGY1s spend 1 hour a week throughout the year in our orthopaedic surgical skills lab led by faculty. Much debate remains on the best way-longitudinal or clustered-for the training and retention of surgical skills.

Although numerous orthopaedic procedures may be simulated in a skills lab, ensuring the transition of these skills to the operating room is of critical importance. With the abundance of technological advances, there have been efforts to bridge this gap. The University of Iowa has evaluated residents’ real-life surgical experiences using a GoPro from a point of view perspective while Dartmouth is currently studying video-assisted feedback using Google Glass.^9,16 Due to the material cost and time constraints of video capture analysis, these innovative concepts are not yet a common practice in surgical assessment of residents.

While the goal of resident evaluation in the operating room to provide objective feedback, it remains a subjective assessment based upon the interaction of the supervising attending with the resident during each procedure. Attending assessments intra-operatively and post-operatively remain the predominant means of providing surgical evaluation. These evaluations become officially recorded as “the milestones” and are the current standard for assessing technical competency. To streamline milestone evaluations our program has utilized an online milestone tracking and reporting tool to make the process simpler. This novel online tool has easy milestone access, no lost paperwork, and allows immediate feedback to our program and residents to note strengths and weaknesses in education. Furthermore, the novelty of the program breaks the individual milestones down into “yes/no” questions the faculty answers to aggregate an overall milestone level. The “yes/no” evaluations are sent to the faculty by the resident on each case, and the faculty simply completes the “yes/no” evaluation on their smartphone.^7,17

Training capable surgeons is the goal of all surgical residency programs but determining the methods that best support technical proficiency is still being studied. As surgical simulation research becomes more widespread, we anticipate accelerated resident progress that can be more objectively measured through milestones.

Discussion

The ACGME NAS is an attempt to standardize resident education using core competencies measures with milestones and ensure a safe working environment for both trainees and patients. While providing quality orthopaedic care for patients is of utmost importance, adequately training our future orthopaedic surgeons should be held in the same regard. Investing in the training of orthopaedic residents with quality intraoperative exposure will help ensure that orthopaedic surgeons are prepared to safely operate independently after completion of residency. Within the duty hour restrictions, residency programs must find more efficient ways to educate residents. Simulation allows mastering procedural steps prior to performing it and has been shown effective in decreasing the learning curve for techniques particularly in arthroscopy.¹² Dwyer et al showed when comparing junior residents after arthroscopy simulation to senior residents with no prior simulation experience, senior residents continued to perform superior to junior residents. This study demonstrated that although simulation is beneficial in initial learning, it is not a replacement for intra-operative and clinical experience.¹⁸

Simulation is an essential part of surgical training and is a worthwhile investment. Our simulation lab includes sawbones, wet labs, arthroscopy simulators and two haptic virtual reality computer-based simulators. In addition, industry sponsored surgical labs are provided quarterly and residents have the opportunity to attend 2 orthopaedic skills courses per year. The combination of these resources has become an essential adjunct to our residents’ education.

Unlike the FLS required for the ACS, there is no standardized method for evaluating technical skills in orthopaedics at this time. Similar to FLS, a Fundamentals of Arthroscopic Surgery Training Program (FAST) may eventually be required in the future to demonstrate competency in techniques associated with arthroscopy.⁷ Even with standardization, it is important to recognize that many residents will subspecialize and these requirements may not be applicable to all clinical practices.

Orthopaedics is a constantly evolving medical field. As subspecialization is becoming more widespread the medical knowledge to learn increases exponentially. In addition, the endless advent of new technology requires continual surgical training even after completion of residency. This requires residencies educate residents how to learn technical skills independently, such as with simulation, based on a foundation of fundamental skills. Surgical training modalities are currently geared towards resident education, but the use of these methods may prove beneficial in maintaining, improving, and acquiring surgical skills for even established orthopaedic surgeons.

The goal of orthopaedic residency training is to graduate competent surgeons feasibly with the available resources at hand. While the ACGME NAS provides tools to measure residency programs’ outcomes, the milestones are only a guide with which to identify the progress and deficits in a resident’s training. Standardized testing provides a window into the breadth of clinical knowledge a resident may have, but may not elucidate the application of that knowledge in a clinical setting. Therefore determining how to adequately evaluate competency, particularly surgically, is still a dilemma orthopaedic training programs face.

Conclusions

A drastic paradigm shift is taking place within medicine which has greatly impacted the educational environment of orthopaedic residents due to the concern of quality of education, competency of graduating residents, and patient safety. The ACGME NAS is one of the many changes that residency programs have seen recently. The ACGME has provided milestones which give more specific expectations as well as a framework for identifying the progress or deficiency of residents, but they are only able to accredit programs and cannot determine the competency, including technical (surgical) ability, of individual graduates. Therefore it is up to each residency program and each specialty board to standardize and determine what represents a competent graduate.

Further work remains to be done in the translation of surgical skills learned in the lab to the OR, the best modalities for evaluation and simulation, and even the definition of competency itself. In order to meet the demands from the public and from regulators, residency programs must respond to demonstrate we are generating safe and effective physicians and surgeons. To do so, we must develop objective measures which are reliable and reportable, and the graduate medical education community is in the infancy of doing so.

Biography

Kirsten Norrell, MD, and Joanne Marasigan, MD, are in the Department of Orthopaedic Surgery, University of Missouri—Kansas City School of Medicine. James Bogener, MD, (above), is Program Director of the Department of Orthopaedic Surgery, University of Missouri—Kansas City School of Medicine.

Contact: norrellk@umkc.edu