“May We Live in Interesting Times”—Society of General Internal Medicine Clinician-educators Respond to New Challenges in Graduate Medical Education

July 1, 2003 marked a watershed moment in graduate medical education in this country. New interns entered residency programs transformed by two sweeping reforms from the Accreditation Council for Graduate Medical Education (ACGME), which acted to raise our profession's accountability for training new physicians.

First, the ACGME approved duty hours standards for all accredited programs.¹ This regulation reflects growing concerns, dating back to the Libby Zion case in 1984, about the effects of fatigue on resident well-being and patient safety. The ACGME first responded in the 1990s with loosely enforced work hours limitations, which varied widely by specialty. As many programs failed to comply (30% of reviewed internal medicine programs cited in 1999²), the public outcry grew louder and was heard in the Occupational Safety and Health Administration and in the halls of Congress.³ By this time, stakeholders could cite a growing body of scientific evidence linking long work hours to poorer performance and increased burnout among residents.⁴ With this backdrop, the ACGME imposed (and is beginning to enforce) an 80-hour per week limit, 1 day off in 7, and relief after a 24-hour shift plus 6 hours for transition.

Second, the ACGME “outcomes project” changed the accreditation currency from process and structure to outcome.⁵ Program directors must now provide more than a schedule of rotations, a written curriculum, and agreements with clinical training venues. They must objectively document that their residents achieve a level of competence in six general dimensions of practice. Two of these, practice-based learning and improvement and systems-based practice, reflect a recent emphasis on evidence-based reflective practice and newer models of team care and disease management.

As directors dramatically restructure their programs to comply with these two formidable unfunded mandates, they struggle with many uncertainties.^6,7 They can take some lessons from New York State, which has legislated work hours restrictions since 1989.⁸ And the ACGME “toolbox” provides some guidance in evaluating the six competencies. Nonetheless, many questions remain unanswered. In this issue, educators respond to these challenges with reports of creative studies, innovative curricula, and robust evaluation strategies.

Hoellein et al.'s study adds to the evidence underpinning the work hours restrictions.⁹ In this observational study of 646 clinic encounters in a single program, patients were less satisfied with their visits with postcall residents, after adjusting for important confounders. Of note, resident satisfaction did not vary with call status.

Wong et al. developed an innovative day-float rotation that responds to both ACGME mandates.¹⁰ A senior medical resident joins the postcall team for morning rounds, assists with the daily tasks, and pursues the team's emerging clinical questions. When the day-float rotation was in operation, ward residents worked from 67 to 81 hours per week, compared to 79 to 90 hours prior to implementation. In addition, by keeping a learning portfolio, the day-float resident documents her practice-based learning competency. Day-float residents collected 6.6 and answered 6.1 questions per rotation. We suspect a very low recording rate, however, given studies showing that residents encounter clinical questions at a much higher rate.¹¹

Pinsky and Fryer-Edwards describe their program's experience with a more ambitious program-wide portfolio system, designed to evaluate and promote reflection around all six ACGME competencies.¹² They identified five elements that may promote successful implementation, including separate working and performance functions, a supportive climate, skill development in faculty and residents, observing progress over time, and fostering mentoring opportunities. Residents in this program apparently offered little resistance to maintaining portfolios. However, skeptical program directors may require more quantitative feasibility data before asking their busy residents to embrace this extra task.

Residents completed quality improvement projects in Ogrinc et al.'s innovative practice-based learning and improvement curriculum.¹³ In a pre-post controlled trial, exposed residents improved their scores on a validated quality improvement knowledge instrument. Furthermore, their project sponsors appreciated the meaningful contributions to care in their clinical venues.

Two investigators raised the bar for procedure training, which falls within the “patient care” competency. Smith et al. developed an innovative medical procedure service, which included a web-based component to teach procedural knowledge and 24-hour availability of qualified faculty (hospitalists and intensivists) to teach, supervise, evaluate, and track procedural experience.¹⁴ The additional billing revenues partially offset the costs of increased faculty coverage. In a 12-month pilot program, the complication rate was 3.7% over 246 procedures. The pneumothorax rate for thoracenteses was only 3.5%, which compares favorably to a 10.6% rate in a pooled analysis of the literature. Watkins and Moran developed a targeted intervention to train residents in Pap smear sampling, which included a skills workshop with a manikin and peer comparison feedback of adequacy data.¹⁵ In a randomized controlled trial, exposed residents were twice as likely to obtain sufficient endocervical cells, adjusting for important confounding variables.

On a practical level, competency-based evaluations add a substantial administrative burden to already stressed program leadership and staff. Triola et al. developed a web-based modular evaluation system that generates competency-based, venue-specific, and training level-specific questions for evaluators and customized evaluation reports for residents, faculty, and program directors.¹⁶ In their program, evaluation compliance increased from 35% in a paper system to 85% at 10 months after transition to this electronic system.

Finally, while investigators often report easily measured endpoints like trainee satisfaction, knowledge, and skills, we are charged to determine “how the design and conduct of medical education programs affect the clinical outcomes produced by doctors.”¹⁷ Educators, in this issue, evaluated the impact of educational interventions on trainee performance, such as procedure complications, Pap smear adequacy, and patient satisfaction. Furthermore, the educational activities themselves may add value to clinical care.¹⁸

Of course, “this is not the end,” as Churchill said. “It is not even the beginning of the end. But it is, perhaps, the end of the beginning.” The impact of these reforms will play out over many years. Much more research is needed to address lingering questions. For instance, many educators worry about the repercussions of disrupting continuity of care to meet work hours standards. What, then, will be the net effect on quality of care, patient satisfaction, resident education, and resident well-being? What is the ideal resident workweek? And important economic questions should rekindle the debate about GME funding. Will hospitals be able to absorb the “replacement costs” of resident hours and maintain their support of residency programs? Will program faculty be able to fully implement competency-based training, while they receive a shrinking allocation of GME financial support and insufficient academic “credit” for this work? Will researchers be able to advance the lagging educational science, despite extremely limited funding opportunities? We do, indeed, live in interesting times.