Effects of Reducing or Eliminating Resident Work Shifts over 16 Hours: A Systematic Review

Abstract

Study Objectives:

The Institute of Medicine (IOM) has called for the elimination of resident work shifts exceeding 16 hours without sleep. We sought to comprehensively evaluate the effects of eliminating or reducing shifts over 16 hours.

Design and Outcome Measures:

We performed a systematic review of published and unpublished studies (1950-2008) to synthesize data on all intervention studies that have reduced or eliminated U.S. residents' extended shifts. A total of 2,984 citations were identified initially, which were independently reviewed by two authors to determine their eligibility for inclusion. All outcomes relevant to quality of life, education, and safety were collected. Study quality was rated using the U.S. Preventive Services Task Force methodology.

Measurements and Results:

Twenty-three studies met inclusion criteria (κ = 0.88 [95% CI, 0.77-0.94] for inclusion decisions). Following reduction or elimination of extended shifts, 8 of 8 studies measuring resident quality of life found improvements. Four of 14 studies that assessed educational outcomes found improvements, 9 found no significant changes, and one found education worsened. Seven of 11 identified statistically significant improvements in patient safety or quality of care; no studies found that safety or care quality worsened.

Conclusions:

In a systematic review, we found that reduction or elimination of resident work shifts exceeding 16 hours did not adversely affect resident education, and was associated with improvements in patient safety and resident quality of life in most studies. Further multi-center studies are needed to substantiate these findings, and definitively measure the effects of eliminating extended shifts on patient outcomes.

Citation:

Levine AC; Adusumilli J; Landrigan CP. Effects of reducing or eliminating resident work shifts over 16 hours: a systematic review. SLEEP 2010;33(8):1043-1053.

SEVEN YEARS HAVE PASSED SINCE THE ACCREDITATION COUNCIL FOR GRADUATE MEDICAL EDUCATION (ACGME) ESTABLISHED DUTY HOUR LIMITS for physicians-in-training. Prior to that time, few high quality studies existed regarding the effects on safety of residents' traditional work schedules.^1,2 In the past few years, however, a growing body of literature has emerged regarding the hazards of prolonged sleep deprivation and of one particular aspect of residents' traditional schedules: the extended duty (> 16 h) shift.^3–10

Data on extended shifts have previously prompted calls for action from the federal government and professional organizations.^11–14 Last year, the Institute of Medicine (IOM) published a comprehensive report on resident duty hours,¹⁵ in which it recommended that residents should work no more than 16 consecutive hours without sleep. This recommendation has been the subject of considerable controversy. While numerous studies demonstrate the adverse consequences of sleep deprivation in industrial and laboratory settings,¹⁵ critics have suggested that insufficient data have been generated from clinical intervention studies to support the call for a change in traditional resident scheduling practices.¹⁶ Concerns raised include both the potential for “unintended consequences” of reducing work hours on patient safety—due to concerns about inadequate handovers of care—as well as the effects of work hour reduction on resident education. In addition, data have emerged suggesting that implementation of the IOM's recommendations could cost up to $1.6 billion dollars, while yielding uncertain benefits.¹⁷ The ACGME is currently reviewing the IOM recommendations to determine whether to make changes to current duty hour standards.¹⁸

Remarkably, in the face of international reductions in resident-physicians' extended shifts and calls for change in the U.S., there has not been a systematic evaluation of the IOM's key recommendation regarding extended shifts. Prior systematic reviews have been conducted of the relationship between resident sleep deprivation and performance,^1,2,10 but the intent of these prior reviews was to not to assess the effects of specific interventions, or to rate the evidentiary strength of the studies conducted. We therefore conducted a systematic review with the specific aim of determining whether safety, education, and quality of life have improved or deteriorated in programs that have reduced or eliminated shifts > 16 h, and to assess the quality of included studies using the U.S. Preventive Services Task Force methodology.

METHODS

Data Sources and Searches

We performed a Medline search of English language articles published from January 1950 to May 2008 using the MESH terms “Internship and Residency,” “Education, Medical, Graduate,” or key words fellow, resident, intern, house officer, house-staff, house staff, or junior doctor. These were combined with MESH terms “Work Schedule Tolerance,” “Sleep,” “Sleep Deprivation,” “Fatigue” and key words work hours, working conditions, shift length, shift work, or night float. We also searched several other relevant databases, including EMBASE, CINHAL, and ERIC. To limit publication bias, we searched databases of unpublished abstracts from scientific, educational, and sociological conferences, unpublished dissertations and theses from North American universities, and contacted authors of relevant papers for additional unpublished work. This search produced 2,984 English language articles, abstracts, and dissertations.

Study Selection

In order to focus our review on the impact of reducing extended shifts for U.S. house staff, we created 4 inclusion criteria. First, we included only peer reviewed, original research. Second, we limited our review to studies that included interns, residents, and/or fellows training at ACGME accredited U.S.-based training programs. Third, because we were interested in assessing the net effects of interventions in clinical settings, we limited our review to trials which included both an intervention and a comparison group; cross-sectional studies and cohort studies that did not include an intervention were excluded. Both randomized and non-randomized trials were included as well as trials that utilized concurrent or historical comparison groups (i.e., pre and post design). To ensure internal validity, control and intervention groups had to include residents in the same specialty and institution. Fourth, we included only studies where the intervention being evaluated included elimination of extended shifts or a decrease in their frequency. Studies with multiple interventions were included as long as they included a decrease in the frequency of extended shifts. We chose to consider shifts > 16 h as extended, as this was a prominent cutoff cited in prior studies of resident work hours and in ongoing calls for work hour reform.^{13–15,19–21}

Data Abstraction and Quality Assessment

Data were abstracted by 2 independent investigators (AL and JA), who reviewed all 2,984 studies using a standardized abstraction form that captured study design, population, setting, intervention type, and outcomes. The 2 investigators were blinded to each others determination about whether or not studies met the pre-determined inclusion criteria. A manual search was performed of the bibliographies of all relevant systematic reviews and original research studies identified. All studies were ranked as Level I, II-1, II-2, II-3, or III using methods described by the U.S. Preventive Services Task Force.²² This methodology provides a standardized metric for assessing study quality based on the design of a study and the rigor of its execution, as follows: Level I: Evidence obtained from at least one properly designed randomized controlled trial; Level II-1: Evidence obtained from well-designed controlled trials without randomization; Level II-2: Evidence obtained from well-designed cohort or case-control analytic studies, preferably from more than one center or research group; Level II-3: Evidence obtained from multiple time series with or without the intervention; dramatic results in uncontrolled trials might also be regarded as this type of evidence; Level III: Opinions of respected authorities, based on clinical experience, descriptive studies, or reports of expert committees.²²

Data Synthesis and Analysis

Of the 2,984 English language citations produced by our search, 22 met inclusion criteria. The pre-discussion inter-rater reliability of inclusion decisions was high (κ = 0.88 (95% CI, 0.77 to 0.94); inter-rater agreement, 99.8%). Details regarding the number of studies excluded at each stage of the abstraction process are presented in Figure 1. Manual search of bibliographies from included studies and relevant systematic reviews produced one additional article which met our inclusion criteria, totaling 23 articles.

Figure 1.

Literature search strategy

Outcomes varied in the 23 studies, but generally fell into one of 3 categories: Resident Quality of Life, Sleep, and Fatigue; Resident Education; or Patient Safety and Quality of Care. Some studies included multiple outcomes in more than one category. For studies that assessed multiple outcomes, the study was considered positive if at least one outcome measure was positive and negative if at least one outcome measure was negative. Studies with both negative and positive results were considered mixed, while studies without statistically significant changes in outcome were considered to demonstrate no change. The studies included in our review involved residents in internal medicine, general surgery, obstetrics/gynecology, and radiology. All findings throughout the results section below are presented with data from the traditional system listed first, and the intervention schedule second.

RESULTS

Quality of Life, Sleep, and Fatigue

Eight published studies examined the impact of decreasing or eliminating extended shifts on resident sleep, fatigue, quality of life, or satisfaction (Table 1). A concurrent, controlled trial of internal medicine residents on a rotation with overnight call (maximum shift length 32 h) and those on a rotation with a night float system (maximum shift length typically16 h once per week) found no difference in anxiety or hostility scores, measured by the Multiple Affect Adjective Checklist on the day after their longest shift, though residents on the rotation with night float were less depressed on a depression scale (19.4 ± 4.4 traditional vs. 17.4 ± 3.4 intervention, P < 0.02).²³

Table 1.

Studies of resident quality of life, sleep, and fatigue

Source	Study Design	Population and Setting	Intervention Type	Study Outcomes	Study Quality
Carey29	Retrospective, “before and after” controlled trial	Ob/Gyn residents at a single hospital	Reduction in extended shifts from Q3 to at most Q7, with most shifts <13 h	Decrease in resident rating of sleep deprivation.	Level II-3
Goldstein26	Prospective, “before and after” controlled trial	Surgical residents at a single hospital	Transition from Q4 call to a night float system with 12- to 14-h shifts	Improvement in overall fatigue, hours of sleep, and availability for family events with night float system.	Level II-3
Gottlieb23	Prospective, concurrent controlled trial	Internal medicine residents at a single VA medical center	Comparison of a Q4 call schedule to night float system with maximum 16-h shifts	No difference for anxiety or hostility scores, but lower depression scores for residents on the night float system.	Level II-1
Hutter25	Prospective, “before and after” controlled trial	Surgical residents at a single hospital	Reduction in call frequency from Q3 to Q4	Decrease in emotional exhaustion and increase in motivation to work on Q4 schedule.	Level II-3
Lockley20	Prospective, randomized controlled trial	Internal medicine interns working in the MICU/CCU at a single hospital	Comparison of traditional Q3 system with 30-h shifts to a system with maximum 16-h shifts	Increased sleep and decreased attentional failures with the 16-h maximum compared to the 30-h maximum	Level I
Lund28	Prospective, “before and after” controlled trial	Ob/Gyn residents at a single hospital	25% overall reduction in call frequency for all residents	Improvement in resident satisfaction with program but no change in other quality of life measures.	Level II-3
Nichols27	Retrospective, “before and after” controlled trial	Ob/Gyn residents at a single hospital	Unspecified reduction in call frequency for PGY2 residents with implementation of night resident system	Residents with reduced call frequency more supportive of co-resident becoming pregnant; no change in overall stress level.	Level II-3
Sawyer24	Prospective, concurrent controlled trial	Surgical interns at a single hospital	Comparison of interns on Q2, Q3, and Q4 schedules	Increased overall satisfaction and sleep, decreased fatigue on the Q4 call schedule.	Level II-1

A randomized controlled trial evaluated internal medicine interns working in the MICU or CCU on a “traditional schedule” with “extended shifts” (maximum 30-h shifts) and an “intervention schedule” (maximum 16-h shifts, including an equal number of 16-h shifts during the day and at night).²⁰ On average, interns on the intervention schedule slept 5.8 more h/week (45.9 ± 5.9 h/week traditional vs. 51.7 ± 6.0 intervention, P < 0.001) and had half the rate of “attentional failures” at night (electro-oculographically documented slow rolling eye movements during waking work hours) (0.69/h traditional vs. 0.33/h intervention, P < 0.02).

A concurrent, controlled trial compared surgical interns assigned to 3 different work schedules: 2 interns on a “Q2” long-call schedule (i.e., a schedule built upon recurrent 24-36-h long shifts resulting in a resident staying overnight in the hospital every 2^nd night, or “Q2”), 3 interns on a “Q3” long-call schedule (i.e., a schedule built upon recurrent long 24-36 h long shifts alternating with shorter day shifts (typically 8-12 h in duration), such that each resident stays overnight in the hospital every 3^rd night, or “Q3”), and 2 interns on a “Q4” long-call schedule (i.e., a schedule built upon recurrent long 24-36 hour long shifts interspersed with 2 shorter day shifts such that each resident stays overnight in the hospital every 4^th night, or “Q4”) with cross coverage provided by interns on a different service every other night.²⁴ Compared to Q2 and Q3 interns, Q4 interns logged significantly more sleep (38 ± 1 [Q2] and 37 ± 1 [Q3] vs. 41 ± 1 [Q4] hours per week, P < 0.05). Compared to Q2 interns, Q4 interns had significantly lower levels of stress (3.0 ± 0.2 [Q2] vs. 1.9 ± 0.3 [Q4], P < 0.05), on-call fatigue (1.8 ± 0.2 [(Q2]) vs. 0.9 ± 0.2 [(Q4]), P < 0.05), off-call fatigue (2.4 ± 0.2 [(Q2] vs. 0.9 ± 0.3 [Q4], P < 0.05), and increased overall satisfaction (2.0 ± 0.2 [Q2] vs. 4.3 ± 0.1 [Q4], P < 0.05) as measured by a standard 0-5 Likert scale.

A pre-post study of surgical residents found a significant decrease in resident emotional exhaustion (Maslach Burnout Inventory score of 29.1 [Q3] vs. 23.1 [Q4], P < 0.02) and significant increases in motivation to work (statistically significant increases in 4 of 5 questions on the Hertzberg Motivational Dimensions questionnaire) after a reduction in extended shifts from Q3 to Q4.²⁵

A study of surgical residents found significant self-identified improvements in overall fatigue (P < 0.01), hours of sleep at home (P < 0.001), and time for reading (P < 0.01) after elimination of extended shifts (transition from a Q4 schedule to a night float system — i.e., a system where residents not working during the day come on at night to cover patients cared for by a daytime-based team — with maximum 12- to 14-h shifts).²⁶ Resident spouses in the same study reported increased availability for family events (P < 0.01).

A pre-post study of Ob/Gyn residents found no significant change in levels of stress based on resident self-assessments (8.03 traditional vs. 7.17 intervention on a visual analog scale, P < 0.08) after a reduction in the frequency of extended shifts with implementation of a night resident system,²⁷ but the intervention group was significantly less resentful of the idea of colleagues becoming pregnant during residency (score of 3.36 traditional vs. 3.87 intervention, P < 0.02).

Another pre-post study of Ob/Gyn residents found significant improvement in a composite measure of resident satisfaction after an intervention that reduced the frequency of overnight call by 30% (3.60 pre vs. 3.84 post on a 5-point Likert scale, P = 0.005), but no significant change in time for sleep, exercise, or reading.²⁸

A third pre-post study of Ob/Gyn residents found a statistically significant improvement in resident rating of sleep deprivation after institution of a night float system that decreased call frequency from Q3 to at most Q7 (Likert score 2.0 ± 1.2 pre vs. 3.3 ± 1.0 post, P < 0.05).²⁹ The authors found no statistically significant changes in 12 other measures of resident stress.

Thus, in 8/8 studies, significant improvements occurred in some indicators of resident quality of life, sleep, and/or fatigue; in no studies were indicators of resident quality of life found to worsen (Table 5).

Table 5.

Published study outcome summary

Outcome Type	Number of studies with outcome	Significant improvement with reduction in extended shifts	No change with reduction in extended shifts	Significant decrement with reduction in extended shifts
Resident Quality of Life	8	8	0	0
Resident Education	14	4	9	1
Patient Safety/Quality of Care	11	7	4	0

Resident Education

We identified 14 published studies looking at 13 unique cohorts of residents. These studies evaluated various measures of resident education, including standardized test scores and resident operative and procedural experience (Table 2). The majority of these studies focused on surgical residents, though one focused on internal medicine residents and 2 on Ob/Gyn residents.

Table 2.

Studies of resident education

Source	Study Design	Population and Setting	Intervention Type	Study Outcomes	Study Quality
Afessa30	Retrospective, “before and after” controlled trial	Internal medicine residents working in the MICU at a single hospital	Transition from a Q3 call system to a model with maximum 14-h work shifts	No change in resident performance on a standardized post rotation examination.	Level II-3
Barden32	Retrospective, “before and after” controlled trial	Surgical residents at a single hospital	Reduction in ICU call frequency from Q2 to Q3; elimination of overnight call for PGY2/3 residents on other rotations via a night float system	No change in operative case volume per graduating chief resident; increase in ABSITE scores with new system.	Level II-3
Carey29	Retrospective, “before and after” controlled trial	Ob/Gyn residents at a single hospital	Reduction in extended shifts from Q3 to at most Q7, with most shifts <13 h	Increase in CREOG in-training examination scores with the new system.	Level II-3
Cockerham38	Prospective, “before and after” controlled trial	Surgical residents at a single hospital	Elimination of call for PGY 4/5 residents via a night float system	Decrease in percent of time spent on non-patient care activities with night float system.	Level II-3
de Virgilio34	Retrospective, “before and after” controlled trial	Surgical residents at a single hospital	25% reduction in call frequency for all residents	No change in ABSITE or Qualifying Examination scores; increase in total number of graduating chief resident operative cases.	Level II-3
Ferguson33	Prospective, “before and after” controlled trial	Surgical residents at a single hospital	Reduction in call frequency from Q3 to Q4	No change in mean operative volume per resident.	Level II-3
Goldstein26	Prospective, “before and after” controlled trial	Surgical residents at a single hospital	Transition from Q4 call to a night float system with 12- to 14-h shifts	No change in total number of operative cases performed by graduating chief residents.	Level II-3
Hutter25	Prospective, “before and after” controlled trial	Surgical residents at a single hospital	Reduction in call frequency from Q3 to Q4	No change in ABSITE scores; decrease in attending physician perception of resident skills.	Level II-3
Jarman37	Prospective, “before and after” controlled trial	Surgical residents at a single hospital	Transition from traditional Q3-Q4 call to a night float system with 14-h shifts	Increase in percentage of cases performed by residents after institution of night float system.	Level II-3
Kelly31	Retrospective, “before and after” controlled trial	Ob/Gyn residents at a single hospital	Transition from traditional Q3-Q4 call to a night float system with the longest shifts averaging 15.5 h	No change in total number of surgical cases on resident service or CREOG in-training examination scores.	Level II-3
Malangoni36	Prospective, “before and after” controlled trial	PGY4/5 surgical residents at a single hospital	50% reduction in call shifts per month per resident	No change in percent of operations performed by residents.	Level II-3
McElearney35	Retrospective, “before and after” controlled trial	Surgical residents at a single hospital	Elimination of extended shifts for PGY1/3 residents via transition to a night float system with 13-h shifts	No change in mean operative volume per resident per month.	Level II-3
Sawyer24	Prospective, concurrent controlled trial	Surgical interns at a single hospital	Comparison of interns on Q2, Q3, and Q4 call schedules	More operative cases per month on the Q4 call schedule.	Level II-1
Welling39	Prospective, concurrent controlled trial	Surgical residents at 2 affiliated teaching hospitals	Comparison of Q4 call schedule to night float system with maximum 13-h shifts	No change in conference attendance by residents.	Level II-1

A prospective study of medical residents working in the MICU found no difference in scores on a post-rotation examination in residents working a night float system with 14-h shifts compared to residents working a traditional Q3 call schedule (81.9% ± 16.6% pre vs. 86.9% ± 8.8% post, P < 0.43).³⁰

A pre-post study of Ob/Gyn residents at a single hospital found increases on CREOG (Ob/Gyn in-service exam) scores after decreasing call frequency from Q3 to Q7 (mean percentile score 60 ± 24 pre-, vs. 71 ± 22 post-intervention),²⁹ Although the authors did not report the results of statistical testing, this change represents a statistically significant improvement (t-test, P = 0.03).

Another pre-post study of Ob/Gyn residents who transitioned from a system with extended shifts every 3^rd or 4^th night (Q3-Q4) to a night float system found no change in CREOG scores (mean percentile scores 37 pre vs. 16.5 post, P = ns) or number of surgical cases performed on the resident service (number of cases 141 pre vs. 173 post, P = ns) after the elimination of extended shifts via the introduction of a night float system.³¹

A concurrent, controlled trial compared surgical interns assigned to Q2, Q3, and Q4 schedules.²⁴ Q4 and Q3 Interns logged significantly more operative cases than Q2 interns (8 ± 1 [Q2] vs. 16 ± 2 [Q3] and 20 ± 3 [Q4] cases per month, P < 0.05). Monthly surveys of faculty blinded to intern schedule noted more operating room participation by Q4 and Q3 interns compared to Q2 interns (1.9 ± 0.1 [Q2] vs. 3.2 ± 0.2 [Q3] and 3.0 ± 0.2 [Q4], P ≤ 0.05) as measured by a standard 0-5 Likert scale.

A pre-post study of general surgery residents that involved eliminating the majority of extended (i.e., > 16 h) shifts for PGY 2 and 3 residents on most rotations by implementing a night float system, and decreasing the frequency of extended shifts from Q2 to Q3 in the intensive care units found a significant increase in resident ABSITE (surgical in-service exam) scores (P < 0.02).³² In this same study, the total number of cases performed by graduating chief residents was not significantly changed (1,015 ± 148 pre- vs. 1,116 ± 176 post-intervention).

A larger pre-post study of surgical residents did not find a significant change in ABSITE scores after decreasing the frequency of extended shifts from Q3 to Q4 (P < 0.09).²⁵ The same study found a statistically significant decrease in attending physicians' perception of residents' technical skill (3.75 pre vs. 2.70 post, P < 0.01), clinical judgment (3.67 pre vs. 2.40 post, P < 0.01), and efficiency (3.57 pre vs. 2.51 post, P < 0.01) after the intervention. A separate analysis of data from the same group found no change in mean operative cases per resident per year (231 vs. 246, P < 0.61).³³

A pre-post study of surgical residents found no change in ABSITE scores (66.4 pre vs. 69.6 post, P < 0.40) or American Board of Surgery Qualifying examinations pass rate (86% pre vs. 89% post) after transition to a new schedule that reduced extended shifts per resident by 25%.³⁴ The mean number of surgical cases performed by graduating chief residents significantly increased after the new schedule (831 pre vs. 1156 post, P < 0.001).

A study of surgery resident operative cases before and after institution of a night float system that eliminated shifts exceeding 16 hours through implementation of a night float system with maximum 13-h shifts found no significant change in the mean number of operative cases performed by residents (23.9 ± 16.3 pre vs. 24.6 ± 16.1 post cases/month, P < 0.71).³⁵

Another study looked at mean operative experience in surgical residents after transition from a traditional Q4 system to a night float system with maximum 12- to 14-h shifts and found no significant change in total cases performed by graduating chief residents (1334 pre vs. 1294 post, P < 0.59), though there was an apparent redistribution of case-load during the residency, with a decrease during the PGY4 year (from 270 ± 41 pre to 207 ± 32 cases post, P < 0.05).²⁶

One study found no change in the percentage of trauma or emergency operations performed by PGY4/5 residents (82% vs. 82%) after transition to a schedule that reduced extended shifts by 50% per resident, from 12.3 to 6.1 extended shifts per month.³⁶

Another study of surgical residents recorded the percentage of operations performed by residents during 3 successive 9-month phases.³⁷ They found that the percentage of cases performed during the baseline Q3-Q4 phase (55.2%) decreased after the institution of an initial 80-h/week restriction (47.3%), but then increased back to baseline after institution of a night float system that limited residents to maximum 14-h shifts (54.5%). The authors do not report statistical test results, but by Fisher exact tests, the change between time points 1 and 2 (P < 0.001) and between points 2 and 3 (P < 0.001) were statistically significant; the percentage of cases cared for by residents in time points 1 and 3 (P = 0.31) did not differ.

A pre-post study of surgical residents that involved decreasing shifts for PGY4/5 residents from 24 to 12 h found a decrease in the overall percentage of time spent on non-patient care activities (including administrative tasks, attending conferences, independent study in hospital, sleep in hospital, and personal in hospital) (33.3% pre vs. 21.3% post for PGY1-3 residents, 39.3% pre vs. 28.0% post for PGY4-5 residents, and 42.0% pre vs. 20.6% post for PGY6 residents), though they do not state whether these differences were statistically significant,³⁸ and insufficient data were provided to allow for independent testing.

A prospective observational study of surgery residents at 2 separate hospitals compared a Q4 call schedule with 24-h call shifts to a night float system with maximum 13-h shifts. Required conference attendance was 100% in both groups, but the authors did not assess any other educational outcomes.³⁹

Overall, in 4/14 studies, significant improvements occurred in indicators of resident educational quality after reduction or elimination of residents' extended shifts. In 9/14 studies, no significant changes were found. Education was found to worsen significantly in 1/14 (Table 5).

Patient Safety and Quality of Care

Eleven published studies included some measure of patient safety or quality of clinical care as an outcome (Table 3). A prospective, randomized, controlled trial of interns working in the MICU and CCU at a single institution compared a Q3 call schedule with maximum 30-h shifts to an intervention schedule that reduced the maximum shift length to 16 h (16-h shifts involved day and night hours in equal proportions).¹⁹ The study found that interns made significantly fewer serious medical errors during the intervention schedule than during the traditional schedule (136.0 traditional vs. 100.1 intervention per 1000 patient-days, P < 0.001), including significantly fewer non-intercepted serious errors (44.8 traditional vs. 28.6 intervention per 1000 patient days, P < 0.001). The rate of all serious errors in the unit as a whole was also significantly lower during the intervention schedule (193.2 traditional vs. 153.4 intervention per 1000 patient-days, P < 0.001).

Table 3.

Studies of patient safety and quality of care

Source	Study Design	Population and Setting	Intervention Type	Study Outcomes	Study Quality
Afessa30	Retrospective, “before and after” controlled trial	Internal medicine residents and 626 patients in a MICU at a single hospital	Transition from Q3 call to a 14-h work-shift model	No difference in MICU or in-hospital mortality or length of stay.	Level II-3
Bhavsar40	Retrospective, “before and after” controlled trial	Internal medicine residents and 1,003 patients admitted to the cardiology service at a single hospital	Elimination of extended shifts for senior residents on the cardiology service (no change for interns)	Improved adherence to ACS guidelines, decreased length of stay; no change in hospital mortality but improvement in 6 month mortality.	Level II-3
de Virgilio34	Retrospective, “before and after” controlled trial	Surgical residents and 11,518 trauma patients treated at a single Level 1 Trauma Center	Decrease in call frequency from an average of once every 4.8 days to once every 6.4 days	No change in complication rate or mortality rate for trauma patients.	Level II-3
Goldstein26	Prospective, “before and after” controlled trial	Surgical residents and surgical patients treated at a single hospital	Transition from Q4 call to a night float system with 12- to 14-h shifts	Improvement in Press-Ganey scores for Surgery Department and quality of care delivered by residents as rated by nurses.	Level II-3
Gottlieb42	Prospective, “before and after” controlled trial	Internal medicine residents and 1,103 patients admitted to a single VA medical center	Transition from a Q4 call schedule to schedule with maximum 16 hour shifts	Decrease in patient length of stay and medication errors with shift work schedule.	Level II-3
Horwitz41	Retrospective, controlled trial with both concurrent and historical controls	Internal medicine residents and 20,924 medicine patients admitted to a single hospital	Elimination of extended shifts for residents (no change for interns)	Decrease in ICU admission and decrease in pharmacist intervention to prevent drug errors after intervention.	Level II-1
Hutter25	Prospective, “before and after” controlled trial	Surgical residents and 3,976 surgical patients treated at a single Level 1 Trauma Center	Reduction in call frequency from Q3 to Q4	No change in complication or mortality rates for surgical cases.	Level II-3
Landrigan19	Prospective, randomized controlled trial	Internal medicine interns and 634 MICU/CCU patients admitted to a single hospital	Comparison of Q3 traditional call schedule with 30-h shifts to system with maximum 16-h shifts	Decrease in serious medical errors made by interns with shift work system compared to Q3 call schedule.	Level I
Malangoni36	Prospective, “before and after” controlled trial	PGY4/5 surgical residents and 3,100 trauma/emergency surgery patients admitted to a single Level 1 Trauma Center	50% reduction in call shifts per month per resident	Decrease in mortality for trauma and emergency surgical patients.	Level II-3
Mann43	Prospective, “before and after” controlled trial	Radiology residents and 26,421 emergency radiology cases reviewed at a single hospital	Elimination of extended shifts for radiology residents via implementation of dedicated night shifts	Decrease in frequency of mis-read films requiring patient call back to the Emergency Department.	Level II-3
Sawyer24	Prospective, concurrent controlled trial	Surgical interns at a single hospital	Comparison of interns on Q2, Q3, and Q4 schedules	No difference in number of errors per call shift.	Level II-1

A prospective study compared MICU residents on a traditional schedule (Q4 call) to those on an intervention schedule with maximum 14-h shifts.³⁰ In multivariate analyses controlling for predicted hospital mortality rate, there was no significant difference in ICU mortality (OR 0.46, 95% CI 0.17 to 1.22, P < 0.12) or in-hospital mortality (OR 0.83, 95% CI 0.44 to 1.58, P < 0.57) post-intervention. Similarly, there was no significant difference in ICU length of stay ratio (0.62 ± 0.54 vs. 0.66 ± 0.74, P < 0.52) or hospital length of stay ratio (0.66 ± 0.64 vs. 1.26 ± 7.98, P < 0.38), though the authors note that the study was not adequately powered to detect differences in mortality or length of stay.

A retrospective study of internal medicine residents rotating on a cardiology service found improvements in adherence to evidence-based guidelines for Acute Coronary Syndrome after introduction of an intervention that eliminated extended (> 16-h) shifts for senior residents through introduction of a night float system (no change in schedule for interns).⁴⁰ Specifically, patients were more likely to receive β-blockers (85.8% pre vs. 93.8% post, P < 0.001), angiotensin-converting enzyme inhibitors/angiotensin II receptor blockers (65.7% pre vs. 71.8% post, P < 0.046), and statins (76.2% pre vs. 84.0% post, P < 0.002) at the time of discharge. The same study found a 10% reduction in patients' median length of stay (3.1 pre vs. 2.8 days post, P < 0.002). No significant differences in risk-adjusted in-hospital mortality (odds ratio [OR] 0.47, 95% confidence interval [CI], 0.18-1.20, P < 0.11) or 6-month major adverse cardiovascular events (OR 1.29, 95% CI, 0.76-2.20, P < 0.35) were found post-intervention, but there was a significant decrease in risk-adjusted 6-month mortality (OR 0.53, 95% CI, 0.28-0.99, P < 0.05) in the post-intervention group.

A retrospective study of over 20,000 patients admitted to an internal medicine teaching service compared 7 quality of care and patient safety outcomes after an intervention that eliminated extended shifts for residents (no change in schedule for interns) through the use of “nocturnalists,” including 2 hospitalists and a pool of moonlighting fellows.⁴¹ Relative to changes that occurred for non-teaching (control) patients, rates of intensive care unit utilization (−2.1%, 95% CI, −3.3% to −0.7%; P = 0.002), discharges to home or rehabilitation facilities vs. discharges elsewhere (+5.3%, 95%CI, +2.6% to +7.6%; P < 0.001), and pharmacist interventions to prevent errors (−1.92 per 100 patient-days, 95%CI, −2.74 to −1.03; P < 0.001) improved. Teaching and non-teaching services had similar changes over time in length of stay (−0.16, 95%CI −0.53 to 0.19; P = 0.37), 30-day readmission rate (−1.99, 95%CI 4.35 to 0.49; P = 0.097), adverse drug–drug interactions (+0.31 per 100 patient-days, 95%CI −0.25 to 0.87; P = 0.30), and in-hospital mortality (−0.25%, 95%CI −1.0% to +0.56%; P = 0.50).

In a study of all house-staff working at a VA medical center, medication errors, laboratory testing, and length of stay were measured. At baseline, interns had a 32-h shift on a Q4 schedule and senior residents had a 32-h shift on a Q8 schedule. This was changed to a system that substantially reduced the frequency of 30-h shifts for interns and senior residents to once every 7 weeks; the longest shifts between these sporadic extended shifts were 16 hours.⁴² Baseline characteristics for patients remained unchanged. Patient length of stay (10.9 ± 14.7 pre vs. 9.3 ± 12.1 days post, P < 0.01), the number of laboratory tests ordered (24.0 ± 46.0 pre vs. 19.0 ± 36.0 post, P < 0.01), and medication errors per patient (16.9 pre vs. 12.0 errors per 100 patient discharges post, P < 0.02) all decreased significantly under the new schedule.

A concurrent, controlled trial compared surgical interns assigned to Q2, Q3, or Q4 schedules.²⁴ Rates of errors committed per call shift did not differ, as measured both by monthly surveys of interns (1.1 ± 0.2 [Q2], 0.8 ± 0.2 [Q3], and 0.8 ± 0.2 [Q4], P = ns) and faculty blinded to intern schedule (1.0 ± 0.1 [Q2], 0.8 ± 0.2 [Q3], and 1.2 ± 0.1 [Q4], P = ns).

A pre-post study of an intervention that reduced call frequency from Q3 to Q4 in a surgical training program prospectively examined mortality and complication rates for 1,665 pre- and 2,311 post-intervention cases. Risk adjusted data were collected by trained nurses using standardized definitions from the previously validated National Surgical Quality Improvement Program.²⁵ No change in mortality or complication rates was found. A similar pre-post study of surgical residents transitioned from a traditional Q4 call schedule to a night float system without extended shifts found statistically significant improvement in 6 different measures of patient care.²⁶ In the same study, Press-Ganey patient satisfaction surveys revealed improved ratings of care across all categories, though the authors do not state whether the improvements in patient ratings were statistically significant, and insufficient data are presented to permit independent calculation.

A pre-post study at a single Level 1 Trauma Center found a 31.9% decrease in mortality (from 72 deaths in 1550 admissions pre to 49 deaths in 1550 admissions post) for trauma patients, following an intervention that decreased frequency of call by 50% for PGY4/5 surgical residents from 12.3 to 6.1 extended shifts per month.³⁶ Although the authors do not comment on the significance of this change, it represents a statistically significant decrease (P = 0.04, 2-sided Fisher exact test). A similar study found no significant change in the overall complication rate (2.54% pre vs. 2.05% post, P < 0.09) or the mortality rate (5.23% pre vs. 5.91% post, P = 0.12) for trauma patients treated at a single Level 1 Trauma Center after an intervention that decreased the frequency of extended shifts by 25% (from an average of once every 6.4 days to once every 4.8 days) for surgical residents.³⁴

A study of radiology residents on an Emergency Radiology rotation at one institution analyzed misreadings by a radiology resident working overnight.⁴³ After implementation of a night-float system that reduced the frequency with which on-call residents were required to work extended shifts (> 16 h), there was a significant decrease in the proportion of misreadings (1.69 pre vs. 1.00 post per shift, χ² = 11.79, P < 0.037).

In sum, in 7/11 studies, significant improvements occurred in processes and outcomes indicating the quality and/or safety of patient care after reduction or elimination of residents' extended shifts, and in 4/11 studies, quality and safety remained unchanged. No studies found that safety or quality of care deteriorated (Table 5).

Unpublished Studies

Two of three unpublished studies identified found that reduction of extended shifts was associated with improvements in quality of life measures and/or neurocognitive performance; one found that safety improved (Table 4).

Table 4.

Overview of unpublished studies

Source	Study Design	Population and Setting	Intervention Type	Study Outcomes	Publication Status
Gohar54	Prospective, concurrent controlled trial	Internal medicine interns and residents	Compared residents working a Q4 call schedule to those on a schedule without extended shifts.	Lower sleepiness score and fewer math and accuracy errors on the intervention schedule.	Not yet published at time of review; subsequently published in J Clin Sleep Med57
Gottlieb55	Prospective, “before and after” controlled trial	Internal medicine residents	Transition from an on-call schedule to a night float system without extended shifts.	Decreased patient length of stay and fewer medication errors under the night float system.	Follow up study including more patients was published two years later40
Graef56	Prospective, concurrent controlled trial of a single resident cohort	Internal medicine residents in a single residency program	Comparison of a single cohort of residents working both one month on a traditional call schedule and one month on a night float schedule	Less resident sleepiness during overnight hours during night float schedule.	Not yet published

DISCUSSION

In a systematic review of the literature, we found that reduction or elimination of resident shifts exceeding 16 hours led to improvements in resident quality of life (8/8 improved), improvements in patient safety and quality of care (7/11 improved; 4/7 unchanged; 0/7 worsened); and generally unchanged medical education (4/14 improved; 9/14 unchanged; 1/14 worsened) (Table 5). Systematic collection of data on the evidence supporting proposed policy change is important both to guide and facilitate that change. Acceptance of the ACGME Duty Hour Standards was impeded by a lack of evidence supporting the specific limits chosen, variation in the implementation of work hour reforms, and uncertainty about their impact.^44–46 Under the ACGME standards, shifts of 24-30 hours have remained the norm,⁴⁷ but in the past five years, data have emerged strongly indicating that such extended shifts pose significant hazards for patients and residents.^3–10 In this systematic review, we gathered together both published and unpublished data regarding programs that reduced or eliminated these extended shifts. While the precise nature of the changes made varied from one center to the next, all interventions included in our study either eliminated or reduced the frequency of shifts exceeding 16 hours (extended shifts). Most led to improvements in patient safety and quality of life, without adverse educational consequences. This new evidence could be of value to policy makers and program directors as they seek to develop evidence-based work hour reforms. Further data comparing specific approaches to eliminating or reducing extended shifts continue to be needed.

Several prior reviews have evaluated the literature regarding resident working conditions. Two early reviews by Asken et al. and Leung et al. found a paucity of high quality studies and reported generally mixed results.^48,49 Two more recent rigorous systematic reviews by Fletcher et al. found that while resident quality of life generally improved with reduction of resident work hours, educational experience was mixed, and insufficient information existed to draw conclusions about patient care.^1,2 While Fletcher's reviews were of high quality, they included studies of a wide variety of different interventions aimed at improving resident working conditions, making it difficult to draw conclusions about any one specific intervention. In contrast to Fletcher, we chose to focus on studies of a single type of evidence-based intervention, the reduction or elimination of extended shifts, many of which were published after Fletcher's reviews. We found that quality of life measures improved in eight of eight studies and patient safety improved in the majority; in no studies did patient safety worsen. Moreover, both surgical case volume and objective measures of medical education were generally unchanged, countering the concern that reductions in extended shifts degrade educational experience (Table 5).

While we did not find evidence supporting concerns that implementation of the IOM standards could lead to safety hazards due to discontinuities of care (a previously demonstrated safety hazard⁵⁰) or a degradation of education, our study does not rule out the possibility that either could potentially occur. In programs that reduce or eliminate 24-hour shifts, substantial efforts will need to be undertaken to develop robust handoff systems⁵¹ and redesign educational curricula⁵² to minimize any adverse consequences of reducing hours. Prior research has demonstrated that discontinuity can pose hazards. Implementation of the IOM recommendations is also likely to be expensive, at least in the near term; Nuckols et al. estimated that hiring the providers needed to supplement the existing resident work force will cost approximately $1.6 billion,¹⁷ though some of the surgical studies included in this review, which demonstrated maintenance of surgical resident case volume even in the face of decreasing work hours, would suggest that additional hiring will not be needed in all settings. If Nuckols' projection regarding hiring is correct, such a cost would be recouped by society (due to savings resulting from decreased adverse events) if an 11% reduction in adverse events accompanies implementation of the IOM's recommendations. This degree of improvement or greater did occur in some of the studies included in this review, but further multi-center studies are needed to determine if adverse events would in fact be reduced to this degree with implementation of work hour reductions across settings.

Our review has a number of limitations. Though many of the studies on quality of life used validated psychological tests, measuring quality of life remains a subjective enterprise. Since residents cannot be blinded to the frequency of the extended shifts that they work, there will always be room for bias in any self-reported measures. While such subjective outcomes were used in a minority of included studies of education and patient safety, they were an important component of quality of life measures. Another limitation is that while studies of quality of life and safety were reasonably distributed across specialties, studies of medical education involved primarily surgical trainees, limiting the external validity of these findings. Moreover, long term educational outcomes were not assessed. A third limitation is that many intervention efforts we studied included changes to resident schedules in addition to extended work shift reduction, and not all studies objectively measured work hours; this limits the strength with which changes in outcome can be causally attributed to reducing or eliminating extended work shifts. In some studies, residency programs implemented enhancements to the resident sign-out process as they reduced extended shifts, which limits our ability to disentangle the effects of reducing extended shifts from this important co-intervention. However, no other intervention was common to all studies included in our review, reducing the likelihood that any other co-intervention was responsible for the effects observed across studies.

Though all of the studies in our review were Level I or II quality studies according to the U.S. Preventive Services Task Force rating schema,²² many had important methodological limitations. Some failed to provide sufficient statistical data to allow for meta-analysis of results. Some were underpowered to detect significant differences in outcomes of interest. Most utilized a pre-post design, which is subject to time series bias (i.e., confounding of study results secondary to ongoing improvements in education or medical care unrelated to the intervention of interest). However, there was a single prospective, randomized controlled trial and several non-randomized concurrent trials which found significant improvements associated with reduction or elimination of extended shifts. The precision with which this review supports a specific work hour limit is also somewhat limited. Our review does not provide evidence on outcomes in 16-hour as opposed to 18-hour shift systems, for example, though it does suggest that shifts of 12-16 hours are associated with significantly better outcomes than shifts of 24 hours or longer. Complicating the issue further, however, the precise method by which extended shifts were eliminated or reduced varied from one study to the next, and the pattern of shift scheduling as well as total work hours is highly likely to affect resident performance and hence the success of an intervention.⁵³ The current study was not designed to differentiate these nuances of approach, but future multi-center studies should attempt to directly compare divergent approaches to work hour reduction. Lastly, our review is subject to publication bias, as reports of successful interventions may be more likely to reach publication than those that fail. To limit publication bias, we searched several different databases of unpublished abstracts and manuscripts. The findings of the unpublished studies (Table 4) were generally commensurate with those of the published studies included in our review.

As policy makers and residency program directors seek to identify solutions to the well-documented hazards of residents' traditional long work hours, elimination or reduction of the frequency of extended shifts will be an important consideration. We found that decreasing or eliminating extended work shifts led to significant improvements in diverse aspects of resident quality of life. Additionally, we found that decreasing or eliminating extended shifts is unlikely to harm surgical resident education, though more studies are necessary to see if this remains true for non-surgical residents. Finally, and perhaps most importantly, decreasing or eliminating extended shifts is likely to lead to reductions in medical errors and improvements in the quality of patient care, though larger studies are necessary to see if these improvements translate into decreased patient morbidity or mortality.

DISCLOSURE STATEMENT

This was not an industry supported study. Dr. Landrigan has consulted for AxDev on a study that was supported by a grant to AxDev from Cephalon and has consulted for Vital Issues in Medicine. The other authors have indicated no financial conflicts of interest.