The Effect of Restricting Residents' Duty Hours on Patient Safety, Resident Well-Being, and Resident Education: An Updated Systematic Review

Abstract

Background

Despite 25 years of implementation and a sizable amount of research, the impact of resident duty hour restrictions on patients and residents still is unclear. Advocates interpret the research as necessitating immediate change; opponents draw competing conclusions.

Objective

This study updates a systematic review of the literature on duty hour restrictions conducted 1 year prior to the implementation of the Accreditation Council for Graduate Medical Education's 2011 regulations.

Methods

The review draws on reports catalogued in MEDLINE and PreMEDLINE from 2010 to 2013. Interventions that dealt with the duty hour restrictions included night float, shortened shifts, and protected time for sleep. Outcomes were patient care, resident well-being, and resident education. Studies were excluded if they were not conducted in patient care settings.

Results

Twenty-seven studies met the inclusion criteria. Most frequently, the studies concluded that the restrictions had no impact on patient care (50%) or resident wellness (47%), and had a negative impact on resident education (64%). Night float was the most frequent means of implementing duty hour restrictions, yet it yielded the highest proportion of unfavorable findings.

Conclusions

This updated review, including 27 recent applicable studies, demonstrates that focusing on duty hours alone has not resulted in improvements in patient care or resident well-being. The added duty hour restrictions implemented in 2011 appear to have had an unintended negative impact on resident education. New approaches to the issue of physician fatigue and its relationship to patient care and resident education are needed.

Introduction

Despite a large number of primary studies and numerous systematic reviews, the impact of resident duty hour restrictions on patients and residents remains unclear. In turn, reviewers find the impact to be favorable, unfavorable, neither, or inconclusive.^1–6 For programs that have implemented these changes, and for those about to, this literature is vexing.

From 2004 to 2013, 9 peer-reviewed systematic reviews of the duty hour literature were conducted (table 1).^1–9 Eight of the reviews examined the impact of duty hour restrictions on patient safety. A total of 2 reported a positive impact,^1,5 2 reported no impact,^3,6 and 4 found that the effect differed across studies.^2,4,7,9 Five reviews examined the impact on resident well-being, with 4 finding positive changes.^1,4,8,9 Resident education is examined in 5 reviews, with 1 concluding the impact is unfavorable,⁵ 2 concluding there is no impact,^4,6 and 2 finding that the impact is inconclusive.^8,9 Despite these equivocal findings, organizations that represent both patients and residents cite empirical support for action. The Bell Commission, which instituted New York State's limit on resident work hours¹⁰; the Institute of Medicine's Committee on Optimizing Graduate Medical Trainee (Resident) Hours and Work Schedule to Improve Patient Safety¹; the Accreditation Council for Graduate Medical Education (ACGME)¹¹; the Fédération des médecins résidents du Québec¹²; and Canada's National Steering Committee on Resident Duty Hours¹³ all point to the literature in their campaigns to limit duty hours. Others, notably residents,^14,15 program directors,¹⁶ and other experts,¹⁷ read the same body of literature and concluded that restrictions may be detrimental.

TABLE 1.

Results of 9 Systematic Reviews of the Impact of Resident Duty Hour Restrictions on 3 Categories of Outcomes

Although much evidence has been produced for this debate, it has been inconclusive so far. The evidence includes systematic reviews; however, the most recent review⁵ assessed that the effect of night float, shortened shifts, or protected time for sleep is out of date. This review, by Reed and colleagues,⁵ was completed 1 year prior to when the ACGME's 2011 duty hour regulations took effect, and before the publication of hundreds of additional potentially pertinent studies. The purpose of this study is to update this systematic review of the duty hour literature.

Methods

As an update, this review modeled the methodology of the original study, including the search query, data sources, eligibility criteria, study selection and data extraction, assessment of study quality, and data synthesis. Here we present a concise description of the methodology: the full description is available in the original study.⁵ Our review followed the PRISMA guidelines for systematic reviews.¹⁸

Eligibility Criteria, Study Selection, and Data Extraction

We included English-language original research studies published in MEDLINE and PreMEDLINE. The literature search for the original review included articles published between January 1, 1989, and May 21, 2010. The search for our review included all relevant literature between May 22, 2010, and February 4, 2014. We included studies that implemented duty hour restrictions by reducing shift length, providing protected time for sleep, and implementing night float. As in the original review, shift length is defined as the number of consecutive hours worked by residents without protected sleep. Protected time for sleep is defined as a period during a work shift in which residents transfer all of their responsibilities (such as admitting and cross-covering patients, and performing procedures) to another individual, so that they can obtain uninterrupted sleep. Night float is defined as a staffing system in which dedicated residents work during the night and not during the day.

As in the original study, we included shift length and night float studies only if the studies occurred in actual practice settings, and described at least 1 outcome related to patient care, residents' health, or residents' education. Studies with any outcome that examined protected time for sleep for residents were included.

The search strategy yielded at total of 1362 unique citations. Uncertainties regarding their fitness for inclusion were resolved through discussion among the investigators. The interrater reliability for study inclusion was excellent (κ =  0.81) as κ > 0.80 indicates near perfect agreement.¹⁹ A total of 203 abstracts were selected for full text review, and 27 articles were included in the study (figure). Both authors independently reviewed the articles and performed data extraction. Disagreements were resolved by discussion and consensus, and the interrater agreement was calculated for study inclusion and for the qualitative characterization of each study's overall effects.

FIGURE.

Summary of Evidence Search and Selection

Data were entered into a structured form to facilitate the abstraction of information about study design, sample characteristics, type of intervention, outcomes, and study quality.

Assessment of Study Quality

The Medical Education Research Quality Study Instrument (MERSQI) was used to evaluate the studies' quality.²⁰ The MERSQI highlights 6 aspects of a study, including the type of outcomes it evaluates, the research design, the sampling strategy, and the data collection and analysis procedures. Since its introduction in 2007, its psychometric properties have been investigated in multiple studies, and evidence is accruing for its validity and reliability.^20–22 These data were extracted in duplicate, and intraclass correlations were calculated to measure interrater reliability.

Data Synthesis

Consistent with the original review, the data were synthesized qualitatively. We grouped the studies by the intervention type (shift length, night float, protected sleep time), and recorded their results categorically as having a favorable impact, unfavorable impact, or no impact on the following 3 classes of outcomes: patient care, resident education, and resident well-being. A study was determined to be favorable if its intervention had achieved a positive outcome, unfavorable if it had a negative outcome, and no impact if the intervention had a neutral outcome. Both authors coded each study's outcome, and any disagreements were resolved by discussion. The interrater reliability for the qualitative categorization of study effects overall was substantial (κ = 0.76). When a study examined the impact on more than 1 outcome it was included in each group. We tallied the frequency of results for each category of outcome and each type of intervention. Due to the heterogeneity of the outcome measures, we did not synthesize the results quantitatively, and calculation of effect size was not performed.

Results

There were 27 studies included in the final review: 13 (48%) reviewed shift length,^23–35 12 (44%) examined night float,^36–47 and 2 (7%) examined protected time for sleep.^48,49 Some studies measured more than 1 outcome, with 10 studies (37%) assessing patient care,^{23–28,31,40,41,48} 17 studies (63%) reporting on resident well-being,^{32–39,41–49} and 14 studies (52%) assessing educational impact.^{26,29–36,41–43,45,46} Across all of the studies and interventions, the impact of duty hour restrictions most frequently had no impact on patient care (5 of 10, 50%),^{23,24,28,41,48} no impact on resident well-being (7 of 17, 41%),^{27,34,35,41,44,46,49} and an unfavorable impact on resident education (9 of 14, 64%).^{29,30,33–36,41,43,46} A minority of studies found a favorable impact on resident well-being (4 of 17, 24%; table 2).^32,33,45,48 When analyzed by intervention type, the most frequent result of shift length changes was no impact (8 of 20, 40%),^{23,24,27,28,31,32,34,35} that of night float was an unfavorable impact (10 of 18, 56%),^{35–39,41–43,46,47} and for protected time for sleep, 1 study reported a favorable outcome for resident wellness (1 of 2, 50%)⁴⁸ but no impact on patient care, while the other study was unfavorable for resident well-being (1 of 2, 50%).⁴⁹

TABLE 2.

Summary of Studies

Study Quality

The mean (SD) MERSQI score for the 27 studies was 10.55 (2.38, maximum 18 points). The interclass correlation for the 2 authors' MERSQI scoring was excellent (intraclass correlation coefficient = 0.90). Thus, the quality of these studies is consistent with the quality of studies in the original review, which reported a mean MERSQI score of 11.90.⁵ As a set, the studies had consistent shortcomings in 4 of the 6 dimensions of MERSQI quality measures. Most frequently, analyses were based on a single institution setting (19 of 27, 70%),^{23–25,28,31–43,45,49} outcomes were assessed through self-reporting (15 of 27, 56%),^{27,30,31,34–39,41–43,45,46,49} no information was provided on the psychometric properties of the measurement tools (19 of 27, 70%),^{23–26,29–36,41–43,45,46,48,49} and measurement focused on lower-level outcomes, such as user reaction or knowledge gains (10 of 27, 37%),^{26,27,29,30,34–36,42,45,46} instead of the higher-level outcomes, such as behavior or patient outcomes.

Shift Length

Table 3a outlines the 13 studies examining shift length and shows the study design, quality score, and outcomes.^23–35 The length of shifts that were studied ranged from 13 to 36 hours. The populations of interest, and subsequently the study outcomes, were not consistent across all of the studies. There were 2 studies reviewing surgical cases, using complication rates as outcomes (total n = 6066)^23,24; 9 studies involving residents with a variety of patient care, resident wellness, and resident education outcomes (total n = 3618)^{26,27,29–35}; and 2 studies involving patients with length of stay, mortality, and readmission rates used as outcomes (total n = 4655).^25,28 Of the 2 studies of surgical cases, 1 found no difference in mortality for surgeries performed on shifts shorter than 16 hours compared with shifts longer than 16 hours, but this study found a slightly higher complication rate for procedures performed during shifts longer than 16 hours.²⁴ The other study found no difference in complication rates for surgeries performed on shifts longer than 16 hours.²³ Two studies found shorter length of stay for patients with the shorter shift length,^25,31 although another study found no difference in length of stay,²⁸ and 3 found no difference in mortality.^24,28,31

TABLE 3a.

Outcomes Associated With Duty Hour Reduction Interventions—Reduce Shift Length

TABLE 3a.

Extended

Regarding resident wellness outcomes, the interventions most frequently had no impact (3 of 5, 60%).^27,34,35 Notably, the only randomized controlled trial in the shift length intervention group found no impact on resident well-being with shifts of 12 versus 30 hours.³⁴ Resident education outcomes were most frequently unfavorable (5 of 8, 63%),^{29,30,33–35} with decreases in staff supervision, opportunities for assessment, attendance at educational activities, and decreased operative cases reported as negative outcomes.

Night Float

Table 3b shows the study design, quality score, and outcomes of the 12 studies examining night float.^36–47 Studies predominantly examined resident well-being (10 of 12, 83%),^{37–39,41–47} and the majority reported unfavorable outcomes (6 of 10, 60%), such as decreased sleep,^36,38,43 higher fatigue,^39,47 and higher stress,⁴³ compared to traditional call. The majority of studies that examined educational outcomes demonstrated worse outcomes with night float (4 of 6, 67%), with decreased attendance at teaching conferences,^36,41–43 less time per week spent with attending physician,⁴³ and less time for independent reading.⁴³ Only 2 studies examined patient care outcomes, and their results are contradictory. One demonstrated decreased diagnostic errors,⁴⁰ and the other reported no differences in length of stay, 30-day readmission rates, 30-day mortality, codes, rapid response team calls, or intensive care unit transfers.⁴¹

TABLE 3b.

Outcomes Associated With Duty Hour Reduction Interventions—Night Float (NF)

TABLE 3b.

Extended

Protected Time for Sleep

Two studies with high-quality ratings examined the effect of protecting time for sleep.^48,49 A randomized controlled trial of 103 residents examined the impact on patient care and reported no significant impact on mortality, length of stay, or readmission rates for the 2657 patients cared for by the residents.⁴⁸ Both studies examined the impact on sleep as a measure of resident well-being, and results are contradictory. One found that protected time improved the quality and quantity of residents' sleep,⁴⁸ while the other found no differences⁴⁹ (table 3c).

TABLE 3c.

Outcomes Associated With Duty Hour Reduction Interventions—Protected Sleep Time (Nap)

Discussion

We reviewed the literature that was published during this period (2010–2014) to clarify the impact of duty hour restrictions on patient safety, resident education, and resident wellness. Across 27 studies with 41 separate analyses, the largest proportion of results pointed to an unfavorable impact (16 of 41, 39%) and the second largest to no impact (15 of 41, 37%). Of the 3 types of outcomes, resident education garnered the highest proportion of unfavorable outcomes, followed by resident well-being. Of the 3 approaches for adapting resident schedules to duty hour restrictions, night float was associated with the highest proportion of unfavorable results.

The majority of previous reviews have examined retrospective outcomes before and after duty hour implementation, based on when the ACGME regulations took effect, and the studies included in these reviews did not state how duty hour regulations were implemented.^3–5,7 The review by Reed et al⁵ and our review highlight that not all strategies to comply with duty hour regulations are equally effective. Reduced shift length is the strategy with the best evidence for improved patient care, although this is still only modest evidence, as there are numerous studies that demonstrate no impact. For the modest improvements, at best, in patient care, the reduced shift length did improve resident wellness and had a negative impact on education in most studies (table 3a).

Night float is an increasingly common strategy, yet only 1 study in our review demonstrated a positive impact on patient care⁴⁰ and 1 demonstrated a favorable impact on residents.⁴⁵ The remainder of the studies demonstrated either an unfavorable impact or no change to resident well-being. Further, 4 of 6 studies (67%) found a negative impact on resident education. This is in contrast to the review by Fletcher et al⁹ that concluded resident quality of life may be improved by duty hour restrictions, and that the impact on education was unclear.

As more prospective studies with defined interventions become available, there is increasing evidence that some strategies (night float) for duty hour regulations may have a negative impact on resident well-being. The vast majority of interventions did not use the suggested 4-night maximum recommended by the Institute of Medicine; therefore, these findings may be attributable to the acute sleep deprivation that has been reported with increased numbers of consecutive nights on call.⁵⁰ Previous reviews, including the 1 we updated, have been supportive of duty hour restrictions, although many have determined that the evidence is unclear.^1–3,6 The original intended purpose of duty hour changes was to enhance patient safety through enhanced supervision and sleep.^1,9,11 Many of the reviews have found no conclusive evidence that duty hour restrictions have improved patient care; therefore, one might conclude that duty hour restrictions have not had their intended impact. There is also a growing body of evidence to suggest that there is no definitive improvement in resident well-being, and that there is a potential negative impact on resident education. It is worth noting that while the issue of supervision after the institution of the duty hour limits is relatively underresearched, 2 of the studies in our review found decreased time with attending physicians and decreased supervision as a result of duty hour regulations. Enhanced supervision was a key feature in the Institute of Medicine report and the ACGME standards; however, it appears to not have been addressed in the majority of interventions, and may actually be decreased by implementing duty hour restrictions.

Limitations of this review are similar to those of the original study. Few of the studies were randomized controlled trials.^34,36,48 However, many used a nonrandomized 2-group design, which is a reasonable design for an educational intervention. Unfortunately, the studies in this review used heterogeneous outcome measures, precluding quantitative synthesis and comparisons. Further, while the most important patient care outcomes are arguably mortality and morbidity, only 60% (6 of 10) of studies that reviewed patient care looked at mortality and morbidity outcomes.^{23–25,28,41,48} Regarding education, the outcome measures were largely related to attendance at teaching sessions and time with faculty. While important, these are rough surrogates for more meaningful educational outcomes.

As an update to a previous literature review with strict inclusion criteria, the studies in our review do not represent all of the literature on this topic. As in the original review, we excluded studies that did not occur in actual practice settings. The purpose of this criterion is to accentuate results that are ecologically valid over results derived from laboratory studies on the effects of sleep deprivation. Further, by limiting our review to studies with defined interventions, the results of each method of limiting duty hours are more apparent. Without a clearly defined intervention to implement duty hours, it is difficult to determine cause and effect, and even associations are less clear due to confounders. Given these limitations, this review does not definitively answer the questions regarding the impact of duty hour restrictions. However, the studies included are the most applicable, current evidence on this topic, and the number of studies in our review is consistent with other reviews of issues in medical education.^51–54

Future directions for this field should include attempts to standardize outcome measures across studies so that there will be more generalizability and studies will be able to be compared more accurately. Additionally, duty hours are only 1 factor in the safety of patients cared for by residents; handover-related communication errors and supervision of residents have also been identified as contributing factors. Future studies should seek to assess the impact of a more comprehensive approach to improving patient care. Recently, a prospective handoff “bundle” intervention study demonstrated a significant reduction in medical errors, and demonstrated proof of the concept of studying the impact on patient care from combining simultaneous interventions.⁵⁵

Conclusion

An updated review of 27 recent and highly applicable studies demonstrates that focusing on interventions to limit resident duty hours alone has not had the expected, consistent improvements to patient care, resident education, or resident well-being. Further duty hour restrictions appear to have had an unintended negative impact on resident education. It is time to reevaluate the profession's approach to the issue of resident hours and its impact on patient care and residency education.