Skip to main content
Journal of General Internal Medicine logoLink to Journal of General Internal Medicine
. 2011 Mar 3;26(8):907–919. doi: 10.1007/s11606-011-1657-1

Patient Safety, Resident Education and Resident Well-Being Following Implementation of the 2003 ACGME Duty Hour Rules

Kathlyn E Fletcher 1,, Darcy A Reed 2, Vineet M Arora 3
PMCID: PMC3138977  PMID: 21369772

Abstract

Context

The ACGME-released revisions to the 2003 duty hour standards.

Objective

To review the impact of the 2003 duty hour reform as it pertains to resident and patient outcomes.

Data Sources

Medline (1989–May 2010), Embase (1989–June 2010), bibliographies, pertinent reviews, and meeting abstracts.

Study Selection

We included studies examining the relationship between the pre- and post-2003 time periods and patient outcomes (mortality, complications, errors), resident education (standardized test scores, clinical experience), and well-being (as measured by the Maslach Burnout Inventory). We excluded non-US studies.

Data Extraction

One rater used structured data collection forms to abstract data on study design, quality, and outcomes. We synthesized the literature qualitatively and included a meta-analysis of patient mortality.

Results

Of 5,345 studies identified, 60 met eligibility criteria. Twenty-eight studies included an objective outcome related to patients; 10 assessed standardized resident examination scores; 26 assessed resident operative experience. Eight assessed resident burnout. Meta-analysis of the mortality studies revealed a significant improvement in mortality in the post-2003 time period with a pooled odds ratio (OR) of 0.9 (95% CI: 0.84, 0.95). These results were significant for medical (OR 0.91; 95% CI: 0.85, 0.98) and surgical patients (OR 0.86; 95% CI: 0.75, 0.97). However, significant heterogeneity was present (I2 83%). Patient complications were more nuanced. Some increased in frequency; others decreased. Outcomes for resident operative experience and standardized knowledge tests varied substantially across studies. Resident well-being improved in most studies.

Limitations

Most studies were observational. Not all studies of mortality provided enough information to be included in the meta-analysis. We used unadjusted odds ratios in the meta-analysis; statistical heterogeneity was substantial. Publication bias is possible.

Conclusions

Since 2003, patient mortality appears to have improved, although this could be due to secular trends. Resident well-being appears improved. Change in resident educational experience is less clear.

Electronic supplementary material

The online version of this article (doi:10.1007/s11606-011-1657-1) contains supplementary material, which is available to authorized users.

INTRODUCTION

In 2003, the Accreditation Council for Graduate Medical Education (ACGME) reduced resident work hours for all US residents.1 Resident well-being and patient safety were the main concerns relating to the excessive duty hours that residents sometimes worked prior to the 2003 changes. Even after these modifications, concerns remained about how compliant residents were with the duty hour rules,2 and one rigorous study suggested that further reductions in maximum shift length could lead to safer patient care.3

The Institute of Medicine (IOM) conducted an investigation in 2008 and published recommendations for further reductions in resident duty hours.4 Partially in response to the IOM report, the ACGME convened a duty hours task force to examine the issues and propose revisions in the duty hour standards. Those proposals were made public in June 2010 and will go into effect in July 2011.5 No changes were made to the 80-h/week limit or to the maximum frequency of call (every 3rd night). However, interns will now be held to a maximum shift length of 16 h. Residents who are post-graduate year (PGY) 2 and above will still be able to work 24-h shifts, but with only 4 additional h for hand-offs (reduced from 6 h).5 As residency programs prepare for the implementation of the new standards, it would be helpful to review how patient care and residents’ lives have changed since the 2003 duty hour rules were implemented. Our objective was to synthesize the research that specifically assessed the relationship of the pre- and post-2003 time periods to patient and resident outcomes.

METHODS

Data Sources

This systematic review was part of a larger project completed at the request of the Accreditation Council for Graduate Medical Education (ACGME). The overall project goal was to examine all aspects of resident duty hours including sleep, fatigue, education, well-being, learning environment, patient safety, moonlighting, supervision, and the effects of the 2003 duty hour standards.6 Here we present the results of the studies examining the possible effect of the 2003 ACGME duty hour regulations on patient and resident outcomes. We followed PRISMA guidelines (Preferred Reporting Items for Systematic Reviews and Meta-Analyses ).7

We conducted an electronic search of the literature in Medline and Embase in June 2009. The Medline search included in-process papers and was updated in May 2010. We updated the Embase search in June 2010. We used an extensive search strategy in Medline, based on prior work8,9 and consultation with a reference librarian. We conducted a similar search in Embase. We used a combination of MESH subheadings and keywords that can be accessed online. We limited the entire list to studies published in 1989 or later and to English language papers. We reviewed the bibliographies of all included studies, and previous reviews, to identify additional studies.

To identify studies not yet published, we searched titles and abstracts from 2008–2010 national meetings of the Accreditation Council of Graduate Medical Education and the Society of General Internal Medicine. We also searched the Research in Medical Education abstracts from the meeting of the Association of American Medical Colleges for the years 2008–2009. We assumed that abstracts written before 2008 would have been published by the time we conducted our search. Finally, we had an expert review our bibliography.

Study Selection

Inclusion criteria were that the study had to contain data collected after the 2003 ACGME duty hour rules went into effect. If studies contained data prior to implementation of the 2003 duty hour rules, but the data had been obtained after instituting changes to achieve compliance with the duty hour rules, we included these studies. The included studies also had to have been conducted to assess the impact of the duty hour reform. Since ACGME duty hour regulations apply only to accredited programs in the United States, we included only studies conducted in the US.

We identified 5,345 citations in our electronic searches, and we reviewed the abstracts of all pertinent articles (Fig. 1). We divided the abstracts between the three study team members for review. During weekly conference calls, we discussed the abstracts about which we were uncertain and came to consensus about inclusion. In all decisions, we erred on the side of inclusion. Abstracts were rejected without further review if they were not research articles or they did not address the study topic. Each article was reviewed by the same study team member who had originally reviewed that paper’s abstract. Again, we discussed uncertainties regarding inclusion of papers as a group. Only one pertinent abstract was found in the meeting proceedings. It did not significantly change the results.10 We then excluded studies that did not include one of the three following outcomes: a direct measure of patient safety, an objective measure of educational outcomes (standardized test scores or experience), and well-being [using the Maslach Burnout Inventory (MBI)].11 This resulted in a final number of 60 studies.

Figure 1.

Figure 1

Flow chart summarizing the number of abstracts/papers reviewed for this project and the reasons for excluding abstracts/papers from further review.

Data Abstraction

We abstracted data from each study into a structured data abstraction form in a database called Research Electronic Data Capture (REDCap).12 This form included information on specialty, sample characteristics, study duration, study design, study quality (see below), and outcomes.

One study team member (KF) reviewed all included papers and abstracted all of the relevant data from them. To validate the abstraction process, the other two study team members each reviewed a randomly selected sample of 15 papers and abstracted all quality-related items from each. Other large systematic reviews have used a similar approach.13 Inter-rater agreement on study quality scores was calculated using a weighted kappa.

Assessment of Study Quality

We used the Medical Education Research Quality Index (MERSQI) to assess study quality. Substantial evidence supports the use of MERSQI scores for evaluating research study quality.14 The MERSQI has a maximum score of 18, with 9.8 as an average score for medical education research studies. For the studies assessing patient outcomes, we rated the study quality again using the United States Preventive Services Task Force criteria for cohort studies. 15 One point is awarded for meeting each of seven criteria for study quality, resulting in a 0–7 score, with 7 representing the highest quality possible.

Synthesis of the Results

We qualitatively synthesized the results of the studies in a deliberative process that included weekly conference calls and research-in-progress presentations. We plotted outcomes versus quality scores in an effort to understand the significance of the trends that we noticed. We considered the risk of bias such as publication bias, which would favor studies with significant outcomes. We were also cognizant of selective reporting of significant outcomes within our included studies, and tried to include and consider the importance of null results as well.

We conducted a meta-analysis assuming random effects (Stata version 11.2, College Station, TX) on the mortality studies that provided enough detail to calculate odds ratios (n = 14). One study16 included patients that were a subset of patients from two other studies,17,18 so we excluded the study with the duplicate patients. We used the mean upper and lower confidence intervals of the combined studies for two of the studies. The first was a study that reported no deaths in either group,19 so confidence intervals were not available. The second study reported one death in the pre-2003 period and none in the post period.20 Because the number of deaths was virtually the same and odds ratios cannot be calculated with a zero in them, we assigned an odds ratio of 1 to that study. Not all of the studies included information on adjusted odds ratios; hence, we used unadjusted odds ratios. When not included, we calculated them. In the studies that included data for more than 1 post-2003 year, we included the most recent year. The mortality outcomes varied across studies (e.g., in-hospital mortality, 30-day mortality). We also performed meta-analysis separately for studies with medical and surgical patients. We assessed publication bias using the Egger’s test for small study effects.

RESULTS

A list of all 60 included studies can be found in the online table. In the following paragraphs, we focus on three overarching themes: patient safety, resident education and resident well-being. For the studies that had quality rated by two investigators, the raw percent agreement was 89% and the kappa was 0.67, representing substantial agreement.75

The 2003 Duty Hour Rules and Patient Safety

Mortality

Twenty studies assessed mortality as an endpoint (Table 1); 12 assessed mortality in surgical patients, 4 in internal medicine patients, and 4 in both. Overall, these studies were of relatively high quality as measured by MERSQI scores between 12 and 16.67. The actual mortality outcomes varied across studies, and included overall, inpatient, and 30-day mortality. No studies demonstrated a worsening in mortality outcomes, and many trended toward improvement in the post-2003 time period.

Table 1.

Studies Assessing Patient Safety Outcomes before and after 2003

Study Quality scores (M/U) Study design Participants N (description) Measure Event rates
Pre-2003 Post-2003
Medicine
Bhavsar 200724 14.67/7 Single site retrospective cohort 1,003 (acute coronary syndrome) Inpatient mortality 4.2% 2.8%
Horwitz 200725 15.33/7 Retrospective cohort 20,924 (internal medicine) In-hospital mortality. Several complications studied 2.4% 2%
Parthasarathy 200726 13/5 Single site pre-post ICUa Intercepted medication order entry errors 3.4% 0.9%
Alshekhlee 200927 16.67/7 Retrospective cohort 377,266 (stroke) Mortality (teaching). Multiple complications assessed 11.65% 10.98%
Prasad 200928 16.67/6 Retrospective cohort 230,151 (ICU) In-hospital mortality (academic/pre year 1) 14% Post year 2: 12.3%†‡
Surgery + medicine
Mycyk 200529 14.67/3 Pre-post single site Adultsa Adverse drug events: number (%) 194 (47.4%) 172 (41.8%)
Shetty 200731 15.33/6 Retrospective cohort 151,1945 (medical and surgical) In-hospital mortality % change (medical): -0.25% % change (surgical): 0.13% § §
Volpp 200718 16.67/6 Retrospective cohort 318,636 (medical and surgical) 30-Day mortality (post-reform year 2): medical patients: OR = 0.74†‡; surgical patients OR = 1.05 § §
Volpp 200717 16.67/6 Retrospective cohort 8,529,595 (medical and surgical- Medicare) 30-Day mortality (post-reform year 2): medical patients: OR = 1.04. Surgical patients: OR = 0.99 § §
Rosen 200932 16.67/6 Retrospective cohort 14,193,320 (medical and surgical Medicare /VA) Patient safety indicators mostly unchanged § §
Volpp 200916 16.67/6 Retrospective cohort 8,848,231 (medical and surgical) 30-Day mortality for highest risk (adjusted): § §
Medicare medical: OR = 0.9    
Surgical: OR =1.01
VA
Medical: OR 1.35
Surgical: OR = 0.80
Obstetrics and gynecology
Bailit 200433 13.8/6 Single institution pre-post Obstetrics a Multiple complications assessed § §
Pediatrics
Landrigan 200836 16/4 Multi-site pre-post 2,324 (pediatric inpatient-days) Medication errors 1.29/100 1.5/100
Surgery
Christmas 200541 14.67/3 Single site pre-post Traumaa Hospital mortality 9% 8%
Kaafarani 200544 14.67/3 Single site retrospective cohort 1,197 (general and vascular) General surgery mortality 4.56% 2.62%
General surgery morbidity 22.52% 20.94%
Vascular surgery mortality 4.02% 4.81%
Vascular surgery morbidity 27.01% 30.29%
Naylor 200519 14.67/5 Single site pre-post 275 (emergent LC) Mortality 0 0
Surgical complications 6.5% 8.9%
Schenarts 200547 14.67/4 Single site pre-post 2,826 (trauma) In-hospital mortality 5.2% 5.8%
Multiple complications assessed
de Virgilio 200652 14.67/5 Single site pre-post 11,518 (trauma) Overall complication rate 2.54% 2.05%
Frankel 200653 14.67/4 Single site pre-post 4,956 (surgical ICU) Readmissions to the SICU 1.4% 3%
Salim 200760 12/6 Single site retrospective cohort 16,854 (trauma) Inpatient mortality 6.4% 6.25%
Complication rate 5.64% 7.28%
Hutter 200854 15/4 Single site pre-post 3,976 (general surgery) Mortality: no change § §
Complications
Yaghoubian 200820 14.67/6 Single site retrospective cohort 2,470 (LC) Mortality 0.07% 0
Complications 5% 2%
Browne 200967 16.67/7 Retrospective cohort study using a national database 48,430 (hip fractures) Mortality (perioperative): § §
No change between teaching (beta = −0.26) and non-teaching hospitals (beta = −0.23)
Selected complications assessed
Gopaldas 200990 14/6 Single site pre-post 1,562 (cardiac surgery) 30-Day mortality 3.9% 1.8%†‡
Morrison 200998 16/6 Retrospective cohort study using a national database 492,173 (trauma) Mortality (inpatient) 5.16% 5.03%
Selected complications assessed
Privette 200971 14.67/5 Single site retrospective cohort 14,610 (general, vascular and trauma) Inpatient mortality 1.96% 1.1%
Complications 14.9% 14.6%
Shonka 200973 13.8/4 Single site retrospective cohort Otolaryngologya Inpatient mortality index unchanged in pre versus post years

Mortality is reported in univariate analyses in the pre- versus post-2003 time periods, unless otherwise noted. Only overall complications are included in this table

M = MERSQI score; U = USPSTF score

ICU = intensive care unit

LC = laparascopic cholecystectomy

OR = odds ratio

aNumber of patients included was not specified

†Statistically significant at the p < 0.05 level or less

‡Multivariate analysis reported and also significant

§See the “Measures” column for outcome details

We conducted a meta-analysis on the 14 studies that included enough detail for the calculations (Fig. 2). The results of the meta-analysis with all included studies revealed a pooled estimate of the odds ratio of 0.9 (0.84, 0.95). Separate meta-analyses were performed for studies assessing mortality in medical (OR 0.91; 0.85, 0.98) and surgical (OR 0.86; 0.75, 0.97) patients, and these also showed significant improvement in the post-2003 period. Significant heterogeneity was present (I2 83%, p < 0.01); I2 > 56% is considered large.76. The Egger’s test failed to show a small study effect, suggesting that publication bias was not present (z = −0.05; p = 0.97).

Figure 2.

Figure 2

Forest plot illustrating the odds ratios for mortality in the post-2003 time period as compared to the pre-2003 time period. Odds ratios are illustrated for medical patients, surgical patients and overall. The two studies by Volpp included both medical and surgical patients. For those studies, the medical patient data were included in the top portion with the other medical studies, and the surgical patient data were included in the bottom portion with the surgical studies. The boxes around the mean represent the study weight, while the lines extending outward from the mean represent the 95% confidence interval around the odds ratio.

Complications

The association of duty hour reform and complications was evaluated in 24 studies (Table 1). A diverse group of complications was evaluated, including surgical complications (e.g., intra-operative, postoperative) and medical [e.g., Intensive Care Unit (ICU) transfers, days on ventilators, adverse drug events]. The preponderance of studies demonstrated that some outcomes improved, some worsened, and some were unchanged.

We highlight the studies of highest quality. All used national databases and controlled for secular trends by comparing teaching intensive and non-teaching intensive hospitals. Browne et al. studied patients with hip fractures, and demonstrated that pneumonia, hematoma, and need for transfusions were significantly more likely in the post-duty hour time period, although measures of effect size were not included.67 In that study, many other complications did not differ between the two time periods. Rosen et al. demonstrated that in Medicare and VA patients, most patient safety indicators (PSIs), which are adverse events that are identifiable in administrative data,77 were equally common in both time periods, although certain PSIs occurred more often in the post-duty hour time period in the VA patients (OR 1.63).78 Silber et al. found no increased risk of prolonged hospitalization in Medicare or VA patients.79

Errors

Two studies assessed actual errors (as opposed to self-reported errors). A study in pediatric residents failed to show improvement in medication errors after changes to reduce duty hours were implemented.80 The other study showed that intercepted medication errors made by internal medicine residents were decreased after duty hour rules were initiated.26

The 2003 Duty Hour Rules and Resident Education

This section focuses on the results of studies objectively measuring education using standardized tests and clinical experience. Heterogeneity in outcomes and incomplete reporting precluded meta-analysis.

Standardized Tests

Ten studies assessed the impact of the duty hour rules on standardized tests (see Table 2). Nine studies were of surgical (or surgical subspecialty) in-training examination scores,49,50,52,54,61,64,69,70,73 and one examined the obstetrics and gynecology in-training examination.35 These were mostly small single site studies with sample sizes ranging from 28 to 238. Several did not report the number of residents included.

Table 2.

Studies assessing educational outcomes

Study Quality scores Study design Resident participants Measure Pre-2003 Post-2003
Obstetrics and gynecology
Blanchard 200434 13.8 Single site pre-post 10 Ob/gyn Many procedures decreased for PGY4sa
Bailit 200535 13.8 Single site cross-sectional survey 28 Ob/gyn residents and graduates No change in CREOG examination scores
Short 200655 12.6 Single site pre-post 35 Ob/Gyn Overall cases: no change
Surgery
Goldstein 200438 11.5 Single site pre-post General surgery PGY5 operative: unchanged
Welling 200439 12.67 Prospective cohort General surgery† Chief cases/month
Night float residents: 7.25
Residents not on night float: 34.75
Bland 200540 14.67 Multi-site pre-post General surgery† Total major procedures 938 932
Feanny 200542 13.33 Single site pre-post 13 General surgery Total emergency abdominal surgeries as primary surgeon 101 (13) 110 (33)
Ferguson 200543 13.33 Single site pre-post General surgery † PGY5 operative experience 339 390
231 246
Overall
McElearney 200545 13.33 Single site pre-post General surgery † PGY5 cases/month 30.6 (8.5) 26.1 (9.6)
Overall cases/month 23.9 (16.3) 24.6 (16.1)
Mendoza 200546 13 Multi-site survey/non-randomized General surgery † Chief resident operative volume at universities
With rules: 250.5 (13.6)
Without: 262.6 (16.1)
Spencer 200548 9 Pre-post single 91 Pediatric surgery Senior residents (cases/day) 1.58 (0.42) 1.84 (0.87)
Vetto 200549 9 Single site pre-post 40 General surgery ABSITE scores increased by 7% post-2003
Zare 200550 12 Multi-site pre-post 238 General surgery ABSITE scores 70 (10.3) 69.8 (10.2)
de Virgilio 200652 14.67 Single site pre-post General surgery † Operative experience: no change
ABSITE score: no change
Hutter 200654 15 Single site pre-post 58 General surgery Operative experience 231 246
ABSITE scores: no change
Carlin 200756 13.8 Pre-post single site 126 General surgery resident-years Operative cases/resident/year 278 (100) 211 (150)a
Damadi 200757 12 Single site pre-post 17 General surgery PGY 5 Operative experience 255 (19) 189 (11)a
Izu 200758 9.5 Single site pre-post 2 General surgery Surgeries covered by a resident 47 (35%) 49 (45%)
Pappas 200759 13.8 Single site pre-post 79 Orthopedic resident-years Operative experience 455 467
Schneider 200761 12.5 Single site pre-post 66 General surgery Overall operative experience 1,112 1,277
ABSITE scores PGY5 78.8 (3.7) 78 (3.1)
Weatherby 200762 12.6 Single site pre-post 8 Orthopedic Operative experience: 954 759
Baskies 200863 13.33 Pre-post single site 109 Orthopedic surgery Operative experience 363 (145) 410 (164)a
Durkin 200864 13.33 Single site pre-post study General surgery † Operative experience 1,052 1,011
ABSITE scores (pre-post): +3.6% correct
Kairys 200865 14.67 National pre-post General surgery † Total cases as chief or juniorurgeon 930 909a
Shin 200866 12.6 Single site retrospective pre-post General surgery † % Cases covered by PGY 4–5 94% 92%
% Cases covered overall 79% 80%
Christmas 200968 13.33 Single site pre-post 22 General surgery Total chief resident cases 494 (116) 333 (116)a
Froelich 200969 13.33 Single site retrospective pre-post 97 Orthopedic surgery resident-years PGY5 operative experience 424 457
PGY5 ITE scores 196 197
Jagannathan 200970 12.5 Multi-site pre-post Neurosurgery† American Board of Neurosurgery Examination (PGY 2–3) 310 (55) 259 (48)a
Privette 200971 14.67 Single site retrospective cohort 48 general surgery Operative experience 985 887
Sarff 200972 8 Single site pre-post study 36 General surgery Operative experience (as surgeon or teaching assistant) 984 1,057
Shonka 200973 13.8 Single site retrospective cohort study Approximately 30 otolaryngology Total cases as surgeon 1,515 1,533
In-training examination scores 6.55 6.53
Sneider 200974 12.5 Multi-site cross-sectional survey General surgery programs† ABSITE PGY5 73.4 (4.4) 71.3 (3.2)
American Board of Surgery pass rate 80.5% (18.7) 91% (8.3)

Participants are residents, unless otherwise specified. Outcome measures include standardized examination scores and/or pass rates and operative experience reported as number of cases, unless otherwise specified. For operative experience, we report numbers for overall and final (chief) year of residency

PGY = post-graduate year

aStatistically significant at the p < 0.05 level or less

†Number not provided

‡See Measures column

Two studies reported an improvement in test scores after the duty hour rules were implemented.61,64 In the first, scores increased for interns only (from 59.5 to 72.4, p = 0.006), but were unchanged among the other residents.61 In the other study, residents’ basic science and overall scores increased (by 4.7% and 3.6%, respectively), while clinical management scores did not.64 Five studies showed no change in examination scores between the pre- and post-duty hour periods.35,52,54,69,73 Two studies demonstrated a decrease in examination scores between the pre- and post-duty hour periods.50,70

Operative Experience

The difference in operative experience before and after reform has been reported in 26 studies (Table 2). The type of operative experience varied between studies. Of the 17 studies that evaluated the relationship between duty hour rules and overall surgical experience or overall experience as the main surgeon, two showed significant decreases, one showed a significant increase, nine showed no change,40,52,54,55,59,64,71,73,81 and several others did not report statistical analyses. Many of these studies were single site studies, and likely underpowered to detect a true difference.

Other studies suggest that the volume of certain procedures may have changed; this outcome would not be captured in studies examining only overall operative experience. One study of cardiothoracic surgical experience showed that overall experience with coronary artery bypass grafting significantly decreased in the post-duty hour time period (from 148 cases to 110 cases combining all years of residency).82 Another study of abdominal trauma surgery42 found that the overall number of operative procedures per graduating resident in the last 2 years of their residency did not differ between the pre- and post-duty hour periods; however, there was a significant decrease in the number of advanced emergency abdominal cases (51 versus 31) and an increase in the number of basic abdominal cases (47 versus 84) when comparing the pre- and post-duty hour time periods.

A study by Coverdill et al. included interviews with surgical faculty members.83 One theme identified in the analysis of the interviews was that routine work was being shifted from residents to faculty. For example, they reported that residents only come to the operating room, while the faculty provided the preoperative and postoperative care. Another study counted the use of the relative value unit (RVU) modifier “-82,” which signifies that no qualified resident was available.71 The use of this modifier increased from 523 in the 2 years prior to duty hour rules to 6,542 in the 2 years post-duty hour rules.71

The 2003 Duty Hour Rules and Resident Well-Being

Eight studies assessed burnout, using the Maslach Burnout Inventory (see Table 3). One of these studies reported burnout in a cross-sectional study23 and reported the same data as part of a pre-post analysis.22 We include only the pre-post information in Table 3. Rates of burnout did not statistically significantly worsen in any study, although there was a non-significant worsening in one study.21 Burnout improved in five studies,22,30,36,51,54 most often as a result of a decrease in emotional exhaustion. Two studies found that a higher number of work hours was related to burnout.30,51 Specifically, in one study, working > 80 h corresponded to rates of burnout near 70%, which decreased to 39% when working <80 h per week.30

Table 3.

Studies assessing burnout in residents before and after 2003

Study Quality scores Study design Participants Measure % Reporting burnout and/or burnout scores
Pre-2003 Post-2003
Surgery
Gelfand 200437 12.5 Single site pre-post 33 Residents and faculty % Residents meeting criteria for burnout on emotional exhaustion scale 50% 47%
Barrack 200651 11.5 Single site pre-post 55 Orthopedic surgery residents Emotional exhaustion score 27.5 22.3
Hutter 200654 15 Single site pre-post 58 General surgery residents Emotional exhaustion score 29.1 23.1a
Internal medicine
Goitein, 200521 13 Single site pre-post 118 Residents % Residents meeting any criteria 68% 76%
Emotional exhaustion 45% 53%
Gopal 200522 12.5 Single site pre-post 227 Residents % Residents any meeting criteria 61% 55%
Emotional exhaustion 42% 29%a
Pediatrics
Landrigan 200836 16 Multi-site pre-post 268 Pediatrics residents % Residents meeting criteria for burnout 75.4% 57%a
Mixed
Martini 200630 12.5 Single site pre-post 118 Residents % Residents meeting criteria for burnout 49% 41%

aStatistically significant at the p < 0.05 level

†Lower scores indicate less burnout

DISCUSSION

Appropriate duty hour reform must consider the interests of all stakeholders involved. In a review of the frameworks used to conceptualize this discussion, Schwartz et al. point out that it is imperative to work from models that take into account the trade-offs associated with public policy issues such as this one.84 A recent review by Jamal and colleagues focused on the effects of duty hour limits on surgeons85, and a second review by Reed et al. examined evidence specifically pertaining to shift length and night float86. Our review differs from and expands upon these prior reviews by providing a comprehensive synthesis of the impact of the 2003 duty hour policy reforms on the most important stakeholder groups: patients and residents of all specialties.

Our major findings were in the areas of patient safety, resident education, and resident well-being. With respect to patient safety, our meta-analysis suggested an improvement in mortality between the pre- and post-2003 time periods. Medical and surgical complications were more variable, with some improving and others worsening. Resident burnout was improved.

The finding that mortality has improved over time must be considered with several important caveats. First, we used unadjusted odds ratios to conduct our analyses. Therefore, we cannot account for differences in patient characteristics between the two time periods. Of particular importance is the fact that we could not take advantage of the adjustments made in the subset of studies that used non-teaching hospitals as controls to account for temporal trends.17,18,27,28 It is important to note that after adjustment those studies largely found no change in mortality between pre- and post-2003. Therefore, our meta-analysis results could easily reflect improvements in quality of care that occurred over the time period studied rather than a direct result of the duty hour rules.

Complications were more nuanced, with some improving and some worsening in the post-2003 time period. One possible explanation for these variable results is that strategies for complying with duty hour reform may lead to improvements in certain types of complications, and a worsening in others. Another possible explanation is that certain complications are more sensitive to fatigue, and these improved post-reform, whereas outcomes more sensitive to discontinuity of care worsened. For example, in one surgical study, bile duct injuries decreased in the post-2003 time period, but conversion from laparoscopic to open cholecystectomy was significantly more common in the post-2003 time.20 Improved manual dexterity from being better rested could account for the former finding, consistent with prior simulation studies.8789 The latter finding of more conversions to open procedures could reflect the impact of less resident experience with laparoscopy in the post-2003 time period. Less continuity of care may also contribute to certain complications. For example, if doctors are less familiar with patients, this could lead to delayed decisions and therapeutic interventions. This phenomenon could partially explain the increase in the number of cardiac surgery patients that remained on ventilators for >48 h in one of our studies.90 A third possibility is that these inconsistencies simply represent variation due to local factors or chance. For example, the complication of postoperative pneumonia was increased in one study67 and lessened in another.52 We are unable to explain the specific patterns found in these studies by any one of these explanations alone, so other factors are likely involved as well. Regardless, many complications appear to be worsening in the post-reform period, and this deserves further study as additional changes to duty hour rules are made.

The impact of duty hour reform on resident experience is also important. Today’s residents will become tomorrow’s independent doctors, and we must be confident that they are ready for practice.91,92 Most studies in our review did not demonstrate significant differences in overall resident operative experience between the pre-2003 and post-2003 time periods. However, the role of residents in surgeries may be evolving to one in which they have less responsibility. In addition, none of the studies assessed residents who had trained entirely after the 2003 reform compared with those who trained before the reform. Moving forward into an era of further restrictions, it will be essential to study not only the number of surgeries performed, but also the specific surgeries performed and the residents’ roles in those surgeries. This will allow us to better understand the full effect of reform on residents’ operative experience. There remains a paucity of data on patient care experience in the non-operative specialties. The non-surgical specialties could easily track the admitting diagnoses of all patients their interns see or the non-operative procedures that they perform. It is important to determine whether other specialties are struggling to maintain training experiences.

Another interesting finding from this review was the improvement in resident well-being following the 2003 duty hour reforms, which has been noted in prior work.9 We focused on burnout in this review, but other studies have corroborated the improvement in well-being by documenting more residents having babies in the post-2003 time period,93 greater ability to attend family events,38 and less perceived stress.94 However, other aspects of well-being such as rates of depression do not seem to have changed between the pre-2003 period and the post-2003 period.21,22,36,95 Prior research has demonstrated links between resident well-being and quality of patient care,96,97 making preservation of resident well-being extremely important. This improvement in well-being may be one explanation for why some patient care parameters are improving in the post-2003 time period.

Our study has limitations. Perhaps the greatest limitation of this review is that our conclusions rest upon studies demonstrating association, not causality. It is likely that other contextual changes unrelated to duty hour rules contribute to the observed effects. These confounders may explain much of the heterogeneity that we observe. However, decisions must frequently be made in the context of incomplete evidence. While a causal relationship between the duty hour rules and outcomes cannot be determined with certainty from the studies cited, we have diligently identified and synthesized the best available evidence. The possibility of publication bias is also a limitation. We reviewed abstracts from recent meetings in order to capture studies that have not yet made it to publication and also asked an expert to review our bibliography for omissions. Other limitations include the wide range of quality of the included studies. To account for this variability, we used the MERSQI to rate and compare study quality objectively. However, since the MERSQI is designed to measure quality across the full range of quantitative study designs, the instrument incorporates only broad aspects of methodological quality and thus does not account for finer methodological differences within study types. The decision about whether to include a study was made by a single reviewer, although we erred on the side of inclusion and discussed studies about which we were unsure. Additionally, most data from each study were abstracted by a single reviewer and could have been inaccurate. Finally, the reviewers were not blinded to the study authors or journals, which could result in bias as well. Despite these limitations, this review was comprehensive, including over 60 studies. This allows conclusions to be drawn that were not possible when the last comprehensive reviews on this subject were published.8,9

Limitations notwithstanding, this review provides a comprehensive synthesis of the evidence base for the 2003 duty hour reforms in the US. The balance of evidence suggests that burnout among residents has decreased. Given the unacceptably high prevalence of burnout among trainees,96 the reduction in burnout represents an important success of the 2003 reforms. In contrast, data on residents’ educational outcomes, such as test scores and clinical experience, with the 2003 reforms are more mixed, preventing the formulation of any firm conclusions. Moreover, while our review included several studies that examined surgical residents’ operative experience before and after duty hour reform, we were unable to identify any study assessing the impact of the 2003 duty hour rules on the clinical experience of non-surgical residents (e.g., the number of patients seen with specific diagnoses or the number of bedside procedures done). As the new 2011 duty hour rules are implemented, it will be important to quantify any changes in the breadth of clinical exposure for all residents. While this review suggests a modest decrease in mortality following the 2003 duty hour limits, we are unable to exclude the possibility of secular trends playing a role. Nevertheless, because several studies reported increased rates of certain complications, special attention should be paid to monitoring these complications during future reforms. Future efforts to evaluate the impact of the 2011 duty hour limits should build upon this evidence base by using rigorous methods to examine the most important outcomes related to patient care and residents’ education.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1 (168.5KB, doc)

Study quality and outcomes assessed for all included studies (DOC 168 kb)

Acknowledgments

Author contributions Dr. Fletcher had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the analysis.Study concept and design: Fletcher, Reed, AroraAcquisition of data: Fletcher, Reed, AroraAnalysis and interpretation of data: Fletcher, Reed, Arora, JacksonDrafting of the manuscript: FletcherCritical revision of the manuscript for important intellectual content: Fletcher, Reed, AroraStatistical analysis: Fletcher, Reed, Arora, JacksonObtained funding: Fletcher, Reed, AroraAdministrative, technical, or material support: Fletcher, Reed, AroraStudy supervision: Fletcher, Reed, Arora

Funding Support This study was funded by a grant from the Accreditation Council of Graduate Medical Education.

Role of the Sponsor The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.

Financial Disclosures Dr. Fletcher reports receiving funding from the VA HSRD and also from NCI. Dr. Reed reports receiving funding from the ABIM Foundation. Dr. Arora reports receiving grant funding from the ABIM Foundation, the ACP Foundation, the Agency for Healthcare Research and Quality, and the National Institutes of Aging. Dr. Fletcher was a reviewer for the Institute of Medicine Report on Resident Duty Hours: Sleep, Supervision and Safety. Dr. Fletcher served voluntarily on the ACGME Committee on Innovation, Learning, and Education. Dr. Arora has provided testimony on duty hours to the Institute of Medicine Committee on Optimizing Graduate Medical Trainee Hours and Work Schedules to Improve Patient Safety and to the ACGME Duty Hours Congress as a representative of the American College of Physicians. Drs. Arora and Reed are members of the Association of Program Directors of Internal Medicine.

Additional Contributions We wish to thank Jessica Schmidt and Andrea Bruckbauer at the Milwaukee VAMC, Alexis Dye, MS, Sherrie Smaxwill and Mark Oium, MS, at the Medical College of Wisconsin, Katya Papatla at Duke University, Patricia Erwin and Kate Featherstone at the Mayo Clinic College of Medicine, and Meryl Prochaska, BA, and Diane Daviera, BS, at the University of Chicago, and Emily Chiu at the University of Michigan for their excellent research assistance. We also wish to thank Jack Littrell, MS, for his assistance with database creation and management, and DeWitt Baldwin, MD, at the ACGME for his assistance with obtaining funding. We are grateful to Jeffrey Jackson, MD MPH, for conducting the meta-analysis and to Dr. Monica Lypson, MD, for her review of the bibliography. Ms. Bruckbauer was a Milwaukee VAMC employee while this project was underway and was also paid through the ACGME grant. Ms. Schmidt is an employee of the Milwaukee VAMC and was also paid through the ACGME grant. Ms. Papatla was paid through the ACGME grant. Ms. Dye, Mr. Oium, Ms. Smaxwill, and Mr. Littrell are paid employees at the Medical College of Wisconsin and volunteered to help with this project. Ms. Erwin and Featherstone are paid employees of the Mayo Clinic College of Medicine. Ms. Prochaska and Daviera are paid employees of the University of Chicago, and Ms. Chiu was paid by the ACGME grant.

Footnotes

Prior Presentations

This work was presented at the 2010 national SGIM meeting and at the 2010 national Society of Hospital Medicine meeting, both times in poster form.

References

  • 1.Philibert I, Friedmann P, Williams WT, Education AWGoRDHACfGM New requirements for resident duty hours.[see comment] JAMA. 2002;288(9):1112–1114. doi: 10.1001/jama.288.9.1112. [DOI] [PubMed] [Google Scholar]
  • 2.Landrigan CP, Barger LK, Cade BE, Ayas NT, Czeisler CA. Interns' compliance with accreditation council for graduate medical education work-hour limits. JAMA. 2006;296(9):1063–1070. doi: 10.1001/jama.296.9.1063. [DOI] [PubMed] [Google Scholar]
  • 3.Landrigan CP, Rothschild JM, Cronin JW, et al. Effect of reducing interns' work hours on serious medical errors in intensive care units. [See comment] New England Journal of Medicine. 2004;351(18):1838–1848. doi: 10.1056/NEJMoa041406. [DOI] [PubMed] [Google Scholar]
  • 4.Ulmer C, Wollman DM, Johns MME. Resident duty hours: Enhancing sleep, supervision, and safety. Washington DC: Institute of Medicine National Academies Press; 2008. [PubMed] [Google Scholar]
  • 5.Nasca TJ, Day SH, Amis ES, Jr., The ADHTF: the new recommendations on duty hours from the ACGME task force. N Engl J Med: NEJMsb1005800. [DOI] [PubMed]
  • 6.Nasca T. ACGME Request for a Proposal for a Comprehensive Literature Review and Analysis of Residency Training and Duty Hours Experience. Chicago, 2009.
  • 7.Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA Statement. Ann Intern Med. 2009;151:264–269. doi: 10.7326/0003-4819-151-4-200908180-00135. [DOI] [PubMed] [Google Scholar]
  • 8.Fletcher KE, Davis SQ, Underwood W, Mangrulkar RS, McMahon LF, Jr, Saint S. Systematic review: effects of resident work hours on patient safety. Ann Int Med. 2004;141(11):851–857. doi: 10.7326/0003-4819-141-11-200412070-00009. [DOI] [PubMed] [Google Scholar]
  • 9.Fletcher KE, Underwood W, 3rd, Davis SQ, Mangrulkar RS, McMahon LF, Jr, Saint S. Effects of work hour reduction on residents' lives: a systematic review. [See comment] JAMA. 2005;294(9):1088–1100. doi: 10.1001/jama.294.9.1088. [DOI] [PubMed] [Google Scholar]
  • 10.Volpp K SJ, Wang Y, Even-Shoshan O, Halenar M, Bellini L, Romano P, Zhu J, Press M, Rosen AK, Itani KMF. The impact of resident duty hour reform on hospital readmission rates. J Gen Intern Med 2010; 5(suppl). [DOI] [PMC free article] [PubMed]
  • 11.Maslach C JS. Maslach Burnout Inventory Manual, 3rd Edition. Palo Alto, CA: Consulting Psychology Press, 1986.
  • 12.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–381. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Baernstein A, Liss HK, Carney PA, Elmore JG. Trends in study methods used in undergraduate medical education research, 1969–2007. JAMA. 2007;298:1038–1045. doi: 10.1001/jama.298.9.1038. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. JAMA. 2007;298(9):1002–1009. doi: 10.1001/jama.298.9.1002. [DOI] [PubMed] [Google Scholar]
  • 15.Harris RP, Helfand M, Woolf SH, et al. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med. 2001;20(3 suppl):21–35. doi: 10.1016/S0749-3797(01)00261-6. [DOI] [PubMed] [Google Scholar]
  • 16.Volpp K, Rosen A, Rosenbaum P, et al. Did duty hour reform lead to better outcomes among the highest risk patients? J Gen Intern Med. 2009;24(10):1149–1155. doi: 10.1007/s11606-009-1011-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Volpp KG, Rosen AK, Rosenbaum PR, et al. Mortality among hospitalized medicare beneficiaries in the first 2 years following ACGME resident duty hour reform. [See comment] JAMA. 2007;298(9):975–983. doi: 10.1001/jama.298.9.975. [DOI] [PubMed] [Google Scholar]
  • 18.Volpp KG, Rosen AK, Rosenbaum PR, et al. Mortality among patients in VA hospitals in the first 2 years following ACGME resident duty hour reform. [See comment] JAMA. 2007;298(9):984–992. doi: 10.1001/jama.298.9.984. [DOI] [PubMed] [Google Scholar]
  • 19.Naylor RA, Rege RV, Valentine RJ. Do resident duty hour restrictions reduce technical complications of emergency laparoscopic cholecystectomy? J Am Coll Surg. 2005;201(5):724–731. doi: 10.1016/j.jamcollsurg.2005.06.271. [DOI] [PubMed] [Google Scholar]
  • 20.Yaghoubian A, Saltmarsh G, Rosing DK, Lewis RJ, Stabile BE, Virgilio C. Decreased bile duct injury rate during laparoscopic cholecystectomy in the era of the 80-hour resident workweek. Arch Surg. 2008;143(9):847–851. doi: 10.1001/archsurg.143.9.847. [DOI] [PubMed] [Google Scholar]
  • 21.Goitein L, Shanafelt TD, Wipf JE, Slatore CG, Back AL. The effects of work-hour limitations on resident well-being, patient care, and education in an internal medicine residency program. [See comment] Arch Int Med. 2005;165(22):2601–2606. doi: 10.1001/archinte.165.22.2601. [DOI] [PubMed] [Google Scholar]
  • 22.Gopal R, Glasheen JJ, Miyoshi TJ, Prochazka AV. Burnout and internal medicine resident work-hour restrictions. [See comment] Arch Int Med. 2005;165(22):2595–2600. doi: 10.1001/archinte.165.22.2595. [DOI] [PubMed] [Google Scholar]
  • 23.Gopal RK, Carreira F, Baker WA, et al. Internal medicine residents reject "longer and gentler" training. J Gen Int Med. 2007;22(1):102–106. doi: 10.1007/s11606-007-0119-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Bhavsar J, Montgomery D, Li J, et al. Impact of duty hours restrictions on quality of care and clinical outcomes. [See comment] Am J Med. 2007;120(11):968–974. doi: 10.1016/j.amjmed.2007.07.026. [DOI] [PubMed] [Google Scholar]
  • 25.Horwitz LI, Kosiborod M, Lin Z, Krumholz HM. Changes in outcomes for internal medicine inpatients after work-hour regulations.[See comment][summary for patients in Ann Intern Med. 2007 Jul 17;147(2):I28; PMID: 17548400] Ann Int Med. 2007;147(2):97–103. doi: 10.7326/0003-4819-147-2-200707170-00163. [DOI] [PubMed] [Google Scholar]
  • 26.Parthasarathy S, Hettiger K, Budhiraja R, Sullivan B. Sleep and well-being of ICU housestaff. Chest. 2007;131(6):1685–1693. doi: 10.1378/chest.06-1398. [DOI] [PubMed] [Google Scholar]
  • 27.Alshekhlee A, Walbert T, DeGeorgia M, Preston DC, Furlan AJ. The impact of accreditation council for graduate medical education duty hours, the July phenomenon, and hospital teaching status on stroke outcomes. J Stroke Cerebrovasc Dis. 2009;18(3):232–238. doi: 10.1016/j.jstrokecerebrovasdis.2008.10.006. [DOI] [PubMed] [Google Scholar]
  • 28.Prasad M, Iwashyna TJ, Christie JD, et al. Effect of work-hours regulations on intensive care unit mortality in United States teaching hospitals*. Crit Care Med. 2009;37(9):2564–2569. doi: 10.1097/CCM.0b013e3181a93468. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Mycyk MB, McDaniel MR, Fotis MA, Regalado J. Hospitalwide adverse drug events before and after limiting weekly work hours of medical residents to 80. Am J Health Syst Pharm. 2005;62(15):1592–1595. doi: 10.2146/ajhp040527. [DOI] [PubMed] [Google Scholar]
  • 30.Martini S, Arfken CL, Balon R. Comparison of burnout among medical residents before and after the implementation of work hours limits. Acad Psychiatry. 2006;30(4):352–355. doi: 10.1176/appi.ap.30.4.352. [DOI] [PubMed] [Google Scholar]
  • 31.Shetty KD, Bhattacharya J. Changes in hospital mortality associated with residency work-hour regulations. [See comment] [summary for patients in Ann Intern Med. 2007 Jul 17;147(2):I16; PMID: 17548402] Ann Int Med. 2007;147(2):73–80. doi: 10.7326/0003-4819-147-2-200707170-00161. [DOI] [PubMed] [Google Scholar]
  • 32.Rosen AK, Loveland SA, Romano PS, Itani KMF, Silber JH, Even-Shoshan OO, Halenar MJ, Teng Y, Zhu J, Volpp KG. Effects of resident duty hour reform on surgical and procedural patient safety indicators among hospitalized veterans health administration and Medicare patients. Med Care. 2009;47:723–731. doi: 10.1097/MLR.0b013e31819a588f. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Bailit JL, Blanchard MH. The effect of house staff working hours on the quality of obstetric and gynecologic care. Obstet Gynecol. 2004;103(4):613–616. doi: 10.1097/01.AOG.0000119225.57285.c1. [DOI] [PubMed] [Google Scholar]
  • 34.Blanchard MH, Amini SB, Frank TM. Impact of work hour restrictions on resident case experience in an obstetrics and gynecology residency program. Am J Obstet Gynecol. 2004;191(5):1746–1751. doi: 10.1016/j.ajog.2004.07.060. [DOI] [PubMed] [Google Scholar]
  • 35.Bailit JL, Weisberger A, Knotek J. Resident job satisfaction and quality of life before and after work hour reform. J Reprod Med. 2005;50(9):649–652. [PubMed] [Google Scholar]
  • 36.Landrigan CP, Fahrenkopf AM, Lewin D, et al. Effects of the accreditation council for graduate medical education duty hour limits on sleep, work hours, and safety. Pediatrics. 2008;122(2):250–258. doi: 10.1542/peds.2007-2306. [DOI] [PubMed] [Google Scholar]
  • 37.Gelfand DV, Podnos YD, Carmichael JC, Saltzman DJ, Wilson SE, Williams RA. Effect of the 80-hour workweek on resident burnout. Arch Surg. 2004;139(9):933–938. doi: 10.1001/archsurg.139.9.933. [DOI] [PubMed] [Google Scholar]
  • 38.Goldstein MJ, Kim E, Widmann WD, Hardy MA. A 360 degrees evaluation of a night-float system for general surgery: a response to mandated work-hours reduction. Curr Surg. 2004;61(5):445–451. doi: 10.1016/j.cursur.2004.03.013. [DOI] [PubMed] [Google Scholar]
  • 39.Welling RE, Boberg JT, Weinberg E, Gulley J. Work hours compliance in a community hospital. Curr Surg. 2004;61(2):241–243. doi: 10.1016/j.cursur.2003.10.009. [DOI] [PubMed] [Google Scholar]
  • 40.Bland KI, Stoll DA, Richardson JD, Britt LD, Members of the Residency Review C-S Brief communication of the Residency Review Committee-Surgery (RRC-S) on residents' surgical volume in general surgery. Am J Surg. 2005;190(3):345–350. doi: 10.1016/j.amjsurg.2005.06.036. [DOI] [PubMed] [Google Scholar]
  • 41.Christmas AB, Reynolds J, Hodges S, et al. Physician extenders impact trauma systems. J Trauma. 2005;58(5):917–920. doi: 10.1097/01.TA.0000162736.06947.E3. [DOI] [PubMed] [Google Scholar]
  • 42.Feanny MA, Scott BG, Mattox KL, Hirshberg A. Impact of the 80-hour work week on resident emergency operative experience. Am J Surg. 2005;190(6):947–949. doi: 10.1016/j.amjsurg.2005.08.025. [DOI] [PubMed] [Google Scholar]
  • 43.Ferguson CM. Mandatory resident work hour limitations.[comment] J Am Coll Surg. 2005;200(4):637–638. doi: 10.1016/j.jamcollsurg.2004.12.018. [DOI] [PubMed] [Google Scholar]
  • 44.Kaafarani HMA, Itani KMF, Petersen LA, Thornby J, Berger DH. Does resident hours reduction have an impact on surgical outcomes? J Surg Res. 2005;126(2):167–171. doi: 10.1016/j.jss.2004.12.024. [DOI] [PubMed] [Google Scholar]
  • 45.McElearney ST, Saalwachter AR, Hedrick TL, Pruett TL, Sanfey HA, Sawyer RG. Effect of the 80-hour work week on cases performed by general surgery residents. Am Surg. 2005;71(7):552–556. [PubMed] [Google Scholar]
  • 46.Mendoza KA, Britt LD. Resident operative experience during the transition to work-hour reform. Arch Surg. 2005;140(2):137–145. doi: 10.1001/archsurg.140.2.137. [DOI] [PubMed] [Google Scholar]
  • 47.Schenarts P, Bowen J, Bard M, et al. The effect of a rotating night-float coverage scheme on preventable and potentially preventable morbidity at a level 1 trauma center. Am J Surg. 2005;190(1):147–152. doi: 10.1016/j.amjsurg.2005.03.026. [DOI] [PubMed] [Google Scholar]
  • 48.Spencer AU, Teitelbaum DH. Impact of work-hour restrictions on residents' operative volume on a subspecialty surgical service. J Am Coll Surg. 2005;200(5):670–676. doi: 10.1016/j.jamcollsurg.2005.01.008. [DOI] [PubMed] [Google Scholar]
  • 49.Vetto JT, Robbins D. Impact of the recent reduction in working hours (the 80 hour work week) on surgical resident cancer education. J Cancer Educ. 2005;20(1):23–27. doi: 10.1207/s15430154jce2001_9. [DOI] [PubMed] [Google Scholar]
  • 50.Zare SM, Galanko JA, Behrns KE, et al. Psychologic well-being of surgery residents after inception of the 80-hour workweek: a multi-institutional study. Surgery. 2005;138(2):150–157. doi: 10.1016/j.surg.2005.05.011. [DOI] [PubMed] [Google Scholar]
  • 51.Barrack RL, Miller LS, Sotile WM, Sotile MO, Rubash HE. Effect of duty hour standards on burnout among orthopaedic surgery residents. Clin Orthop Relat Res. 2006;449:134–137. doi: 10.1097/01.blo.0000224030.78108.58. [DOI] [PubMed] [Google Scholar]
  • 52.Virgilio C, Yaghoubian A, Lewis RJ, Stabile BE, Putnam BA. The 80-hour resident workweek does not adversely affect patient outcomes or resident education. Curr Surg. 2006;63(6):435–439. doi: 10.1016/j.cursur.2006.03.006. [DOI] [PubMed] [Google Scholar]
  • 53.Frankel HL, Foley A, Norway C, Kaplan L. Amelioration of increased intensive care unit service readmission rate after implementation of work-hour restrictions. J Trauma. 2006;61(1):116–121. doi: 10.1097/01.ta.0000222579.48194.2b. [DOI] [PubMed] [Google Scholar]
  • 54.Hutter MM, Kellogg KC, Ferguson CM, Abbott WM, Warshaw AL. The impact of the 80-hour resident workweek on surgical residents and attending surgeons. Ann Surg. 2006;243(6):864–871. doi: 10.1097/01.sla.0000220042.48310.66. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Short AC, Rogers SJ, Magann EF, Rieg TS, Shapiro A, Morrison JC. The 80-hour workweek restriction: How are OB/GYN resident procedure numbers affected? J MaterFetal Neonatal Med. 2006;19(12):801–806. doi: 10.1080/14767050601023277. [DOI] [PubMed] [Google Scholar]
  • 56.Carlin AM, Gasevic E, Shepard AD. Effect of the 80-hour work week on resident operative experience in general surgery. Am J Surg. 2007;193(3):326–329. doi: 10.1016/j.amjsurg.2006.09.014. [DOI] [PubMed] [Google Scholar]
  • 57.Damadi A, Davis AT, Saxe A, Apelgren K. ACGME duty-hour restrictions decrease resident operative volume: a 5-year comparison at an ACGME-accredited university general surgery residency. J Surg Educ. 2007;64(5):256–259. doi: 10.1016/j.jsurg.2007.07.008. [DOI] [PubMed] [Google Scholar]
  • 58.Izu BS, Johnson RM, Termuhlen PM, Little AG. Effect of the 30-hour work limit on resident experience and education. J Surg Educ. 2007;64(6):361–364. doi: 10.1016/j.jsurg.2007.08.006. [DOI] [PubMed] [Google Scholar]
  • 59.Pappas AJ, Teague DC. The impact of the accreditation council for graduate medical education work-hour regulations on the surgical experience of orthopaedic surgery residents. J Bone Joint Surg (American) 2007;89(4):904–909. doi: 10.2106/JBJS.F.01083. [DOI] [PubMed] [Google Scholar]
  • 60.Salim A, Teixeira PGR, Chan L, et al. Impact of the 80-hour workweek on patient care at a level I trauma center. Arch Surg. 2007;142(8):708–714. doi: 10.1001/archsurg.142.8.708. [DOI] [PubMed] [Google Scholar]
  • 61.Schneider JR, Coyle JJ, Ryan ER, Bell RH, Jr, DaRosa DA. Implementation and evaluation of a new surgical residency model. J Am Coll Surg. 2007;205(3):393–404. doi: 10.1016/j.jamcollsurg.2007.05.013. [DOI] [PubMed] [Google Scholar]
  • 62.Weatherby BA, Rudd JN, Ervin TB, Stafford PR, Norris BL. The effect of resident work hour regulations on orthopaedic surgical education. J Surg Orthop Adv. 2007;16(1):19–22. [PubMed] [Google Scholar]
  • 63.Baskies MA, Ruchelsman DE, Capeci CM, Zuckerman JD, Egol KA. Operative experience in an orthopaedic surgery residency program: the effect of work-hour restrictions. J Bone Joint Surg (American) 2008;90(4):924–927. doi: 10.2106/JBJS.G.00918. [DOI] [PubMed] [Google Scholar]
  • 64.Durkin ET, McDonald R, Munoz A, Mahvi D. The impact of work hour restrictions on surgical resident education. J Surg Educ. 2008;65(1):54–60. doi: 10.1016/j.jsurg.2007.08.008. [DOI] [PubMed] [Google Scholar]
  • 65.Kairys JC, McGuire K, Crawford AG, Yeo CJ. Cumulative operative experience is decreasing during general surgery residency: a worrisome trend for surgical trainees? J Am Coll Surg. 2008;206(5):804–811. doi: 10.1016/j.jamcollsurg.2007.12.055. [DOI] [PubMed] [Google Scholar]
  • 66.Shin S, Britt R, Britt LD. Effect of the 80-hour work week on resident case coverage: corrected article. J Am Coll Surg. 2008;207(1):148–150. doi: 10.1016/j.jamcollsurg.2008.05.004. [DOI] [PubMed] [Google Scholar]
  • 67.Browne JA, Cook C, Olson SA, Bolognesi MP. Resident duty-hour reform associated with increased morbidity following hip fracture. J Bone Joint Surg Am. 2009;91:2079–2085. doi: 10.2106/JBJS.H.01240. [DOI] [PubMed] [Google Scholar]
  • 68.Christmas AB, Brintzenhoff RA, Sing RF, et al. Resident work hour restrictions impact chief resident operative experience. Am Surg. 2009;75(11):1065–1068. [PubMed] [Google Scholar]
  • 69.Froelich J, Milbrandt JC, Allan DG. Impact of the 80-hour workweek on surgical exposure and national in-training examination scores in an orthopedic residency program. J Surg Educ. 2009;66(2):85–88. doi: 10.1016/j.jsurg.2008.12.007. [DOI] [PubMed] [Google Scholar]
  • 70.Jagannathan J, Vates GE, Pouratian N, et al. Impact of the accreditation council for graduate medical education work-hour regulations on neurosurgical resident education and productivity. J Neurosurg. 2009;110(5):820–827. doi: 10.3171/2009.2.JNS081446. [DOI] [PubMed] [Google Scholar]
  • 71.Privette AR, Shackford SR, Osler T, Ratliff J, Sartorelli K, Hebert JC. Implementation of resident work hour restrictions is associated with a reduction in mortality and provider-related complications on the surgical service: a concurrent analysis of 14, 610 patients. Ann Surg. 2009;250:316–321. doi: 10.1097/SLA.0b013e3181ae332a. [DOI] [PubMed] [Google Scholar]
  • 72.Sarff M, Ellis MC, Vetto JT. case log review produces translational change in surgical oncology education. J Cancer Educ. 2009;24:176–179. doi: 10.1080/08858190902876213. [DOI] [PubMed] [Google Scholar]
  • 73.Shonka DC, Jr, Ghanem TA, Hubbard MA, Barker DA, Kesser BW. Four years of accreditation council of graduate medical education duty hour regulations: have they made a difference? Laryngoscope. 2009;119(4):635–639. doi: 10.1002/lary.20144. [DOI] [PubMed] [Google Scholar]
  • 74.Sneider EB, Larkin AC, Shah SA. Has the 80-hour workweek improved surgical resident education in New England? J Surg Educ. 2009;66(3):140–145. doi: 10.1016/j.jsurg.2008.10.005. [DOI] [PubMed] [Google Scholar]
  • 75.Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. doi: 10.2307/2529310. [DOI] [PubMed] [Google Scholar]
  • 76.Higgins J, Thompson S. Quantifying heterogeneity in a meta-analysis. Statist Med. 2002;21:1539–1558. doi: 10.1002/sim.1186. [DOI] [PubMed] [Google Scholar]
  • 77.AHRQ. Patient Safety Indicators Overview. AHRQ Quality Indicators., vol 2010. Rockville, MD: Agency for Healthcare Research and Quality, 2006.
  • 78.Volpp KG, Rosen AK, Rosenbaum PR, et al. Did duty hour reform lead to better outcomes among the highest risk patients? J Gen Intern Med 2009. [DOI] [PMC free article] [PubMed]
  • 79.Silber JH, Rosenbaum PR, Rosen AK, et al. Prolonged hospital stay and the resident duty hour rules of 2003. Med Care. 2009;47(12):1191–1200. doi: 10.1097/MLR.0b013e3181adcbff. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 80.Landrigan CP, Fahrenkopf AM, Lewin D, et al. Effects of the accreditation council for graduate medical education duty hour limits on sleep, work hours, and safety. [See comment] Pediatrics. 2008;122(2):250–258. doi: 10.1542/peds.2007-2306. [DOI] [PubMed] [Google Scholar]
  • 81.Ferguson CM, Kellogg KC, Hutter MM, Warshaw AL. Effect of work-hour reforms on operative case volume of surgical residents. Curr Surg. 2005;62(5):535–538. doi: 10.1016/j.cursur.2005.04.001. [DOI] [PubMed] [Google Scholar]
  • 82.Connors RC, Doty JR, Bull DA, May HT, Fullerton DA, Robbins RC. Effect of work-hour restriction on operative experience in cardiothoracic surgical residency training. J Thorac Cardiovasc Surg. 2009;137(3):710–713. doi: 10.1016/j.jtcvs.2008.11.038. [DOI] [PubMed] [Google Scholar]
  • 83.Coverdill JE, Finlay W, Adrales GL, et al. Duty-hour restrictions and the work of surgical faculty: results of a multi-institutional study. Acad Med. 2006;81(1):50–56. doi: 10.1097/00001888-200601000-00014. [DOI] [PubMed] [Google Scholar]
  • 84.Schwartz A, Pappas C, Bashook P, et al. Conceptual frameworks in the study of duty hours chanves in graduate medical education: a review. Acad Med. 2011;86:18–29. doi: 10.1097/ACM.0b013e3181ff81dd. [DOI] [PubMed] [Google Scholar]
  • 85.Jamal M, Rousseau M, Hanna W, Doi S, Meterissian S, Snell L. Effect of ACGME duty hours restrictions on surgical residents and faculty: A systematic review. Acad Med. 2011;86:34–42. doi: 10.1097/ACM.0b013e3181ffb264. [DOI] [PubMed] [Google Scholar]
  • 86.Reed D, Fletcher K, Arora V. Systematic review: association of shift length, protected sleep time and night float with patient care, residents' health and education. Ann Intern Med. 2010;153:829–842. doi: 10.7326/0003-4819-153-12-201012210-00010. [DOI] [PubMed] [Google Scholar]
  • 87.Taffinder NJ, McManus IC, Gul Y, Russell RCG, Darzi A. Effect of sleep deprivation on surgeons' dexterity on laparoscopy simulator. Lancet 1998; 352(October): 1191. [DOI] [PubMed]
  • 88.Grantcharov TP, Bardram L, Funch-Jensen P, Rosenberg J. Laparoscopic performance after one night on call in a surgical department: prospective study. Br Med J. 2001;323(24):1222–1223. doi: 10.1136/bmj.323.7323.1222. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 89.Eastridge BJ, Hamilton EC, O'Keefe GE, et al. Effect of sleep deprivation on the performance of simulated laparoscopic surgical skill. Am J Surg. 2003;186(2):169–174. doi: 10.1016/S0002-9610(03)00183-1. [DOI] [PubMed] [Google Scholar]
  • 90.Gopaldas RR, Huh J, Bakaeen FG, et al. The impact of resident work-hour restrictions on outcomes of cardiac operations. J Surg Res. 2009;157(2):268–274. doi: 10.1016/j.jss.2009.03.042. [DOI] [PubMed] [Google Scholar]
  • 91.Charap M. Reducing resident work hours: unproven assumptions and unforeseen outcomes. [See comment] Ann Int Med. 2004;140(10):814–815. [PubMed] [Google Scholar]
  • 92.Barone JE, Ivy ME. Resident work hours: the five stages of grief. [See comment] Acad Med. 2004;79(5):379–380. doi: 10.1097/00001888-200405000-00002. [DOI] [PubMed] [Google Scholar]
  • 93.Jones AM, Jones KB. The 88-hour family: effects of the 80-hour work week on marriage and childbirth in a surgical residency. Iowa Orthop J. 2007;27:128–133. [PMC free article] [PubMed] [Google Scholar]
  • 94.Karamanoukian RL, Ku JK, DeLaRosa J, Karamanoukian HL, Evans GRD. The effects of restricted work hours on clinical training. Am Surg. 2006;72(1):19–21. doi: 10.1177/000313480607200105. [DOI] [PubMed] [Google Scholar]
  • 95.Stamp T, Termuhlen P, Miller S, et al. Before and after resident work hour limitations: an objective assessment of the well-being of surgical residents. Curr Surg. 2005;62(1):117–121. doi: 10.1016/j.cursur.2004.09.013. [DOI] [PubMed] [Google Scholar]
  • 96.Shanafelt TD, Bradley KA, Wipf JE, Back AL. Burnout and self-reported patient care in an internal medicine residency program. [See comment] [summary for patients in Ann Intern Med. 2002 Mar 5;136(5):I29; PMID: 11874329] Ann Int Med. 2002;136(5):358–367. doi: 10.7326/0003-4819-136-5-200203050-00008. [DOI] [PubMed] [Google Scholar]
  • 97.Fletcher KE, Parekh V, Halasyamani L, et al. Work hour rules and contributors to patient care mistakes: a focus group study with internal medicine residents. J Hosp Med (Online) 2008;3(3):228–237. doi: 10.1002/jhm.288. [DOI] [PubMed] [Google Scholar]
  • 98.Morrison CA, Wyatt MM, Carrick MM. Impact of the 80-hour work week on mortality and morbidity in trauma patients: an analysis of the national trauma data bank. J Surg Res. 2009;154(1):157–162. doi: 10.1016/j.jss.2008.06.004. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Below is the link to the electronic supplementary material.

ESM 1 (168.5KB, doc)

Study quality and outcomes assessed for all included studies (DOC 168 kb)


Articles from Journal of General Internal Medicine are provided here courtesy of Society of General Internal Medicine

RESOURCES