The Impact of the 80-Hour Resident Workweek on Surgical Residents and Attending Surgeons

Abstract

Objective:

To assess the impact of the 80-hour resident workweek restrictions on surgical residents and attending surgeons.

Summary Background Data:

The ACGME mandated resident duty hour restrictions have required a major workforce restructuring. The impact of these changes needs to be critically evaluated for both the resident and attending surgeons, specifically with regards to the impact on motivation, job satisfaction, the quality of surgeon training, the quality of the surgeon's life, and the quality of patient care.

Methods:

Four prospective studies were performed at a single academic surgical program with data collected both before the necessary workforce restructuring and 1 year after, including: 1) time cards to assess changes in components of daily activity; 2) Web-based surveys using validated instruments to assess burnout and motivation to work; 3) structured, taped, one-on-one interviews with an external PhD investigator; and 4) statistical analyses of objective, quantitative data.

Results:

After the work-hour changes, surgical residents have decreased “burnout” scores, with significantly less “emotional exhaustion” (Maslach Burnout Inventory: 29.1 “high” vs. 23.1 “medium,” P = 0.02). Residents have better quality of life both in and out of the hospital. They felt they got more sleep, have a lighter workload, and have increased motivation to work (Herzberg Motivation Dimensions). We found no measurable, statistically significant difference in the quality of patient care (NSQIP data). Resident training and education objectively were not statistically diminished (ACGME case logs, ABSITE scores). Attending surgeons perceived that their quality of their life inside and outside of the hospital was “somewhat worse” because of the work-hour changes, as they had anticipated. Many concerns were identified with regards to the professional development of future surgeons, including a change toward a shift-worker mentality that is not patient-focused, less continuity of care with a loss of critical information with each handoff, and a decrease in the patient/doctor relationship.

Conclusion:

Although the mandated restriction of resident duty hours has had no measurable impact on the quality of patient care and has led to improvements for the current quality of life of residents, there are many concerns with regards to the training of professional, responsible surgeons for the future.

Four different, prospective study instruments were used to assess the impact of changes in resident work-hours on surgical residents and attending surgeons. Despite improved quality of life for the residents and a decrease in their “emotional exhaustion,” many have concerns with regards to the training of professional, responsible surgeons for the future.

Since the Accreditation Council of Graduate Medical Education (ACGME) mandated resident work-hour restrictions, both proponents and critics have presented emphatic arguments for or against the changes. Proponents say that the restrictions are necessary for the physical and mental health of residents, and to reduce errors that can result from sleep deprivation. Critics point to the fact that shorter work-hours will prevent residents from learning as a clinical episode evolves through the longitudinal course of the patient's illness. Critics feel that this loss of continuity of care will require more frequent handoffs of care, potentially leading to additional errors. Critics are also concerned that the new system will not ingrain a necessary sense of accountability and professionalism in today's residents.¹ The overall effect on the training of future surgeons, and the quality of care of the surgical patient is unclear. Data need to be collected to answer these crucial questions.

The ACGME has mandated nationwide restrictions on resident work-hours as of July 1, 2003. The new requirements include an 80-hour weekly limit, averaged over 4 weeks; at least 10 hours of rest between duty periods; a 24-hour limit to continuous duty plus up to 6 more hours for continuity of care and education; 1 day in 7 free from patient care; and in-house call no more than once every 3 nights averaged over 4 weeks.²

We designed a study using 4 different methodologies to examine the issues surrounding these workforce changes in our surgical department. The goal of our studies was to prospectively evaluate the impact of the 80-hour resident workweek on both surgical residents and attending surgeons by assessing changes in motivation, job satisfaction, the quality of surgeon training, the quality of the surgeon's life, and the quality of patient care.

METHODS

Four prospective studies were performed on all surgical residents and on all attending surgeons who have resident contact at a single academic surgical department. Data were collected both before the necessary workforce restructuring, and 1 year after. The prevailing system was every third night call in house. To be in compliance with the new regulations, multiple new systems were instituted, including a combination of cross-covering, physician extenders, night float residents, call from home, and every 4th night call with the next day free from clinical activities.

The 4 complementary methodologies used for these analyses included:

1. Time cards

Time cards were constructed to assess changes in components of daily activity. They consisted of a 4 × 6 in. card with 18 questions to be filled out each day for 14 days. The questions were designed to quantify the hours and components of day-to-day activities (eg, time spent rounding, in the operating room, at teaching conferences, studying/reading, in the outpatient setting etc).

2. Web-based surveys

Web-based surveys were constructed to assess perceptions of quality of care, quality of life, and job satisfaction, using validated instruments whenever possible. The overall survey was a 47-item, structured questionnaire containing 5 different sections.

Five questions assessed demographic and background information. Nine questions were based on previously published, validated questions.^3–5 They addressed perceptions of the effect of work-hour limitations on quality of life, quality of training, and quality of patient care using 5-point Likert scales. Six questions were designed to dovetail with, and provide quantitative data for, the structured interviews with the external PhD investigator (see below). Five questions were based on the Herzberg Motivational Dimensions to assess the subject's motivation to work.⁶

Twenty-two questions included the entire Maslach Burnout Inventory Human Services Survey (MBI), an established survey instrument that has been extensively tested and validated.^7–11 The MBI questions address 3 axes: depersonalization (feeling an impersonal approach and interaction with people), emotional exhaustion (feeling emotionally overextended by work), and personal accomplishment (feeling of personal achievement and competence at work).¹⁰ The MBI questions consist of a series of 22 brief questions (“I feel emotionally drained,” “I feel used up at the end of the workday,” “I have accomplished many worthwhile things in this job,” etc.) A standardized coding template was used to tabulate the results in each axes as “high,” “medium,” or “low,” and results were benchmarked to the published cutoffs based on large national surveys, with the scale specific to medical workers.⁷ High scores in depersonalization, and emotional exhaustion and a low score in personal accomplishment are associated with high rates of burnout. The opposite is associated with engagement, which is linked to job satisfaction.⁸

The Web-based survey and timecards were assessed by health services researchers experienced in survey design, and piloted on a focus group of surgical residents and attending surgeons.

3. Interviews

An outside PhD investigator who specializes in organizational behavior and workforce issues (K.C.K.) performed structured, taped, one-on-one interviews with surgical residents and attending surgeons. This allowed for an in-depth ethnographic exploration of the critical issues surrounding the work-hour changes. A question script was followed, and interviews provided semi-quantitative results, as well as allowed the investigator to probe further to gain insight into the underlying, prevailing issues and concerns. Interviews were taped, transcribed, abstracted, and analyzed in a semi-quantitative fashion. Direct quotes are included to highlight the findings.

4. Quantitative data

Objective, quantitative data were statistically analyzed to assess the quality of patient care, resident operative experience, and resident knowledge before, and 1 year after the work-hour changes.

The quality of patient care was assessed using prospectively gathered, risk-adjusted data collected by audited, trained, nurse reviewers using standardized definitions from the National Surgical Quality Improvement Program (NSQIP). The NSQIP has been rigorously tested and validated as a quality assessment tool;^12–16 30-day mortality and complication rates were compared.

Resident operative experience was quantified using residents' surgical operative logs. Each surgical resident in an ACGME accredited program is required to submit their operative cases via a web based mechanism every year, and these logs were tabulated to assess the number of cases done by each resident that year.

Resident knowledge was measured using the American Board of Surgery In Training Examination (ABSITE) scores. The ABSITE is a written, standardized examination given to the residents each year to provide an objective assessment of their knowledge. Standardized scores were analyzed to take into account that raw scores are dependent on how hard the test happened to be that year.

Work-hours were tabulated using an automated computerized system that periodically assesses work-hour compliance. Residents had to answer specific questions about the hours they had worked before they were allowed to log into the hospital's computer system that day.

Data were analyzed with χ² analysis of contingency tables for nominal data, and t tests (unpaired and paired, when appropriate) for interval data. A P value less than 0.05 determined statistical significance. All aspects of these studies were reviewed and approved by the institutional review board of the Human Research Committee at the Massachusetts General Hospital. Responses were tracked, but anonymity was maintained. General reminders were given for follow-up, but nonresponders were not specifically targeted per the institutional review board's request.

RESULTS

Response Rates

A total of 58 surgical residents and 58 surgical attending physicians were included in the study. Overall response rate for the time card surveys and Web-based survey was 61%.

Work-Hours and Compliance With ACGME Restrictions

Work-hours averaged 99.5 hours per week prior to the ACGME mandated work-hour restrictions, and 78.9 hours after. Ongoing evaluations showed that most rotations were in compliance with the ACGME requirements, and those that were not were restructured so that they became compliant.

Quantitative Results

NSQIP Results

NSQIP results from 1665 cases (1327 General surgery, 338 Vascular surgery) for the year preceding the work-hour changes were compared with 2311 cases (1875 General surgery, 436 Vascular surgery) for the year following the work-hour changes; χ² analyses of contingency tables for complication rates and mortality rates on each of these services showed no statistically significant difference and no identifiable trends. Therefore, objective assessment of the quality of care showed no measurable difference from the work-hour changes.

ABSITE Scores

The ABSITE scores show an overall slight improvement for each level of resident, a finding that almost reaches statistical significance (paired t test, P = 0.09) (Table 1).

TABLE 1. American Board of Surgery in Training Exam (ABSITE) Scores, Before the Work-Hour Changes (2003) and After (2004)

PGY 3 results were not included, as two thirds of the PGY 3 residents who took the examination were exclusively doing research and so were not involved in patient care, and as such they were not affected by the work-hour changes.

Resident Operative Volume

Overall, there was no aggregated change in the operative case volume per resident per year (231 cases in 2003 vs. 246 cases in 2004, P = 0.61), as previously published (Table 2). ¹⁷ However, the PGY 5 residents did have a statistically significant increase in the number of operative cases (339 cases in 2003, vs. 390 in 2004, P = 0.05).

TABLE 2. Number of Surgical Procedures Performed by the Surgical Residents Each Year, as Documented in the ACGME-Required Surgical Operative Log

Time Cards

Time cards were used to assess whether the components of day-to-day activities changed appreciably after the work-hour changes. Significant findings for the residents are shown in Table 3. After the work-hour changes, residents are more apt to have lunch, to spend time in teaching conferences, and to spend time reading. However, residents tended to spend less time rounding in the afternoon, fewer days in the outpatient office setting, and were less likely to be present in the OR before the patient went to sleep (Table 3). There was no measurable difference in the amount of sleep the residents got, the number of pages they received, or the amount of “scut” that they did.

TABLE 3. Resident Time Card Results

Time cards for the attending physicians showed that attending physicians spent less time involved in meetings or administration after the work-hour changes (46 minutes/day before, 31 minutes/day after, P < 0.01). Attending physicians also felt that residents were less prepared for the cases (scale of 1–10, 3.7 before, and 2.3 after the work-hour changes P < 0.001), were less prepared for teaching conferences (scale of 1–10, 2.3 before, and 1.3 after the work-hour changes, P < 0.001) and the covering resident at night knew the patient less well (scale of 1–10, 7.0 before and 5.1 after the work-hour changes, P < 0.001).

Web-Based Surveys

Web-based surveys assessed the perceptions of the residents and staff on the impact of the work-hour changes. Residents felt that seniors, fellows, the Chief Resident, attending physicians, and leaders of the surgical residency program were more supportive of the 80-hour workweek than expected (P = 0.001 to 0.03 for the groups). Residents thought that the work-hour changes would make the overall workload “somewhat improved,” but the perception afterward was “no change” in overall workload (P < 0.001). They thought the amount of sleep they would get after the work-hour changes would be “much improved,” whereas after the changes they found it to be only “somewhat improved” (P = 0.04). Residents felt that the work-hour changes would make the operative experience “somewhat worse,” when after the changes they perceived “no change” in the operative experience (P < 0.001). In response to the question: “my work made it difficult to maintain the kind of relationships that I like with my family or significant other,” residents responded “often” before the work-hour changes, but this improved to “sometimes” after the work-hour changes (P < 0.001). Similarly, in response to the question: “because of my work, I didn't have enough time to participate in nonwork activities I find relaxing and enjoyable,” residents responded “often” before the work-hour changes, but this improved to “sometimes” after the work-hour changes (P < 0.001).

Several of the residents' perceptions did not change significantly from before the work-hour changes to 1 year after. The residents felt that there would be and was “no change” in the quality of surgical training, that the quality of life in the hospital would be and was “somewhat improved,” that the quality of life outside of the hospital would be and was “somewhat” to “much improved,” and that the impact on the continuity of patient care would be and was “no change” (P = not significant for all of the above).

The attending surgeons were less likely to agree with each of the following statements after the work-hour changes (P < 0.001 for each): 1) “Surgical residents caring for my patients at night know them well.” 2) “There is adequate senior resident coverage at night.” 3) “Currently, the postoperative care that the surgical residents give my patients is excellent.”

Several of the attending physicians' other perceptions did not change from before the work-hour changes, to 1 year after the work-hour changes. The attending physicians consistently felt that the changes made for the 80-hour workweek would be “somewhat worse” for the quality of the training of future surgeons, the quality of their life in the hospital, the quality of their life outside of the hospital, the quality of patient care, their overall workload, their experience in the operating room, and resident learning experiences (procedures, autonomy, supervision, conferences, reflection, reading, etc.).

The attending surgeons perceived that the surgical interns are most at risk due to the work-hour changes. The attending physicians felt that the surgical interns after the 80-hour workweek, had less technical skill, less clinical judgment, less sense of responsibility for the patient, less preparedness for cases, and less efficiency, compared with the interns before the work-hour changes (Table 4).

TABLE 4. Web-Based Survey Results Showing Attending Surgeons' Perceptions on the Impact on the Surgical Interns Due to the 80-Hour Workweek Restructuring

Burnout

For the residents, the Maslach Burnout Inventory Human Service Survey showed a statistically significant decrease in “emotional exhaustion” perceived after the work-hour changes (Table 5). Emotional exhaustion is rated as “low,” “medium,” and “high,” and the residents moved from the “high” range to the “medium” range after the work-hour changes, a statistically significant and clinically significant finding (P = 0.02). “Depersonalization” score decreased from 14.8 to 11.9, a result that approached statistical significance (P = 0.09), and, however, stayed within the “high” range. These findings suggest that the work-hour changes have led to decreased propensity for job burnout in surgical residents.

TABLE 5. Summary of Maslach Burnout Inventory: Human Services Survey for the Surgical Residents, Before the Work-Hour Changes (2003) and 1 Year After (2004)

For the attending surgeons, the Maslach Burnout Inventory Human Service Survey assessment of “personal accomplishment,” “emotional exhaustion,” and “depersonalization” scored in the “medium” range for every category at both time points, and so did not significantly change after the work-hour changes (data not shown).

Motivation to Work

Five questions were derived from the Herzberg Motivational Dimensions and were used to assess motivation to work. As shown in Table 6, after the work-hour changes, the residents had a statistically significant increased motivation to work. Specifically, after the work-hour changes, the residents felt significant improvements in their motivation to do their job, with regards to overall job satisfaction, independent decision making, sense of appreciation, acknowledgment and recognition, and with regards to the fact that their job allowed them to achieve what they want.

TABLE 6. Herzberg Motivational Dimensions

The attending surgeons showed no significant changes in their motivation to work, as assessed from before the work-hour changes and compared with 1 year after the changes (Table 6).

Interviews With External PhD Investigator

Structured, taped, one-on-one interviews with an external PhD investigator who specializes in organizational behavior and workforce issues (K.C.K.) identified many significant recurrent themes. A total of 22 residents (8 interns, 8 PGY 5s, 2 Super Chiefs, 4 mid-level residents) and 20 attending surgeons were interviewed both before the work-hour changes and 1 year after.

The Positives

Interviews with both residents and attending physicians pointed out that the transition mandated by the work-hour restrictions was smoother than expected. Residents and attending physicians felt that physician extenders, including nurse practitioners and physician assistants were successfully incorporated. The night float system, which had not previously been used at this institution, worked better than expected. The new system was deemed to be good for the present seniors, who now take much less in-house call.

The Negatives

The interns were identified as the group at greatest risk from the work-hour changes, by both the attending physicians and senior residents. Both groups point out that there has developed less of a bond between the interns and the attending physicians, senior residents, nurses, and patients. As one attending comments, “I can't even tell you the name of my intern right now.” It was also pointed out how the patients frequently had previously identified the intern as “their doctor,” likely since the intern was omnipresent (attending comment). However, after the work-hour changes, the patients were more likely to refer to their caregivers as “the team” and could no longer identify the individual members of the team by name.

The interviews also identified a concern about the interns' development as a surgeon. As one attending said, “There is a change in attitude in the freshman class. Now they are entitled to work less, to be less tough...now they're softies.” The older residents felt the same way: “When the current intern class are senior residents, that's when you're going to see the difference.”

Another concern was the development of a shift worker mentality in the residents, an attitude that was a change from the previous patient-focused culture. As an attending points out: “These are our patients. We take care of them. We're responsible for them, and you don't go home until the patient is okay. And if you don't go home at all, it's okay too. But that is not what is going on anymore.”

The loss of continuity of care, and the loss of critical information with each handoff, was another recurrent theme as a byproduct of the work-hour changes. As stated by one attending, “It's like the game ‘telephone'...where I tell the intern in the morning, and they sign it out to the PA (physician assistant) and the PA now reports it back to me. But there is something that invariably gets missed there.” An almost identical comment came from a senior resident.

With regards to the quality of patient care, most attending physicians and residents felt during the interviews that the patient care has not suffered, but this was due to the fact that “the ball gets rolled uphill” (senior resident comment). The senior residents feel that they were “picking up the slack” for the junior residents, and the attending physicians think that they were “picking up the slack” for the residents. The end result was felt to be no change in the quality of patient care.

DISCUSSION

We present the results from four, concurrent, prospective studies at a single academic institution that were designed to provide in-depth analyses of the impact of the 80-hour resident workweek restrictions on both surgical residents and attending surgeons.

Impact on Residents

We found an improvement in the current quality of life of the residents after the work-hour changes. Residents had decreased “burnout” scores with significantly less “emotional exhaustion” (Maslach Burnout Inventory). They perceived a better quality of life both in and out of the hospital, an increase in their ability to maintain relationships with significant others, and to participate in nonwork activities they find enjoyable. Residents felt they got more sleep and had a lighter workload, although these effects were less than they had anticipated before the work-hour changes took place. Residents were found to have an increased motivation to work (Herzberg Motivation Dimensions). Resident training and education objectively were not statistically diminished, and showed signs of improvement. Residents documented that they spent more time reading and in teaching conferences, and this was consistent with a trend toward improvement in ABSITE scores. The PGY 5s had an increase in their case volume as documented by their ACGME surgical case logs, while all other residents maintained the same volume of cases despite the reduction in work-hours. Both before and 1 year after the work-hour changes, the residents felt these work-hour restrictions would not affect the quality of training of future surgeons. As one resident said, “I think that it has helped me to truly enjoy life as a whole.”

Impact on Attending Physicians

The impact on the attending physicians and their perceptions about the work-hour restrictions were quite different from the residents' perceptions. Attending surgeons perceived that their quality of their life inside and outside of the hospital was “somewhat worse” because of the work-hour changes, as they had anticipated, and they found that the quality of patient care, and the continuity of patient care would be and was “somewhat worse.” After the work-hour changes, attending physicians were less likely to agree that the surgical residents taking care of their patients at night know them well, that senior resident coverage at night is adequate, that the postoperative care that the surgical residents give their patients is excellent. They consistently felt that the quality of the training of future surgeons, and resident learning experiences (procedures, autonomy, supervision, conferences, reflection, reading, etc) were “somewhat worse.” For the attending physicians, there was no measurable change in job burnout, as assessed with the Maslach Burnout Inventory, or motivation to work, measured using the Herzberg Motivational Dimensions.

The data revealed other significant themes, including identifying the interns as being at greatest risk from the work-hour restrictions. Concerns were also expressed about the loss of continuity of care, about less resident responsibility for the patient, and about the development of a shift worker mentality by the residents.

Comparisons With Published Results

Our findings of an overall improvement in resident quality of life are consistent with several other published studies.3–5,18–20 Our findings of a negative impact of the work-hour restrictions on continuity of care, professionalism, and the development of a shift worker mentality have also been previously reported, or commented on, by others.1,3,20–23

Our findings of an improvement in the burnout scores for residents are novel findings. A study using the Maslach Burnout Inventory to assess the impact of the work-hour changes on residents (n = 26) and attending physicians (n = 7) failed to show statistically significant findings.²⁴ Our study shows a statistically significant and clinically significant improvement in “emotional exhaustion” with mean responses moving from the “high” group to the “medium” group (P = 0.02), and a nonsignificant improvement in “depersonalization” (P = 0.09). This may be due to the larger sample size in our study or to the fact that our study chose time points exactly 1 year apart (as opposed to 6 months apart) to minimize seasonal variability, and the psychologic impact of the academic calendar year on the attending physicians and particularly the residents. Another study looked at “psychologic distress” and “perception of stress” using validated psychometric surveys (Symptom Checklist-90-R and Perceived Stress Scale).²⁵ Our findings of decreased “emotional exhaustion” (MBI) in the residents after the work-hour changes are consistent with their findings of reduced “psychologic distress,” although not consistent with their findings of unchanged “perception of stress.”

Our findings looking at objective data from the NSQIP showed no statistically significant changes in the quality of care. This finding is similar to previously published single institution results.²⁶ Even with greater power (or even the 20,000+ cases/year in the complete NSQIP private sector data set), other potentially confounding, temporal variables could likely not be adequately controlled for in a multivariate model, in the attempt to attribute any perceived effect specifically to the 80-hour workweek changes.

Our study is consistent with previously published studies that resident operative volume was not statistically different after the work-hour changes.^27,28 One single institution study also specifically looked at ACGME case logs, and also found no overall change in case volume.²⁷ Our study, in contrast, identified a statistically significant increase in cases done by PGY 5s, whereas this University of Virginia study showed a statistically significant decrease in PGY 5 cases. This may be due to the fact that our system's use of a night float made for fewer call nights for our PGY 5s, so that they would more likely be available for daytime cases. At the University of Virginia, this change in PGY 5 cases was attributed to the shifting of postcall afternoon cases to other residents.

Our analysis of ABSITE scores shows a nonsignificant trend of improvement over all levels of residents (P = 0.09). A previously published single-institution study of ABSITE scores also showed no significant changes but did not identify any trends or suggestions of improvement.²⁵ However, the analyses of ABSITE scores have limitations, since the test changes each year (1 year might be a harder test than the other) making comparison of percentile correct inaccurate, and standardized results are subject to the confounding factor that all programs were required to make work-hour changes. A study from a New York hospital that implemented work-hour changes in 1998, before many programs had implemented work-hour changes, showed significant improvements in overall mean ABSITE scores, especially in the junior residents.²⁰

The greatest strength of this study is also potentially its greatest limitation. This single institution study allowed for detailed, in-depth analyses with multiple instruments and techniques, including extensive one-on-one interviews; however, the single institution aspect limits the generalizability of the findings. Many of these findings may reflect our institutional bias and may be affected by how we chose to restructure our services in response to the work-hour restrictions. “We have been doing it this way since 1811” is a common saying, both in truth and in jest, in our hallways at the Massachusetts General Hospital. Such scrutiny of anything new may be reflected in the attending physicians' perceptions that everything after these new work-hour restrictions is, on a whole, “somewhat worse.”

The other major limitation of this study is the apparent limitations of being able to actually measure the components of resident knowledge, resident training, psychologic well-being, quality of life, and quality of patient care. Although we attempted to use the most sensitive, quantitative metrics available, many of the issues brought up in the one-on-one interviews would not have been measurable with any techniques that we currently have. Sensitive, specific research instruments will be necessary to tease out whether more information is lost in handoffs and transitions of care, or in the sleepy brain of a postcall resident. As Drazen and Epstein described, “we risk exchanging our sleep-deprived healers for a cadre of wide-awake technicians.”¹ Further studies will need to be undertaken to attempt to assess these important questions. We present in this paper a snapshot of where things stand 1 year after the work-hour changes. The crucial analyses will be many years from now when today's trainees become tomorrow's attending surgeons.

CONCLUSION

The data presented above demonstrate many concerns with regards to the professional development of future surgeons, including a change toward a shift-worker mentality that is not patient-focused, less continuity of care with loss of critical information with each handoff, and a decrease in the patient/doctor relationship. Although the mandated restriction of resident work-hours has had no measurable effect on the quality of patient care, there have been significant improvements in the current quality of life of residents.

Discussions

Dr. Keith D. Lillemoe (Indianapolis, Indiana): This presentation is one of the ever-increasing numbers of papers designed to address the impact of the ACGME work-hour restrictions on various aspects of surgical training and surgical outcomes. I congratulate the authors for having the insight to plan this study so that they would have a true control group from the pre-work-hour restriction time period and then waiting a full year for the effects of these restrictions to mature. In my opinion, this is the most comprehensive evaluation of work-hour changes with the most complete and varied measurements yet published. It is a classic. The full results could not be completely presented in this 10-minute time limit, so I urge all of you to read it when it is published in the next few months.

The only limitation I see is that it is limited to one single institution, and again, in my opinion, that hospital and that training program is one of the finest in the world. Therefore, I wonder if the results can actually be translated to all programs around the country.

The results are somewhat predictable and reassuring. Resident burnout is diminished, they have a better quality of life, get more sleep, have a lighter workload, and increased motivation to work. All this is with no measurable statistically significant effect on patient care, based on NSQIP data, case volumes, or ABSITE scores. Not surprisingly, surgical faculty are less impressed and feel care is somewhat worse and might be much worse without the increase in their own efforts. Although I think we would all applaud the decrease in time spent in meetings or in administration that the faculty noted in response to these changes.

I think it is a great study. But I do have a few questions concerning the methodology.

The first is the response rate to your study. Today you gave the response rate of about 61%, but in the manuscript, you document that the initial survey response was about 37% and 40% in the two groups, whereas in the follow-up surveys they were 77% or 83%, over double the response rate. This suggests to me that perhaps we are mixing a little bit of apples and oranges in the interpretation and the results. Can you comment how the doubling of the response rate in the post-80-hour workweek time period may have affected your results?

Secondly, I appreciate that the NSQIP data showed no difference in overall complications or mortality. But as Dr. Warshaw said, this is somewhat a blunted analysis. Perhaps there may be more subtle findings that may show a breakdown in the communication with increased need for sign-out or expanded cross coverage. Was there an increase in sentinel events or near misses? Has the length of stay or hospital costs increased because of the perceived decrease in efficiency of the surgical interns?

Then, how did you deal with the data for comparison in the attitudes, the ABSITE scores, with respect to various resident years? By that, I mean did you compare before and after groups from the same year with each other, or did you compare the pre-interns with the post-interns in your analysis? This again, that may have altered your statistical comparisons.

Finally, a couple of philosophical questions. First, since concern exists for the interns as being a greatest risk, are you and Dr. Ferguson planning any intervention to change your training program to help avoid future problems? Secondly, clearly you demonstrated “so far so good” with respect to the effects of the work-hour restrictions. But can we be so confident that this will be the same in 3 to 5 years when the current intern class who are considered to have worse technical skill, clinical judgment, sense of responsibility and efficiency, are the residents who are going to be the key players in patient care and those residents who grew up in the old system are long gone? I assume that you are going to repeat this study in a few years time to see what kind of effects have taken place.

Finally, just a compliment I would like to pay to Dr. Hutter and his group in the Center for Clinical Effectiveness in Surgery. With both this study and your excellent paper yesterday, you have clearly raised the bar in terms of clinical outcomes research.

Dr. Matthew M. Hutter (Boston, Massachusetts): Dr. Lillemoe, thank you very much for your kind comments, and I would like to address your questions.

Yes, this is a single institution study and as such it is not necessarily generalizable to all hospitals. The fact that it is a single institution study is probably one of its greatest strengths, as well as one of its greatest weaknesses. The strength is that it allowed us to really delve into the issue, and not just scratch the surface. We were able to approach this topic from many different angles: we used timecards, we used web-based surveys incorporating validated instruments, we used quantitative information where available and had a Ph.D. investigator perform focused, structured interviews. This intensity could not be replicated in a multi-institution study. Although we think that our experience as presented in this study is probably reiterated across the country at other hospitals, the generalizability of this study is potentially a significant limitation.

Another question was about the response rate. The response rate overall was 61%. The response rate did not double over the course of the study, however the response rate for the timecards decreased, while that for the web-based survey increased from one year to the next. I am sorry if that was not clear.

You also asked about other issues—sentinel events, length of stay or other metrics one could look at. We did look at many of these issues. There was no change in sentinel events from before to after the work hour changes. Length of stay has gone down, although it is hard to attribute that specifically to the 80-hour workweek changes. It is more likely to be able to be attributed to the increased use of clinical pathways, which have lead to a dramatic decrease in length of stay—particularly our bowel surgery pathway.

Next you had a methodological question with regards to how we analyzed the groups. We did do analysis by postgraduate year. PGY 1s were compared to the PGY 1s for the next year, and the same with PGY 2s, 3s etc. We did not compare responses by an individual per se as they progressed along their training.

In answer to the philosophical questions that you had, yes, we did identify that the interns were the group that seemed to be most affected by the work hour changes. And yes, we are quite concerned about what this might mean when today's interns are tomorrow's seniors. What have we done about this? Well first of all, we put the interns back on call with “their” seniors. When we first instituted these changes we had split up call so that the intern was no longer taking call so that the intern was no longer taking call with the senior on their team. An important mentoring relationship was lost with these changes, so we changed it back. We have also developed a skills lab, as well as a skills curriculum to help remedy this situation. We have developed a night float systems, so that the interns would be around during the day in order to participate in the care of the patients, and not just put out fires at night. We have hired more nurse practitioners. We have a long way to go, but these are some of the things that we have started to do already to improve the education of the interns.

The loaded question that you put forth is, “where will we stand in three to five years”? Only time will tell.

The ACGME work hour restrictions are here to stay. And I applaud the ACGME in doing this. If they didn't do it, then some other government mandated program would be making us do it. Work hour changes are not going away, and we need to deal with that, stop whining, and make the most if it.

One aspect that we really need to change is our attitude. The residents have been forced into this 80-hour workweek—they did not necessarily choose it. As attending surgeons, it is never productive to say that, because of these changes, the residents are not going to be good surgeons. They are surgical residents because they want to be good surgeons, and it is extremely harmful and counterproductive for them to hear that they may never live up to this because they didn't do it “the old way.”

Dr. Timothy C. Flynn (Gainesville, Florida): I do commend this, like Dr. Lillemoe did, to the membership because it is a piece of work that is probably not ever going to be repeated. It has been 2 and a half years since the ACGME established limitations on resident duty hours, and the GME community as a whole is just beginning to understand the impact of these rules on resident education in the teaching hospitals' processes of care. This paper, as unique as it is, is one that will make a significant contribution to that understanding. And while the response rates for some areas studied were not high, I really don't think this in any way diminishes the value of your report, and it may help us provide information to manage the complex systems that are at teaching hospitals.

As you all know, this topic is of no small interest. There have been papers on the impact of the ACGME rules at virtually every major surgical meeting that I have attended over the last 2 years.

A recent meta-analysis in Sleep, published this month, reviewed 60 papers on the effect of sleep loss on physicians, and implied that the current rules allow for situations where sleep loss can be detrimental and degrade performance. An editorial suggested that the current rules endangered patients by exposing them to physicians who work 24-hour shifts.

In reviewing the literature on duty hours, it is interesting to compare the reports that are written by surgeons and non-surgeons on these issues. Surgeons tend to focus on many of the issues that you discussed, the negative effects on continuity of care, the purported developing shift work mentality, and the risk of repeated hand-offs. Non-surgeons typically are impressed with the experimental data that show declines in cognitive function, negative effects on the individual physician of prolonged sleeplessness.

We should keep in mind where these rules came about. In my opinion, the major stimulus was the IOM report “to err is human,” which suggests that hospital care is hazardous to your health. And I think there is no small number of things that we are seeing that are reaping the whirlwind of this report, including the “pay for performance” activities that we are all going to have to live under.

So what does it all mean? First, as you say, we are never going back to the “good old days.” Second, we have got to figure out how to make this system work to produce the type of surgical practitioners we would like to take care of us. This paper offers some hopes and many challenges. I am encouraged by your observation that burnout motivation scores were improved. Other studies have shown that we take otherwise enthusiastic and idealistic interns and make them bitter and disillusioned by about September of that intern year.

My first question is this: How can we use this improved sense of self-value to create a surgeon well-grounded in the values and social context of caring? Could you not capitalize on this to address the issues of your concern about the reputed decline in dedication to patient care and professionalism?

You noted no change in the NSQIP data. And as noted, this was a relatively blunt instrument and can be interpreted two ways. Limiting the duty hours has not contributed positively to safety, one of the stated goals of the reduction of duty hours. On the other hand, you could say, well, it really hasn't hurt it. And as you note in your paper, you suggest that the faculty are picking up the slack. How do you put this in context with your observations?

Lastly, you devoted a good deal of time in your paper to the interviews of faculty and residents. One of the things that really struck me was the difference in perception of the current situation between the residents and the faculty. The faculty, before the rules went into, as noted, thought that the changes would be somewhat worse. And sure enough, after the rules were put into effect, things were described as somewhat worse. Residents, especially junior residents, seemed to be much more positive. As one resident said, “I think that it has helped me to, like, truly enjoy life as a whole.” You can tell this is a member of the new generation by the gratuitous use of the word “like,” which seems to be every fourth word for my children.

The general thrust of the faculty comments are that the intern class is less dedicated to patient care and quality of work. I would submit that this is an unfair criticism of this group of individuals. There is no doubt that the current group of medical school graduates is different. In this context, I take some exception to your including in your paper a quote by a faculty member about the intern class “now they are entitled to work less, to be less tough, now they are softies.” And while that may be a quote from any number of people, perhaps some in this room, this attitude is not going to be perceived positively by this generation of medical students.

What makes you think that the concerns you expressed are a reflection of the duty hours themselves and not just an older generation looking back romantically on “the good old days” and of the generational difference that we know exists? Are not many of the faculty concerns more about how residents function as providers of patient care and less about their education?

The difficulty we as surgeons are having in no small part is a service versus education problem in our complex systems. In my opinion, our job is to educate healthy, positive, competent physicians. We need to get over the fact that the ground rules have changed and get on with the tasks. You have identified several areas of concerns, issues that people all over the country are talking about. We need to use our considerable talent to solve it.

Dr. Matthew M. Hutter (Boston, Massachusetts): We don't know what the future is going to hold. We are here to provide some information—a snapshot—of what things look like at one point before the work hour changes and what things look like a year later. What we found does raise a lot of questions—questions that I don't think we will have answers to for quite some time. I don't think I will be able to answer all of your questions right now.

One of the overriding questions that you were referring to was the concept of service versus education. At our institution, the work hour restrictions allowed us to take a step back and look at each aspect of each rotation in our residency and ask, “Is this worthwhile for their education or not?” It forced us to look objectively, to separate the wheat from the chaff, and the resultant changes in the residency rotations were for the better. Oour goal was to cut out the scut—the service aspect of the rotations—while preserving the educational components. This introspection was a very positive aspect of the ACGME mandated work hour changes.

We were also able to preserve case volume, as shown in our study. But case volume is not what makes a doctor. Now we need to go back and think about how are we are going to create a doctor, to create a compassionate healer as opposed to a technician. And a lot of that will come through continuity of care, from following a patient through all aspects of their illness and recovery. Continuity of care does not necessarily mean being in the hospital 120 hours a week, which is how we provided continuity of care before. A new paradigm for resident education will need to be developed. We see in this study a move away from a patient focused culture to a shift worker mentality. Residents probably lean more from experiencing the continuum of a patient's care—watching a patient as they present, are diagnosed, as they undergo an operation, and how they do in the postoperative setting. This is very hard to incorporate into the system we have now, however this is something that we need to do.

You took exception to the quote from an attending interview that was in the manuscript that “the new interns are entitled to work less, to be less tough, now they are softies.” This is the attitude that a lot of people have. I personally do not think that this attitude is constructive. The residents want to be good doctors. And we need to help them be good doctors. By telling them they are never going to be any good is absurd. You would never say that to your own child. The residents are our children and we want to see them grow and develop. I present that as a fact, as an attitutde that some have, but I don't necessarily condone the statement.

Dr. Robert T. J. Holl-Allen (West Midlands, England): May I gently criticize that the 56-hour week does not cover attendings. They have no limitation on the hours they have got to work I emphasize the word “got.”

I can speak because I am a member of a family who is involved in the new system, and it involves a lot of shift work, where they work so many hours per day or per night. And this has led to a lot of problems, particularly with handover, continuity of care, but above all, the training of surgeons.

It has reached, I think, a crisis level. Because our president was on national television only a week or so ago indicating that if the week stayed as it was the 56 hours or it went down as we are anticipating in 2 years time to 48 hours, the training of surgeons was now being compromised very severely. And we do not have the luxury of simulators in every hospital. We have very few.

And this has been recognized by my son, who is a courtroom attorney, and myself, both involved in medicolegal work. We are being inundated. And it is quite clear that at the level that the residents are now are inadequately trained and are making major errors of decision because they are not getting the experience, what is going to happen if it goes down to 48 hours?

At the moment, there are sufficient doctors to cover most of the shifts, but the government, of course, has the control over the finances. And only on the airplane coming out did I read that a letter has been sent out by the government to ask hospitals to stop operating to save money. They cannot afford to do certain operations.

This, I think, indicates what is going to happen to surgical training with the reduction in hours. And I think you have to fight to keep your 80-hour week. It seems that the residents are getting a reasonable life, reasonably hard work and I can say I was a resident over here for a while and I think that you have got to keep that 80-hour week. You can't get any less. Because you can learn from the mistakes we are making, or being made in the United Kingdom, in terms of cutting the number of hours. And I think the training will be severely compromised the more hours that are cut.

And there is talk talking to a French surgeon friend of mine that on the continent they want to go down to 40 hours. That would be an unmitigated disaster.

Dr. Matthew M. Hutter (Boston, Massachusetts): I think that we in the US have a lot to learn from what is going on elsewhere with regards to surgeon work hours. The United Kingdom, Germany and the rest of the European community, as well as Sweden, have already adopted much more restrictive work hour changes. This work force restructuring that we have just embarked on in the US is an ethnographic experiment of a huge magnitude. The more information we have—whether from experiments and studies in our own institutions in the US, or from examining what is happening overseas—the better off we will be. 56? 48? 40 hour work weeks? Who knows what is ahead for use or for other surgeons in other countries.

Dr. Thomas F. Dodson (Atlanta, Georgia): I have no barbs and arrows for Dr. Hutter, Dr. Warshaw, Dr. Ferguson, and their colleagues. I rise to congratulate you for your excellent paper.

I think you made one really important point among other things that you have said, and it is a key thing, and I would just repeat it because the flaw that I see in this process are physicians of my age group I am 60 years old who denigrate or criticize our residents for going home at the end of their shift, at the end of their 30-hour workweek, and make them feel as if they are not living up to the standards that we set in the past. If I could change one thing in this 2 and a half days of discussion we have talked about culture change, if I could change one thing in the culture, this would be the thing I would change: I would stop nonconstructive criticism of the current generation.

I would add one note, and then I have two questions. The note is that not all is sanguine in New England, and Irani and co-authors at the Brigham in the Journal of Surgery in August of 2005, did a mailed survey to 19 New England programs, with a response rate of 36%, but they had some interesting findings: 60% of the residents reported doing fewer operations and 39% reported that the requirements had worsened the quality of their training. Although they did have some positive remarks along the way, those are two very negative findings from a mailed survey to 19 New England programs.

My two questions are these: You noted in your paper that the attendings felt the quality of life was “somewhat worse,” as well as the quality of patient care, and continuity of care. Mary Flingensmith from Washington University in St. Louis published a paper in the Journal of the American College of Surgeons in 2004 where she documented that the staff work was increasing; it was actually over 70 hours, and she hypothesized that over time the staff work would exceed that of the residents' work. I just wonder what your thoughts are on how we might improve the attendings' perception that things are really not going so well.

My second question and it is one where “the rubber meets the road” when people leave the hospital mid-day, the biggest potential problem is communicational during the hand-off. How do you hand off the service from one individual to another? How do we avoid making the errors that come when somebody doesn't know the patients as well? There have been a lot of suggestions about this: PDAs lists, so on and so forth. I would be interested in how you assure that you have no problems with communication at hand-off.

Dr. Matthew M. Hutter (Boston, Massachusetts): Dr. Dodson, I agree agree with your first comment with regards to non-constructive criticism. That is something we need to stop.

With regards to negative findings in the Irani paper—that the residents reported that the requirements had worsened the quality of their training—I think that the residents feel this way because we, the attendings, are telling them that. Again, this is not constructive and needs to stop.

You ask, “How do you change the attendings' perceptions?” Well I think you show them the data, like I have attempted to do here. The residents are coming to conference more, they are reading more, they tend to do better on the ABSITE, and they have increased motivation to work and decreased emotional exhaustion. There are many good aspects of the work hour changes and we need to highlight that.

We also need to find out what the problems are and make them better. Telling people they are doing a bad job is not constructive—it is just mean. If they are doing a bad job, it is our fault. As our program director Charlie Ferguson would say, “if they are not doing a good job, it is because we are not teaching them the right way.” That is our job—to teach them.

“Handoffs” were identified from this study as a major issue. Our follow-up work from this study focuses on handoffs. We, and others, are yet to come up with the answer of how to mitigate this most serious side effect of the work hour changes, but we are trying to find some solutions.

Dr. L. D. Britt (Norfolk, Virginia): Myself excluded, I found it interesting we have three former chairs of the RRC in the audience now (Dr. Polk, Dr. Diethelm, and Dr. Bland) and there are four others at this meeting who are on the RRC. I am sure it was an oversight that not one was invited as an invited discussant.

However, my question is this: The residents have novel ways of fudging the hours. Even New York after a few hours, a few years down the road, they had 100% noncompliance of their 80-hour workweek. How did you specifically validate that the residents were actually doing 80 hours? That is my concern. And I think studies early are too premature unless you have some fancy way of validating that the residents are actually going home.

Dr. Matthew M. Hutter (Boston, Massachusetts): I am sorry for the oversight in choosing discussants. No offense was meant.

How did we actually validate that residents were doing 80 hours? Well, we did not specifically track them with an RFID device or other surveillance technology, though that might be the only way to get an accurate assessment. We did however try to address this with a computerized information technology query that we developed. The hospital computer system is the lifeblood of the residents these days. Everything is done on the computers. Before they can access the computers, they had to fill out a questionnaire with regards to the duty hours. Those results were compared to whether they had used the computer system when they said they were not there. Whether this is accurate or not is difficult to ascertain. We also compared the results from the timecard survey to validate the computer results, and we did find concordance. Our results show that, for the most part, we have been compliant.