Sleep, Supervision, Education and Service: Views of Junior and Senior Residents

Abstract

Objectives

To assess sleep time and views about faculty supervision and educational activities of residents training only under 2003 duty hours standards

Design

A survey was delivered with the 2010 American Board of Surgery In-Training Examination (ABSITE). Twelve items explored sleep patterns, supervision, and educational activity times. Survey response relationships to gender, resident level, and program variables were explored through factorial ANOVA and effect size testing. Alpha was set to <0.001 and effect size (omega-squared) significance at ≥ 1% of variance explained to limit statistically significant but practically unimportant results. Survey participation was voluntary and responses were processed separately from ABSITE scoring.

Setting

General Surgery residencies

Participants

6161 Categorical Surgery Residents; 2545 first postgraduate year (PGY1) and PGY2 trainees took the Junior exam (IJE) and 3616 PGY3 and above residents took the Senior exam (ISE).

Results

Response rates were ≥ 95%. Sleep during extended call was significantly less for IJE but IJE sleep mirrored ISE sleep on night float, day assignments, and days off. Faculty supervision was judged Adequate or more by over 90% of both groups. IJE significantly more often rated operative caseloads and operating time as inadequate; caseloads and OR time also linked significantly to program type. IJE reported significantly higher inpatient, but not outpatient, time. Most IJE and ISE agreed that care continuity opportunities were Adequate and judged workloads as Adequate or above. While many IJE and ISE rated educational time as Adequate or better, 25% of each group scored it as Insufficient or worse.

Conclusions

Resident discretionary time is not devoted primarily to sleep. Residents consider increased faculty supervision unnecessary. IJE believe their time could be better apportioned across educational settings. Decreased workloads and increased educational time are desired by substantial minorities of IJE and ISE, arguing for further interventions to preserve education over service.

INTRODUCTION

In December 2008, the Institute of Medicine (IOM) Committee on Optimizing Graduate Medical Trainee (Resident) Hours and Work Schedules to Improve Patient Safety reported to the Congress on “Resident Duty Hours: Enhancing Sleep, Supervision, and Safety”. The Committee’s stated goals were “to recommend ways to improve conditions for safety during training while maintaining the necessary educational experience to ensure long-term patient safety after trainees are on their own” (1, 2). During their deliberations, the Committee found a paucity of high-quality data concerning the impact of existing duty hours standards as mandated in 2003 by the Accreditation Council on Graduate Medical Education (ACGME) (1, 3, 4). To help inform future duty hours debates, in January 2010 we queried residents in ACGME-accredited General Surgery residencies about their sleep habits, educational activities, and faculty supervision. Given the mandatory five-year duration of General Surgery residency combined with two years of research or other professional development activity embedded within many programs, 2010 became the first year in which virtually all residents completed all of their graduate surgical education under the 2003 ACGME standards. To maximize response rates, the voluntary survey questions were delivered along with the 2010 American Board of Surgery In-Training Examination (ABSITE).

MATERIALS AND METHODS

A survey of residents given with the 2009 Internal Medicine In-training Examination (IM-ITE) was modified and delivered with the 2010 American Board of Surgery In-Training Examination (ABSITE). The initial IM-ITE items were drafted based upon review of 2003 duty hours related literature and were modified by the IM-ITE steering committee. Further refinement was performed by a subgroup representing the American College of Physicians, the Alliance for Academic Internal Medicine and the University of Pennsylvania. The final IM-ITE survey items were adapted for use on the ABSITE by ABS psychometricians, representatives from the Association of Program Directors in Surgery (APDS) and from the University of Pennsylvania. Final ABSITE survey items were reworded where necessary to reflect graduate surgical education terminology (e.g. “24+6” instead of “long call”) and were limited in number by available answer sheet space.

Twelve survey items explored resident sleep on different commonly used work schedules (n=4), clinical supervision (n=1), and time for learning activities (n=7) (Table 1). Residents also answered 12 questions related to quality of care and patient safety, findings from which are the subject of a separate study. Residency type (university versus independent, based upon an ABS master list), size (small 1–3, medium 4–6, large >6, based upon finishing chiefs per year) and location (Northeast, Southeast, Midwest, Southwest, West, based upon ABS region definitions) were linked to all examinees using residency program codes. Examinee gender and postgraduate year (PGY) level were self-reported. The ABSITE is comprised of two distinct tests. First postgraduate year (PGY1) and PGY2 trainees take the Junior exam (IJE group) and residents of PGY3 and higher are given the Senior exam (ISE group). Survey responses can thereby also be categorized by junior or senior resident level. Data were filtered to capture responses from only Categorical Surgery residents to eliminate any data skewing by the heterogeneous perceptions of Preliminary residents who ultimately will pursue a wide variety of specialties. Data analyses included descriptive statistics and group mean calculations for survey item responses. Relationships of responses to gender and resident level and to residency program variables were explored through factorial ANOVA. Because of the multiple and complex variables studied and the resultant large number of comparisons being made, as well as the large sample size, alpha was set to <0.001 to limit statistically significant but practically unimportant results. For the same reasons, effect size also was tested (omega-squared) with significance set at ≥ 1% of variance explained.

Table 1.

ABSITE 2010 Survey Items on Supervision, Education, and Sleep

How would you rate your typical general surgery rotations with respect to each of the factors listed below? Rate each of the eight factors listed below on a scale from 1 to 5.	Insufficient	Minimal	Adequate	Above Average	Excessive
Supervision by attending physicians
Workload
Number of operations
Sufficiency of operating room time
Time available for educational activities (reading, lectures, other didactics)
Adequacy of inpatient/ward time
Adequacy of outpatient/office time
Continuity of care
In a typical 24 hour period, how many hours of sleep (total) will you get when you are working each of the following (listed below):	None	1–2 hours	3–4 hours	5–6 hours	7+ hours
Extended shifts (e.g., 24 + 6)
Overnight (e.g., night float)
Day shifts
Days off

In 2010, the ABSITE was delivered in a paper-and-pencil format to most residencies but a subset of programs received a computer-based version via the Internet. Programs using the online test were volunteers recruited by the ABS and APDS through a process unrelated to the survey study. Survey questions were distributed after exam completion and survey participation was voluntary. Survey data were processed separately from ABSITE scoring. Reports provided by the ABS to the authors contained deidentified data in accordance with ABS data-sharing policy. The survey was IRB approved (University of Pennsylvania). The authors and not the Board are solely responsible for analytic accuracy and results interpretation.

RESULTS

The ABSITE was administered to 7588 examinees: 3878 IJE (51%) and 3710 ISE (49%). Filtering out Preliminary residents left a study group of 6161 Categorical General Surgery trainees: 2545 IJE (41%) and 3616 ISE (59%). Just over 80% of residents, whether IJE or ISE, took the pencil-and-paper exam while just under 20% were tested online (data not shown). Gender or program codes were missing for 7% and 1% of residents respectively, and those individuals’ responses were excluded from related subgroup analyses. Gender plus residency type and size distributions are shown in Table 2; the overall, IJE, and ISE groups are indistinguishable. In 2010 women comprised about 40% of Categorical residents. University-sponsored programs contained two-thirds of trainees, Independent programs about 30%, and military programs about 3%. About one-half of residents were enrolled in medium-size programs while the rest were nearly evenly split between small and large programs. Northeastern and Midwestern residencies accounted for about one-half of all residents, Southeast and West about one-third, and Southwest about 10% (data not shown).

Table 2.

Distribution of Gender and Program Type and Size by Resident Level

Variable
	All	IJE*	ISE**
Total	6161	2545	3616
Gender
Women	40%	41%	40%
Men	60%	59%	60%
Program Type
University	68%	68%	67%
Independent	29%	28%	30%
Military	3%	4%	2%
Program Size
Small	23%	23%	23%
Medium	49%	50%	49%
Large	28%	28%	28%

Totals may exceed 100% due to roundiing

^*IJE = Junior exam group, PGY-1 and PGY-2 residents

^**ISE = Senior exam group, PGY-3, PGY-4, and PGY-5 residents

Residents were asked to estimate their typical total daily hours of sleep on several common duty hours assignments: extended call (24+6), night float, day duty only, and day off. Sleep during extended call totaled < 3 hours for 70% of IJE and 60% of ISE and this difference reached significance both by ANOVA and effect size. Sleep was similarly precious for IJE and ISE on night float duty, <3 hours for nearly 60% of both groups (NS). Nearly two-thirds of IJE and ISE slept 5 or more hours while on day assignments (NS). Finally, about 80% of both groups reported 7 or more hours of sleep on days off (NS). Gender plus program type, size, and location were not significantly related to resident sleep time (Table 3).

Table 3.

Group Means and Statistics for Survey Items Reaching Significance

Item	Junior/Senior Resident		p-value	% Variance
	IJE*	ISE**
Number of operations	3.20 (.75)	3.38 (.66)	<.001	1.6
Sufficiency operative time	3.13 (.77)	3.32 (.68)	<.001	1.8
Adequacy inpatient/ward time	3.51 (.61)	3.39 (.58)	<.001	1.0
Extended shifts (e.g., 24 + 6)	2.20 (.90)	2.41 (.96)	<.001	1.1

Item	Program Type			p-value	% Variance
	University	Independent	Military
Number of operations	3.24 (.70)	3.52 (.67)	2.89 (.69)	<.001	3.0
Sufficiency operative time	3.16 (.72)	3.47 (.70)	2.92 (.68)	<.001	2.5

^*IJE = Junior exam group, PGY-1 and PGY-2 residents

^**ISE = Senior exam group, PGY-3, PGY-4, and PGY-5 residents

Residents were asked to rate faculty supervision during typical General Surgery rotations on a five-point scale ranging from Insufficient to Excessive (Table 1). Oversight was judged Adequate or Above Average by just over 90% of IJE and ISE, Excessive by about 5% and Insufficient or Minimal by the remainder. Slight differences between IJE and ISE did not reach significance. Impressions of faculty supervision also were not significantly affected by resident gender or by residency type, size, and location (Table 3).

Residents were asked to assess the balance between education and service on typical General Surgery rotations through ratings for seven elements of their programs: workload; procedure volume; time available for educational activities; time in operative, inpatient, and outpatient settings; and continuity of care. Each element was scored on a five-point scale ranging from Insufficient to Excessive (Table 1). Workload was deemed Adequate to Above Average by 95% of IJE and ISE and Excessive by 5% or fewer (NS). One-half of both IJE and ISE judged their operative experiences as Adequate, but IJE more often rated their caseloads as Minimal or Insufficient while ISE found their caseloads to be Above Average. Differences between IJE and ISE reached significance by both ANOVA and effect size. Unsurprisingly, patterns for Operating Room (OR) time resembled those for operative caseloads, judged Adequate by one-half of all residents but significantly more often Minimal or Insufficient for IJE and Above Average for ISE. IJE and ISE also assessed their inpatient care time demands significantly differently, being seen as Above Average to Excessive for nearly one-half of IJE versus one-third of ISE. Ambulatory care time, however, did not vary significantly by PGY level, being viewed as Adequate to Above Average by about 80% of IJE and ISE (NS). Finally, IJE and ISE agreed about opportunities for continuity of patient care, approximately 60% Adequate, 20% Minimal to Insufficient, and 20% Above Average to Excessive (NS). Resident assessments of operative experience and of OR time also were impacted significantly by program type when tested by ANOVA and effect size. Variance explained was 3.0% for caseload and 2.5% for OR time, the largest effect sizes found in our study, and a more powerful correlation than with PGY level. Independent program trainees were more satisfied with their procedural volumes and available OR time than were University-sponsored trainees, while Military residents were the least satisfied (Table 3). At least 70% of IJE and ISE perceived educational activity time to be Adequate to Above Average and 30% or less of each group found time available for education to be Minimal or Insufficient (NS).

DISCUSSION

Much has been written about the current (2003) ACGME duty hours standards and their effects. Early studies focused on predicted impacts including patient care, educational program, and resident quality of life. (5–6). Many subsequent studies reported pre-implementation versus post-implementation comparisons of clinical and educational outcomes or surveys of resident personal and professional satisfaction. Studies have varied widely in their scope and power (single versus multispecialty, single versus multi-institutional, residents plus or minus faculty), duration (single versus multiple years pre- or post-duty hours reform) and content (subjective survey data, objective performance metrics) (7–15). Predictably, results have been heterogeneous (e.g., differing by specialty) and even internally conflicted (e.g., simultaneous perception diminished care quality but increased care continuity) (14, 16). Confounding factors have included resident PGY level, gender, and program type (10, 11, 13) along with ongoing duty hours adaptations by residencies and teaching hospitals. Unfortunately, the nature and structure of graduate medical education largely preclude randomized controlled trials. In their December 2008 report, the IOM Committee on Optimizing Graduate Medical Trainee (Resident) Hours and Work Schedules to Improve Patient Safety repeatedly cited the absence of high-quality definitive data to inform their deliberations (1).

Despite limited data, the IOM Committee chose to recommend additional duty hours restrictions plus related GME and healthcare system interventions (1). In response to the IOM report, the ACGME undertook its own literature review and sought fresh testimony from multiple stakeholders. Proposed revisions to the 2003 duty hours standards were released by the ACGME for comment in June 2010 and were finalized in September; revised standards will take effect July 1, 2011 (17). Resident survey comparison studies to date have involved residents whose graduate education included experiences both before and after implementation of the 2003 standards. Future assessments of the impact of the 2011 duty hours standards will require that credible data under 2003 standards be available as a baseline. By surveying General Surgery residents in 2010, we assured that virtually all of our respondents, even those completing multiple research years, had started their graduate surgical education on or after July 1, 2003. Our responses from over 6000 Categorical General Surgery residents therefore can be regarded as the 2003 duty hours standards baseline.

Consistent with the focus of the December 2008 IOM report (1), we targeted our survey towards resident sleep and supervision while also assessing educational program quality. In 1988, Bartle described the “rested” General Surgery resident as averaging 6.5 hours of sleep per day (18). Just prior to 2003 duty hours implementation, General Surgery trainees reported mean weekly work hours of 101–105 (5, 12) except in New York where the Bell Commission duty hours regulations were already in place and mean weekly hours were 88 (16). Since 2003, most residents and their programs have reported compliance with the 80 hour per week ACGME regulation. The 20% resultant total work hours reduction has translated into a substantially smaller increment in resident sleep hours per week, despite mandatory education within residencies about sleep and fatigue (6, 19). Senior General Surgery residents from three large university-sponsored programs surveyed in the spring of 2005 reported 44.1 hours of sleep in the week prior to survey completion (6.3 hrs/day) (14). We did not ask residents to estimate their average sleep on a weekly basis but instead examined daily sleep on the most common current rotation schedules. Our data, from residents whose training reflects only the 2003 standards, document minimal sleep (≤ 2 hours) on extended call or night float assignments, improving to 5–6 hours on daytime-only duty and to ≥ 7 hours on mandatory days off for most residents. Average daily sleep for current residents seems unlikely to exceed 7 hours, representing at best a 10% improvement from the pre-2003 era. As suggested by Barden in 2002 relative to the New York Bell Commission work hours regulations (20), it appears that our current residents are choosing not to invest much of the discretionary time gained from duty hours limits into sleep but our study does not offer insights into what motivates their choices. Sleep patterns were not significantly linked to gender or to residency type or size. Senior residents on extended call did achieve significantly more sleep than junior residents (≤ 2 hours for 70% IJE versus 60% ISE). Though the incremental sleep increase for senior residents may be small, additional sleep for those responsible for more complex decision-making seems a desirable outcome.

Supervision of residents, particularly at the PGY-1 level, was a focus of the 2008 IOM report and is addressed in detail by the 2011 ACGME standards (1, 21). Expanded attention to supervision of residents closes the loop on a finding by a New York City grand jury of inadequate Internal Medicine resident supervision in the Libby Zion case, the 1984 death of a young woman in a teaching hospital that ultimately triggered resident work hours limits (22). Supervision of General Surgery residents generally has been less suspect for several reasons. The ACGME Program Requirements for Surgery demand an explicitly described supervisory chain of command for every patient and they hold the attending surgeon fully accountable for each patient’s care (23). Documentation of attending physician involvement required by healthcare payers is the most rigorous for surgical patients. Medicare is the standards leader, mandating that the attending teaching physician be present for the key moments of every major procedure and for the entirety of each minor procedure as well as providing evidence of involvement in preoperative and postoperative care (24). Finally the rich traditions of the Halstead model, upon which General Surgery residency is based, include a progressive assumption of responsibility at each level within the residency for oversight of all more junior trainees. Our data indicate that faculty supervision of General Surgery residents in the context of the 2003 ACGME standards is viewed as at least adequate by more than 90% of residents. Ratings of faculty supervision were unaffected by resident seniority, gender and program type, size, and location. Surgical faculty more often find themselves challenged to provide sufficient opportunities for resident autonomy than to supervise trainees sufficiently.

A goal of the IOM Committee was to enhance sleep and supervision while “maintaining the necessary educational experience to ensure long-term patient safety after trainees are on their own” (2). Longitudinal sampling of patient safety metrics was clearly outside our study scope. We did undertake to inform discussion about the General Surgery residency educational experience by establishing a baseline of resident perceptions of several important educational elements of their programs in the 2003 ACGME duty hours standards era. Happily, about 80% of junior residents and 90% of senior residents rated their number of operations and sufficiency of Operating Room time as Adequate or better. Senior residents were statistically significantly more satisfied suggesting that efforts to increase caseloads should include and perhaps preferentially target junior residents. Program type also significantly influenced operative adequacy assessments, identifying greater challenges in this educational activity for military and University-sponsored programs than for Independent programs. Program size and location did not independently impact resident views of their operative experiences. Given the growing numbers of women residents, it is noteworthy that gender did not influence perceptions of operative volume. Inpatient care opportunities were deemed Adequate or greater by over 90% of residents, and the distribution was skewed significantly towards Above Average for Junior residents. Outpatient care opportunities were judged as slightly less strong than inpatient, but were seen as Adequate or better by about 80% of Junior and Senior residents. Resident level and gender, and program variables, failed to significantly influence outpatient educational ratings.

The remaining three educational questions we explored arguably are the most important to the future of General Surgery graduate education. First, we asked for resident perceptions of continuity of care, considered a vital part of surgical education and surgical patient care. Much of the apprehension voiced by surgical disciplines to the IOM report and 2011 ACGME standards (25–26) takes origin from concerns about continuity of care by residents, and continuity of care has been identified as a major concern when surgical faculty members are surveyed (10). Despite the attention given to this issue at meetings and in publications, it appears to resonate far less strongly with residents training post 2003 standards implementation. In our study, at least 80% of junior and senior residents responded that continuity of care was Adequate or better and their views were statistically indistinguishable. We did not confirm the higher value placed on continuity by senior residents that has been reported by others (13). Gender and residency program variables also did not impact resident views of care continuity. Our findings could reflect much larger sample size than prior work or that our survey population has completed their education entirely in the context of the 2003 ACGME standards.

Secondly, we asked residents to score the time available for educational activities, formal (e.g. lectures) and informal (e.g., reading) and to asses their overall workloads. We anticipated little change in formal educational activities since the educational experiences required for General Surgery board eligibility did not decrease with the 2003 ACGME standards. Further, since substantial time has been transferred back to the control of residents, we anticipated they would have more time for self-directed learning. We were uncertain whether workload assessments would decrease, reflecting increased use of physician extenders and hospitalists, or increase, reflecting continued high volumes of complex patients cared for in fewer duty hours. While most residents regard time for educational activities as Adequate or better, we were disturbed to find that at least 25% of Junior and Senior residents judge the available time as Minimal to Insufficient. Time for educational activities was deemed Adequate or better by fewer Junior and Senior residents than OR time, case numbers, inpatient time, and outpatient time, consistent with sacrifice of structured education to patient care demands. Gender and residency program variables did not impact perceptions of educational activity time. Overall workload was deemed adequate by approximately 60% of Junior and Senior residents and Above Average by about 35% of both groups and workload perceptions were not affected by gender or residency program variables. Our findings about educational time are consistent with Willis’ survey of General Surgery Program Directors in which nearly 2/3 of program directors had reassigned residents to higher acuity inpatient services, more than one-half had increased Trauma and Night Float rotations, more than one-half had reduced or eliminated multiple rotations including essential content areas such as Minimally Invasive Surgery, and one-third had reduced core conference time (27). Other studies have reported decreased faculty teaching and teaching among residents (8) and decreased interest in teaching by residents (6). In sum, our work and the literature suggest that at least ongoing intensive monitoring and perhaps immediate specific interventions must be undertaken to maintain the necessary breadth of General Surgery residency educational activities. While workload is not yet judged clearly excessive by most residents, assuring an appropriate balance of education to service within General Surgery residencies clearly will require system approaches rather than more duty hours changes.

CONCLUSION

We undertook a nationwide survey of General Surgery residents in January 2010, focusing on the subjects of sleep, supervision, and education to service balance, as examined in the IOM December 2008 report on resident duty hours (1). Over 95% of residents responded, establishing our findings as a highly credible baseline of perceptions of Categorical General Surgery residents whose graduate education has occurred only after implementation of the 2003 ACGME duty hours standards. Extended call and night float schedules are associated with two or fewer hours of self-reported sleep for the majority of residents, while daytime assignments and days off produce five or more and seven or more hours of sleep per day respectively. Increased resident discretionary time has not translated in large measure to increased sleep time when compared to historical controls. Faculty supervision of residents at all levels is perceived to be sufficient. Many educational elements (operative volume, inpatient time, and outpatient time) are viewed as adequate or better by most residents. While workload is judged adequate by the majority of residents, nearly one-third find it to be above average. This finding, combined with the perception of 25% of residents that time for educational activities is minimal to insufficient, raises concern about preserving the education to service balance in today’s General Surgery residencies. System solutions seem more likely than more complex duty hours limits to address education to service balance concerns.