ACGME Duty Hour Compliance in a General Surgery Residency Program: Challenges and Solutions in a Teaching Hospital

Abstract

Background:

The inception of work hour restrictions for resident physicians in 2003 created controversial change within surgery training programs. On a recent ACGME survey at our institution, we noted a discrepancy between low recorded duty hour violations and surgery resident’s perception of poor duty hour compliance. We sought to identify factors that lead to duty hour violations and to encourage accurate reporting among surgical trainees.

Methods:

A3/Lean methodology, an industry derived systematic problem-solving approach, was used to investigate barriers to accurate duty hour reporting by residents within the department of surgery at an academic institution. In partnership with our Graduate Medical Education office, we encourage a 6-month period where residents were asked to accurately record duty hours and provide descriptive explanations of violations without punitive effects on residents or the program. We performed a 6 month before and after analysis of duty hours violations following the A3/Lean implementation. Quantitative analysis was used to elucidate trends in violations by post graduate year and rotation. Qualitative evaluation by key thematic areas revealed resident attitudes and opinions about duty hour violations.

Results:

Residents reported fear of personal and programmatic punitive measures, desire to retain control of their surgical education, and frustration with the administrative burden following violations as deterrents to honest duty hour reporting. The intervention was successful in changing logging behavior with 10 total violations prior to A3 meeting and 179 violations afterward (p = 0.003). This was largely driven from an increase in Short Break violations (4 vs. 134, p = 0.021). Analysis of violations revealed trends by post-graduate year, rotation and weekend cross-coverage. Key findings including lower than anticipated 80-hour work week violations despite high rates of short break violations. The ability to participate in procedures voluntarily and a sense of professional responsibility emerged as the prevailing themes among residents describing violations.

Conclusions:

Systematic evaluation of duty hour reporting within a surgical training program can identify structural and cultural barriers to accurate duty hour reporting. Accurate reporting can identify program specific trends in duty hour violation that can be addressed though programmatic intervention.

Introduction

In 2003, with revisions in 2011, the Accreditation Council for Graduate Medical Education (ACGME) enacted duty hour restrictions for medical trainees to promote patient safety and a healthy occupational environment for residents (1, 2). Multiple investigations of the effects of these restrictions on surgery training programs have noted heterogenous results (3, 4). The majority of studies have found no change in patients’ post-surgical outcomes (5–7) and either no change or a clinically insignificant decrease in operative volume following work hour restrictions (8, 9). Conversely, in multiple survey-based investigations, surgery residents report decreased educational and operative opportunities following work hour restrictions (10–12).

More recently, The Flexibility in Duty Hour Requirements for Surgical Trainees (FIRST) trial performed in 2014–2015 demonstrated a less restrictive structure on work hour limitations did not negatively impact patient outcomes or surgical resident well-being (13). This resulted in the ACGME modification of duty hours restrictions to now include an 80-hour work week maximum, a mandatory break between shifts (8 hours), a required one day off per week on average over the course of a month, and a 24-hour maximum shift length regardless of trainee level.

Most surgery training programs have residents self-report duty hours on a weekly basis. This information is aggregated at the program level and reported to the ACGME for review. The tension of accurate reporting while maximizing educational experience and patient care has led to duty hour reporting compliance issues in surgery programs across the nation (14–16).

Our general surgery training program participates in the annual ACGME resident survey, which includes anonymous questions regarding duty hour compliance. At the end of the 2017 academic year, our results indicated approximately 20% of general surgery residents felt they were having difficulty adhering to work hour regulations at our institution. However, very few violations were being recorded across our residency program. This highlighted two important issues- first, there was a perceived difficulty in adhering to mandatory ACGME work hour restrictions; and second, there was hesitation among residents to log working hours honestly.

To address these important issues, our surgery department, with close support of our Graduate Medical Education (GME) team, adapted A3 thinking (17, 18) and root cause analyses to evaluate compliance with ACGME work hour restrictions. These techniques were first derived in the manufacturing industry (19) and have been used to improve the functionality and efficiency of outpatient surgical clinics, inpatient discharge processes and OR turnover time (20–22). In this report, we highlight key aspects of our investigation of duty hour violations at our institution. We include early results of duty hour compliance and our employment of Lean principles (23) to maximize the rotational structure of our residency program to maintain excellent surgical and clinical training.

Methods

Key aspects of A3/Lean Investigation

Following our institutional ACGME survey results noting concern for duty hour violations, our pre-existing Program Evaluation Committee created a working group to specifically address this issue (Figure 1: Timeline of Interventions and Evaluation). The working group consisted of the general surgery residency program director, representative general surgery residents and faculty, and the Associate Dean of Graduate Medical Education. Immediate working group recommendations included direct changes to the resident consult service to increase resources for a notoriously demanding position. They also suggested implementation of a professional scheduling software. Next, the working group employed a formal A3 investigation at the resident level, guided by an institutional specialist unaffiliated with the Department of Surgery and sanctioned by the GME office. At the A3 meeting, the entire general surgery residency group was present without faculty for a discussion of duty hour compliance. The aim of the A3 meeting was specifically to determine potential causes of duty hour violations and find opportunities for improvement, as well as determine how to encourage truthful reporting of duty hour among residents. Following the A3 meeting, a 6-month evaluation period was undertaken in which accurate reporting of duty hours was emphasized without individual or programmatic penalization for violations.

Figure 1:

Timeline of A3 Meeting and Evaluation Period. A3 Meeting occurred in October 2017. Pre-intervention period (April 2017- October 2017). Post-intervention Period (October 2017- April 2018)

In addition to the 6-month period following the A3 Meeting (Oct 2017- April 2018), data was also retrospectively evaluated over the 6-month period directly prior to the meeting (April 2017- Oct 2017) to determine if logging behavior changed as a result of the intervention. All work hours were self-logged by residents using existing tracking software (New Innovations) over both periods. At the conclusion of this data collection period, quantitative and qualitative data analyses were performed.

Quantitative Analysis

We compared overall duty hour violations 6 months before and 6 months after the A3/Lean Meeting. We assumed, a priori, that duty hour violations would be more accurate following the A3 meeting. We thus concentrated further descriptive analysis on the 6-month period following the intervention. We analyzed 4 key ACGME duty hour restriction requirements for analysis- 80-hour weekly maximum (“80 Hour”), 8-hour mandatory break between shifts (“Short Break”), required one day off per week on average per month (“Day Off”), and a 24+4 hour shift length maximum (“28 Hour”). We abided by the strict ACGME definition for each violation as specified in the 2017 ACGME Program Requirements (24). All violations triggered by New Innovations were individually reviewed by a Chief Resident to assure accuracy of scheduling software. Descriptive analysis was undertaken by ACGME violation type, clinical rotation, and PGY year. Two-sided student T-test, Wilcoxon Rank Sum test or Fisher’s Exact test were used as appropriate. A p-value of less than 0.05 was considered significant for all hypothesis tests. Statistical analysis was conducting using SAS Version 9.4 (Cary, North Carolina).

Qualitative Analysis

After triggering a violation in New Innovations, residents were encouraged to give free response descriptions describing the circumstances surrounding the violation. These free responses were analyzed qualitatively using content analysis methods (25). The principle goal was to elucidate explicit and implicit factors affecting resident attitudes toward duty hour logging/violations. We also evaluated the role of specific rotations and other extraneous factors in accumulating duty hour violations. The data was first examined as a whole by authors DG and BG independently, as suggested by Tesch and colleagues (26). Within the available content, codes that described key factors contributing to the violation and prevailing resident attitudes or opinions were noted (27). The investigators then met to review findings and in a collaborative fashion, codes were systematically grouped and hierarchically subcategorized to create overall related categories of data derived from the available resident responses (28). A subgroup analysis was also performed by evaluating responses based on PGY level, rotation, and violation type.

Results

A3/Lean Meeting

Problem Statement and Background

With all general surgery residents present and facilitated by a small group of institutional experts, we utilized A3 thinking and the ‘Five Whys’ to fully engage in root cause analysis (29). Our main problem statement included the discrepancy between reported work hours by general surgery residents and actual hours worked by the residency group. The initial background discussion revealed that most general surgery residents felt they often violated strict ACGME definitions of duty hours. However, there was general consensus that work requirements fluctuated greatly by rotation and post graduate year. Many people felt that duty hour requirements normalized when averaged across rotations. A shared sentiment among the group was the absolute priority of adequate technical practice and case exposure during the surgical training period.

Root Cause Analysis

Several areas were identified as to why work hour violations occur. The desire to complete all patient care tasks for the day, including operating, was the main issue discussed by residents. This was described as an individual responsibility to complete the administrative and clinical work associated with each patient on a service. Specific reasons for duty hour violations included (in no order): high patient volume on surgical services, lack of back up coverage in the event a resident was absent from work, system inefficiencies especially in operating room turn-over, communication challenges due to a lack of ability to directly contact nursing and ancillary staff, dependence on attending surgeon’s schedule for afternoon and evening rounds, and desire to participate in all available operative and educational opportunities.

Residents cited they did not accurately report duty hours to avoid the additional time and recriminations associated with duty hour violations. Residents feared losing educational and operative experiences in addition to consequences for the program if duty hour violations accumulated. The general surgery resident group then came to a consensus that honest reporting of duty hours would require the following: no programmatic or personal ramifications from the institution or GME, identifiable data would remain ‘internal’ to the Department of Surgery, and mechanisms would be put in place to improve the ease of reporting and reduce redundancy in explanations of violations.

Proposed Solutions

Following the meeting, the Department of Surgery leadership and our institutional GME group reviewed the results of the A3 data and agreed to a 6-month pilot period of data collection. During this time, duty hours and violations logged by the general surgery residents would be used for quality improvement and remain free from punitive consequence.

Quantitative Analysis

There were 36 clinical residents during the study period (Table 1). Initially, 207 violations were recorded in New Innovations over the 12-month study period (6 months (April- Oct 2017) retrospective period prior to A3 meeting and 6 months (Oct 2017- April 2018) prospective period following meeting). After individual review of each registered violation, 18 violations were excluded from analysis as they were triggered by the software program but did not represent true violations. A total of 189 violations were included for analysis. There were 10 violations prior to the implementation of the A3 working group recommendations and 179 after, representing a statistically significant change in logging behavior (p = 0.003). This was driven by a large increase in the number of Short Break violations (4 vs. 134, p = 0.021).

Table 1:

ACGME Violations by Post Graduate Year

Post Graduate Year		ACGME Violation
	Number of Residents	ACGME 28+	ACGME 80 Hour	ACGME Day Off	ACGME Short Break	Total
PGY 5	6	0	0	1	25	26 (14.5%)
PGY 4	5	15	3	1	30	49 (27.4%)
PGY 3	7	2	0	2	8	12 (6.7%)
PGY 2	7	1	3	0	28	32 (17.9%)
PGY 1	11	11	6	0	43	60 (33.5%)
Total	36	29 (16.2%)	12 (6.7%)	4 (2.2%)	134 (74.9%)	179

Of the 179 violations after the A3 meeting, 134 (74.9%) were Short Break violations. Under the ACGME rules, Short Break violations (8 hours between shifts) is a recommendation to promote duty hour compliance with the 80 hour work week and the correct days off, but itself, does not represent a duty hour violation (24). For the other 3 violations types, 29 (16.2%) were +28 Hour violations, 12 (6.7%) were 80 Hour violations and 4 (3.1%) were Day Off violations. An estimated ‘opportunity’ for each violation to occur across the residency group was calculated by the number of residents (36) multiplied by the number of possible violations per month (dependent on violation type) multiplied over the study period (6 months) (estimation calculations in Appendix 1). In gross percentage based on the estimated ‘opportunity’ for each violation, the 80 Hour violation occurred 12/216 times (5.6%); the Short Break violation occurred 134/5,616 (2.4%); 28 Hour violation occurred 29/5,616 (0.5%); and the Day off violation occurred 4/864 (0.4%). Over the study period, 25 of 36 residents (69.4%) logged at least 1 violation. Violations are further described by Post Graduate Year (Table 1) and Rotation (Table 2).

Table 2:

ACGME Violations by General Surgery Rotation

Rotation	ACGME Violation
	ACGME 28 Hour	ACGME 80 Hour	ACGME Day Off	ACGME Short Break	Total
Community Hospital (PGY3)	1	0	0	2	3
Colorectal Surg (PGY1/PGY3/ PGY5)	2	2	1	8	13
Emergency General Surg (PGY1/PGY2/ PGY5)	3	4	0	10	17
Endoscopy (PGY1)	0	0	0	1	1
Surgical Oncology (PGY1/PGY2/ PGY3)	1	0	0	1	2
Hepatobiliary Surg (PGY3/PGY5)	0	0	0	5	5
Night Float 1 (PGY1)	0	0	0	9	9
Night Float 2 (PGY1)	0	0	0	2	2
Minimally Invasive Surg (PGY1/PGY3/ PGY5)	5	0	1	5	11
Pediatric Surgery (PGY1/PGY4)	3	0	0	4	7
Endocrine Surgery (PGY4)	3	0	0	3	6
Endocrine/ Hepatobiliary Surg Surgery (PGY1)	0	1	0	0	1
Surgical Intensive Care (PGY1)	0	0	0	1	1
Trauma Surg (PGY2/PGY4)	3	3	0	5	11
Transplant Surgery (PGY1/PGY2/PGY4	7	2	1	30	40
Veterans Administrative (PGY1/PGY2/ PGY5)	0	0	1	46	47
Cardiac ICU (PGY1)	0	0	0	2	2
Vascular Surgery (PGY1/PGY2/ PGY3)	1	0	0	0	1
Total	29 (16.2%)	12 (6.7%)	4 (2.2%)	134 (74.9%)	179

Abbreviations- Post Graduate Year (PGY)

The ACGME 80-hour violation occurred across 5 services- Emergency General Surgery (n=4), Trauma Surgery (n=3), Transplant Surgery (n=2), Colorectal Surgery (n=2) and Hepatobiliary/Endocrine Surgery (n=1). The 80-hour work week violation occurred most often in PGY1 (n=6). It also clustered for PGY4 (n=3) and PGY2 (n=3). Chief residents had no recorded violations of the 80-hour rule. The Emergency General Surgery rotation was specifically difficult for junior residents to adhere to the 80-hour work week (n =4 violations in PGY1 and PGY2 over the study period).

The ACGME 28+ hour work limitation largely affected the PGY4 (n=15, 51.2%) and PGY1 (n=11, 37.9%) trainees. This violation occurred more frequently on the weekend (21 violations vs 11 violations, p = 0.001) when residents were cross-covering additional services. The violations were heterogenous across 10 of the 18 represented specialties with slight concentrations in transplant surgery (n=7) and minimally invasive surgery (n=5). The ACGME Day-Off violation was also heterogenous with violations occurring in PGY5, PGY4 and PGY3 years and without noticeable trend among specialties. The ACGME Short Break violation occurred at every PGY level and across every represented rotation. There were expected increased short break violations in transplant surgery (n=30) and our Veterans Administrative General Surgery rotation (n=46) in which residents take home call.

Qualitative Analysis

The free response comments from trainees logging duty hour violations (approximately 103 of 189 logged violations included comments) were analyzed with four principle categories emerging: educational and procedural opportunity, clinical task completion, patient-physician relationship, and system inefficiency. The prevailing theme was that of educational and procedural opportunity, cited in nearly half (53/103) of all resident comments. Quantitative results of comments within each particular theme are displayed in Table 3.

Table 3:

Thematic Areas from Qualitative Analysis by Post-Graduate Year and Violation Type

Evaluation Category	Education & Procedure (total: 53)	Clinical Task Completion (total: 30)	Patient-Physician Relationship (total:13)	System Inefficiency (total: 7)
PGY Year
Junior Resident	16	23	3	5
Senior Resident	37	7	10	2
Violation Type
80 Hour	3	5	1	1
28 Hour	9	12	3	0
Day Off	1	0	1	0
Short Break	40	13	8	6

Junior Residents- (Post Graduate Year (PGY) 1 and PGY2), Senior Residents (PGY3-PGY5)

Educational and procedural opportunity

Opportunities such as emergent cases, cases combined with other subspecialties, and the labor-intensive rotations like transplant surgery with multiple operative settings including trips for organ procurement, were cited most often as contributory to working over hour limits. The desire to finish a case in progress and to be present for re-operations of previous patients known to the trainee also contributed. The overall sentiment indicated residents felt compelled and motivated to stay beyond hour restrictions to pursue these opportunities. In these settings, it was noted to be a personal choice to stay as opposed to a requirement. The desire to pursue educational and procedural opportunities transcended PGY status. It was most commonly cited as the reason for a violation by all residents regardless of training level. Representative comments included: “Acting transplant fellow on call, stayed for an opportunity to assist in a pancreas procurement and to do a kidney transplant as the acting primary surgeon.”

Clinical Task Completion

The second most common theme (mentioned in 30/103 comments) among resident feedback accompanying violations included clinical task completion. This encompasses inpatient responsibilities from rounding and documentation as well as outpatient responsibilities in the surgical clinics and consults from the Emergency Department. Clinical tasks were more commonly cited by junior residents, PGY-1 and PGY-2, than senior residents as a justification for a violation. A sample of comments associated with this theme include “22 consults and several traumas overnight with plenty of left-over documentation.”

Patient-Physician Relationship

There was strong evidence of the emphasis of the patient-physician relationship by residents when evaluating violation comments. Residents stated they felt compelled to be present to provide care for the complete patient encounter. Multiple comments indicated these included times surrounding patient’s end of life. Trainees also mentioned cultivating and maintaining relationships with the families of their patients. These behaviors seemed to be particularly evident among the senior level residents, reflective of experience and increased responsibility.

A recurring secondary theme within that of the patient-physician relationship was personal ownership. Verbiage used by trainees reflected a strong personal commitment to the patient and a serious dedication to their own training as well as that of their fellow residents. There was a sense of teamwork and acknowledgment that surgical training is beyond that of a job. The frequent use of “my patient” or “my team” over “the patient” or “the team” was clearly evident. “I facilitated several conversations with family, palliative care, ethics, and trauma team throughout the day regarding transition to [DNR] status for a recent [ICU patient].” “I had a patient who suffered unspeakable abuse and I felt obligated to continue caring for this patient and family.”

System Inefficiency

System inefficiencies that accompany any large medical institution were mentioned in several resident comments. Interestingly, this was the least demonstrated theme among the aggregate comments (less than 7% of all overall comments). While system inefficiencies exist, this theme did not seem to be strongly influential in resident duty hour violations among surgical residents queried. There were also no consistent sub-themes noted within system inefficiencies including electronic medical records or specific health services within the hospital system.

Discussion

In this study we describe a novel application of A3 thinking and root cause analysis to the challenge of ACGME work hour restrictions and compliance among surgical residents. Eliminating the punitive culture surrounding violations afforded the opportunity for self-evaluation, regulation, and improvement in a highly effective resident driven manner at our institution. Maintaining resident autonomy and protection of educational opportunities was a key aspect of this study. It encouraged resident participation and implementation. Supported by a collaborative GME and Department of Surgery relationship, the collection of accurate data following our A3 meeting intervention revealed temporal duty hour violation ‘hot spots.’ We were also able to determine PGY specific violations, including increased short break violations in senior residents, in addition to highlighting rotations that seemed to lead to violations at all residency levels. These included weekend violations for cross covering junior residents. These findings were then used to institute meaningful changes to our rotational scheduling and standard work. Examples included standardized rounding times and redistribution of cross coverage assignments to help improve workflow and eliminate redundancy, particularly for our junior residents assigned weekend shifts.

Combining A3, quantitative and qualitative data revealed several key themes that have led to actionable suggestions for the program. First, the recognition that junior and senior residents face different challenges and require different solutions regarding duty hour violations. Junior residents appear to be bound by shift scheduling and as a result, report violations more often as a consequence of cross coverage, variable rounding times with senior residents and attendings, and administrative burden. Senior residents report more rotation specific violations associated with professional accountability, such as operative complications or extensive family and patient conversations. This likely reflects an emphasis on the growth of responsibility as senior residents progress through training. These results support that there may not be a universally beneficial change available within a residency program, but through methodological evaluation, solutions exist for the challenge of duty hour reporting and compliance for residents at all training levels.

We found that the majority of violations logged at our institution were short break violations. Despite the significant increase in these violations registered in our study, we had relatively low 80-hour work week violations (~5%). Daily and weekly violations appeared to normalize over a 4-week period in many cases, particularly for senior residents. Furthermore, on qualitative analysis, the residents greatly valued many of the experiences that occurred during ACGME violations as they nearly always involved a direct operative experience or opportunity to deepen a physician-patient relationship. Repeatedly throughout the study our residents evidenced their strong dedication to the pursuit of high-quality education and the ownership and personal responsibility they feel for the patients they serve.

The success of trainee driven change within the healthcare field has been reported in many instances from decreasing utilization costs through unnecessary lab draws to improving system efficiency in the patient discharge process (30, 31). We advocate for the use of A3 thinking and Lean methodology to evaluate for areas of change within residency programs as it is an accessible opportunity for collaboration and quality improvement education for residents as well as the attending staff. The needs and demands of each training program are unique; however, we have provided a framework to encourage investigation and creative thinking to improve efficiency of resident training/scheduling within the bounds of ACGME duty hour restrictions.

Our investigation has several limitations which may affect generalizability of our results to other programs. We have a strong institutional experience with A3 thinking and Lean methodology and utilized institutional experts in this field to assist our investigation. Our study was not randomized and only partially de-identified at the resident level, which may have affected accurate reporting for some individuals despite our intervention design that eliminated recriminations. While 70% of all residents reported at least one violation and we demonstrated a significant change in logging behavior, we cannot confirm that this data represents ‘true’ work hour behavior as our intervention, and the reporting system, relies on self-logging. We also did not evaluate different demographic specific factors that could potentially effect duty hour violations, such as having a family. We believe this is an important future direction associated with the large issue of work hour regulation. Despite these limitations, this study allowed us to develop meaningful scheduling changes in our program to assist with duty hour compliance.

Conclusions

We demonstrate one of the first published studies using A3 thinking and root cause analysis to identify barriers to accurate ACGME duty hour reporting among general surgery residents. We further provide investigational tools to improve duty hour compliance in other training programs. As a result of this intervention, our residency program experienced a significant improvement in reporting and was able to determine a more honest assessment of violations occurring within the program. A3 thinking is an iterative process of continual re-evaluation. Future aims of this work will be to continue to evaluate and to improve our duty hour compliance while maintaining excellent clinical and surgical training.

Appendix

Description of ACGME Duty Hour Violations

1. 80-hour weekly maximum: clinical and educational work hours must be limited to no more than 80 hours per week, averaged over a four-week period, inclusive of all in-house clinical and educational activities, clinical work done from home, and all moonlighting.

2. Mandatory break between shifts: Residents should have 8 hours off between scheduled work and education periods. Residents must have at least 14 hours free of clinical work and education after 24 hours of in-house call.

3. Required day off per week: Residents must be scheduled for a minimum of 1 day in 7 free of clinical work and required education (when averaged over 4 weeks). At-home call cannot be assigned on these free days.

4. 24-hour shift length maximum: Clinical and educational work periods for residents must not exceed 24 hours of continuous scheduled clinical assignments. Up to four hours of additional time may be used for activities related to patient safety, such as providing effective transitions of care, and/or resident education. Additional patient care responsibilities must not be assigned to a resident during this time.

Estimated Total Violations Possible Over Study Period By ACGEM Violation Type

1. 80 Hour violation = 36 residents * 1 violation/month * 6 months (80-hour violation is average over 4-week period). (estimated total violations = 216)

2. Short Break violation = 36 residents * (30 days – 4 days off/month) * 6 months (estimated total violations = 5,616)

3. Day off violation = 36 residents * 4 weeks * 6 months (estimated total violations = 864)

4. 28 Hour violation = 36 residents * (30 days – 4 days off/month) * 6 months (estimated total violations = 5,616)