Automation Improves Schedule Quality and Increases Scheduling Efficiency for Residents

Abstract

Background

Medical resident scheduling is difficult due to multiple rules, competing educational goals, and ever-evolving graduate medical education requirements. Despite this, schedules are typically created manually, consuming hours of work, producing schedules of varying quality, and yielding negative consequences for resident morale and learning.

Objective

To determine whether computerized decision support can improve the construction of residency schedules, saving time and improving schedule quality.

Methods

The Optimized Residency Scheduling Assistant was designed by a team from the University of Michigan Department of Industrial and Operations Engineering. It was implemented in the C.S. Mott Children's Hospital Pediatric Emergency Department in the 2012–2013 academic year. The 4 metrics of schedule quality that were compared between the 2010–2011 and 2012–2013 academic years were the incidence of challenging shift transitions, the incidence of shifts following continuity clinics, the total shift inequity, and the night shift inequity.

Results

All scheduling rules were successfully incorporated. Average schedule creation time fell from 22 to 28 hours to 4 to 6 hours per month, and 3 of 4 metrics of schedule quality significantly improved. For the implementation year, the incidence of challenging shift transitions decreased from 83 to 14 (P < .01); the incidence of postclinic shifts decreased from 72 to 32 (P < .01); and the SD of night shifts dropped by 55.6% (P < .01).

Conclusions

This automated shift scheduling system improves the current manual scheduling process, reducing time spent and improving schedule quality. Embracing such automated tools can benefit residency programs with shift-based scheduling needs.

What was known and gap

Medical resident scheduling is complex, due to the need to balance educational goals, workforce considerations, and accreditation requirements.

What is new

A computerized scheduling mechanism was designed and tested to set up schedules for residents in a pediatric emergency department.

Limitations

Single site study limits generalizability; outcomes are limited to scheduling performance, not to the impact on patient care, learning, or resident well-being.

Bottom line

The automated shift scheduling system improved scheduling efficiency and schedule quality.

Introduction

Medical residents have unique and complex scheduling needs that relate to their training. Residents must abide by the Accreditation Council for Graduate Medical Education (ACGME) duty hour restrictions, which include guidelines regarding days off, time between shifts, and more. There also are hospital- and program-specific rules governing schedules and residents' personal preferences for days off and vacation. Designing a schedule by hand that addresses all of these factors is time-consuming and error-prone. Without assistance, it is difficult to achieve even a feasible schedule, here defined as one that satisfies all strict requirements.

Poor-quality resident schedules can yield negative consequences for both patients and staff. Uneven shift distribution results in poor morale, raising the risk for resident burnout.¹ Additionally, poor scheduling forces residents into irregular, suboptimal sleep patterns, which contribute to resident fatigue, a profound problem in residency programs. Fluctuations in scheduled sleep periods force residents to work against their circadian rhythms, lowering the magnitude of physiological factors related to wakefulness.² Fatigue also depresses fine-motor skills and cognition,^3–6 thus endangering patient care. Furthermore, fatigued residents have an increased risk of negative health events, including motor vehicle accidents.^7–9 By increasing resident fatigue, poor scheduling places the hospital at risk for both diminished patient care and adverse health events for residents.

To allow for smarter scheduling, residency programs may benefit from computerized assistance, using a systems-based approach to generate high-quality schedules. This is particularly so in environments staffed by multiple residency programs. For example, the pediatric emergency department (ED) is staffed by residents from pediatrics, family medicine, combined medicine and pediatrics, and emergency medicine. Each group has unique educational goals and out-of-hospital program requirements that must be incorporated. To address these challenges, the pediatric ED of the University of Michigan C.S. Mott Children's Hospital collaborated with the Center for Healthcare Engineering and Patient Safety at the University of Michigan College of Engineering. The outcome of this collaboration was the development of a computerized scheduling tool: the Optimized Residency Scheduling Assistant (ORSA). In simple spreadsheets, schedulers provide data to ORSA, such as the month's list of residents, the residents' program and year, and day off requests. Then, ORSA analyzes all the data and returns feasible, optimized schedules.

The ability of existing scheduling tools to incorporate nuanced rules is typically limited, as is the ability to modulate multiple conflicting objectives. Other more-advanced decision support tools typically focus on nursing, a field with much different scheduling parameters than residency, and typically optimize a single schedule parameter, such as shift preference or cost.^10–13 However, multiple important criteria often exist in resident scheduling, and a multicriteria function is more appropriate. Additionally, relationships between criteria are difficult for schedulers to quantify and often change from month to month, and an unchanging mathematically “optimal” solution may not exist.¹⁴ Therefore, ORSA was built to allow the user to adjust several scheduling metrics according to current needs.

Methods

The scheduling tool consists of an integer programming model. Decision variables reflected whether to assign a particular resident to a particular shift on a particular day. One such variable was defined for each resident/shift/day combination. Mathematical constraints then enforced “hard” scheduling rules; for example, a constraint enforced that residents must have at least 10 hours off between consecutive shifts. Further “soft” constraints optimized other scheduling parameters: for example, a constraint minimized the disparity between different residents' number of night shifts per month. The system was implemented and solved using C++ with ILOG CPLEX API version 12.1 (IBM Corp, Armonk, NY) on a computer with an Intel Xeon 3.20 GHz processor and 8 GB of memory.

To initiate the scheduling, ORSA received a list of the month's residents, each resident's number of weeks in the ED, outside educational requirements (table 1), and requested days off. The program then automatically generated a feasible schedule. The scheduler reviewed the schedule to identify undesirable characteristics, then specified additional requests for ORSA to incorporate. It either identified a new schedule that met the additional requests or guaranteed that no such schedule existed. Approximately 1 to 2 hours per month were spent iteratively adding requests and generating new schedules.

TABLE 1.

Description of Program-Specific Outside Educational Requirements

The C.S. Mott Children's Hospital pediatric ED is a level 1 pediatric trauma center located in Ann Arbor, Michigan. It served nearly 19 000 patients in the 2010–2011 academic year (AY), and more than 23 000 patients in AY 2012–2013. Resident schedules from AY 2010–2011 and 2012–2013 were chosen for review. The AY 2010–2011 was the most recent year that the schedule was constructed completely by hand, and AY 2012–2013 was the first complete year scheduled using ORSA. The intervening 2011–2012 year was a year of transition and was omitted.

During both study years, the ED had 6 required resident shifts, each filled by 1 resident. There were 2 day shifts (7 am–4 pm, 9 am–6 pm), 2 evening shifts (4 pm–1 am, 5 pm–2 am), and 2 night shifts (8 pm–5 am, 11 pm–8 am). Two additional optional shifts (10 am–7 pm and 12 pm–9 pm) were filled as frequently as possible.

Residency programs that staffed the ED included pediatrics, combined medicine and pediatrics, emergency medicine, and family medicine. Educational requirements outside of the ED differed for each group (table 1). To be feasible, schedules had to allow residents to attend their outside requirements and still abide by ACGME rules. These rules dictated that residents have a minimum of four 24-hour periods off per month, a maximum of 80 hours worked per week, a minimum of 10 hours between separate shifts or responsibilities, and a maximum of 6 consecutive night shifts.

To assess schedule quality, we evaluated 4 measures: total shift disparity, night shift disparity, occurrence of shifts immediately following outside clinic responsibilities (postclinic shifts), and occurrence of challenging shift transitions (suboptimal sleep patterns [SSPs]). Total shift disparity and night shift disparity refer to variance in shift number among residents in any given month. Postclinic shifts were chosen as a negative quality metric because of the difficulty of preceding an ED shift with clinic duty or other outside requirement. The SSPs were defined as consecutive shift assignments that yield a difficult sleep schedule for residents and were determined by informally surveying senior residents on challenging shift transitions (table 2).

TABLE 2.

Examples of Shift Combinations Yielding Suboptimal Sleep Patterns and Favorable Sleep Patterns

This project was declared exempt from Institutional Review Board oversight.

Data collected included the monthly resident complement, total shifts per resident, night shifts per resident, postclinic shifts, and SSPs. These variables were calculated for each month in the study years, then averaged within each year. Total shift distribution variance per month and night shift distribution variance per month were also calculated and averaged within each year. Student t tests were used to compare the data between the study years. Pediatrics chief residents were informally surveyed throughout the project on the amount of time necessary to create a schedule, both manually and utilizing ORSA. Statistical analysis was completed using Minitab 16 (Minitab Inc, State College, PA).

Results

In AY 2010–2011, each month's schedule required 12 to 16 hours to build manually, plus 10 to 12 additional hours of later corrections. In AY 2012–2013, the schedule took 2 to 3 hours of file-building, plus 2 to 3 hours of modification with the engineering team. There was little-to-no error correction time needed. In sum, the total time to build a monthly schedule was between 22 and 28 hours by hand and 4 to 6 hours using ORSA. Using ORSA resulted in a 79% to 82% time reduction per month.

The mean number of residents working in the ED per month was 15.7 (SD = 2.06) in AY 2010–2011 and 11.2 (SD = 1.18) in AY 2012–2013 (table 3). Each resident worked a mean of 15.8 (SD = 1.68) total shifts per month in AY 2010–2011 and 16.4 (SD = 1.62) total shifts per month in AY 2012–2013. The mean number of monthly night shifts per resident was 4.7 (SD = 0.98) in AY 2010–2011 and 5.0 (SD = 1.50) in AY 2012–2013. As some residents took a 2-week vacation block during their ED rotation, for each statistic, the number of residents was normalized per month, such that a resident who was working in the ED for 15 days in a 30-day month was counted as 0.5 residents.

TABLE 3.

Average Number of Residents per Month, Total Shifts per Resident, and Night Shifts per Resident (Academic Years 2010–2011 and 2012–2013)

The SSPs decreased by 85.7% from AY 2010–2011 to AY 2012–2013 (mean change, −0.54 SSPs/resident per month; P < .001; 95% CI −0.29 to −0.79; table 4). The number of postclinic shifts decreased by 66.7% (mean change, −0.36 postclinic shifts/resident per month; P = .002; 95% CI −0.16 to −0.57).

TABLE 4.

Quality Metrics Normalized for Monthly Resident Compliment

While there was no significant difference in total shift disparity between years (percentage change, −25%; P = .49; 95% CI −50%–0%), there was statistically significant reduction in night shift disparity, which decreased by 55.6% (P < .001; 95% CI −33.3% to −77.8%).

Discussion

The advent of ORSA significantly improved several measures of schedule quality. While total shifts and night shifts per resident remained constant between study years, SSPs, postclinic shifts, and night shift disparity all significantly decreased. The change in total shift disparity was not significant. This is not surprising, as the easiest quality measure to check by hand is a count of each resident's monthly shifts, and this is where schedulers often invest time in improving manual schedules.

These improvements in schedule quality came while decreasing the time required for production, as ORSA reduced schedule creation time by more than 20 hours per month. This reflects a benefit of computer-assisted scheduling; schedules are highly accurate and require little to no later correction.

The ORSA program is a unique addition to the world of medical shift scheduling. It is capable of creating schedules that are adaptable to each month's different resident complement and optimization priorities. Currently, this flexibility is relatively unique among scheduling tools. Additionally, many current tools are focused on nursing, a field with very different scheduling constraints than residency programs. Most nursing-focused tools optimize 1 specific schedule parameter, such as shift preference or cost.^11–13 An ideal scheduling tool incorporates ad hoc adjustments, prioritizing different metrics according to that month's needs. By not optimizing a single predetermined trait, ORSA allows a scheduler to choose what parameters to optimize each month. Furthermore, the commercial packages specifically aimed at physician scheduling typically aid the user in creating a schedule manually or with limited basic decision support. Nuanced optimization (for example, the avoidance of SSPs) is difficult to incorporate, as is the ability to balance multiple conflicting objective functions.

Improved schedule quality addresses SSPs that contribute to resident fatigue. Fatigue is a common problem of residency programs and can place both patients and staff at increased risk for negative health events. By addressing SSPs that worsen resident fatigue, better scheduling has the potential to decrease fatigue and its negative associations.

Limitations of our study include its single program nature, and the fact that our study is limited to schedule quality metrics. We cannot definitively state that automated scheduling improves patient care or resident morale. Another limitation is our statistical power, as our sample size was limited to 2 academic years. Finally, SSPs were defined based on experience and resident feedback; there is no evidence specifically associating these shift transitions with poor patient care or educational outcomes.

Future research goals stem from the recognition that each residency program has unique requirements in scheduling. An ideal infrastructure supports this detailed customization.

Conclusion

Automated shift-scheduling tools, like ORSA, improve the current manual scheduling process. Automated schedules are made faster and are of higher quality than schedules made by hand, and they are able to efficiently incorporate changing scheduler preferences from month to month. Improving schedule quality has the potential to improve patient safety, resident education, and morale.