A Randomized Trial of Nighttime Physician Staffing in an Intensive Care Unit

Abstract

Background

Increasing numbers of intensive care units (ICUs) are adopting the practice of nighttime intensivist staffing despite the lack of experimental evidence of its effectiveness.

Methods

We conducted a 1-year randomized trial in an academic medical ICU of the effects of nighttime staffing with in-hospital intensivists (intervention) as compared with nighttime coverage by daytime intensivists who were available for consultation by telephone (control). We randomly assigned blocks of 7 consecutive nights to the intervention or the control strategy. The primary outcome was patients' length of stay in the ICU. Secondary outcomes were patients' length of stay in the hospital, ICU and in-hospital mortality, discharge disposition, and rates of readmission to the ICU. For length-of-stay outcomes, we performed time-to-event analyses, with data censored at the time of a patient's death or transfer to another ICU.

Results

A total of 1598 patients were included in the analyses. The median Acute Physiology and Chronic Health Evaluation (APACHE) III score (in which scores range from 0 to 299, with higher scores indicating more severe illness) was 67 (interquartile range, 47 to 91), the median length of stay in the ICU was 52.7 hours (interquartile range, 29.0 to 113.4), and mortality in the ICU was 18%. Patients who were admitted on intervention days were exposed to nighttime intensivists on more nights than were patients admitted on control days (median, 100% of nights [interquartile range, 67 to 100] vs. median, 0% [interquartile range, 0 to 33]; P<0.001). Nonetheless, intensivist staffing on the night of admission did not have a significant effect on the length of stay in the ICU (rate ratio for the time to ICU discharge, 0.98; 95% confidence interval [CI], 0.88 to 1.09; P = 0.72), ICU mortality (relative risk, 1.07; 95% CI, 0.90 to 1.28), or any other end point. Analyses restricted to patients who were admitted at night showed similar results, as did sensitivity analyses that used different definitions of exposure and outcome.

Conclusions

In an academic medical ICU in the United States, nighttime in-hospital intensivist staffing did not improve patient outcomes. (Funded by University of Pennsylvania Health System and others; ClinicalTrials.gov number, NCT01434823.)

Most studies suggest that intensivist physicians improve patient outcomes in intensive care units (ICUs).^1-3 It is thus tempting to conclude that a “dose– response effect” might exist, such that greater exposure to intensivists would be associated with even better outcomes.⁴ Indeed, some authors argue that 24-hour presence of seasoned intensivists at the bedside of patients would improve diagnostic and therapeutic efficiency, particularly for high-risk patients.^5-7 As a result, many ICUs, including one third of academic ICUs in the United States⁸ and nearly three quarters of ICUs in Europe,^9,10 use in-hospital intensivist staffing at night.

Before-and-after studies of nighttime intensivist staffing in the United States, United Kingdom, and Canada have yielded mixed results.^11-13 A recent multicenter, retrospective cohort study showed that among 22 U.S. ICUs with “low-intensity” daytime physician staffing (i.e., patients were not routinely cared for by intensivists during the day), ICUs that employed in-hospital intensivists at night had lower risk-adjusted mortality than did ICUs without nighttime intensivists.¹⁴ Among 27 ICUs that used recommended “high-intensity” daytime physician staffing (i.e., mandatory involvement of intensivists as primary physicians or consultants),¹⁵ no such benefits were shown.¹⁴ Given the limitations of observational studies, concerns about the costs of nighttime intensivist staffing,^16,17 and the uncertain effect of nighttime intensivist staffing on the education and training of residents,¹⁸ we conducted a randomized clinical trial of nighttime intensivist staffing in a U.S. academic medical ICU that had high-intensity daytime staffing and continuous coverage by medical residents.

Methods

Study Design and Oversight

We conducted the Study to Understand Nighttime Staffing Effectiveness in a Tertiary Care ICU (SUNSET-ICU) in the medical ICU (MICU) of the Hospital of the University of Pennsylvania. The trial compared nighttime (7 p.m. to 7 a.m.) staffing with in-hospital intensivists plus the usual complement of medical residents (intervention) with nighttime staffing with in-hospital residents alone (control). Nighttime intensivists were attending physicians who were board-certified or board-eligible in critical care medicine. At night during the control periods, daytime intensivists and fellows were available for consultation by telephone. The study was approved by the institutional review board at the University of Pennsylvania, which granted a waiver of the requirement for informed consent. The protocol of the study is available with the full text of this article at NEJM.org. The first author vouches for the accuracy and completeness of the data and for the fidelity of the study to the protocol.

Study Setting and Population

The MICU of the Hospital of the University of Pennsylvania is a 24-bed ICU and regional referral center for acutely ill patients. There are additional beds in another ICU that are occasionally occupied by MICU patients. The MICU is a “closed” unit; responsibility for the care of all admitted patients is transferred to one of two teams, each of which comprises one intensivist, one critical care fellow, six medical residents, and one advanced practitioner, all of whom are typically present from 7 a.m. through at least 6 p.m. Daytime intensivists rotated in 14-day blocks, and three medical residents staffed the ICU nightly. Residents were expected to review all new admissions and critical events with a fellow, an intensivist, or both, in person or by telephone within 1 hour. The default ratio of patients to nurses in this unit is 2:1, with a 1:1 ratio for a median of 2 patients daily (typically patients who require nursing-intensive clinical protocols).

We enrolled all patients who were admitted to the MICU service over the course of 1 year, from September 12, 2011, through September 11, 2012, and conducted in-hospital follow-up for an additional 90 days. For patients readmitted to the MICU within the same hospitalization, we analyzed only the first admission. We excluded patients who were admitted during the winter holiday block (December 17, 2011 through January 2, 2012), because we did not perform randomization of nighttime staffing during that period. We also excluded patients for whom we did not have Acute Physiology and Chronic Health Evaluation (APACHE) III scores (which we used for severity adjustment¹⁹) owing either to an ICU length of stay shorter than 4 hours (making them ineligible for APACHE calculation) or to missing data and patients who had a brief ICU admission that did not include nighttime hours (Fig. 1).

Figure 1. Eligibility of the Patients and Their Exposure to Intervention or Control Model.

APACHE denotes Acute Physiology and Chronic Health Evaluation, and ICU intensive care unit.

Randomization and Interventions

A day was defined throughout the study as 7:00 a.m. through 6:59 a.m. the following calendar day. We randomly assigned 1-week blocks (7 consecutive days) to the control or intervention nighttime staffing model. The allocation of weeks was determined by the generation of an electronic number, with an assignment probability of 50% to each group. We stratified randomization within 2-week blocks that were determined by the daytime intensivists' schedules to mitigate potential confounding by daytime intensivists. We randomly assigned staffing by week rather than by night to enrich the analyses with patients who were exposed predominantly to the intervention or the control staffing model throughout their stay in the ICU, so that results would better reflect the effects of the intervention in practice.

Daytime staffing was constant. On control nights, the two daytime intensivists and critical care fellows maintained primary responsibility for all patients and were available by telephone to in-hospital residents and nurses. On intervention nights, a single intensivist was physically present in the MICU from 7 p.m. to 7 a.m. and assumed primary responsibility for all patients. The intensivist was expected to perform timely in-person evaluations and write notes for all newly admitted patients and other patients whose condition deteriorated. Nighttime intensivists, like residents, had a call room adjacent to the MICU for use when they were not actively engaged in patient care. Nighttime intensivists were drawn voluntarily from the pool of daytime intensivists, excluding those on service, and each nighttime intensivist typically covered no more than 1 night per week.

Study Variables

The primary exposure variable was nighttime staffing on the first day of a patient's admission. The primary outcome was the patient's length of stay in the ICU, defined as the time from ICU admission to ICU discharge. Secondary outcomes were the patient's length of stay in the hospital, defined as the time from ICU admission to hospital discharge; mortality in the ICU; in-hospital mortality; the probability of a patient being discharged from the hospital to home; and readmission of a patient to the ICU within 48 hours after discharge from the unit to a hospital ward.^9,20 Patients who were transferred to an inpatient hospice facility were categorized in primary analyses as having died at the time of discharge from the ICU. Patients who were transferred to home hospice were categorized in primary analyses as alive at the time of discharge, because these patients typically live longer, and dying at home is commonly viewed as a favorable outcome.²¹

Complementary Data

Daytime intensivists and fellows reported their nighttime workload in daily diaries. Nighttime intensivists completed paper-based questionnaires at the end of each shift that assessed their activities. Residents who were assigned to the MICU during the first 6 months of the study provided their perceptions about nighttime intensivist staffing by completing a Web-based survey (see the Methods section in the Supplementary Appendix, available at NEJM.org).

Statistical Analysis

We used Cox multivariable models to assess the effects of the intervention on the length of stay in the ICU, censoring a patient's data at the time of death, at the time of transfer to another ICU or acute care hospital, or after 90 days. We prespecified that all analyses would be adjusted for APACHE III score and the source of ICU admission (e.g., outside hospital, in-hospital floor, or emergency department). We used similar analyses to examine the length of stay in the hospital. We assessed the proportional-hazards assumption by testing whether the Schoenfeld residuals were associated with time.²²

We also assessed relationships between the primary exposure variable and the four dichotomous secondary outcomes, using generalized linear regression with a log link, normal distribution, and robust variance estimator to estimate the relative risks.²³ For analyses of readmission to the ICU, we defined the exposure as the staffing on the night of or immediately preceding the patient's discharge from the ICU; this analysis was restricted to patients who were discharged to a general ward. All analyses were performed according to the intention-to-treat principle, such that if a night was assigned to the intervention but the assigned intensivist was unavailable (e.g., owing to illness), the night was still analyzed as an intervention night.

We conducted several prespecified restricted and sensitivity analyses. First, we repeated our main analyses among patients who were admitted at night, since we reasoned that such patients would be most susceptible to the effects of nighttime staffing. For these restricted analyses, we included patients who were admitted during the period from 5:00 p.m. through 4:59 a.m., because these patients were routinely evaluated by intensivists on intervention nights. Second, the availability of beds on the wards could affect the length of stay in the ICU; therefore, for patients who were discharged to a general ward, we recalculated the length of stay in the ICU as the time from ICU admission until the patients were deemed ready for ICU discharge, as determined by an electronic request for a bed in the general ward. Third, we recoded the discharge disposition of all patients who were discharged to home-based hospice care as deaths, a practice consistent with that in previous studies.¹⁴

Fourth, we redefined the exposure variable as a time-varying covariate representing the cumulative proportion of nights exposed to the intervention.^24,25 On each day of a patient's stay in the ICU, the cumulative exposure was calculated as the number of intervention nights divided by the total number of nights up to that day. Fifth, we repeated the analyses including only the patients whose entire length of stay in the ICU involved either 100% or 0% exposure to the intervention, thus maximizing the contrast between the groups. Finally, to account for the interrelatedness of length of stay in the ICU and death in the ICU, we evaluated a composite outcome in which ICU deaths were ranked as the worst possible outcome on a length-of-stay scale.²⁶

We also tested whether the effect of the intervention was modified in two prespecified subgroup analyses. First, we examined whether the effects of nighttime intensivist staffing differed according to the patients' severity of illness by testing the interaction between the primary exposure variable and patients' APACHE III scores. Second, we tested whether the effects of nighttime intensivist staffing differed when the ICU was staffed with less experienced residents (i.e., the “July effect”²⁷), by comparing the period of “relatively experienced residents” (September 12, 2011, through June 30, 2012) with the period of “relatively inexperienced residents” (July 1, 2012, through September 11, 2012).

We estimated that with a sample of 1408 patients (4 eligible patients per day for 1 year, excluding the winter holiday block), the primary analysis would have 86% power to detect a rate ratio of 1.2 for the outcome of time to ICU discharge, at a two-sided significance level of 0.05. All the analyses were performed with the use of Stata software, version 12.0 (StataCorp) and SAS software, version 9.3 (SAS Institute).

Results

The study period included 352 nights, 175 of which (50%) were randomly assigned to the intervention; nighttime intensivists staffed 166 (95%) of the intervention nights. A total of 1598 patients were included in the analyses (Fig. 1), of whom 970 (61%) were admitted at night (Table 1, and Table S1 in the Supplementary Appendix). The median APACHE III score (with scores ranging from 0 to 299 and higher scores indicating more severe illness) was 67 (interquartile range, 47 to 91), and the median length of stay in the ICU was 52.7 hours (interquartile range, 29.0 to 113.4). A total of 381 patients (24%) died in the hospital, including 293 (18%) who died in the ICU.

Table 1.

Characteristics of the Study Population.^*

Characteristic	Intervention (N = 820)	Control (N = 778)
Male sex — no. (%)	454 (55)	422 (54)
Age — yr
Median	60	60
Interquartile range	48–69	49–70
APACHE III score†
Median	67	67
Interquartile range	47–93	47–91
Interventions during stay in the ICU — no. (%)
Mechanical ventilation	338 (41)	300 (39)
Vasopressor administration	220 (27)	191 (25)
Hemodialysis	123 (15)	109 (14)
Source of admission — no. (%)
Emergency department or direct admission	383 (47)	367 (47)
Intrahospital transfer from general floor	295 (36)	306 (39)
Intrahospital transfer from another ICU	53 (6)	36 (5)
Interhospital transfer	89 (11)	69 (9)
Admission at night, 5:00 p.m.–4:59 a.m. — no. (%)	500 (61)	470 (60)

^*There were no significant differences (at P<0.05) between the intervention and control groups in any of the characteristics listed here.

^†APACHE III scores range from 0 to 299, with higher scores indicating more severe illness.

Nighttime intensivists were generally younger than the daytime intensivists (Table S2 in the Supplementary Appendix), although most (82%) also worked as daytime intensivists during the study period. Nighttime intensivists completed post-shift questionnaires on 116 intervention nights (66%) and reported evaluating a median of 4 (interquartile range, 3 to 5) new patients and 2 (interquartile range, 1 to 3) previously admitted patients per night (Table S3 in the Supplementary Appendix). During the control nights, the at-home intensivists received a median of 2 (interquartile range, 1 to 3) calls each night, and the critical care fellows received a median of 2 (interquartile range, 0 to 3) calls each night (Table S4 in the Supplementary Appendix).

Patients who were admitted on intervention days had greater cumulative exposure to nighttime intensivists than did patients who were admitted on control days (median, 100% of nights [interquartile range, 67 to 100] vs. median, 0% [interquartile range, 0 to 33]; P<0.001). Staffing with nighttime intensivists did not have a significant effect on the length of stay in the ICU (rate ratio for ICU discharge, 0.98; 95% confidence interval [CI], 0.88 to 1.09; P = 0.72) (Fig. 2A). In this study, the rate ratio refers to the instantaneous rate of discharge from the ICU in the intervention group divided by the instantaneous rate of discharge from the ICU in the control group, such that a rate ratio greater than 1 would indicate that the intervention shortened the time to ICU discharge. The findings were similar in analyses that were restricted to patients admitted at night (hazard ratio, 0.98; 95% CI, 0.84 to 1.13; P = 0.74) (Fig. 2B), and in several sensitivity analyses (Table 2). The results were also similar in the rank-based analysis of length of stay in the ICU, in which deaths were coded as the longest possible length of stay (P = 0.51).

Figure 2. Kaplan–Meier Curves for Time to Discharge from the ICU.

The time to discharge from the ICU is shown for all the patients (Panel A) and for only patients who were admitted to the ICU at night (Panel B), according to whether they were admitted to the ICU on a night with in-house intensivist staffing (intervention) or on a night with traditional nighttime coverage by residents (with daytime intensivists available for consultation by telephone) (control).

Table 2.

Primary, Restricted, and Sensitivity Analyses of the Effect of the Intervention.^*

Analysis and Exposure Variable	Population†	Outcome Measured	Rate Ratio (95% CI)‡	P Value
Primary
Intensivist staffing on first night	All patients	Time to ICU discharge	0.98 (0.88–1.09)	0.72
Restricted
Intensivist staffing on first night	Patients admitted at night	Time to ICU discharge	0.98 (0.84–1.13)	0.74
Sensitivity
Intensivist staffing on first night	All patients	Time to request for bed in general ward	0.97 (0.87–1.08)	0.56
Intensivist staffing on first night	All patients	Time to ICU discharge, with patients transferred to hospice facility categorized as having died	0.97 (0.87–1.09)	0.61
Cumulative proportion of nights exposed to the intervention§	All patients	Time to ICU discharge	0.92 (0.81–1.04)	0.18
100% vs. 0% intensivist staffing	Patients with all or no exposure to intervention	Time to ICU discharge	0.98 (0.86–1.12)	0.81

^*All the analyses were adjusted for APACHE III score and source of ICU admission (e.g., outside hospital, in-hospital floor, or emergency department).

^†The total population (all patients) comprised 1598 patients. A total of 970 patients were admitted at night from (5:00 p.m. through 4:59 a.m.), and 1072 patients had either all or no exposure to the intervention.

^‡In this study, the rate ratio refers to the instantaneous rate of discharge from the ICU in the intervention group divided by the instantaneous rate of discharge from the ICU in the control group, such that a rate ratio greater than 1 would indicate that the intervention shortened the time to ICU discharge.

^§On each day of a patient's stay in the ICU, the cumulative exposure was calculated as the number of intervention nights divided by the total number of nights up to that day.

Nighttime intensivist staffing also had no significant effect on the length of stay in the hospital (median, 174 hours [interquartile range, 91 to 361] in the intervention group vs. 166 hours [interquartile range, 84 to 328] in the control group; rate ratio, 0.91; 95% CI, 0.82 to 1.02; P = 0.12) or on ICU mortality, hospital mortality, readmission to the ICU among ICU survivors, or discharge to home (Table 3). Analyses that were restricted to patients admitted at night also showed no significant effects of nighttime intensivist staffing.

Table 3.

Secondary Outcomes.^*

Outcome	Intervention	Control	Relative Risk with Intervention (95% CI)†	P Value
	no./total no. (%)
ICU mortality
All patients	154/820 (19)	139/778 (18)	1.07 (0.90–1.28)	0.45
Patients admitted at night	90/500 (18)	81/470 (17)	1.02 (0.82–1.27)	0.88
Hospital mortality
All patients	203/820 (25)	177/778 (23)	1.08 (0.93–1.25)	0.33
Patients admitted at night	122/500 (24)	103/470 (22)	1.06 (0.88–1.28)	0.51
Discharge home from the hospital
All patients	317/820 (39)	314/778 (40)	0.95 (0.85–1.07)	0.41
Patients admitted at night	190/500 (38)	197/470 (42)	0.94 (0.82–1.09)	0.44
ICU readmission within 48 hr‡
All patients	32/602 (5)	21/630 (3)	1.56 (0.84–2.89)	0.16
Patients discharged at night	20/324 (6)	14/356 (4)	1.36 (0.53–3.47)	0.52

^*For all secondary outcomes except for readmissions, the exposure to intervention or control staffing was considered to be the exposure to the staffing model that was in place on the day of admission. For readmissions, the exposure to intervention or control was considered to be the exposure to the staffing model that was in place at time of discharge or, for daytime discharges, during the preceding night. Night admissions were considered to be admissions between 5:00 p.m. and 4:59 a.m. The reference for all the analyses is the control group.

^†A relative risk of more than 1 indicates that a greater proportion of patients in the intervention group had the outcome, whereas a relative risk of less than 1 indicates that a greater proportion of patients in the control group had the outcome.

^‡The analyses of ICU readmissions were restricted to patients who were discharged to floor locations within the hospital.

The patients' APACHE III score did not modify the effect of the intervention on the length of stay in the ICU (P = 0.28 for interaction) (Table S5 in the Supplementary Appendix). The effects of the intervention on the length of stay in the ICU were also similar during periods in which residents were more experienced and those in which residents were less experienced (Table S6 in the Supplementary Appendix). There was significant heterogeneity in the effect of the intervention on readmission to the ICU during the two periods (P = 0.03 for interaction). However, the intervention was not associated with significantly fewer readmissions during the inexperienced-resident period (relative risk for readmission, 0.58; 95% CI, 0.10 to 3.39), and the heterogeneity was due, in part, to a nonsignificantly higher readmission rate with the intervention during the experienced-resident period (relative risk for readmission, 1.94; 95% CI, 0.87 to 4.30).

The Web-based surveys were completed by 41 of 91 eligible residents (45%). A majority of residents reported that the presence of nighttime intensivists improved the quality of care as perceived by the resident, provided support to residents, permitted appropriate resident autonomy, and improved the educational experience (Table S7 in the Supplementary Appendix).

Discussion

In this single-center randomized trial of in-hospital nighttime intensivist staffing in an academic medical center in the United States, we found no evidence that this staffing model, as compared with nighttime telephone availability of the daytime intensivist, had a significant effect on length of stay in the ICU or hospital, ICU or in-hospital mortality, readmission to the ICU, or the probability of discharge to home. We also observed no significant benefits of the intervention in subgroups of patients for whom we had hypothesized the greatest effects: patients admitted at night, those with the most severe illness at the time of ICU admission, and those admitted during the period when the residents were least experienced. These findings are consistent with a multicenter observational study that suggested that in hospitals with high-intensity daytime intensivist staffing, the presence of nighttime intensivists did not reduce mortality.¹⁴ The current trial extends this work by removing the potential for ICU-level and patient-level confounding and by documenting the lack of significant effects on a broad range of outcomes.

There are several possible explanations for the lack of significant benefit of nighttime intensivists in this study. First, there may be limited room for improvement in ICUs that have daytime intensivist staffing, particularly if the benefits of daytime intensivist staffing derive from better ICU-wide processes of care.^9,28 Second, nighttime intensivist staffing may be associated with discontinuity of care for some patients, offsetting benefits for other patients. Third, in the staffing model and setting that we studied, bedside intensivists may not add to the quality of care provided by well-trained resident physicians who have telephone access to intensivists. Finally, nighttime intensivists may truly have an effect on mortality in a small number of patients, but such patients may be so few in number that detecting these benefits would require a much larger study. Future research that investigates these and other potential explanations could inform broader debates about the best ways to use a limited intensivist workforce.^29,30

We also found that most residents believed that nighttime intensivists improved their educational experience and provided desirable support for decision making. These findings are tempered by the positive framing of the questions in our survey and the modest response rate. Nonetheless, academic centers may wish to consider residents' perspectives in choosing to adopt or retain this staffing model.

A strength of this randomized trial is that it took place in an ICU in which 61% of admissions occurred at night. If nighttime intensivists were effective, it is likely that they would be particularly effective in an ICU with such a large nighttime workload. In addition, by randomly assigning weeks rather than individual nights, we ensured that our contrast would meaningfully represent the presence or absence of year-round nighttime intensivist staffing.

An important limitation of this study is that it was performed in a single, academic medical ICU in the United States that had round-the-clock coverage by reasonably well-trained residents. Our results may be generalizable to U.S. academic ICUs with high-intensity daytime staffing, which have been among the early adopters of nighttime intensivist staffing in the United States. However, our study does not address whether nighttime intensivist staffing may provide benefits in community ICUs, ICUs without high-intensity daytime staffing, ICUs with fewer or less well-trained residents, or ICUs in other countries.

Second, our nighttime workforce may differ with respect to age, frequency of shifts, or other characteristics from workforces that are employed or considered elsewhere. It is uncertain whether different nighttime staffing models would affect patient outcomes.

Third, although we evaluated several outcomes, the presence of nighttime intensivists may affect other outcomes such as physician burn-out, staff satisfaction, patient and family experiences, objectively measured educational outcomes, and the incidence of malpractice claims. In addition, outcomes external to the ICU were not measured, such as the possible role of nighttime intensivists in helping hospitals meet quality benchmarks.¹⁵

In summary, this randomized trial in an ICU with high-intensity staffing during the day failed to identify benefits to adding intensivists at night. The compelling face validity of nighttime intensivist staffing has probably spurred the widespread adoption of this staffing model.^8,10 However, nighttime intensivist staffing may also be one of several expensive medical practices that have been adopted without a supportive evidence base.³¹ Because the adoption of nighttime intensivist staffing by hospitals with plentiful resources may siphon intensivists away from hospitals with fewer resources,^17,18 rigorous evaluation of the model is needed in settings that were not evaluated in this study.