Association of Shift-Level Organizational Factors with Nosocomial Infection in the Neonatal Intensive Care Unit

Abstract

Objective

To evaluate the association between shift-level organizational data (unit occupancy, nursing overtime ratios [OTRs], and nursing provision ratios [NPRs]) with nosocomial infection (NI) among infants born very preterm in the neonatal intensive care unit (NICU).

Study design

This was a multicenter, retrospective cohort study, including 1921 infants 23^0/7-32^6/7 weeks of gestation admitted to 3 tertiary-level NICUs in Quebec between 2014 and 2018. Patient characteristics and outcomes (NIs) were obtained from the Canadian Neonatal Network database and linked to administrative data. For each shift, unit occupancy (occupied/total beds), OTR (nursing overtime hours/total nursing hours), and NPR (number of actual/number of recommended nurses) were calculated. Mixed-effect logistic regression models were used to calculate aOR for the association of organizational factors (mean over 3 days) with the risk of NI on the following day for each infant.

Results

Rate of NI was 11.5% (220/1921). Overall, median occupancy was 88.7% [IQR 81.0-94.6], OTR 4.4% [IQR 1.5-7.6], and NPR 101.1% [IQR 85.5-125.1]. A greater 3-day mean OTR was associated with greater odds of NI (aOR 1.08, 95% CI 1.02-1.15), a greater 3-day mean NPR was associated lower odds of NI (aOR 0.96, 95% CI 0.95-0.98), and occupancy was not associated with NI (aOR, 0.99, 95% CI 0.96-1.02). These findings were consistent across multiple sensitivity analyses.

Conclusions

Nursing overtime and nursing provision are associated with the adjusted odds of NI among infants born very preterm in the NICU. Further interventional research is needed to infer causality.

Infants born very preterm at <33 weeks of gestational age account for more than 60% of neonatal intensive care unit (NICU) resource use¹ and are at high risk of nosocomial infection (NI).² In infants born very preterm, NI is associated with increased lengths of stay, costs, mortality, and short- and long-term morbidities.^3,4 Clinical-care bundles for NI prevention focus on hand hygiene practices, maintenance of central lines, and patient skin care.⁴ However, organizational factors may affect the ability to comply with these recommendations.5, 6, 7

Three important organizational factors include unit occupancy, nursing overtime, and nursing provision ratios (NPRs). Greater unit occupancy may contribute to overcrowding and lower staffing ratios (among all providers), leading to lower compliance in infection-prevention measures and greater risk of NI.⁸ Nursing overtime has been associated with greater fatigue, burnout, and reductions in attention, which have been associated with lower compliance with NI -prevention measures.9, 10, 11, 12 Lower nursing ratios also have been associated with several adverse patient outcomes in adult, pediatric, and neonatal settings.¹³ The NPR is the ratio of actual nurses per shift divided by the estimated required number of nurses (estimated using a nurse-to-patient ratio guideline). The use of NPRs allows health care management to account for variations in patient acuity in a unit (unlike using a ratio of total nurses per patient, which assumes each patient should receive the same nursing ratio).¹⁴ NPRs may reflect staffing adequacy and have been associated with mortality.¹⁵ Several methods have been used to model these exposures, including aggregating data by year, month, and day, and recent studies suggest that shorter observation periods may provide more precise estimates.^16,17 These organizational factors also can be difficult to evaluate because of their interdependent nature: overtime can be used to meet NPRs in periods of high occupancy.^10,18, 19, 20, 21 Consequently, there is a need to better understand the association of these 3 factors with NI in the NICU.21, 22, 23 The objective of this study was to assess the association between occupancy, nursing overtime, and NPRs with the risk of NI among infants born very preterm in the NICU using shift-level data. We hypothesized that all 3 organizational factors would be associated with NI.

Methods

Study Design, Inclusion, and Exclusion Criteria

This was a retrospective cohort study of infants born 23^0/7-32^6/7 weeks of gestational age, admitted to 3 tertiary-level NICUs in Québec (Canada) within 3 days after birth from April 1, 2014, to December 30, 2018. Infants <33 weeks were included in this study, as they are high-risk patients with high resource use and compose a large proportion of patient-days in the NICU.²⁴ Exclusion criteria were moribund or palliative care upon admission; major congenital anomalies; length of stay <3 days (typically because of mortality); and diagnosis of early-onset sepsis (positive blood and/or cerebrospinal fluid culture drawn within the first 3 days after birth). The research ethics boards of each participating site approved this project.

Setting

The 3 participating NICUs each had 20, 56, and 65 funded beds (Table I). All the NICUs have pediatric subspeciality consultant services with pediatric surgeons available on site. The median number of admissions (all gestational ages) per year was 450, 860, and 985; infants <33 weeks of gestational age accounted for 70% of patient-days in each unit. Two of the units had an open-bay design and one had single-patient rooms (the unit with 65 beds). Maximum occupancy is based on funded beds, and all units can accommodate for more than the maximum occupancy with additional beds above what is funded (due to twin/triplet rooms that are counted as a single bed but can accommodate 2-3 patients, and/or overflow rooms).

Table I.

Characteristics by site

Characteristics	Site 1	Site 2	Site 3	Overall
Site
Number of beds	65	56	20	–
Most common length of nursing shifts, h	8	8	8	–
Number of medical teams during the daytime	4	3	1	–
Single-patient rooms	Yes	No	No	–
Organizational
Unit occupancy, %	91.3 [86.7, 95.4]	90.5 [85.7, 95.8]	78.3 [66.7, 88.3]	88.7 [81.0, 94.6]
OTR, %	5.1 [3.4, 7.5]	2.0 [0.0, 5.0]	5.6 [1.8, 10.4]	4.4 [1.5, 7.6]
Shifts with overtime, No.	4584 (88.1)	2397 (46.1)	3280 (63.0)	10261 (65.7)
Number of overtime hours among shifts with overtime	14.5 [7.5, 23.0]	7.9 [4.0, 14.7]	7.3 [1.7, 11.3]	8.8 [4.7, 16.8]
NPR, %	103.7 [96.2, 121.5]	80.5 [75.0, 86.5]	128.6 [113.9, 152.3]	101.1 [85.5, 125.1]
Patient
Number of infants	920	653	348	1921
Number of infants <28 wk of gestational age	283 (30.8)	181 (27.7)	69 (19.8)	533 (27.7)

NPR, nursing provision ratio; OTR, nursing overtime ratio.

Organizational variables were calculated as the average for each day (mean of the 3 shifts from 8:00 am to 7:59 am) for the study period. Data are presented as median [IQR] or No. (%) as appropriate. Each site had 5205 shifts in the dataset, totaling 15 615 shifts.

Organization of work was similar between units, as previously described.²⁴ The majority of nurses worked 8-hour shifts. Each unit recorded patient acuity and occupancy 3 times per day (12:00 am, 8:00 am, and 4:00 pm) in a provincial administrative database. At the beginning of each shift, the assistant head nurse evaluated the required number of registered nurses based on the expected admissions, planned activities within the unit, and patient acuity was estimated using provincial guidelines.¹⁴ When nursing staffing needs increased, management initially relied on off-duty nurses and, if necessary, on a pool of floating trained neonatal nurses. If staffing needs were not met on the basis of the evaluation by the nurse in charge, overtime (voluntary and mandatory) could then be used. The participating NICUs did not rely on nurses from external agencies.

Data Collection

Patient data were obtained from the Canadian Neonatal Network database. At each site trained abstractors follow a standard protocol²⁵ to enter infant information into a data-entry program with built-in error checking that has high reliability and internal consistency.²⁶ Patient data included gestational age at birth, birth weight, sex, small for gestational age status (birth weight <10th percentile for gestational age and sex),²⁷ multiple births, mode of delivery, 5-minute Apgar score <7, Score for Neonatal Acute Physiology version II >20,²⁸ receipt of antenatal steroids (≥1 doses), outborn status (delivery in a hospital without a tertiary-level NICU), and NICU mortality. We did not have data on central line use and dates.

Data on the number of patients in each acuity category and occupancy at the beginning of each 8-hour shift were obtained from the provincial administrative database (SitNeo, ∼4% missing data, imputed using data from previous shift). Data on the number of nursing hours worked (regular and overtime hours) by bedside nurses (which includes admitting nurses and/or nurses designated for break relief but does not include nurse managers) for each 8-hour period were obtained from Logibec (LGI Healthcare Solutions), the administrative software used for scheduling and payroll in each institution (no missing data). Nursing overtime was defined as per the nursing collective agreement in the province and is similar to the rest of Canada and the US^29,30: hours worked beyond the scheduled work shifts (based on the regular length of shifts the nurse works, either 8-hour or 12-hour).³¹ This represented hours worked before or after the scheduled shift (maximum of 16 consecutive hours) or the hours worked greater than 37.5 hours per week. Distinction between mandatory and voluntary overtime was not available.

Exposures, Data Linkage, and Observation Period

All 3 organizational factors were computed for each shift. Occupancy rate was defined as the total number of infants in the NICU at the beginning of each shift divided by the total number of funded beds at the site. The overtime ratio (OTR) was the sum of all nursing overtime hours divided by the sum of total nursing hours for that shift.³⁰ The NPR was defined as the total number of nursing hours worked divided by the total number of required nursing hours based on patient acuity categories. Acuity categories were based on provincial guidelines: unstable (1.0 nurse), intensive care (0.7 nurses), intermediate care (0.3 nurses), or continuing care (0.25 nurses).¹⁴ The NPR was calculated as a percentage of recommended nurses per shift, with a value of <100% indicating being at less than the recommended number of nurses. Similarly, OTR was converted to a percentage such that larger values indicate a greater proportion of overtime hours worked by nurses. Patient data from the Canadian Neonatal Network were linked to the administrative data on the organizational factors using each infant’s admission site and admission date.

Occupancy, OTR, and NPR were calculated for each shift from admission until the 45th day of admission, NI, death, or discharge (whichever occurred first) for each infant. Among survivors without NI and still in the NICU, data were censored at the 45th day of admission to account for the greater-risk period for NI, as most events occur within this period on the basis of previous evidence on the timing of infection in the NICU.³² Infants who experienced an NI only contributed observations until their date of first NI because the time frame for recurrent infections is unclear and the data cannot distinguish between a new or recurrent infection.

Outcome

The primary outcome was NI, defined as the isolation of a pathogenic organism in blood culture or cerebrospinal fluid culture, drawn at more than 3 days after birth in a symptomatic infant (ie, apnea, bradycardia, temperature instability, or feeding difficulties). A second positive blood culture was not mandatory because of technical issues associated with drawing an additional sample among very small infants.³³ The cases included both catheter-associated and noncatheter-associated infections. Events of “culture negative sepsis” were not included. The date of infection was defined as the date when the diagnostic sample was collected. Only the first NI episode per infant was included.

Conceptual Framework for Modeling Approach

The exact sequence and timeline of events leading to NI is still not well defined.⁴ Previous studies focusing on organizational factors suggest that the 3-day period before NI may be an appropriate observation window.⁶ Consequently, the risk of infection on each day of a patient’s admission may vary on the basis of the organizational variables in the 3 days previously, which can be addressed with multiple observation points for each patient. For each day of an infant’s observation period, we calculated the mean organizational variables of the 1, 2, and 3 days before that observation day based on an 8:00 am to 7:59 am period (Figure 1).

Figure 1.

Conceptual framework for analysis using the 3-day mean. Visual representation of conceptual framework for calculating a 3-day mean from 8:00 am to 7:59 am for each observation point. Infant A represents the data used for the analysis of multiple observations throughout an infant’s admission, which generates a moving average as indicated by the arrows.

Analysis

This was a convenience sample size based on data availability. The Pearson correlation coefficient (r) was used to assess the correlation between the 3 organizational variables. Patient and unit characteristics were presented using descriptive statistics. We compared the mean organizational variables on the 1-, 2-, and 3-day intervals before NI (among infants with NI) with the same interval among infants without NI (using the 1-, 2-, and 3-day interval before discharge or end of observation). Unadjusted comparisons were conducted using Wilcoxon rank-sum tests for continuous variables and χ² tests for categorical variables.

Mixed Model Analysis for Repeated Measurements (Multiple Observations per Patient)

Several methods have been described to evaluate the association of organizational factors with NI.^6,34 To include each observation point (ie, each day the infant was in the NICU) for each infant (infant was the unit of analysis), we used a mixed-effect logistic regression model to account for repeat observations of patients (patient identification included as a random effect).³⁵ For each observation point of each infant, the exposures were the means of each organizational variable (occupancy, OTR, and NPR) over the 3 days before and the outcome was NI on the observed day for the infant (yes/no). Mixed effect models using multiple observations per patient allow us to evaluate the association between a changing exposure (3-day mean) and an event (NI) while including each observation point (each day) for the same patient.

The analysis was adjusted using 2 models. Model I included each individual organizational variable, site, and patient covariates (gestational age [continuous], small for gestational age, Score for Neonatal Acute Physiology version II score >20, sex) as fixed effects. Model II included all 3 organizational variables in a single model to account for the effects of the others, in addition to site and the aforementioned patient covariates. Site was included as a fixed effect to account for correlations in unit characteristics that may impact the exposures (eg, unit size and overtime hours) and infection risk. Random intercepts using patient identification accounted for the dependence of repeated measures among participants. Sensitivity analyses were conducted using different exposure timeframes (ie, 1-day and 2-day means), using site as a random effect,³⁶ among infants <28 weeks, and among only infants that had NI (since models were using each patient as repeat observations, this helps to assess if organizational variables affect the timing of NI). In addition, we reran the analysis with the exclusion of infants with coagulase-negative staphylococci (CoNS) infections.

Machine Learning

To quantify the relative importance of organizational factors in predicting the risk and timing of NI, we used generalized mixed effect-regression trees (GMERTs), which is a machine-learning technique.³⁷ GMERTs allow the inclusion of random effects to account for unobserved variables and uses binary indicators for the presence or absence of exposures (or creates a binary cut-off for a continuous exposure) to predict an outcome. In addition, GMERT reduces the dimensionality of the tree by ranking the predictors according to the relative variable importance indicator (VIMP).³⁸ The VIMP is a normalized score between 0% and 100% and variables with low values are removed from the tree since they have little predictive power. The interpretations of the VIMP are in relation to the most important one (ex: a value is 50% would mean that the variable is 50% as important as the main predictor). All patient covariates, site, and organizational variables (repeated observations of 3-day means) were used as initial inputs in the GMERT. The model was specified with a minimum split of 20 observations (ie, 20 observations required to split using a node), a minimum bucket of 6 (ie, a minimum of 6 final leaves), and a maximum depth of 6 layers to the tree. Fifty iterations were forced at the start of the iteration process of the mixed effect regression tree (MERT) function, with a maximum number of iterations allowed based on the MERT function of 100. Fifty iterations were forced at the end of the iteration process for the MERT function, and 50 macro-iterations were forced at the end of the iteration process of the GMERT function. Statistical analyses were conducted using R, version 4.3.1 (R Foundation for Statistical Computing) and packages: glmer, pacman, zoo, compareGroups, car, and GGally.

Results

There were 2157 infants eligible for inclusion in the study. Infants were excluded for being moribund upon admission (n = 37), having a major congenital anomaly (n = 53), having a length of stay less than 72 hours (n = 110), or were diagnosed with early-onset sepsis (n = 36). Among the 1921 infants included (Figure 2), 220 experienced an NI (11.5%) (Table II) at a median age of 11 days [IQR 7-21]. Overall, median gestational age was 30 weeks, 5.5% of infants died, and median length of stay was 50 days [IQR 26-85]. Among the 220 NI, the organisms were CoNS (126, 57%), non-CoNS gram-positive bacteria (36, 16%), gram-negative bacteria (42, 19%), and other fungal/viral infections (16, 7.3%).

Figure 2.

Exclusion criteria flow chart.

Table II.

Characteristics of full cohort, infants with NI, and infants without NI

Characteristics	Total n = 1921	Infants with No NI n = 1701	Infants with NI n = 220	P value∗
Patient
Male	1014 (52.8)	890 (52.4)	124 (56.4)	.294
Birth weight, g	1310 [955, 1660]	1370 [1010, 1700]	900 [704, 1120]	<.001
SGA10	219 (11.4)	188 (11.1)	31 (14.1)	.223
Gestational age, wk	30 [27, 31]	30 [28,31]	26 [25,28]	<.001
Gestational age below 28 wk	533 (27.7)	391 (23.0)	142 (64.5)	<.001
SNAP-II score >20	267 (13.9)	203 (12.0)	64 (29.1)	<.001
Apgar at 5 min <7	579 (30.7)	482 (28.8)	97 (45.1)	<.001
Antenatal steroids	1688 (91.1)	1493 (91.3)	195 (90.3)	.727
Cesarean delivery	1316 (68.7)	1166 (68.8)	150 (68.2)	.916
Outborn status	256 (13.3)	232 (13.6)	24 (11.0)	.310
NI	220 (11.5)	–	–	–
Age at NI, d	11 [7, 21]	–	–	–
Length of stay, d†	50 [26, 85]	46 [24, 80]	90 [55, 116]	<.001
Length of stay >45 d	1041 (54.2)	869 (51.1)	172 (78.2)	<.001
Death‡	105 (5.5)	65 (3.8)	40 (18.2)	<.001
Follow-up time, d	40 [18, 45]	45 [23, 45]	11 [7, 21]	<.001
Organizational variables (average of all shifts)§
Unit occupancy, %	88.7 [81.0, 94.6]	–	–	–
OTR, %	4.4 [1.5, 7.6]	–	–	–
Shifts with overtime	10 261 (65.7)
Number of overtime hours among shifts with overtime, h	8.8 [4.7, 16.8]	–	–	–
NPR, %	101.1 [85.5, 125.1]	–	–	–
Organizational variables (1-day intervals/day 44-45)¶
Unit occupancy, %	–	90.8 [85.1, 95.2]	92.3 [87.6, 97.0]	.001
OTR, %	–	4.5 [1.8, 7.4]	5.5 [3.2, 8.5]	.001
NPR, %	–	100.1 [83.7, 119.3]	97.8 [85.7, 107.9]	.063
Organizational variables (2-day intervals/day 43-45)¶
Unit occupancy, %	–	90.8 [85.1, 95.1]	92.5 [87.4, 97.0]	.001
OTR, %	–	4.7 [2.5, 7.1]	5.5 [3.0, 8.3]	.001
NPR, %	–	99.1 [83.8, 119.6]	98.1 [85.5, 106.8]	.030
Organizational variables (3-day intervals/day 42-45)¶
Unit occupancy, %	–	90.7 [85.1, 95.0]	92.1 [86.8, 96.9]	.001
OTR, %	–	4.7 [2.6, 7.0]	5.6 [3.5, 8.6]	<.001
NPR, %	–	98.6 [84.0, 119.9]	98.3 [85.2, 106.9]	.023

SGA, small for gestational age; SNAP-II, Score for Neonatal Acute Physiology-II.

Missing data: sex, n = 1; SGA 10th percentile, n = 1; SNAP-II score >20, n = 3; Apgar at 5 minutes, n = 32; antenatal steroids, n = 69; cesarean delivery, n = 6.

Data are presented as No. (%) or median [IQR]. Infants with missing data were excluded from the denominator for each variable.

^∗P value obtained using Wilcoxon rank-sum test for continuous variables and the χ² test for categorical variables.

^†Length of stay in the neonatal intensive care unit can exceed 45 days even if the observation period for analysis was limited to 45 days.

^‡Death can occur before or after 45 days among included patients and can occur after NI among infants with NI.

^§Organizational variables were calculated as the average for all shifts for the study period for all units combined using the mean of the 3 shifts 8:00 am to 7:59 am each day throughout the study period. The median [IQR] for OTR was calculated for all shifts, including shifts with 0 hours of overtime. n = 15 615 total shifts in the dataset.

^¶Organizational variables were calculated as a mean of the 1-3 days before NI or discharge/end of observation period, using the first shift to define the time when an NI occurred on a given date (ie, the averages were then calculated 8:00 am to 7:59 am).

After converting the organizational variables into daily means (8:00 am to 7:59 am) during the study period, the median occupancy was 88.7% [IQR 81.0-94.6], median OTR was 4.4% [IQR 1.5-7.6], and median NPR was 101.1% [IQR 85.5-125.1]. Across all sites, there was a low positive correlation between occupancy and OTR (r = 0.204, P < .001), a moderate negative correlation between occupancy and NPR (r = −0.643, P < .001), and a negligeable correlation between OTR and NPR (r = 0.064, P < .01).

Table II shows the median organizational variables on the 1, 2, and 3 days before NI in infants with NI compared with those without NI (1, 2, or 3 days before death, discharge, or the 45th day). In the unadjusted analysis, days preceding infection had greater occupancy, greater OTR, and lower NPR than days without.

Table III shows the analysis using multiple observations per patient (daily 1-, 2-, and 3-day means of organizational variables). When adjusted for patient covariates and site, high OTR and low NPR over the 1-, 2-, and 3-day intervals were associated with greater odds of NI. After adjustment for all 3 organizational variables, site, and patient covariates, OTR was associated with NI in the model using 3-day means. NPR was associated with NI in the models using the 2- and 3-day means. Sensitivity analyses using site as a random effect, among infants <28 weeks, and among only infants that had NI had similar findings (results not shown). Analyses excluding CoNS infections also showed similar results (Table IV).

Table III.

Mixed-effects logistic regression analysis using multiple observations for each patient for association of organizational factors with odds of NI (n = 1921; 60 957 observations)

Exposure	Crude OR [95% CI]	Model I OR [95% CI]	Model II OR [95% CI]
1-day intervals
Unit occupancy	1.02 [1.02-1.03]	1.03 [1.00-1.05]	1.01 [0.98-1.04]
OTR	1.04 [1.04-1.04]	1.05 [1.00-1.10]	1.03 [0.98-1.07]
NPR	0.99 [0.97-1.01]	0.98 [0.96-0.99]	0.98 [0.97-1.00]
2-day intervals
Unit occupancy	1.03 [1.00-1.06]	1.03 [1.01-1.06]	1.00 [0.98, 1.03]
OTR	1.06 [1.05-1.08]	1.07 [1.02-1.12]	1.04 [0.98-1.09]
NPR	0.98 [0.96-1.00]	0.97 [0.95-0.98]	0.97 [0.95-0.99]
3-day intervals
Unit occupancy	1.03 [1.00-1.07]	1.03 [1.01-1.06]	0.99 [0.96-1.02]
OTR	1.11 [1.04-1.19]	1.12 [1.06-1.18]	1.08 [1.02-1.15]
NPR	0.99 [0.99, 0.99]	0.96 [0.94-0.98]	0.96 [0.95, 0.98]

Model I adjusted for patient covariates (gestational age, sex, SGA10, SNAP-II score >20), and site as fixed effects; Model II adjusted for patient covariates, site, and all 3 organizational factors (unit occupancy, OTR, NPR) as fixed effects.

Patient identification was included as a random effect in all models.

Occupancy is occupied/total beds, OTR is nursing overtime hours/total nursing hours, and NPR is number of actual nurses on the unit/number of recommended nurses. The intervals were means of the organizational exposures from either 1, 2, or 3 days before the date of each observation throughout an infant’s stay, from admission until infection, discharge with no infection, or the 45th day with no infection (whichever came first).

Table IV.

Mixed-effects logistic regression analysis using random effect of site^∗ and multiple observations for each patient for association of organizational factors with odds of NI, excluding infants with a CoNS infection from the NI group (n = 1795; 59,120 observations)

Exposure	Crude OR [95% CI]	Model I OR [95% CI]	Model II OR [95% CI]
1-day intervals
Unit occupancy	1.01 [0.95-1.08]	1.02 [0.97-1.06]	1.00 [0.95, 1.05]
OTR	1.07 [1.00-1.14]	1.06 [0.99-1.15]	1.08 [1.00-1.17]
NPR	0.99 [0.96-1.02]	0.99 [0.97-1.01]	0.99 [0.97- 1.02]
2-day intervals
Unit occupancy	1.02 [0.98-1.06]	1.03 [0.98-1.08]	1.01 [0.95-1.07]
OTR	1.08 [1.00-1.16]	1.09 [0.99-1.19]	1.07 [0.97-1.18]
NPR	0.99 [0.96-1.05]	0.99 [0.97-1.01]	0.99 [0.96-1.02]
3-day intervals
Unit occupancy	1.03 [0.98-1.07]	1.03 [0.97-1.08]	1.00 [0.94-1.07]
OTR	1.13 [1.05-1.23]	1.13 [1.04-1.22]	1.14 [1.01-1.27]
NPR	0.98 [0.96-1.00]	0.98 [0.96-0.99]	0.98 [0.95-1.02]

Occupancy is occupied/total beds, OTR is nursing overtime hours/total nursing hours, and NPR is number of actual nurses on the unit/number of recommended nurses. The intervals were means of the organizational exposures from either 1, 2, or 3 days before the date of each observation throughout an infant’s stay, from admission until infection, discharge with no infection, or the 45th day with no infection (whichever came first).

Model I adjusted for patient covariates (gestational age, sex, small for gestational age status [10th percentile], Score for Neonatal Acute Physiology-II score >20); Model II adjusted for patient covariates and all 3 organizational factors (occupancy, OTR, NPR) as fixed effects. Patient identification was included as a random effect in all models, and site was included as a random effect in Models I and II.

^∗Note that site was included as random effect in this sensitivity analysis due to failed model convergence when using site as fixed effect (due to low number of NIs, n = 94).

GMERT Model

The classification tree generated from the GMERT model using repeated observations of 3-day means is provided in Figure 3. The GMERT model corroborated the findings of the primary analysis and showed that the highest variable of importance in predicting NI in the NICU was OTR followed by occupancy, NPR, and gestational age (Figure 4).

Figure 3.

GMERT for algorithm predicting infection risk based on organizational factors and patient characteristics (n = 1921; 60 957 observations. n is the number of observations from the data that were converted to multiple observations per patient using the 3-day mean of the organizational variables from 3 days before the date of each observation throughout an infant’s stay, from admission until infection, discharge with no infection, or the 45th day with no infection (whichever came first). fsite, site of admission; ga, gestational age; GMERT, generalized mixed-effect regression tree; NPR, nursing provision ratio; OCC, occupancy; OTR, nursing overtime ratio; snapgrp:0, Score for Neonatal Acute Physiology-II score <20; sex:0, male; sga10:0, not small for gestational age status (10th percentile).

Figure 4.

Plot of VIMP for organizational factors and patient characteristics from machine-learning model VIMP score is calculated for each value as the sum of the decrease in impurity (calculated when a variable appears as a primary split and when it appears as a surrogate). Score for relative VIMP is calculated by taking each score of VIMP and dividing by the maximum VIMP among all the variables, multiplied by 100. Each variable can have a relative VIMP score between 0% and 100%. The dashed line represents the mean score for the relative VIMP.

Discussion

In this multicenter cohort study of 1921 infants born at <33 weeks of gestational age, greater OTR and lower NPR over 2- and 3-day periods were associated with greater odds of NI on the following day. These results were consistent across multiple analytic approaches and sensitivity analyses and highlight that periods of high nursing overtime and low nursing ratios correlate with increased risk of NI in the NICU. Occupancy often lost significance when accounting for the other 2 organizational variables (Model II), which is potentially attributable to some correlation between variables and confounders.

The results of this study are similar to previous work on nursing overtime in adult patients and add to a growing body of literature on the clinical effects of nursing overtime in the NICU.^6,18,22 Although we could not ascertain which infant received care from a nurse working overtime, our data suggest that periods of greater nursing overtime represent a high-risk period in the NICU for NI. This is consistent with other studies showing that increased overtime is associated with greater fatigue and lower performance in work-related tasks, potentially contributing to lower compliance with infection-prevention measures.^10,39 In addition, greater overtime has been associated with lower safety culture and greater worker burnout, which have both been linked with NI.^9,40,41

Greater NPR was associated with lower odds of NI when adjusted for patient confounders and remained significant after including occupancy and OTR in the model. A previous single-center study found that NPR was not associated with NI when including OTR⁶; however, the larger sample size of this study may have been more appropriate to account for several variables in the model. The association of NPR with NI is consistent with previous large observational studies on adverse outcomes^5,42,43 but few of them also included nursing overtime. In one NICU-based multicenter study, NPR was not associated with morbidity/mortality after adjustment for occupancy, although this may have been the result of limited variation in their data (IQR 89-122 vs this study’s IQR of 85-125).²⁴

Although managers may use overtime to maintain adequate nursing ratios,¹⁰ the relationship between OTR and NPR is complex, hence the low correlation between the variables in our data. In addition, overtime use is affected by other factors such as vacations, sick leave, and number of full-time positions.¹⁸ Reports on occupancy and NI have shown mixed results: some have correlated greater occupancy with NI, whereas others have not.^6,8 Ultimately, the evaluation of organizational factors with outcomes in the NICU is complex and may be limited when using conventional statistical modeling. The machine-learning GMERT model showed results similar to the conventional modeling (ie, OTR as the greatest contributor to risk of NI) and also showed that the top 3 predictors were the organizational factors (all ranked above clinical risk factors). Although gestational age is an important risk factor for developing NI during admission, the GMERT model highlights that organizational factors play an important role in predicting when the infant will have the infection. Although this should be interpreted as exploratory, it demonstrates a novel method to evaluate organization of care.

In a public health care setting with finite resources, these results highlight the ongoing challenges of resource allocation and organization of care. Considering that meeting 100% of recommended nursing ratios without overtime is unrealistic, managers must strike a balance between using overtime to maintain adequate nursing ratios while considering the clinical risk of prolonged periods of high overtime. In addition, nursing overtime and understaffing have other effects on employee retention, work satisfaction, and burnout.^9,11 Several strategies to reduce nursing overtime have been evaluated: use of 12-hour shifts, improving staff moral to reduce work absenteeism, and adjusting the patient flow in the unit (reorienting transfers, elective procedures).⁴⁴ However, large multicenter studies on ways to implement and evaluate the effects of these strategies are required. The daily staffing levels in this study were typically greater than what has been reported in other acute care settings in the US, with reports of a 6.5% OTR in adult ICUs,⁴⁵ 31%-68% of NICU infants being understaffed in some NICUs,⁵ and an average 91% NPR in NICUs.⁴⁶

This study has limitations. The observational nature of the study is at risk of ecological bias as the result of the exposure being at a unit-level for a patient-level outcome. It is assumed that the unit-level measures captured how much each patient is exposed to occupancy, overtime, and nursing provision. In reality, the unit-level exposures may correlate with unmeasured patient-level exposures, which actually influence infection risk (eg, overtime may correlate with nursing skill mix, and nurse training can affect infection rates⁴⁷). Future research should assess patient-level exposures to nursing ratios and overtime worked by the nurses caring for each patient. Residual confounding from nursing-level factors (eg, sociodemographic characteristics, experience working on a NICU/skill mix) may exist,⁴⁷ data on changes in nursing ratios during the 8-hour shift because of new admissions were not available, and data on mandatory vs voluntary overtime were not available.^19,48 The study sample had a high proportion of infants with CoNS infections, although these were removed in a sensitivity analysis to assess for consistency. We did not have data on use of central line and central-line infections. Data on daily fluctuations in patient risk factors were not available to include in the model as time-varying confounders. Finally, emotional, and psychological factors vary among individuals during periods of high stress, and we were unable to survey nurses on their stress and compliance with infection prevention measures.⁹

Conclusions

In this multicenter study, 3-day periods of greater nursing overtime and lower nursing ratios throughout hospitalization were associated with increased risk of NI in infants born very preterm. This study adds to the growing data on the effects of organization of care in the NICU and highlights the need to integrate organization of care for NI prevention. Future studies investigating infant outcomes and organizational factors should consider accounting for nursing overtime and nursing provision.

CRediT authorship contribution statement

Marissa Fazio: Writing – review & editing, Writing – original draft, Visualization, Methodology, Formal analysis, Conceptualization. Elias Jabbour: Writing – review & editing, Writing – original draft, Supervision, Methodology, Formal analysis, Conceptualization. Sharina Patel: Writing – review & editing, Conceptualization. Valérie Bertelle: Writing – review & editing, Data curation, Conceptualization. Anie Lapointe: Writing – review & editing, Data curation, Conceptualization. Guy Lacroix: Writing – review & editing, Methodology, Formal analysis, Conceptualization. Sophie Gravel: Writing – review & editing, Data curation, Conceptualization. Michèle Cabot: Writing – review & editing, Data curation, Conceptualization. Bruno Piedboeuf: Writing – review & editing, Data curation, Conceptualization. Marc Beltempo: Writing – review & editing, Writing – original draft, Supervision, Resources, Methodology, Investigation, Funding acquisition, Formal analysis, Data curation, Conceptualization.

Declaration of Competing Interest

This project was funded by an Early Career Investigator Grant from the Canadian Institutes of Health Research (CIHR) Institute of Human Development, Child, and Youth Health (IHDCYH) and an Early Career Investigator Grant from the Montreal Children’s Hospital Foundation. M.B. holds a FRSQ Clinical Research Scholar Career Award Junior 1 and funding from the Ministry of Health of Quebec. The Canadian Neonatal Network is supported by a grant from the CIHR funding the Canadian Preterm Birth Network (PBN 150642). The funding agencies had no role in the design or conduct of the study; the collection, management, analysis, or interpretation of the data; the preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication. The authors report no conflicts of interest.