Key Points
Question
Does the workload of neonatal intensive care unit nurses influence the likelihood that a nurse will miss necessary care for assigned infants?
Findings
In this study of 136 nurses caring for 418 infants during 332 shifts, increased infant-to-nurse ratio during a shift was associated with increased missed nursing care in about half of the measured missed care items. When a measure of subjective workload was considered, the associations of ratios were mostly attenuated; increased subjective workload was consistently associated with increased missed care.
Meaning
Focusing exclusively on infant-to-nurse ratios to address missed care may be limiting; nurses’ subjective workload is typically unmeasured but has promise for tailored workload interventions.
This study evaluates the association of nurse shift-level workloads in the neonatal intensive care unit with missed nursing care using nurse-reported data.
Abstract
Importance
Quality improvement initiatives demonstrate the contribution of reliable nursing care to gains in clinical and safety outcomes in neonatal intensive care units (NICUs); when core care is missed, outcomes can worsen.
Objective
To evaluate the association of NICU nurse workload with missed nursing care.
Design, Setting, and Participants
A prospective design was used to evaluate associations between shift-level workload of individual nurses and missed care for assigned infants from March 1, 2013, through January 31, 2014, at a 52-bed level IV NICU in a Midwestern academic medical center. A convenience sample of registered nurses who provided direct patient care and completed unit orientation were enrolled. Nurses reported care during each shift for individual infants whose clinical data were extracted from the electronic health record. Data were analyzed from January 1, 2015, through August 13, 2018.
Exposures
Workload was assessed each shift with objective measures (infant-to-nurse staffing ratio and infant acuity scores) and a subjective measure (the National Aeronautics and Space Administration Task Load Index [NASA-TLX]).
Main Outcomes and Measures
Missed nursing care was measured by self-report of omission of 11 essential care practices. Cross-classified, multilevel logistic regression models were used to estimate associations of workload with missed care.
Results
A total of 136 nurses provided reports of shift-level workload and missed nursing care for 418 infants during 332 shifts of 12 hours each. When workload variables were modeled independently, 7 of 12 models demonstrated a significant worsening association of increased infant-to-nurse ratio with odds of missed care (eg, nurses caring for ≥3 infants were 2.51 times more likely to report missing any care during the shift [95% credible interval, 1.81-3.47]), and all 12 models demonstrated a significant worsening association of increased NASA-TLX subjective workload ratings with odds of missed care (eg, each 5-point increase in a nurse’s NASA-TLX rating during a shift was associated with a 34% increase in the likelihood of missing a nursing assessment for his or her assigned infant[s] during the same shift [95% credible interval, 1.30-1.39]). When modeling all workload variables jointly, only 4 of 12 models demonstrated significant association of staffing ratios with odds of missed care, whereas the association with NASA-TLX ratings remained significant in all models. Few associations of acuity scores were observed across modeling strategies.
Conclusions and Relevance
The workload of NICU nurses is significantly associated with missed nursing care, and subjective workload ratings are particularly important. Subjective workload represents an important aspect of nurse workload that remains largely unmeasured despite high potential for intervention.
Introduction
Quality improvement initiatives consistently demonstrate the positive effect of bedside nursing care on clinical and safety outcomes in neonatal intensive care units (NICUs). For example, high levels of nurse adherence to central line maintenance bundles substantially reduce neonatal bloodstream infections,1,2 and consistent nurse adherence to precise oxygen titration protocols reduces risk of retinopathy of prematurity.3,4,5,6 Given the importance of nursing care to infant outcomes in NICUs, a clear understanding of the working conditions influencing nurses’ capability to deliver care in a highly reliable way is critical for sustaining improvement gains.
Nurse workload, defined as the “the amount of performance required to carry out nursing activities”7(p464) and frequently measured using resource-based objective metrics, such as infant-to-nurse staffing ratios or patient acuity scores, has been correlated with infant outcomes in NICUs,8 including risk of hospital-acquired infection,9 adverse events,10 and in-hospital mortality.11,12,13 However, explanation of the mechanisms underlying these associations is lacking. Does higher workload in the form of more patients, higher patient acuity, increased cognitive or temporal demands, or some combination of these interfere with a nurse’s ability to deliver highly reliable care?
Missed nursing care or necessary patient care that is omitted or substantially delayed14 is a commonly theorized outcome of high nurse workload and an emerging measure of nursing care process reliability that may partly explain the effects of workload on outcomes. Researchers have correlated nurse workload with missed nursing care in adult populations; higher nurse workload measured as perceived staffing adequacy,15 staffing ratio,16,17,18,19 and nursing care hours per patient day20,21 is frequently associated with increased missed care. Evaluation of a relationship between nurse workload and missed care in NICUs is minimal22 despite evidence that neonatal nurses miss or have lapses in providing essential care.23 A 2015 study of missed care among NICU nurses in 7 states identified oral care for infants undergoing mechanical ventilation, attendance at team rounds, and engaging parents in care as missed most frequently among 35 essential activities, and early evidence suggests that missed care in NICUs is associated with infant outcomes, including increased time to achieve full feedings and length of stay.24,25
This study’s objective was to evaluate the association of NICU nurse workload with missed nursing care by analyzing multilevel data from individual nurses reporting on missed nursing care for individual infants during 332 shifts of 12 hours each. We hypothesized that nurses exposed to higher shift-level workload in the form of more assigned patients, higher acuity scores of assigned patients, and/or higher subjective ratings of workload on a validated scale would be significantly more likely to report missing care for their assigned infants during the same shift.
Methods
Study Design
We conducted a prospective study during the 11 months from March 1, 2013, through January 31, 2014. Each quarter we initiated a continuous 6-week data collection cycle for a total of 4 cycles; at the end of every shift we collected confidential nurse reports of staffing ratio, subjective workload, and missed care that was specific to each assigned infant. We also collected infant acuity data from the electronic health record that corresponded to each infant-specific report of missed care based on the medical record number, date, and time. The institutional review board at the first author’s institution reviewed and approved the study protocol. Written informed consent was obtained from participating nurses; a waiver of informed consent was granted for the use of deidentified infant data obtained through an honest broker. A Certificate of Confidentiality was obtained as an added measure of protection.
Setting, Sample, and Recruitment
The study occurred in a 52-bed level IV NICU in a Midwestern academic medical center. Registered nurses were eligible if they had completed unit orientation, provided direct patient care for more than 80% of their clinical effort, and were permanently employed in the NICU. Lists of eligible nurses were obtained from hospital administrators. Participants were recruited via email solicitations, brochures, flyers, and information sessions. Enrollment remained open until the fourth cycle; our goal was to obtain as many corresponding nurse-infant shifts of data as possible, with an enrollment target of 80% of eligible nurses participating. Research incentives were provided to compensate participants for their time.
We included data from all infants during active cycles because the missed care measures were not condition specific and represented standard nursing care. However, we only included an infant’s data in an analytic model if the infant was at risk for the missed care outcome on the particular shift as indicated by nurse report.
Outcomes
Missed nursing care was assessed by asking nurses to report on omission of 11 essential neonatal nursing care practices using items developed in earlier work.24 Self-report is the prevailing method for obtaining information about care that was not delivered and therefore is absent from clinical documentation and other objective data sources. Nurses reported on the frequency of care missed for each infant assigned to them based on a Likert-type scale (never, rarely, occasionally, or frequently missed or not applicable). Because the frequency of each care item varied each shift according to an individual infant’s needs, we relied on nurses’ estimates of missed care frequencies. Responses were dichotomized into missed or not missed for each item because of low endorsement of higher levels of missed care response options, and we created a global dichotomous missed care indicator for each infant on each shift if a nurse reported missing any of the 11 items during the shift.
Primary Exposures and Covariables
Nurse workload was assessed using objective and subjective measures.26 Objective measures included infant-to-nurse staffing ratio and infant acuity scores. Nurses reported on the maximum number of infants cared for at a single point during the shift. Ratios were verified by a count of unique missed care forms received each shift and by validation against administrative data sources. We categorized the staffing ratio as 1, 2, or 3 or more infants to 1 nurse. Infant acuity scores were estimated each shift by nurses as part of an electronic health record–based patient classification system for determining an infant’s required nursing care hours per day.27 The score incorporates clinical indicators of nursing care intensity such as ventilation modality, frequency and mode of feedings, number and type of infusions, and procedures. The range for each indicator varies depending on the number and type of items assessed, with higher scores indicative of higher nursing care intensity.
Subjective workload, an emerging measure of ICU nurse workload,28,29 was assessed each shift using a paper version of the National Aeronautics and Space Administration Task Load Index (NASA-TLX), a scale developed to measure domains of subjective workload in high-risk industries such as aviation30,31 and nuclear energy32 and, increasingly, in health care.33 The NASA-TLX is primarily a measure of how an individual experiences the situational demands of work, including cognitive and mental demands, physical demands, time pressure, and overall required effort to accomplish goals. Overall workload scores were calculated as a raw sum of scores from these 4 items; scores for each item ranged from 1 (low) to 20 (high), and overall workload scores ranged from 4 (low) to 80 (high).28 Covariables included nurse educational degree (bachelor’s degree or higher), nurse certification status (holds a professional certification or not), new graduate status (<1 year since obtaining nursing licensure), unit census at shift start, and shift start time (7 am or 7 pm).
Data Sources and Collection
Data collection procedures, summarized herein, are described in detail elsewhere.34 At each shift’s end, nurses reported subjective workload experienced during the course of the shift, infant-to-nurse ratio, and missed care for assigned infants. Shift-level infant acuity data were retrospectively extracted from the electronic health record and merged with missed care data. Forms were reviewed for completeness, and data were entered into a secure database maintained by data managers. Nurses who did not return expected forms were contacted within 24 hours; data were considered missing after 48 hours. Data were deidentified before release to investigators.
Statistical Analysis
Data were analyzed from January 1, 2015, through August 13, 2018. Descriptive statistics were used to describe the sample, shift characteristics, and frequency of missed nursing care items. Pearson product moment and Spearman rank correlations were calculated to assess baseline associations among workload variables, and tetrachoric correlations were calculated for missed care item associations.
Modeling associations between workload and missed care required advanced analytic techniques owing to the data’s hierarchical complexity. Our data included multiple nested and nonnested contexts simultaneously: infants nested within nurses within a single shift and infants nested within different nurses during the course of multiple shifts. In this analysis, we focus on associations at the level of the nurse and infant within a shift; shifts represented a sequential time variable. A logistic cross-classified multilevel model (CCMM) was fit for each of the 11 missed care items and to the global missed care measure. In a CCMM, the repeated observations (shifts) simultaneously belong to 2 inconsistently nested contexts,35,36,37,38 in this case, infants and nurses.
To test for effects of workload on missed care, we estimated multiple CCMMs. In the first set of models, we tested for an association of each individual workload measure with an infant’s odds of missed care for each of the 12 missed care outcomes. Next, we tested effects of objective workload measures—staffing ratios and acuity scores—on each outcome. Finally, we tested a set of models that incorporated objective and subjective workload measures as associated with missed care outcomes. We restricted analyses to infants with acuity scores of less than 191, the 99th percentile, because most infants with scores above 191 were extreme outliers with inverted maximum ratios (eg, 1 infant receiving extracorporeal circulation membrane oxygenation and cared for by 2 nurses). To aid interpretation, we rescaled acuity and subjective workload variables to indicate change in odds for each 5-unit increase in score.
We first ran iterative generalized least squares models using the estimates as start values for the CCMMs to reduce computation time.39 Then, Bayesian estimation procedures were implemented via Markov chain Monte Carlo methods using the Metropolis-Hastings algorithm.40 Bayesian estimation was favored to reduce bias in parameters and standard errors (SEs).40,41,42
Odds ratios and 95% credible intervals (CrIs), which are the confidence intervals generated using Bayesian procedures (specifically, 2.5th and 97.5th quantiles of the posterior distributions) are presented for fixed-effect parameters. Model diagnostics on null models showed no evidence of assumption violations, outliers, or influential points. The false discovery rate was addressed using the Benjamini-Yekutieli procedure for correlated data.43 Analyses were conducted using MLwiN multilevel modeling software (version 2.30) within Stata (version 15; StataCorp), and α was corrected to .004.41
Results
Descriptive Findings
Of 202 eligible nurses, 136 (67.3%) reported on the care of 418 infants during 332 shifts of 12 hours each. A mean of 17 nurses reported on care of 32 infants for each shift. All participants submitted more than 93% of expected surveys, resulting in less than 1% missing survey data. In total, 10 428 nurse-infant shifts of workload and missed care data were available for analysis. Characteristics of nurse participants are detailed in Table 1. No significant differences in key characteristics were found between eligible nurses who did and did not enroll (n = 66) with the exception of experience; enrolled nurses had a mean (SE) of 3.4 (1.3) fewer years of nursing experience compared with nonenrolled peers (P = .01).
Table 1. Characteristics of Nurse Participants.
| Characteristic | Data (N = 136) |
|---|---|
| Baccalaureate degree or higher, No. (%) | 114 (83.8) |
| Specialty certification, No. (%) | 44 (32.4) |
| Primary shift, No. (%) | |
| Day | 49 (36.0) |
| Evening | 2 (1.5) |
| Night | 45 (33.1) |
| Rotating | 40 (29.4) |
| Nursing experience, mean (SD), y | 9.0 (9.5) |
| Tenure in current NICU, mean (SD), y | 6.4 (7.6) |
| Weekly hours worked, mean (SD) | 32.4 (6.2) |
| No. of shifts of data contributed during study, mean (SD) | 41.8 (17.3) |
| No. of unique infants cared for during study, mean (SD) | 39.9 (15.8) |
Abbreviation: NICU, neonatal intensive care unit.
The mean infant-to-nurse ratio on a shift was nearly 2:1 and ranged from 1 to 4 infants per nurse (Table 2); most shifts were characterized by a 2:1 ratio. Infant acuity scores ranged from a low of 7 to a high of 800 for infants receiving extracorporeal circulation membrane oxygenation; the median score was 49, and the 99th percentile was 191 (the cut point for inclusion in models). The mean NASA-TLX subjective workload rating per shift was 46 (range, 4-80; median, 45). Correlations among workload variables were low to moderate and statistically significant, including staffing ratio and acuity score (−0.32; P < .001), staffing ratio and NASA-TLX score (0.19; P < .001), and acuity score and NASA-TLX score (0.19; P < .001).
Table 2. Workload Characteristics.
| Summary Statistics | Data |
|---|---|
| Infant acuity score | |
| Mean (SD) | 62.51 (56.51) |
| Median (IQR) | 49 (36-75) |
| NASA-TLX overall workload scorea | |
| Mean (SD) | 45.54 (19.21) |
| Median (IQR) | 45 (29-60) |
| Maximum ratio of infants to nurse, No. (%) of shifts | |
| 1:1 | 1022 (9.8) |
| 2:1 | 8730 (83.5) |
| 3:1 | 675 (6.4) |
| 4:1 | 25 (0.2) |
| Correlation coefficient (P value) | |
| Staffing ratio and acuity score | −0.32 (<.001) |
| Staffing ratio and NASA-TLX score | 0.19 (<.001) |
| Acuity score and NASA-TLX score | 0.19 (<.001) |
| Acuity score by maximum staffing ratio, mean/medianb | |
| 1 | 137.0/104 |
| 2 | 69.8/62 |
| ≥3 | 62.7/54 |
| NASA-TLX score by maximum staffing ratio, mean/median | |
| 1 | 38.3/38 |
| 2 | 44.5/44 |
| ≥3 | 60.0/65 |
Abbreviations: IQR, interquartile range; NASA-TLX, National Aeronautics and Space Administration Task Load Index.
Scores range from 4 (low) to 80 (high).
Scores range from 7 to 191, with higher scores indicating higher nursing care intensity.
Missed care was reported on 326 (98.2%) of the 332 shifts and in 2502 (24.0%) of 10 428 corresponding nurse-infant shifts of missed care data. All missed care items were significantly correlated (eTable in the Supplement). When nurses reported missing care for an infant during a shift, the amount of applicable care missed ranged from 9% to 100%, with a median value of 22% and 90th percentile of 50%. Frequency distributions of the types of care missed by nurses are shown in Table 3. When care was reported as missed, it was most often missed rarely or occasionally. Overall, nurses most frequently missed hourly checks of intravenous line sites (1066 [20.4%] of applicable shifts) and adherence to the central line–associated bloodstream infection prevention bundle (695 [15.5%] of applicable shifts). Nurses least frequently reported missing standard safety checks of alarms and equipment (188 [1.8%] of applicable shifts).
Table 3. Distributions of Missed Care When Item Was Applicable to an Infant's Care.
| Missed Care Item | No. of Applicable Nurse-Infant Shifts | Response, No. (%) of Shifts | |||
|---|---|---|---|---|---|
| Rarely Missed | Occasionally Missed | Frequently Missed | Missed | ||
| Hourly IV site assessment | 5224 | 869 (16.6) | 173 (3.3) | 24 (0.5) | 1066 (20.4) |
| Adherence to central venous catheter infection prevention bundle | 4476 | 582 (13.0) | 95 (2.1) | 18 (0.4) | 695 (15.5) |
| Oral feedings | 4278 | 240 (5.6) | 57 (1.3) | 23 (0.5) | 320 (7.5) |
| Patient assessment | 10 397 | 643 (6.2) | 78 (0.8) | 30 (0.3) | 751 (7.2) |
| Obtain laboratory samples | 2838 | 163 (5.7) | 28 (1.0) | 8 (0.3) | 199 (7.0) |
| Parent involvement | 5662 | 303 (5.4) | 54 (1.0) | 22 (0.4) | 379 (6.7) |
| Adherence to ventilator-associated respiratory infection prevention bundle | 2978 | 131 (4.4) | 20 (0.7) | 8 (0.3) | 159 (5.3) |
| Thorough handoff | 10 301 | 303 (2.9) | 19 (0.2) | 24 (0.2) | 346 (3.4) |
| Verification of 6 rights during medication administration | 8564 | 240 (2.8) | 20 (0.2) | 19 (0.2) | 279 (3.2) |
| Double-check of high-risk medications | 3090 | 88 (2.8) | 7 (0.2) | 5 (0.2) | 100 (3.2) |
| Standard safety checks of equipment and alarms | 10 351 | 141 (1.4) | 23 (0.2) | 25 (0.2) | 188 (1.8) |
Abbreviation: IV, intravenous.
Workload and Missed Care
Table 4 displays effects of workload measures on missed care items when modeled separately. Seven of 12 ratio models demonstrated a statistically significant worsening effect of an increased infant-to-nurse ratio on the odds of missed care, with effects most pronounced when nurses cared for 3 or more infants during a shift compared with a 1:1 assignment (eg, nurses caring for ≥3 infants were 2.51 times more likely to report missing any care during the shift [95% CrI, 1.81-3.47]). Small increases in acuity scores were significantly associated with increased odds of missed care in 3 of 12 models (any missed care, 1.06 [95% CrI, 1.04-1.07]; missed parent involvement, 1.04 [95% CrI, 1.02-1.07]; and missed verification of 6 rights during medication administration (6 rights), 1.03 [95% CrI, 1.10-1.06]). All 12 NASA-TLX models indicated significant worsening effects of increased subjective workload ratings on the odds of missed care (eg, each 5-point increase in a nurse’s NASA-TLX rating during a shift was associated with a 34% increase in the likelihood of missing a nursing assessment for his or her assigned infant[s] during the same shift [95% CrI, 1.30-1.39]) (Table 4).
Table 4. Effects of Workload Variables on Missed Care Outcomes When Modeled Separately.
| Outcome | Workload Measures | |||
|---|---|---|---|---|
| Staffing Ratio | Infant Acuity Score, OR (95% CrI)b,c | NASA-TLX Rating, OR (95% CrI)b,c | ||
| No. of Infants per Nursea | OR (95% CrI)b | |||
| Any missed care | 2:1 | 1.23 (0.97-1.57) | 1.06 (1.04-1.07)d | 1.22 (1.19-1.24)d |
| ≥3:1 | 2.51 (1.81-3.47)d | |||
| Missed hourly IV site assessment | 2:1 | 2.98 (2.17-4.02)d | 0.99 (0.97-1.01) | 1.22 (1.18-1.25)d |
| ≥3:1 | 7.25 (4.16-11.84)d | |||
| Missed central venous catheter infection prevention bundle | 2:1 | 2.44 (1.60-3.41)d | 0.99 (0.97-1.01) | 1.21 (1.16-1.25)d |
| ≥3:1 | 6.39 (3.31-10.87)d | |||
| Missed oral feeding | 2:1 | 2.06 (0.71-5.12) | 1.05 (1.01-1.09) | 1.21 (1.16-1.27)d |
| ≥3:1 | 4.58 (1.48-11.70)d | |||
| Missed patient assessment | 2:1 | 2.65 (1.65-4.23)d | 1.01 (0.99-1.03) | 1.34 (1.30-1.39)d |
| ≥3:1 | 7.15 (4.05-12.21)d | |||
| Missed obtaining laboratory samples | 2:1 | 1.22 (0.73-1.98) | 1.00 (0.97-1.02) | 1.16 (1.10-1.22)d |
| ≥3:1 | 1.37 (0.45-3.07) | |||
| Missed parent involvement | 2:1 | 0.77 (0.47-1.21) | 1.04 (1.02-1.07)d | 1.16 (1.12-1.21)d |
| ≥3:1 | 1.61 (0.83-2.90) | |||
| Missed ventilator-associated respiratory infection prevention bundle | 2:1 | 1.47 (0.84-2.50) | 1.03 (1.00-1.07) | 1.18 (1.10-1.26)d |
| ≥3:1 | 3.51 (1.11-8.22) | |||
| Missed handoff | 2:1 | 1.07 (0.64-1.69) | 1.00 (0.98-1.03) | 1.23 (1.18-1.29)d |
| ≥3:1 | 3.59 (1.85-6.47)d | |||
| Missed 6 rights during medication administration | 2:1 | 1.55 (0.93-2.75) | 1.03 (1.01-1.06)d | 1.23 (1.17-1.29)d |
| ≥3:1 | 1.70 (0.69-3.53) | |||
| Missed double-check of high-risk medications | 2:1 | 0.91 (0.50-1.58) | 1.03 (0.99-1.06) | 1.13 (1.04-1.21)d |
| ≥3:1 | 0.83 (0.08-2.79) | |||
| Missed standard safety checks of equipment and alarms | 2:1 | 1.75 (0.83-3.71) | 1.00 (0.98-1.03) | 1.20 (1.14-1.26)d |
| ≥3:1 | 4.26 (1.65-9.71)d | |||
Abbreviations: IV, intravenous; NASA-TLX, National Aeronautics and Space Administration Task Load Index; OR, odds ratio; 95% CrI, 95% credible interval.
Reference category was 1 infant per nurse (1:1 ratio).
Each model controlled for nurse education, certification, experience, unit census, and shift start time.
Rescaled to indicate change in odds per 5-unit increase in score.
P < .004, α corrected for false discovery rate.
Twelve models estimating joint effects of objective workload measures—staffing ratios and acuity scores—on the odds of missed care are highlighted in Table 5. The pattern of results was generally consistent with independent modeling, with a significant worsening effect of increased ratio in 9 of 12 models and significant effects of small increases in acuity in 5 models, including any missed care (1.07; 95% CrI, 1.06-1.09), missed oral feeding (1.06; 95% CrI, 1.02-1.10), missed patient assessment (1.03; 95% CrI, 1.02-1.05), missed parent involvement (1.05; 95% CrI, 1.02-1.08), and missed verification of 6 rights of medication administration (1.05; 95% CrI, 1.02-1.07). Compared with models in Table 4, effect sizes and significance levels of ratios were increased when modeled with acuity scores.
Table 5. Effects of Workload Variables on Missed Care Outcomes.
| Outcome | Objective Workload Variables | All Workload Variables | |||||
|---|---|---|---|---|---|---|---|
| Staffing Ratio | Infant Acuity Score, OR (95% CrI)b,c | Staffing Ratio | Infant Acuity Score, OR (95% CrI)b,c | NASA-TLX Rating, OR (95% CrI)b,c | |||
| No. of Infants per Nursea | OR (95% CrI)b | No. of Infants per Nursea | OR (95% CrI)b | ||||
| Any missed care | 2:1 | 2.01 (1.57-2.55)d | 1.07 (1.06-1.09)d | 2:1 | 1.30 (1.04-1.66) | 1.05 (1.04-1.06)d | 1.20 (1.17-1.23)d |
| ≥3:1 | 4.53 (3.15-6.22)d | ≥3:1 | 1.66 (1.18-2.30)d | ||||
| Missed hourly IV site assessment | 2:1 | 3.36 (2.34-4.64)d | 1.02 (1.00-1.03) | 2:1 | 2.29 (1.55-3.17)d | 0.99 (0.98-1.02) | 1.20 (1.16-1.24)d |
| ≥3:1 | 8.41 (4.67-13.78)d | ≥3:1 | 3.28 (1.76-5.54)d | ||||
| Missed central venous catheter infection prevention bundle | 2:1 | 2.67 (1.76-3.88)d | 1.01 (0.99-1.03) | 2:1 | 1.82 (1.22-2.66)d | 0.99 (0.97-1.01) | 1.19 (1.14-1.24)d |
| ≥3:1 | 6.99 (3.54-12.35)d | ≥3:1 | 2.79 (1.39-4.93)d | ||||
| Missed oral feeding | 2:1 | 2.42 (0.75-7.69) | 1.06 (1.02-1.10)d | 2:1 | 1.06 (0.39-2.54) | 1.04 (1.01-1.08) | 1.20 (1.14-1.26)d |
| ≥3:1 | 5.66 (1.60-17.55)d | ≥3:1 | 1.51 (0.47-4.08) | ||||
| Missed patient assessment | 2:1 | 3.54 (2.27-5.52)d | 1.03 (1.02-1.05)d | 2:1 | 2.02 (1.23-3.18)d | 1.00 (0.98-1.02) | 1.34 (1.29-1.38)d |
| ≥3:1 | 10.04 (5.65-16.79)d | ≥3:1 | 2.70 (1.47-4.67)d | ||||
| Missed obtaining laboratory samples | 2:1 | 1.27 (0.75-2.14) | 1.00 (0.97-1.03) | 2:1 | 0.88 (0.45-1.56) | 0.98 (0.94-1.01) | 1.18 (1.12-1.26)d |
| ≥3:1 | 1.46 (0.47-3.40) | ≥3:1 | 0.69 (0.20-1.60) | ||||
| Missed parent involvement | 2:1 | 1.10 (0.68-1.80) | 1.05 (1.02-1.08)d | 2:1 | 0.77 (0.44-1.25) | 1.03 (1.01-1.06)d | 1.15 (1.10-1.20)d |
| ≥3:1 | 2.53 (1.23-4.71) | ≥3:1 | 1.20 (0.53-2.32) | ||||
| Missed ventilator-associated respiratory infection prevention bundle | 2:1 | 1.93 (1.03-3.36) | 1.05 (1.01-1.09) | 2:1 | 1.42 (0.71-2.55) | 1.03 (0.99-1.06) | 1.16 (1.09-1.24)d |
| ≥3:1 | 4.87 (1.45-11.92)d | ≥3:1 | 2.56 (0.71-6.38) | ||||
| Missed handoff | 2:1 | 1.15 (0.63-1.99) | 1.01 (0.98-1.04) | 2:1 | 0.89 (0.50-1.50) | 0.99 (0.97-1.02) | 1.22 (1.15-1.27)d |
| ≥3:1 | 3.96 (1.90-7.63)d | ≥3:1 | 1.70 (0.79-3.24) | ||||
| Missed 6 rights during medication administration | 2:1 | 2.38 (1.31-4.02)d | 1.05 (1.02-1.07)d | 2:1 | 1.50 (0.84-2.53) | 1.02 (1.00-1.05) | 1.23 (1.18-1.28)d |
| ≥3:1 | 2.88 (1.19-5.65) | ≥3:1 | 1.06 (0.41-2.22) | ||||
| Missed double-check of high-risk medication | 2:1 | 1.04 (0.54-1.78) | 1.03 (0.99-1.07) | 2:1 | 0.86 (0.40-1.63) | 1.01 (0.97-1.05) | 1.13 (1.05-1.23)d |
| ≥3:1 | 1.02 (0.09-3.59) | ≥3:1 | 0.68 (0.06-2.41) | ||||
| Missed standard safety checks of equipment and alarms | 2:1 | 1.94 (0.95-3.70) | 1.02 (0.98-1.05) | 2:1 | 1.18 (0.61-2.21) | 0.99 (0.96-1.02) | 1.19 (1.13-1.26)d |
| ≥3:1 | 5.10 (2.07-10.87)d | ≥3:1 | 1.80 (0.70-3.86) | ||||
Abbreviations: IV, intravenous; NASA-TLX, National Aeronautics and Space Administration Task Load Index; OR, odds ratio; 95% CrI, 95% credible interval.
Reference category was 1 infant per nurse (1:1 ratio).
Each model controlled for nurse education, certification, experience, unit census, and shift start time.
Rescaled to indicate change in odds per 5-unit increase in score.
P < .004, α corrected for false discovery rate.
Full models estimating associations between objective and subjective workload variables and the odds of missed care are juxtaposed in Table 5. Four of 12 ratio models demonstrated a significant worsening effect of increased ratio on the odds of missed care during a shift, particularly for a ratio of at least 3:1 compared with 1:1; all ratio effect sizes were substantially decreased and CrIs were narrowed. Effects of subjective workload ratings were consistent with independent modeling and demonstrated statistically significant worsening effects of increased ratings on the occurrence of missed care for all items. For example, each 5-point increase in a nurse’s NASA-TLX rating during a shift remained associated with a 34% increase in the likelihood of missing a nursing assessment for his or her assigned infant(s) during the same shift (95% CrI, 1.29-1.38). Few meaningful effects of acuity were observed, and post hoc testing for nonlinear acuity effects did not change results.
Discussion
Consistent with our hypothesis, we found a statistically significant association between nurse workload and the odds of missed nursing care for individual infants, although the effects varied across workload measures and modeling strategies. Staffing ratios appeared to be strongly associated with some missed nursing care items when modeled alone, and ratio effects were amplified when modeled with acuity. However, when modeled with nurses’ subjective workload ratings, the association between staffing ratios and missed nursing care was mostly attenuated. Subjective workload ratings were repeatedly and significantly associated with all missed nursing care items, irrespective of modeling strategies, and demonstrated consistency across models. We observed few meaningful associations between acuity scores and missed nursing care in any modeling approach.
Like many studies of missed care,16,17,18,19 we demonstrated a significant association between increased ratios and increased odds of missed care, but this association was strongest when subjective workload was not included in the models. One potential explanation for this pattern is theoretical: subjective workload mediates, or explains, the effects of ratios on missed care. We did not examine mediation because formal tests in CCMMs are not yet developed, but partial mediation is possible given the low but significant correlation we found between workload measures. Subjective workload may be farther along the causal pathway between staffing ratios and missed care; how one experiences workload may directly influence actions and response.43 Although this relationship requires clarification, the literature offers insight on the influence of other system factors on nurses’ subjective ratings. In a mixed-methods study of workload challenges associated with falls prevention efforts in adult inpatient units, Lopez and colleagues44 found that prevention efforts were not substantially affected by staffing ratios but were associated with NASA-TLX ratings, which were correlated with physical environment layout and work processes. Moving forward, rigorous evaluation of the causal pathway between nurse workload and missed care will be important for tailoring interventions to varying unit and shift contexts.
The changing results across modeling strategies underscore the importance of measuring multiple domains of workload in the same analysis. Most NICU workload studies lack adjustment for time-varying acuity, which in this analysis strengthened staffing ratio associations when these variables were modeled jointly. In addition, some missed care items appear to be differentially sensitive to workload measures. For example, ratio associations vary across models for some items but are consistently high and statistically significant for routine assessment, hourly intravenous line checks, and adherence to central line infection prevention protocols, possibly because these care needs are scheduled and time sensitive. In these instances, a 3:1 compared with a 1:1 infant-to-nurse ratio substantially increases the risk of missed care.
Strengths and Limitations
Study strengths include high rates of enrollment and response, repeated measures of workload and missed care, use of innovative techniques to model data as closely to its complex hierarchical structure as possible, and use of a widely disseminated and empirically supported measure of subjective workload. Limitations include self-reported data from a convenience sample and potential social desirability bias owing to the study topic, although the latter is likely to make our results underrepresentative of actual effects. We cannot rule out reverse causality or the possibility that nurses’ reports of missed care influenced their subjective workload ratings. The site was part of an academic medical center serving a primarily surgical neonatal population and therefore may not represent conditions in most NICUs. Replication in a larger and more diverse sample of units is needed to ascertain generalizability.
Conclusions
Nurse workload is associated with missed nursing care in NICUs. However, interventions focused exclusively on staffing ratios and/or acuity estimation may not substantially reduce missed nursing care. Subjective workload is an emerging aspect of nurse workload that is largely unmeasured yet presents new possibilities for tailored workload interventions.
eTable. Pairwise Tetrachoric Missed Care Item Correlations, ρ (SE)
References
- 1.Zachariah P, Furuya EY, Edwards J, et al. . Compliance with prevention practices and their association with central line-associated bloodstream infections in neonatal intensive care units. Am J Infect Control. 2014;42(8):847-851. doi: 10.1016/j.ajic.2014.04.020 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Schulman J, Stricof R, Stevens TP, et al. ; New York State Regional Perinatal Care Centers . Statewide NICU central-line-associated bloodstream infection rates decline after bundles and checklists. Pediatrics. 2011;127(3):436-444. doi: 10.1542/peds.2010-2873 [DOI] [PubMed] [Google Scholar]
- 3.Lau YY, Tay YY, Shah VA, Chang P, Loh KT. Maintaining optimal oxygen saturation in premature infants. Perm J. 2011;15(1):e108-e113. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Chow LC, Wright KW, Sola A; CSMC Oxygen Administration Study Group . Can changes in clinical practice decrease the incidence of severe retinopathy of prematurity in very low birth weight infants? Pediatrics. 2003;111(2):339-345. doi: 10.1542/peds.111.2.339 [DOI] [PubMed] [Google Scholar]
- 5.Sink DW, Hope SA, Hagadorn JI. Nurse:patient ratio and achievement of oxygen saturation goals in premature infants. Arch Dis Child Fetal Neonatal Ed. 2011;96(2):F93-F98. doi: 10.1136/adc.2009.178616 [DOI] [PubMed] [Google Scholar]
- 6.van Zanten HA, Pauws SC, Beks EC, Stenson BJ, Lopriore E, Te Pas AB. Improving manual oxygen titration in preterm infants by training and guideline implementation. Eur J Pediatr. 2017;176(1):99-107. doi: 10.1007/s00431-016-2811-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Morris R, MacNeela P, Scott A, Treacy P, Hyde A. Reconsidering the conceptualization of nursing workload: literature review. J Adv Nurs. 2007;57(5):463-471. doi: 10.1111/j.1365-2648.2006.04134.x [DOI] [PubMed] [Google Scholar]
- 8.Sherenian M, Profit J, Schmidt B, et al. . Nurse-to-patient ratios and neonatal outcomes: a brief systematic review. Neonatology. 2013;104(3):179-183. doi: 10.1159/000353458 [DOI] [PubMed] [Google Scholar]
- 9.Rogowski JA, Staiger D, Patrick T, Horbar J, Kenny M, Lake ET. Nurse staffing and NICU infection rates. JAMA Pediatr. 2013;167(5):444-450. doi: 10.1001/jamapediatrics.2013.18 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Lamy Filho F, Silva AA, Lopes JM, Lamy ZC, Simões VM, Dos Santos AM. Staff workload and adverse events during mechanical ventilation in neonatal intensive care units. J Pediatr (Rio J). 2011;87(6):487-492. doi: 10.2223/JPED.2140 [DOI] [PubMed] [Google Scholar]
- 11.Callaghan LA, Cartwright DW, O’Rourke P, Davies MW. Infant to staff ratios and risk of mortality in very low birthweight infants. Arch Dis Child Fetal Neonatal Ed. 2003;88(2):F94-F97. doi: 10.1136/fn.88.2.F94 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Hamilton KE, Redshaw ME, Tarnow-Mordi W. Nurse staffing in relation to risk-adjusted mortality in neonatal care. Arch Dis Child Fetal Neonatal Ed. 2007;92(2):F99-F103. doi: 10.1136/adc.2006.102988 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Watson SI, Arulampalam W, Petrou S, et al. ; Neonatal Data Analysis Unit (NDAU) and the Neonatal Economic, Staffing, and Clinical Outcomes Project (NESCOP) Group . The effects of a one-to-one nurse-to-patient ratio on the mortality rate in neonatal intensive care: a retrospective, longitudinal, population-based study. Arch Dis Child Fetal Neonatal Ed. 2016;101(3):F195-F200. doi: 10.1136/archdischild-2015-309435 [DOI] [PubMed] [Google Scholar]
- 14.Kalisch BJ, Landstrom GL, Hinshaw AS. Missed nursing care: a concept analysis. J Adv Nurs. 2009;65(7):1509-1517. doi: 10.1111/j.1365-2648.2009.05027.x [DOI] [PubMed] [Google Scholar]
- 15.Bragadóttir H, Kalisch BJ, Tryggvadóttir GB. Correlates and predictors of missed nursing care in hospitals. J Clin Nurs. 2017;26(11-12):1524-1534. doi: 10.1111/jocn.13449 [DOI] [PubMed] [Google Scholar]
- 16.Cho SH, Kim YS, Yeon KN, You SJ, Lee ID. Effects of increasing nurse staffing on missed nursing care. Int Nurs Rev. 2015;62(2):267-274. doi: 10.1111/inr.12173 [DOI] [PubMed] [Google Scholar]
- 17.Ausserhofer D, Zander B, Busse R, et al. ; RN4CAST Consortium . Prevalence, patterns and predictors of nursing care left undone in European hospitals: results from the multicountry cross-sectional RN4CAST study. BMJ Qual Saf. 2014;23(2):126-135. doi: 10.1136/bmjqs-2013-002318 [DOI] [PubMed] [Google Scholar]
- 18.Ball JE, Murrells T, Rafferty AM, Morrow E, Griffiths P. “Care left undone” during nursing shifts: associations with workload and perceived quality of care. BMJ Qual Saf. 2014;23(2):116-125. doi: 10.1136/bmjqs-2012-001767 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Friese CR, Kalisch BJ, Lee KH. Patterns and correlates of missed nursing care in inpatient oncology units. Cancer Nurs. 2013;36(6):E51-E57. doi: 10.1097/NCC.0b013e318275f552 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Kalisch BJ, Tschannen D, Lee KH. Do staffing levels predict missed nursing care? Int J Qual Health Care. 2011;23(3):302-308. doi: 10.1093/intqhc/mzr009 [DOI] [PubMed] [Google Scholar]
- 21.Dabney BW, Kalisch BJ. Nurse staffing levels and patient-reported missed nursing care. J Nurs Care Qual. 2015;30(4):306-312. doi: 10.1097/NCQ.0000000000000123 [DOI] [PubMed] [Google Scholar]
- 22.Lake ET, Staiger D, Edwards EM, Smith JG, Rogowski JA. Nursing care disparities in neonatal intensive care units. Health Serv Res. 2018;53(suppl 1):3007-3026. doi: 10.1111/1475-6773.12762 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Pillay T, Nightingale P, Owen S, Kirby D, Spencer A. Neonatal nurse staffing and delivery of clinical care in the SSBC Newborn Network. Arch Dis Child Fetal Neonatal Ed. 2012;97(3):F174-F178. doi: 10.1136/adc.2011.300224 [DOI] [PubMed] [Google Scholar]
- 24.Tubbs-Cooley HL, Pickler RH, Younger JB, Mark BA. A descriptive study of nurse-reported missed care in neonatal intensive care units. J Adv Nurs. 2015;71(4):813-824. doi: 10.1111/jan.12578 [DOI] [PubMed] [Google Scholar]
- 25.Tubbs-Cooley HL, Pickler RH, Meinzen-Derr JK. Missed oral feeding opportunities and preterm infants’ time to achieve full oral feedings and neonatal intensive care unit discharge. Am J Perinatol. 2015;32(1):1-8. doi: 10.1055/s-0034-1372426 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Carayon P, Gürses AP. A human factors engineering conceptual framework of nursing workload and patient safety in intensive care units. Intensive Crit Care Nurs. 2005;21(5):284-301. doi: 10.1016/j.iccn.2004.12.003 [DOI] [PubMed] [Google Scholar]
- 27.Daraiseh NM, Vidonish WP, Kiessling P, Lin L. Developing a patient classification system for a neonatal ICU. J Nurs Adm. 2016;46(12):636-641. doi: 10.1097/NNA.0000000000000419 [DOI] [PubMed] [Google Scholar]
- 28.Tubbs-Cooley HL, Mara CA, Carle AC, Gurses AP. The NASA Task Load Index as a measure of overall workload among neonatal, paediatric and adult intensive care nurses. Intensive Crit Care Nurs. 2018;46:64-69. doi: 10.1016/j.iccn.2018.01.004 [DOI] [PubMed] [Google Scholar]
- 29.Hoonakker P, Carayon P, Gurses A, et al. . Measuring workload of ICU nurses with a questionnaire survey: the NASA Task Load Index (TLX). IIE Trans Healthc Syst Eng. 2011;1(2):131-143. doi: 10.1080/19488300.2011.609524 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Hart SG, Staveland LE. Development of NASA-TLX (Task Load Index): results of empirical and theoretical research In: Hancock PA, Meshkati N, eds. Human Mental Workload. Amsterdam, the Netherlands: North Holland Press; 1988. doi: 10.1016/S0166-4115(08)62386-9 [DOI] [Google Scholar]
- 31.Hart SG. NASA Task Load Index (NASA-TLX): 20 years later. In: Human Factors and Ergonomics Society Annual Meeting Proceedings Thousand Oaks, CA: Sage Publications; 2006;50(9):904-908. [Google Scholar]
- 32.Hwang SL, Yau YJ, Chen JH, et al. . Predicting work performance in nuclear power plants. Saf Sci. 2008;46(7):1115-1124. doi: 10.1016/j.ssci.2007.06.005 [DOI] [Google Scholar]
- 33.Jacobson CJ Jr, Bolon S, Elder N, et al. . Temporal and subjective work demands in office-based patient care: an exploration of the dimensions of physician work intensity. Med Care. 2011;49(1):52-58. doi: 10.1097/MLR.0b013e3181f53542 [DOI] [PubMed] [Google Scholar]
- 34.Tubbs-Cooley HL, Pickler RH, Mark BA, Carle AC. A research protocol for testing relationships between nurse workload, missed nursing care and neonatal outcomes: the Neonatal Nursing Care Quality Study. J Adv Nurs. 2015;71(3):632-641. doi: 10.1111/jan.12507 [DOI] [PubMed] [Google Scholar]
- 35.Rasbash J, Goldstein H. Efficient analysis of mixed hierarchical and cross-classified random structures using a multilevel model. J Educ Behav Stat. 1994;19(4):337-350. doi: 10.3102/10769986019004337 [DOI] [Google Scholar]
- 36.Raudenbush SW. A crossed random effects model for unbalanced data with applications in cross-sectional and longitudinal research. J Educ Behav Stat. 1993;18(4):321-349. doi: 10.3102/10769986018004321 [DOI] [Google Scholar]
- 37.Bhat CR. A multi-level cross-classified model for discrete response variables. Transp Res, Part B: Methodol. 2000;34(7):567-582. doi: 10.1016/S0191-2615(99)00038-7 [DOI] [Google Scholar]
- 38.Browne WJ, Goldstein H, Rasbash H. Multiple membership multiple classification (MMMC) models. Stat Model. 2001;1(2):103-124. doi: 10.1177/1471082X0100100202 [DOI] [Google Scholar]
- 39.Leckie G. Cross-classified multilevel models using Stata: how important are schools and neighbourhoods for students’ educational attainment? In: Garson D, ed. Hierarchical Linear Modeling: Guide and Applications. Thousand Oaks, CA: SAGE Publications Ltd; 2012:311-332. [Google Scholar]
- 40.Browne WJ. An illustration of the use of reparameterisation methods for improving MCMC efficiency in crossed random effect models. Multilevel Modelling Newsletter. 2004;16(1):13-25. [Google Scholar]
- 41.Leckie G, Charlton C. runmlwin: A program to run the MLwiN multilevel modelling software from within Stata. J Stat Softw. 2013;52(11):1-40.23761062 [Google Scholar]
- 42.Goldstein H, Rasbash J, Plewis I, et al. . A User’s Guide to MLwiN. 2nd ed London, UK: Institute of Education; 2000:286. [Google Scholar]
- 43.Benjamini Y, Yekutieli D. The control of the false discovery rate in multiple testing under dependency. Ann Stat. 2001;29(4):1165-1188. doi: 10.1214/aos/1013699998 [DOI] [Google Scholar]
- 44.Lopez KD, Gerling GJ, Cary MP, Kanak MF. Cognitive work analysis to evaluate the problem of patient falls in an inpatient setting. J Am Med Inform Assoc. 2010;17(3):313-321. doi: 10.1136/jamia.2009.000422 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eTable. Pairwise Tetrachoric Missed Care Item Correlations, ρ (SE)