Abstract
Background:
Physiologic monitor alarms occur at high rates in children’s hospitals; ≤1% are actionable. The burden of alarms has implications for patient safety and is challenging to measure directly. Nurse workload, measured using a version of the National Aeronautics and Space Administration Task Load Index (NASA-TLX) validated among nurses, is a useful indicator of work burden which has been associated with patient outcomes. A recent study demonstrated that 5-point increases in NASA-TLX were associated with a 22% increased risk in missed nursing care.
Objective:
To measure the relationship between alarm count and nurse workload using the NASA-TLX.
Methods:
We conducted a repeated cross-sectional study of pediatric nurses in a tertiary care children’s hospital to measure the association between NASA-TLX workload evaluations (using the nurse-validated scale) and alarm count in the 2 hours preceding NASA-TLX administration. Using a multivariable mixed effects regression accounting for nurse-level clustering, we modeled the adjusted association of alarm count on workload.
Results:
The NASA-TLX was assessed in 26 nurses during 394 nursing shifts over a 2-month period. In adjusted regression models, experiencing >40 alarms in the preceding 2 hours was associated with a 5.5 point increase (95% CI 5.2 to 5.7, p<0.001) in subjective workload.
Conclusion:
Alarm count in the preceding 2-hours is associated with a significant increase in subjective nurse workload that exceeds the threshold associated with increased risk of missed nursing care and potential patient harm.
Introduction
High nurse workload has been associated with adverse patient outcomes including increased mortality,1–3 hospital acquired harm,4 nurse-patient miscommunication,5 and missed delivery of nursing care.6 Workload is generally conceptualized as the ratio of physical and cognitive demands to an individual’s available resources7,8 and nursing workload has been understood as the “performance required to carry out nursing activities.”9,10 Originally developed for the aerospace industry, the National Aeronautics and Space Administration Task Load Index (NASA-TLX) is a validated measure of subjective workload that is sensitive to individual differences used to assess work demands.11,12 The NASA-TLX has been applied to nursing6,7,10,13 and a shortened 4-dimension scale comprised of mental demand, physical demand, temporal demand, and effort has been validated to specifically measure nurse-specific workload.12 Recent work demonstrated that each 5 point increase in nurse workload on the nurse-specific, 4-dimension NASA-TLX was associated with a 22% increase in the likelihood of self-reported omission of patient care tasks such as double-checking high-risk medications and adhering to infection prevention bundles during the same shift.6
Physiologic monitor alarms may contribute to nurse subjective workload. Physiologic monitoring of vital sign parameters (including heart rate, respiratory rate, and oxygen saturation) is commonplace on pediatric units, with estimates of up to 48% of non-intensive care unit pediatric patients receiving continuous monitoring of vital signs.14 Abnormal vital signs trigger an alarm in the patient room and, in many institutions, relay a message to the bedside nurse’s institutional mobile telephone. Pediatric nurses are responsible for responding to up to 155 alarms per monitored patient per day.14,15 Analysis of alarms indicates that only approximately 0.5% - 1% of alarms are considered actionable or informative on pediatric wards.16,17 High alarm count is associated with slower nurse response time to alarms,16 but the broader association between alarms and nurse workload has not been characterized in clinical practice. The objective of this project was to directly measure the relationship between alarm count and subjective nurse workload using the NASA-TLX.
Methods
Patient Safety Learning Lab
This work was undertaken by the Agency for Healthcare Research and Quality (AHRQ) funded Patient Safety Learning Laboratory (PSLL) at a tertiary children’s hospital. A primary aim of the PSLL is to re-engineer the system of monitoring hospitalized children within our hospital, aiming to reduce non-informative alarms and accelerate nurse response to critical events. Two general (non-intensive care) pediatric inpatient unitsare participating in the PSLL’s longitudinal quality improvement work. Evaluation of the relationship between alarm count and nurse workload was part of the initial problem analysis phase of this improvement initiative.
The hospital Committees for the Protection of Human Subjects (the IRB) determined that the PSLL’s problem analysis portfolio (which included this project) was consistent with quality improvement activity and did not meet criteria for human subjects’ research.
Participants and Setting
Nurses on the two participating general pediatric inpatient units participated. All patient rooms include General Electric Dash monitors for physiologic monitoring. Default alarm thresholds for physiologic parameters are defined by age, however can be adjusted by nurses to account for patient physiology. Monitor alarms are relayed to bedside nurses’ institutional mobile telephones (Ascom d62; AscomHolding AG). On the units we studied, approximately 40% of patients are continuously monitored.
Measuring nurses’ workload
We prospectively measured nurse subjective workload using the NASA-TLX (Figure 1) in a cohort of nurses whose workload we could assess repeatedly across a wide range of alarm exposure rates. This allowed us to use analytic methods designed to assess both within-nurse and between-nurse differences in workload at different alarm count exposure rates.11 The NASA-TLX provides a scale to quantifying workload at the task or job level, while accounting for individual differences.11
Figure 1:
Copy of NASA-TLX paper survey provided to nurses
We approached nurses from the 2 general pediatric units partnering with the PSLL for participation in workload evaluations. An in-person data collector obtained participating nurses’ demographic information and guided them through completing their first NASA-TLX assessments. Subsequently, nurses were asked to complete the NASA-TLX at least once per shift between July 8, 2019 and August 31, 2019, using personalized paper forms available in envelopes with their names on them in a bin on their unit. In order to measure workload at all hours and days of the week, we reviewed nursing assignments approximately one week in advance and, based on these advanced assignments, programmed a system to automatically deliver personalized text messages to each assigned nurse during their shifts. The personalized text messages requested that nurses complete a workload evaluation, reflecting on the past two-hours of the shift. We aimed to obtain at least 10 evaluations per nurse to allow for adjustment based on within-in nurse differences.
We collected data on all 6 dimensions of workload that comprise the original NASA-TLX designed for the aerospace industry. However, in pediatric hospital settings like ours, the 4-dimension version of the NASA-TLX validated in pediatric nurses is more appropriate than the original version.10 This modified version of the NASA-TLX evaluates the following dimensions: mental demand, physical demand, temporal demand (i.e. time pressure), and effort required in the preceding 2 hours.6,10 In our analysis we utilized the 4-dimension instrument; we calculated the sum of scores in these 4 dimensions and scaled to 100 to facilitate interpretation of the results and to inform subsequent analyses.6 We performed sensitivity analysis using the original 6-dimension scale (scaled to 100).
In addition to nurse demographics, we collected information about the work environment and patient factors that potentially contribute to subjective workload, including unit, time of shift (day vs. night), time within a shift the NASA-TLX was completed, nurse to patient ratio, patient acuity, the number of assigned patients receiving physiologic monitoring, and alarm exposure in the preceding 2 hours. Nurses recorded the time of NASA-TLX completion and listed the medical record numbers of all patients they were caring for at the time of NASA-TLX completion. Patient acuity was evaluated using the Pediatric Rothman Index (PRI) score,18 a measure of projected probability of clinical deterioration computed based on vital signs, lab results, and nursing assessments for each patient. At the time of data collection, PRI scores were embedded into the EHR. PRI scores <65 denoted patients at risk for clinical deterioration or transfer to intensive-care unit;19 scores were dichotomized such that patients with PRI scores <65 were categorized as having “high acuity” and patients with PRI scores ≥65 were categorized as “normal or lower acuity.”
Alarm count
We extracted monitor alarm counts (the absolute number of abnormal vital signs that triggered a monitor to alarm) from the hospital’s clinical data warehouse. We paired NASA-TLX assessments with the total count of alarms for the nurse’s assigned patients during the preceding 2 hours (the period evaluated in the NASA-TLX). We also included alarms from patients whom nurses were only covering for part of the 2 hour window (e.g. when covering another nurse’s patients because the other nurse is on a break).
Employing an approach utilized in our prior work evaluating alarm-count and response time, we evaluated alarm counts categorically because we did not anticipate a linear relationship between alarms and workload.16 When infrequent, nurses are likely able to respond to alarms, whereas at high levels we hypothesized they would be impose additional workload burden. We visually examined a locally weighted scatterplot smoother (LOWESS) to inform alarm count categorization. We divided alarm counts over the preceding 2 hours into 3 categories: 0 to 5 alarms to represent a low/typical alarm rate (50% of time periods), 6 to 39 alarms to represent an elevated alarm rate (50 to 90%), and ≥40 alarms to represent a very high alarm rate (the top 10% in terms of alarm frequency).
Statistical Analysis
To model this dynamic system in which numbers of repeat observations varied for each nurse, we utilized a hierarchical linear mixed-effects regression model that accounted for clustering of observations within nurse and clustering of nurses within units by including unit- and nurse-specific random effects. To select model variables, we examined the association of NASA-TLX workload score (the primary outcome) with each of the nurse, patient, and work environment variables as fixed effects in bivariate analysis and performed a Wald test to evaluate the linear hypothesis. Variables with p≤0.20 in these bivariate, unadjusted analysis were included in the subsequent multivariable model of scaled, composite workload scores. Because alarm counts and the number of monitored patients being cared for measured overlapping constructs, we included only alarm counts in our multivariable analysis. Informed by the literature and awareness of the complex relationships between staffing ratios and patient acuity from our prior work20 and the work of others,6 we included a 2-way interaction between patient acuity and nurse to patient ratio in our model. We reported scaled, adjusted estimates of workload using predictive margins for each alarm count category.
We used REDCap hosted by our institution for data management21,22 and Stata version 16.1 (StataCorp LLC) for statistical analysis.
Results:
Association between alarm count and nurses’ workload
We measured the subjective workload of 26 nurses 394 times using the NASA-TLX. After guiding nurses through completion of initial NASA-TLX evaluations in person (n=39), we transitioned to delivering automated text messages to prompt NASA-TLX completion. We delivered 586 automated messages and subsequently collected 355 workload evaluations. Our response rate (based on responses to automated text messages) was 61%. In attempting to understand non-response, it became clear that the pre-shift assignments that we based our automated messaging strategy on were subject to change due to nurses being called off or calling out of a shift, reassignments of nurses to different units, or assignment to serve as charge nurse. Nonetheless, participating nurses completed a median of 11 NASA-TLX evaluations (IQR: 6 to 17, range: 1 to 35) during the study period. Nurses had a median 3 years of nursing experience (IQR: 2 to 5 years) and 6 nurses (23%) were in their first year of practice. Most nurses completed at least one workload evaluation during a nightshift (22 nurses, 85%) and 25 nurses (96%) evaluated a time period in which they cared for ≥ 1 high acuity patient (as defined as having Rothman score <65).
In terms of shift characteristics (Table 1), 46% of NASA-TLX evaluations were completed during day shifts, and 54% of evaluations during night shifts. In 63% of assessed shifts nurses cared for high acuity patients. Nurses cared for a median 4 patients (IQR: 3 to 4, range 2 to 5). Nurses cared for at least one patient receiving continuous monitoring in 74% of shifts. Nurses received a median of 6 alarms (IQR: 0 to 21) in the 2 hours prior to NASA-TLX completion. The median reported NASA-TLX workload score was 68.6 (IQR: 57.3 to 79.0).
Table 1: Shift Characteristics.
Descriptive characteristics of the shifts in which NASA-TLX (n=394) was completed to assess workload. Nurses completed up to NASA-TLX workload evaluations per shift.
| Workload Evaluations No. (%) |
||||
|---|---|---|---|---|
| Total (N) | 394 | |||
| Years of nursing experience | ||||
| <2 years | 122 (31%) | |||
| 2 to 5 years | 215 (55%) | |||
| >5 years | 57 (15%) | |||
| Unit | ||||
| Unit 1 | 124 (32%) | |||
| Unit 2 | 270 (69%) | |||
| Shift type | ||||
| Day | 180 (46%) | |||
| Night | 214 (54%) | |||
| Time (within a shift) of completion | ||||
| <3 hours | 130 (33%) | |||
| 3 to 7 hours | 105 (27%) | |||
| 8 to 12 hours | 159 (40%) | |||
| No. of patients cared for | ||||
| 1 to 3 patients | 171 (43%) | |||
| 4 or 5 patients | 223 (57%) | |||
| No. of monitored patients cared for | ||||
| None | 103 (26%) | |||
| 1 patient | 116 (29%) | |||
| 2 to 4 patients | 175 (44%) | |||
| Caring for ≥ 1 high acuity patient | 249 (63%) | |||
| NASA-TLX Workload Score | ||||
| 0 to 50 | 73 (19%) | |||
| 51 to 75 | 186 (47%) | |||
| 75 to 100 | 135 (34%) | |||
| No. of Alarms in the preceding two hours | ||||
| 0 to 5 alarms | 195 (50%) | |||
| 6 to 40 alarms | 159 (40%) | |||
| >40 alarms | 40 (10%) | |||
In unadjusted analyses, all variables met a priori criteria of p≤0.2 for inclusion in the multivariable model (Table 2). In multivariable mixed effects modeling, after controlling for other contributors to workload, experiencing a very high alarm rate increased workload scores by 5.5 points (95% CI: 5.2 to 5.7). Nurses with 2–5 years’ experience, mid-shift (hours 3 to 7 of a 12-hour shift), increasing from 1–3 low acuity patients to 4–5 low acuity patients, and caring for at least one patient with high acuity (among nurses caring for 1–3 patients) were associated with increased workload. In Table 3, we modeled different conditions to illustrate the effect of alarms on nurse workload using predictive margins.
Table 2:
Predicted change in NASA-TLX workload score (scaled to 100)
| Unadjusted | Adjusted | |||||||
|---|---|---|---|---|---|---|---|---|
| predicted change (95% CI) | p-value | predicted change (95% CI) |
p-value | |||||
| category | composite | category | composite | |||||
| Alarms in the preceding 2 hours | <0.001 | <0.001 | ||||||
| 0 to 5 alarms | reference | reference | ||||||
| 6 to 39 alarms | +6.9 (3.1 to 10.7) | <0.001 | +1.9 (0.9 to 2.9) | <0.001 | ||||
| ≥40 alarms | +11.8 (5.6 to 17.9) | <0.001 | +5.5 (5.2 to 5.7) | <0.001 | ||||
| Years of nursing experience | <0.001 | <0.001 | ||||||
| <2 years | reference | reference | ||||||
| 2 to 5 years | +15.5 (11.9 to 19.1) | <0.001 | +6.4 (5.4 to 7.4) | <0.001 | ||||
| >5 years | −6.4 (−11.5 to −1.3) | 0.01 | −12.1 (−17.2 to −7.0) | <0.001 | ||||
| Unit | <0.001 | Random effect | ||||||
| Unit 1 | reference | |||||||
| Unit 2 | +15.0 (11.3 to 18.6) | <0.001 | ||||||
| Shift type | 0.4 | 0.09 | ||||||
| Day | reference | reference | ||||||
| Night | +1.6 (−2.1 to 5.2) | 0.4 | −4.8 (−10.2 to 0.7) | 0.09 | ||||
| Time (within a shift) | 0.003 | 0.9 | ||||||
| <3 hours | reference | reference | ||||||
| 3 to 7 hours | +8.2 (3.5 to 13.0) | 0.001 | +3.5 (0.5 to 6.5) | 0.02 | ||||
| 8 to 12 hours | +3.9 (−0.4 to 8.1) | 0.07 | +0.5 (−6.7 to 7.6) | 0.9 | ||||
| No. of monitored patients | <0.001 | not included | ||||||
| None | reference | |||||||
| 1 patient | +6.1 (1.5 to 10.8) | 0.01 | ||||||
| 2 to 4 patients | +14.3 (10.0 to 18.5) | <0.001 | ||||||
| No. and acuity of patients | <0.001 | 0.03 | ||||||
| 1 to 3 patients, none high acuity | reference | reference | ||||||
| 1 to 3 patients, ≥1 with high acuity | +13.1 (7.6 to 18.6) | <0.001 | +8.1 (1.6 to 14.7) | 0.02 | ||||
| 4 to 5 patients, none high acuity | +8.5 (2.6 to 14.3) | 0.005 | +7.8 (0.6 to 15.1) | 0.03 | ||||
| 4 to 5 patients, ≥1 with high acuity | +12.1 (6.9 to 17.3) | <0.001 | +5.7 (2.0 to 9.4) | 0.002 | ||||
Table 3:
Adjusted Estimates of Workload Scores (scaled to 100), by Alarm Frequency
| 0 to 5 Alarms | 6 to 39 Alarms | ≥40 Alarms | ||||
|---|---|---|---|---|---|---|
| predicted score (95% CI) | predicted score (95% CI) | predicted score (95% CI) | ||||
| Years of nursing experience | ||||||
| <2 years | 61.0 (51.7 to 70.3) | 62.9 (52.5 to 73.2) | 66.4 (57.4 to 75.5) | |||
| 2 to 5 years | 67.3 (58.3 to 76.4) | 69.2 (59.2 to 79.3) | 72.8 (64.1 to 81.6) | |||
| >5 years | 49.5 (36.0 to 63.0) | 51.4 (36.8 to 65.9) | 54.9 (41.7 to 68.2) | |||
| Shift type | ||||||
| day | 64.3 (57.8 to 70.9) | 66.3 (58.7 to 73.8) | 69.8 (63.6 to 76.1) | |||
| night | 61.4 (49.0 to 73.9) | 63.3 (49.8 to 76.9) | 66.9 (54.7 to 79.2) | |||
| Time (within a shift) of completion | ||||||
| <3 hours | 61.4 (55.7 to 67.0) | 63.3 (56.6 to 70.0) | 66.9 (61.4 to 72.3) | |||
| 3 to 7 hours | 66.2 (57.7 to 74.7) | 68.1 (58.6 to 77.7) | 71.7 (63.4 to 79.9) | |||
| 8 to 12 hours | 61.6 (47.7 to 75.6) | 63.6 (48.6 to 78.5) | 67.1 (53.5 to 80.8) | |||
| Patients and Acuity | ||||||
| 1 to 3 lower acuity patients | 57.5 (44.3 to 70.7) | 59.4 (45.2 to 73.6) | 63.0 (50.0 to 75.9) | |||
| 1 to 3 patients, including ≥ 1 with high acuity | 64.6 (57.4 to 71.8) | 66.5 (58.3 to 74.8) | 70.1 (63.2 to 77.0) | |||
| 4 or 5 lower acuity patients | 64.4 (57.3 to 71.4) | 66.3 (58.2 to 74.4) | 69.8 (63.0 to 76.7) | |||
| 4 or 5 patients, including ≥ 1 with high acuity | 63.0 (51.6 to 74.5) | 64.9 (52.4 to 77.4) | 68.5 (57.3 to 79.7) |
Sensitivity analysis using the 6-dimension NASA-TLX demonstrated similar results. In multivariable models using the six-domain scale, high alarm count increase workload scores by 5.8 points (95% CI: 5.7 to 5.8). While the coefficient values for the other covariates changed slightly, none changed in statistical significance or direction.
Discussion
We sought to evaluate the relationship between alarm count and subjective workload using the NASA-TLX. Accounting for other contributors to work load, high alarm count (≥40 alarms) was associated with a >5-point increase in subjective workload. The association between high alarm counts on workload is clinically significant; in terms of magnitude it is similar to the increase in workload experienced when caring for 4 to 5 patients (as compared with caring for 1 to 3 patients). This finding is consistent with a simulation based study, which found that reduction in alarms resulted in a 5 point decrease in NASA-TLX workload scores.23
Our findings support the negative impact of high alarm count on nurses’ provision of care. In work by Tubbs-Cooley et al each 5-point increase in nurse subjective workload was associated with a 22% increased risk in self-reported missed nursing care. Both increased alarm count16 and subjective workload23 have been associated with slowed alarm response time. Numerous other adverse outcomes are associated with high alarm burden including poor nurse-patient communication,5 missed delivery of nursing care,6 hospital acquired infection,4 and increased mortality.1–3 More broadly, the inverse relationship between increased workload and task performance has been established in the human factors literature.24,25
Multiple factors influence workload and thus (as highlighted in Table 3) the increased burden imposed by high alarm count may have different implications in different contexts. For example, nurses with 2–5 years of experience report high levels of subjective workload at low alarm counts compared to nurses with >5 years of experience. The human factors literature suggests that professional experience may confer “attentional spare margin,” such that individual with experience have additional “spare capacity” that reduces the workload associated with tasks.26 Moreover “limited capacity” model posits that as individuals approach their capacity “threshold” (the point at which additional task-demands cause performance to decline), their perceived workload increases.27–30 These theories suggest that for nurses without spare capacity, who are already experiencing high levels of workload, the addition of alarm-associated workload may have serious implications for patient safety. Of note, nurses with 2–5 years of experience report higher levels of subjective workload than nurses with <2 years of experience. A variety of factors may contribute to the lower subjective workload of new nurses including reporting bias, additional supervision and support given to new nurses (generally by nurses with 2–5 years of experience), and/or reduced or lower complexity patient assignments. The impact of additional workload on nurse performance when workload is already high (i.e. the nurse with 2–5 years’ experience) may differ from the performance impact of additional workload for a more experienced nurse whose baseline workload is lower. Similarly, there may be particularly vulnerable moments within a clinical shift when an interruption from an uninformative alarm is more burdensome. For instance observational work has demonstrated the association of phone interruptions with medication administration errors.20 Among the multitude of complex, interacting factors that influence workload, alarm-counts are modifiable. Reducing non-actionable alarms may be one of the most modifiable, pragmatic targets for improving nurse workload.
To our knowledge this is the first study to evaluate the relationship between alarms count and subjective nursing workload in clinical practice. Our reproducible approach allows for quantitative evaluation of the consequences of alarm fatigue. As part of our broader quality improvement work, workload will serve as a proximal outcome by which to evaluate future interventions focused on improving monitoring systems.
These findings must be contextualized within the limitations of our approach. We surveyed a limited number of nurses within a single, pediatric hospital. Intuitional policies and practices may influence both the frequency and experience of alarms. We used automated text messages to request nurses complete workload evaluations. Delivery of automated text messages was scheduled based on pre-shift assignments. While this approach allowed us to evaluate workload throughout shifts, we are unable to systematically evaluate why nurses did or did not complete workload evaluations. Some non-response was due to changes in staffing. However because nurses were prompted to complete the NASA-TLX during their clinical shifts, at times shift characteristics may have influenced completion. Busier nurses may have delayed or failed to complete evaluations of workload. Alternatively, nurses may have been eager to report workload during particularly challenging times. By obtaining frequent, repeated evaluations of workload on a routine basis we aimed to overcome these sampling challenges. We adopted a human factors approach that accounted for nurse, unit, and patient level factors; however, because nurses did not cross units, there may be unit-level factors we were unable to account for in our analysis. Finally, the literature with regard to workload and expected workload of nurses is emerging. The original NASA-TLX measures 6 dimensions of workload; for both validity and interpretability within the existing nursing literature, we report scores from a 4-dimension version that has been validated among nurses. Sensitivity analysis using the 6-component NASA-TLX found similar results.
Conclusion:
Increased physiologic monitor alarm exposure was associated with clinically and statistically significant increases in subjective nurse workload that may lead to missed nursing care tasks and patient harm. Reducing non-actionable physiologic monitor alarm burden should be a high priority to improve patient safety.
Acknowledgments
Funding Source: This project was supported by grant number R18HS026620 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.
Footnotes
Financial Disclosure: The authors have no financial relationships relevant to this article.
Conflict of Interest: The authors have no potential conflicts of interest to disclose.
References
- 1.Filho FL, da Silva AM, Lopes JMA, Lamy ZC, Simoes VMF, dos Santos AM. Staff workload and adverse events during mechanical ventilation in neonatal intensive care units. J Pediatr (Rio J). 2011;87(6):487–492. doi: 10.2223/JPED.2140 [DOI] [PubMed] [Google Scholar]
- 2.Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA. 2002;288(16):1987–1993. doi: 10.1001/jama.288.16.1987 [DOI] [PubMed] [Google Scholar]
- 3.Kiekkas P, Sakellaropoulos GC, Brokalaki H, et al. Association between nursing workload and mortality of intensive care unit patients. Journal of Nursing Scholarship. 2008;40(4):385–390. doi: 10.1111/j.1547-5069.2008.00254.x [DOI] [PubMed] [Google Scholar]
- 4.Rogowski JA, Staiger D, Patrick T, Horbar J, Kenny M, Lake ET. Nurse staffing and NICU infection rates. JAMA Pediatr. 2013;167(5):444. doi: 10.1001/jamapediatrics.2013.18 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Carayon P, Gurses AP. Nursing workload and patient safety—A human factors engineering perspective. In: Hughes R, ed. Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Agency for Healthcare Research and Quality (US); 2008:Chapter 30.Accessed March 1, 2020. https://www.ncbi.nlm.nih.gov/books/NBK2657/ [PubMed] [Google Scholar]
- 6.Tubbs-Cooley HL, Mara CA, Carle AC, Mark BA, Pickler RH. Association of nurse workload with missed nursing care in the neonatal intensive care unit. JAMA Pediatrics. 2019;173(1):44. doi: 10.1001/jamapediatrics.2018.3619 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Hoonakker P, Carayon P, Gurses AP, et al. Measuring workload of ICU nurses with a questionnaire survey: the NASA Task Load Index (TLX). IIE Transactions on Healthcare Systems Engineering. 2011;1(2):131–143. doi: 10.1080/19488300.2011.609524 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Xie B, Salvendy G. Review and reappraisal of modelling and predicting mental workload in single-and multi-task environments. Work & Stress. 2000;14(1):74–99. doi: 10.1080/026783700417249 [DOI] [Google Scholar]
- 9.Morris R, MacNeela P, Scott A, Treacy P, Hyde A. Reconsidering the conceptualization of nursing workload: literature review. J Adv Nurs. 2007;57(5):463–471. doi: 10.1111/j.1365-2648.2006.04134.x [DOI] [PubMed] [Google Scholar]
- 10.Tubbs-Cooley HL, Mara CA, Carle AC, Gurses AP. The NASA Task Load Index as a measure of overall workload among neonatal, paediatric and adult intensive care nurses. Intensive and Critical Care Nursing. 2018;46:64–69. doi: 10.1016/j.iccn.2018.01.004 [DOI] [PubMed] [Google Scholar]
- 11.Hart SG, Staveland LE. Development of NASA-TLX (Task load index): Results of empirical and theoretical research. In: Advances in Psychology. Vol 52. Elsevier; 1988:139–183. doi: 10.1016/S0166-4115(08)62386-9 [DOI] [Google Scholar]
- 12.Rubio S, Diaz E, Martin J, Puente JM. Evaluation of subjective mental workload: a comparison of SWAT, NASA_TLX, and Workload profile methods. Applied Psychology. 2004;53(1):61–86. doi: 10.1111/j.1464-0597.2004.00161.x [DOI] [Google Scholar]
- 13.Colligan L, Potts HWW, Finn CT, Sinkin RA. Cognitive workload changes for nurses transitioning from a legacy system with paper documentation to a commercial electronic health record. International Journal of Medical Informatics. 2015;84(7):469–476. doi: 10.1016/j.ijmedinf.2015.03.003 [DOI] [PubMed] [Google Scholar]
- 14.Schondelmeyer AC, Brady PW, Goel VV, et al. Physiologic monitor alarm rates at 5 children’s hospitals. J Hosp Med. Published online April25, 2018. doi: 10.12788/jhm.2918 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Schondelmeyer AC, Bonafide CP, Goel VV, et al. The frequency of physiologic monitor alarms in a children’s hospital: Monitor alarms in a children’s Hospital. J Hosp Med. 2016;11(11):796–798. doi: 10.1002/jhm.2612 [DOI] [PubMed] [Google Scholar]
- 16.Bonafide CP, Lin R, Zander M, et al. Association between exposure to nonactionable physiologic monitor alarms and response time in a children’s hospital: Monitor alarms and response time. J Hosp Med. 2015;10(6):345–351. doi: 10.1002/jhm.2331 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Bonafide CP, Localio AR, Holmes JH, et al. Video analysis of factors associated with response time to physiologic monitor alarms in a children’s hospital. JAMA Pediatr. 2017;171(6):524. doi: 10.1001/jamapediatrics.2016.5123 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Rothman MJ, Tepas JJ, Nowalk AJ, et al. Development and validation of a continuously age-adjusted measure of patient condition for hospitalized children using the electronic medical record. Journal of Biomedical Informatics. 2017;66:180–193. doi: 10.1016/j.jbi.2016.12.013 [DOI] [PubMed] [Google Scholar]
- 19.da Silva YS, Fiedor Hamilton M, Horvat C, et al. Evaluation of electronic medical record vital sign data versus a commercially available acuity score in predicting need for critical intervention at a tertiary children’s hospital. Pediatric Critical Care Medicine. 2015;16(7):644–651. doi: 10.1097/PCC.0000000000000444 [DOI] [PubMed] [Google Scholar]
- 20.Bonafide CP, Miller JM, Localio AR, et al. Association between mobile telephone interruptions and medication administration errors in a pediatric intensive care unit. JAMA Pediatr. 2020;174(2):162. doi: 10.1001/jamapediatrics.2019.5001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics. 2009;42(2):377–381. doi: 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Harris PA, Taylor R, Minor BL, et al. The REDCap consortium: Building an international community of software platform partners. Journal of Biomedical Informatics. 2019;95:103208. doi: 10.1016/j.jbi.2019.103208 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Shanmugham M, Strawderman L, Babski-Reeves K, Bian L. Alarm-related workload in default and modified alarm settings and the relationship between alarm workload, alarm response rate, and care provider experience: quantification and comparison study. JMIR Hum Factors. 2018;5(4):e11704. doi: 10.2196/11704 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Cummings ML, Nehme CE. Modeling the impact of workload in network centric supervisory control settings. In: Kornguth S, Steinberg R, Matthews M, eds. Neurocognitive and Physiological Factors during High-Tempo Operations. Ashgate; 2010:23–40. [Google Scholar]
- 25.Hebb DO. Drives and the C.N.S. (conceptual nervous system). Psychological Review. 1955;62(4):243–254. doi: 10.1037/h0041823 [DOI] [PubMed] [Google Scholar]
- 26.Leedal JM, Smith AF. Methodological approaches to anaesthetists’ workload in the operating theatre. British Journal of Anaesthesia. 2005;94(6):702–709. doi: 10.1093/bja/aei131 [DOI] [PubMed] [Google Scholar]
- 27.Kahneman D. Attention and Effort. Englewood Cliffs, NJ: Prentice Hall; 1973. [Google Scholar]
- 28.Hockey GRJ. Compensatory control in the regulation of human performance under stress and high workload: A cognitive-energetical framework. Biological Psychology. 1997;45(1–3):73–93. doi: 10.1016/S0301-0511(96)05223-4 [DOI] [PubMed] [Google Scholar]
- 29.Mallat C, Cegarra J, Calmettes C, Capa RL. A curvilinear effect of mental workload on mental effort and behavioral adaptability: An approach with the pre-ejection period. Hum Factors. 2020;62(6):928–939. doi: 10.1177/0018720819855919 [DOI] [PubMed] [Google Scholar]
- 30.Hancock PA, Szalma JL, eds. Performance under Stress: Human Factors in Defence. Burlington, VT: Ashgate; 2008. [Google Scholar]