Staffing levels and nursing‐sensitive patient outcomes: Umbrella review and qualitative study

Abstract

Objective

To derive a comprehensive list of nursing‐sensitive patient outcomes (NSPOs) from published research on nurse staffing levels and from expert opinion.

Data Sources/Study Setting

Published literature reviews and their primary studies analyzing the link between nurse staffing levels and NSPOs and interviews with 16 experts on nursing care.

Study Design

Umbrella review and expert interviews.

Data Collection/Extraction Methods

We screened three electronic databases for literature reviews on the association between nurse staffing levels and NSPOs. After screening 430 potentially relevant records, we included 15 literature reviews, derived a list of 22 unique NSPOs from them, and ranked these in a systematic fashion according to the strength of evidence existing for their association with nurse staffing. We extended this list of NSPOs based on data from expert interviews.

Principal Findings

Of the 22 NSPOs discussed in the 15 included literature reviews, we rated the strength of evidence for four as high, for five as moderate, and for 13 outcomes as low. Four additional NSPOs that have not been considered in literature were identified through expert interviews.

Conclusions

We identified strong evidence for a significant association between nurse staffing levels and NSPOs. Our results may guide researchers in selecting NSPOs they might wish to prioritize in future studies. In particular, rarely studied NSPOs as well as NSPOs that were only identified through expert interviews but have not been considered in literature so far should be subject to further research.

What is already known on this topic

• Many studies have examined the association between nurse staffing levels and nursing‐sensitive patient outcomes (NSPOs).

• There is a high variability in methods and measurement approaches, and inconsistencies in results across primary studies.

What This Study Adds

• We conducted an umbrella review on the association between nurse staffing levels and NSPOs in adult acute care covering 15 included literature reviews and derived a comprehensive list of 22 NSPOs.

• We provide an overview of which outcomes are used how often and used a pre‐defined, systematic approach to rate the strength of evidence existing for each NSPO’s association with nurse staffing.

• In addition, using expert interviews, we identified four NSPOs not yet mentioned in existing literature.

1. INTRODUCTION

Nurses constitute the largest occupational group in hospitals, delivering the highest amount of bedside patient care. ¹ At the same time, hospital policies in a number of countries, such as the US, Canada, and Germany, have included reductions in nurse staffing levels, contributing to a deterioration in working conditions and potentially endangering quality of care. ² Minimum staffing regulations have been widely discussed and partially implemented as a way for regulators to affect working conditions for nurses and the quality of care. ³ To underline the importance of adequate staffing levels and for designing and evaluating staffing regulations, it is crucial to depict the link between nurse staffing levels and patient outcomes that are sensitive to nursing care, otherwise known as nursing‐sensitive patient outcomes (NSPOs).

Since the publication of the seminal studies by Needleman et al ⁴ and Aiken, ⁵ many more studies have examined the association between nurse staffing levels and a broad range of NSPOs, applying highly heterogenous methods and mostly finding evidence of a systematic relationship. Starting with the oft‐cited systematic literature review by Kane et al, ⁶ several literature reviews have also been published on the topic. One study synthesized data from literature reviews related to this link and found high variability in methods and measurement approaches, and inconsistencies in results across primary studies. ⁷ As of yet, however, there is no overview of which NSPOs are used how often or of the strength of evidence existing for each NSPO’s empirically observed association with nurse staffing. In addition, path dependency may be an issue in existing research—that is, new studies may be relying on previously used NSPOs, leaving other relevant NSPOs undetected.

To address these gaps in the research, we conducted an umbrella review—a systematic review of published literature reviews—on the association between nurse staffing levels and NSPOs in adult acute care. ⁸ We systematically rated the methodological quality of the included literature reviews and extracted information on each literature review's scope. Based on our findings, we derived a comprehensive list of NSPOs and used a pre‐defined, systematic approach to rate the strength of evidence existing for each NSPO’s association with nurse staffing. Additionally, we conducted expert interviews to expand our list of NSPOs, accounting for NSPOs not yet mentioned in existing literature.

2. METHODS

2.1. Umbrella review

We conducted this umbrella review in accordance with widely used recommendations for undertaking systematic reviews. ⁹

2.1.1. Data sources and literature search

We identified literature reviews published between January 2007 and June 2018 through searches of the following three electronic databases: MEDLINE, the Cochrane Library of Systematic Reviews, and CINAHL. Our search algorithm is detailed in Appendix S1 and included key words such as “nurse staffing”, “nurse workload”, “nurse patient ratio”, “patient nurse ratio”, “patient outcome”, “quality of care”, “outcome assessment”, and “hospitals”. We also conducted manual searches of reference lists to locate published articles missed by the database searches.

2.1.2. Selection of publications

We included literature reviews that (a) synthesized evidence on the association between nurse staffing levels and NSPOs and (b) focused on nursing care for adults in acute hospitals. We excluded literature reviews if they (a) did not relate to the research question, (b) were not available in English or German, or (c) focused only on intensive care, pediatric care, or psychiatry, as these units largely differ from other units regarding patient characteristics and the relevance of NSPOs.

Two researchers (KD and UKH) independently assessed the eligibility of literature reviews using the title and abstract for initial screening, followed by a review of the full text. Disagreements at both stages were resolved by discussion and settled by consensus including a third researcher (VW).

2.1.3. Assessment of the methodological quality of included studies

We assessed the methodological quality of the included literature reviews using a modified version of the AMSTAR 2 tool originally developed by Shea et al (2007). AMSTAR 2 was designed to assess the quality of reviews of intervention studies. ¹⁰ Because our focus was not on interventions but rather correlations, we made some modifications to the appraisal criteria.

As emphasized by Shea et al ¹⁰ , calculating an overall score based on all AMSTAR 2 criteria would not account for the high impact that flaws in particularly important dimensions might have. We therefore assigned a weight of one (smallest impact) to three (largest impact), indicating the item's influence on the overall methodological quality of the literature review (see Appendix S2). Subsequently, we calculated a weighted quality score (between 0 and 1) for each literature review. We then grouped each literature review into one of three categories indicating its overall quality: high (weighted score ≥ 2/3), moderate (weighted score < 2/3 but ≥ 1/3), and low (weighted score < 1/3) (see Table 1).

TABLE 1.

Number of included primary studies, methodologies, patient outcomes, quality assessment score and further descriptors reviews

Authors	Number of included primary studies	Research objective	Design	Period	Patient setting	Input variables—nurse staffing measures a	Output variables—considered NSPOs (min 1, max 29)	Quality Assessment Category [weighted score]
Bae and Fabry (2014) 36	11 b	To evaluate systematically the effect of nurse overtime and long work hours on nurse and patient outcomes	Systematic literature review	2000‐2013	Acute care, nursing homes, and other health care settings	Shift length (daily) work hours Weekly work hours Number of shifts per week Breaks Overtime Voluntary overtime Mandatory overtime On call	Medication errors Falls Decubitus ulcers Failure to rescue Nurse perceptions of quality Hypoglycemic events Perceived adverse events Errors or near misses Pneumonia deaths Patient dissatisfaction Nosocomial infection Abdominal aortic aneurysm postoperative deaths Mortality Central‐line‐associated blood stream infection Urinary tract infection Ventilator‐associated pneumonia Gastrointestinal bleedings Shock cardiac arrest Sepsis	Moderate [0.463]
Chin (2013) 37	12	To examine the empirical evidence on the relationship between nurse staffing and quality of care in acute care settings	Systematic literature review	2002‐2012	Acute care adult setting	Nurse staffing: Nurse‐to‐patient ratio Nursing hours per patient day RN hours per patient day Total number of nursing care hours Total of hours of RNs, LPNs and NAs Total hours of nursing care provided Total hours worked by nurses Ratio of licensed nurses to patients Skill mix: Ratio of RNs to other nursing staff Ratio of RN hours to all nursing care hours	Quality of care (30‐day) mortality Failure to rescue Falls Sepsis Gastrointestinal bleeding Pressure sore/pressure ulcer/decubitus Congestive heart failure Prolonged length of stay Pneumonia Deep vein thrombosis Urinary tract infections (Wound) infection Shock and cardiac arrest Length of stay	Moderate [0.352]
Driscoll et al (2018) 17	31	To examine the association between nurse staffing levels and nursing‐sensitive patient outcomes in acute specialist units	Systematic review and meta‐analysis	2006‐2017	Acute care (specialist units)	Ratio of the number of nurses (per shift or over a 24‐h period) divided by the number of beds occupied by patients over the same period Number of nursing hours per patient bed days	Mortality Failure to rescue Shock Cardiac arrest Unplanned extubation Hospital‐acquired pneumonia Respiratory failure Surgical bleeding Heart failure/fluid overload Catheter‐associated urinary tract infections Pressure sores Patient falls Nosocomial bloodstream infections Medication error Length of stay Hospital‐acquired sepsis Deep vein thrombosis Central nervous system complication Death Wound infection Pulmonary failure Metabolic derangement Pain control Unplanned readmission	Moderate [0.597]
Donaldson and Shapiro (2010) 38	6 c	To examine the impact of California's nurse‐to‐patient ratios on patient care cost, quality, and outcomes in acute care hospitals	Literature Synthesis	2004‐2009	Acute care	RN hours of direct care per patient day RN productive hours of care per patient day RN registry hours of direct care per patient day Ratio RN to patient Hours of RN care per patient day Skill mix LVN	Falls incidence Falls with injury % prevalence hospital‐acquired pressure ulcer (HAPU) % prevalence restraint use Length of stay Incidence failure to rescue Incidence HAPU Incidence hospital‐acquired infections Incidence DVT Incidence postop pneumonia Incidence postop sepsis Reported medication errors	High [0.889]
Griffiths et al (2014) 39	35 d	To provide evidence on which patient safety outcomes are associated with nurse and health care assistant staffing levels and skill mix by answering the following two questions: What outcomes are associated with tasks undertaken by registered nurses, health care assistants (HCA), and other staff? Which outcomes should be used as indicators of safe staffing?	Systematic literature review	1993‐2014	Acute care adult setting	Nurse‐to‐patient ratios	Mortality Fall/injury Pressure ulcer Adverse drug event Pneumonia Urinary tract infection Wound infection Sepsis Upper gastrointestinal bleeding Pulmonary failure Shock/cardiac failure Central nervous system complications Deep vein thrombosis Bloodstream infection Physical/metabolic derangement Length of stay Cost Hospital‐acquired infections Failure to rescue Readmission Patient satisfaction Drug administration errors Lack of or inadequate nursing care Discharge preparation Emergency department (ED) visits MRSA infection	High [0.685]
Heslop and Lu (2014) 40	38	To report a concept analysis of nurse‐sensitive indicators within the applied context of the acute care setting	Concept analysis	2000‐2012	Acute care setting	Hours of nursing care per patient day Nurse staffing (staff mix, skill mix, and staff ratio)	Mortality Falls and falls with injury Pressure ulcer Nosocomial selective infection Nosocomial urinary tract infection Medication error Pneumonia Vein system complication Failure to rescue Restraint Sepsis Gastrointestinal bleeding Shock Patient/family satisfaction with nursing care Patient/family satisfaction with pain management Length of stay Waiting time for nursing care Unplanned hospital visits postdischarge Vital signs status, self‐care ability Symptom resolution/reduction	Moderate [0.344]
Hill (2017) 41	5	To provide evidence on whether registered nurse staffing levels affect patient mortality in acute secondary care settings	Systematic literature review	1994‐2014	Acute care setting	Nurse staffing levels, incl. nurse‐to‐patient ratios and nursing time	Mortality	Low [0.315]
Kane et al (2007) 6	96 (28 e )	To examine the association between RN staffing and patient outcomes in acute care hospitals	Systematic literature Review and meta‐analysis	1990‐2006	Acute care setting	RNs to patients	Hospital‐related mortality Failure to rescue Cardiac arrest Shock Unplanned extubation Respiratory failure Deep venous thrombosis Upper gastrointestinal bleeding Surgical bleeding Patient falls Pressure ulcers Nosocomial infection Urinary tract infection Hospital‐acquired pneumonia Nosocomial bloodstream Length of stay	High [0.764]
Min and Scott (2016) 42	14 f	To identify techniques used to measure nurse staffing and to evaluate the reliability, validity, and limitations of nursing hours per patient day (NHPPD)	Literature review	2002‐2015	Acute care setting	Nurse‐to‐patient ratios Full‐time equivalents Nursing hours per patient day Skill mix Nurse‐perceived staffing adequacy Nurse‐reported number of assigned patients	Risk‐adjusted 30‐d mortality Failure to rescue Rates of patient falls Injury falls Pneumonia Decubitus/pressure ulcer Mortality Quality scores	Moderate [0.407]
Olley et al (2018) 43	11 g	To evaluate and summarize available research on nurse staffing methods and relate these to outcomes under the following three overarching themes: Management of clinical risk, quality, and safety Development of a new or innovative staffing methodology Equity of nursing workload	Systematic literature review	2010‐2016	Acute care setting	Nurse‐to‐patient ratios	Mortality/inpatient deaths within 30 d of admission	Low [0.315]
Recio‐Saucedo et al (2017) 44	14	To undertake a systematic review of the literature on the impact of missed nursing care on outcomes in adults in acute hospital wards or in nursing homes	Systematic literature review	N/A (oldest study is from 2004, most recent study from 2016)	Acute care setting and nursing homes	Missed care/nursing care left undone based on surveys	Medication errors Bloodstream infections Pneumonia Urinary tract infections (UTIs) Nosocomial infections Patient falls Pressure ulcers Patient and/or carer experience and satisfaction ratings Patient safety Quality of nurse‐delivered care Critical incidents Adverse events Mortality 30‐day hospital readmission	Moderate [0.611]
Shekelle (2013) 45	17	To examine the evidence on the effects of interventions aimed at increasing nurse‐patient ratios on patient illness and death	Systematic literature review	2009‐2012	Acute care setting	Nurse‐to‐patient ratios	Mortality Failure to rescue Decubitus/pressure ulcer Pneumonia Sepsis Falls	Moderate [0.407]
Stalpers et al (2015) 1	17 h	To examine the relationships between characteristics of the nurse work environment and five nursing‐sensitive patient outcomes in hospitals	Systematic literature review	2004‐2012	Acute Care setting	Total nursing hours RN hours Proportion of registered nurses Temporary nurses Turnover	Delirium Malnutrition Pain Patient falls Pressure ulcers	Moderate [0.630]
Twigg et al (2015) 46	9	To review the literature on economic evaluations of nurse staffing and patient outcomes to see whether there is consensus that increasing nursing hours/skill mix is a cost‐effective way of improving patient outcomes	Systematic literature review	Until 2013	Acute Care setting	Ratio of required to actual nurse staffing hours Nursing hours per patient day Total RN hours per patient day Non‐RN hours per patient day Raising RN proportion to 75th percentile/raising nursing hours to 75th percentile Nursing FTE per patient day Split between overtime and nonovertime hours RN vacancy rate	Length of stay Presence of complications Acuity level Pulmonary failure Metabolic derangement Wound infection Deep vein thrombosis Pneumonia Urinary tract infection Pressure ulcer Sepsis Shock/cardiac arrest Gastrointestinal bleed Transfer to Intensive Care Unit Avoided deaths Avoided adverse outcomes Mortality Lives saved Failure to rescue Unplanned extubation Respiratory failure Falls Nosocomial infection Nosocomial blood stream infection Life years gained (based on differences in failure to rescue pre‐ and postintervention) Unplanned readmissions in 30 d ED visits in 30 d Quality of discharge teaching scale Readiness for hospital discharge scale	Moderate [0.630]
Unruh (2008) 47	21	To assess the impact of hospital nurse staffing levels on given patient, nurse, and financial outcomes	Literature Review	1980‐2006	Acute Care setting	Number of (registered/nonregistered) nurses or (registered/nonregistered) nursing hours per number of patients or patient days RN skill mix	Blood stream infections Cardiac arrest Shock Complications Falls Failure to rescue Medication errors Mortality Pneumonia Postoperative infections Patient satisfaction Pulmonary compromise Restraint use Skin breakdown Thrombosis Urinary tract infections Upper gastrointestinal bleeding	Low [0.093]

Abbreviations: DVT, deep vein thrombosis; ED, emergency department; FTE, full‐time equivalent; HAPU, hospital‐acquired pressure ulcer; HCA, health care assistant; LPN, licensed practical nurse; LVN, licensed vocational nurse; NA, nursing assistant; NHPPD, nursing hours per patient day; RN, registered nurse.

^aBased on reported list of included NSPOs (inclusion criteria). If not explicitly stated, we derived and aggregated staffing measures based on results and evidence tables from the included studies.

^b24 studies included in total, 11 reporting on patient outcomes.

^c12 studies included in total, six reporting on patient outcomes.

^d46 studies included in total, 35 related to our research question.

^e28 of 96 studies included in the meta‐analysis reported adjusted odds ratios and included studies were available.

^f17 studies included in total, 14 related to our research question.

^g21 studies included in total, 11 related to our research question.

^h29 studies included in total, 17 related to our research question (nurse staffing as independent variable).

To account for the fact that there might be more than one plausible weighting and categorization scheme, we report both the specific score as well as the quality categories (Table 1) and conducted sensitivity analyses using equal weights for all items as well as an alternative weighting approach proposed by an independent researcher (see Appendix S3 and S4).

2.1.4. Data extraction and derivation of the strength of evidence

We used a self‐developed, structured data extraction sheet to record characteristics of the included literature reviews. Apart from a purely narrative review, our aim was to provide researchers with a broad and comparative overview of the existing empirical evidence on each NSPO. We identified a prior study by Cislak et al ¹¹ applying a categorization of the strength of evidence of different variables based on the share of significant results and the number of literature reviews considering the respective variable. Adopting and expanding this approach, we evaluated the strength of evidence with respect to each NSPO’s association with nursing care as follows and as shown in Figure 1.

FIGURE 1.

Procedure for assessing the strength of evidence of nursing‐sensitive patient outcomes (NSPOs) based on results from the included literature reviews. Y = 1 if the association between nurse staffing and an NSPO was coded as “significant” based on evidence from the literature review; otherwise Y = 0

For each literature review, we coded the association between nurse staffing levels and an NSPO as “significant” (Y = 1) if any one of the following conditions was fulfilled:

1. At least two‐thirds of the primary studies cited in the original literature review reported a significant association between the level of nurse staffing and the NSPO.

2. The findings of a meta‐analysis indicated a significant association between the level of nurse staffing and the NSPO.

3. The conclusion of the literature review stated that there was at least a moderate association between the level of nurse staffing and the NSPO.

To assign a greater weight to the findings of literature reviews that included a larger amount of evidence, we weighted the literature reviews according to the number of primary studies they considered per outcome (ie, weight of three: considered more than five primary studies, two: considered three to five primary studies, and one: considered one to two primary studies). Using these weighted scores, we calculated the proportion of literature reviews that found a significant association between nurse staffing levels and an NSPO. Our final assessment of the strength of evidence with respect to each NSPO relied on the weighted share of significant associations, and a minimum requirement for the quantity of evidence (ie, the number of underlying primary studies), categorized as follows:

• High: The association between nurse staffing and an NSPO was coded as “significant” (Y = 1) for at least 2/3 of the weighted literature reviews (with a total of at least three primary studies discussing the NSPO, without accounting for duplicates across literature reviews).

• Moderate: The association between nurse staffing and an NSPO was coded as “significant” (Y = 1) for at least 1/3 but less than 2/3 of literature reviews (with a total of at least three primary studies discussing the NSPO).

• Low: None of the abovementioned conditions were met but the NSPO was discussed in at least one literature review.

Two reviewers (KB, LK) independently evaluated the strength of evidence. Discrepancies between their assessments were solved by discussion until consensus was reached.

Addressing the common issue that any type of categorization is dependent on pre‐determined thresholds, we carried out sensitivity analyses applying varying decision criteria and thresholds on each stage of the categorization (see Appendix S5). Even though the alternative categorization approaches resulted in some NSPOs being assigned to a different category than the original one, the results of the categorization are relatively robust and varying thresholds do not alter the overall interpretation. Of course, it has to be noted that the strength of evidence classification should only be used to gain an overview of existing evidence on each NSPO and does not (necessarily) reflect the actual degree of nursing sensitivity.

For each literature review, we identified the primary studies that assessed the nurse staffing‐NSPO link, entered them into a matrix (see Appendix S6), and derived the degree of overlap between the evidence in the literature reviews.

2.2. Expert interviews

2.2.1. Sampling and recruitment

We invited a total of 21 individuals by e‐mail to be interviewed as part of our study. We chose them purposively so that they would represent a range of medical professions and be capable of assessing how variation in nursing care is visible in patient outcomes during a hospital stay or postdischarge. Ultimately, 16 of these individuals chose to participate in the study. The sample consisted of nurses and other health care professionals with several years of practical experience and varying academic levels. The participants were from different parts of Germany (see Appendix S7 for further details).

2.2.2. Interviews and data collection

Three researchers (UKH, SF, GM) conducted the interviews individually face to face or by telephone using a structured interview guide. At the beginning of each interview, the interviewer explained the aim of the study and the interview in particular, and the interviewee provided oral informed consent. Subsequently, each interviewee was given the following interview stimulus, which connects to the interviewees’ experience: “A patient has been discharged from the hospital. How can you assess whether he or she received good or bad nursing care during his or her hospital stay?” In addition, we used several broader follow‐up questions (see Appendix S8).

The interviewer noted down key points from each interview in a hand‐written protocol, pseudonymized the data, and transferred them to an Excel file containing data from all of the interviews.

2.2.3. Data analysis

We used the inductive content analysis approach proposed by Mayring. ¹² One researcher coded and converted the content of the interviews into patient‐related indicators measuring the quality of nursing care and grouped these indicators until overarching categories of NSPOs emerged. Subsequently, the coding and grouping of indicators was validated by further researchers (SF, UKH, VW, KD) until a consensus was reached.

3. RESULTS

3.1. Umbrella review

Our search yielded 250 results on MEDLINE, 51 on the Cochrane Library of Systematic Reviews, and 129 on CINAHL. After duplicates were removed, there were 401 unique records. Our manual search of reference yielded four additional publications. We screened the 405 unique records and additional publications by titles and abstracts. After exclusions, we assessed 29 full‐text publications for eligibility, of which 15 met the inclusion criteria and were subject to methodological quality assessment. The PRISMA flowchart in Figure 2 provides a summary of the screening process and results.

FIGURE 2.

PRISMA flow chart

Table 1 summarizes the 15 literature reviews by presenting the number of primary studies they included, as well as their research objectives, designs, screening periods, patient settings, the nurse staffing measures, the NSPO(s) they covered, and the quality score we assigned.

Twelve of the literature reviews were systematic, one was a concept analysis, and two were nonsystematic. The earliest primary studies included date from 1990, and the most recent from 2017. The objective of most of the literature reviews was to examine empirical evidence on the relationship between nurse staffing levels and a set of NSPOs. While most of the literature reviews synthesized evidence from acute care settings, some were restricted to specialist units, and one considered evidence from nursing homes in addition to acute care settings. In terms of staffing measures, most of the literature reviews focused primarily on variations in nurse‐to‐patient ratios or nursing‐hours‐per patient‐day ratios. Five literature reviews limited their scope to registered nurse staffing, while the other studies included results on different classifications of the nursing profession (see Appendix S9 for further details). Some literature reviews additionally considered variables related to skill mix. The range of NSPOs ranged from one to 29.

The number of primary studies included in each literature review varied, ranging from five to 96. In total, there were 201 unique primary studies. Of these, 77% were cited by one literature review, 16% by two literature reviews, and 4% by three literature reviews. The primary studies Cho et al ¹³ and Needleman et al ⁴ were cited most often (ie, by five literature reviews each) followed by the studies Aiken et al, ⁵ Donaldson et al, ¹⁴ Halm et al, ¹⁵ and Shuldham et al ¹⁶ (ie, each cited by four literature reviews; see Appendix S6). Overall, the overlap of primary studies among the literature reviews was 23%.

Among the 15 literature reviews included in our umbrella review, we assigned three quality assessment scores of high, nine scores of moderate, and three scores of low using our adapted AMSTAR 2 tool.

In our research context, we considered the adequacy of the literature search (Item 4), the transparency with respect to the characteristics of included and excluded studies (Items 7 and 8), addressing the risk of bias in included studies (Items 9, 12 and 13), appropriate methods for statistical combination of results (Item 11), as well as the adequate investigation of publication bias when performing meta‐analyses (Item 15) to be particularly important (corresponding reasons are included in Appendix S2). Therefore, we assigned each of the eight items addressing these main quality criteria a weight of three. Three of them (ie, Items 11, 12, and 15) were only applicable to the two meta‐analyses. ⁶ , ¹⁷ On average, each of the five remaining items was at least partially fulfilled by nine literature reviews. The seven items we assigned a weight of 1 or 2 were at least partially met by an average of eight literature reviews.

Sensitivity analyses using equal weighting of all quality criteria as well as the alternative weighting scheme proposed by an independent researcher (see Appendix S3) yielded average percentage deviations from the baseline quality scores of 8.5% and 10.3%, respectively, and few quality category assignment changes (see Appendix S4).

Table 2 provides a condensed overview of the 22 NSPOs we extracted from evidence in the included literature reviews and gives our assessment of each NSPO’s strength of evidence, the number of literature reviews referring to this NSPO, the authors and year of publication of each literature review, and the number of primary studies considering each NSPO (with and without overlaps). We did not include NSPOs in the list if they were insufficiently precise (eg, cost), focused too specifically on a patient group (eg, unplanned extubation), could not be clearly attributed to nursing care (eg, cardiopulmonary resuscitation), were difficult to measure accurately (eg, malnutrition), or some combination of these. While most NSPOs are unambiguously defined (eg, mortality and readmission), particularly survey‐based measures might differ in their exact specification across studies. However, since these NSPOs—potentially also due to ongoing movements toward more patient‐oriented health systems ¹⁸ —have gained importance in empirical studies, ¹⁹ , ²⁰ we included them despite the lack of an unique definition. When aggregating existing evidence, we differentiate between perceived (lack of) quality of nursing care and patient (dis‐)satisfaction. The former is a measure of overall quality of nursing care received by patients in a hospital or a hospital unit ²¹ , ²² , ²³ or its improvement over time. ²⁴ Patient (dis‐)satisfaction summarizes the patients’ overall satisfaction with their hospital stay ²⁵ , ²⁶ , ²⁷ and can include detailed aspects like satisfaction with nursing care, satisfaction with information or instruction, and whether the patient would recommend the hospital to family and friends. ²⁷ , ²⁸ , ²⁹

TABLE 2.

Overview of aggregated nursing‐sensitive patient outcomes (NSPOs) and their strength of evidence ^a

#	NSPO	Sources	Number of lit. reviews (out of 15)	Primary studies considered b	Strength of evidence
1	Length of stay	Chin (2013) 37 Donaldson and Shapiro (2010) 38 Driscoll et al (2018) 17 Griffiths et al (2014) 39 Heslop (2014) 40 , c Kane et al (2007) 6 Twigg et al (2015) 46 , c	7	15 [13]	High
2	Patient dissatisfaction	Bae and Fabry (2014) 36 , c Griffiths et al (2014) 39 Heslop (2014) 40 , c Recio‐Saucedo et al (2017) 44 Stalpers et al (2015) 1 Unruh (2008) 47	6	10 [8]	High
3	Poor quality of nurse‐delivered care	Bae and Fabry (2014) 36 , c Chin (2013) 37 Griffiths et al (2014) 39 Min and Scott (2016) 42 Recio‐Saucedo et al (2017) 44	5	7 [4]	High
4	Readmission	Driscoll et al (2018) 17 Griffiths et al (2014) 39 Heslop (2014) 40 , c Recio‐Saucedo et al (2017) 44 , c Twigg et al (2015) 46	5	3 [2]	High
5	Failure to rescue	Bae and Fabry (2014) 36 Chin (2013) 37 Donaldson and Shapiro (2010) 38 Driscoll et al (2018) 17 Griffiths et al (2014) 39 Heslop (2014) 40 , c Kane et al (2007) 6 Min and Scott (2016) 42 Shekelle (2013) 45 Twigg et al (2015) 46 , c Unruh (2008) 47	11	39 [26]	Moderate
6	Medication error	Bae and Fabry (2014) 36 Donaldson and Shapiro (2010) 38 Driscoll et al (2018) 17 Griffiths et al (2014) 39 Heslop (2014) 40 , c Recio‐Saucedo et al (2017) 44 Unruh (2008) 47	7	16 [15]	Moderate
7	Mortality	Bae and Fabry (2014) 36 Chin (2013) 37 Driscoll et al (2018) 17 Griffiths et al (2014) 39 Heslop (2014) 40 , c Hill (2017) 41 Kane et al (2007) 6 Min and Scott (2016) 42 Olley et al (2018) 43 , c Recio‐Saucedo et al (2017) 44 Shekelle (2013) 45 Twigg et al (2015) 46 , c Unruh (2008) 47	13	58 [41]	Moderate
8	Pneumonia	Bae and Fabry (2014) 36 Chin (2013) 37 Donaldson and Shapiro (2010) 38 Driscoll et al (2018) 17 Griffiths et al (2014) 39 Heslop (2014) 40 , c Kane et al (2007) 6 Min and Scott (2016) 42 Recio‐Saucedo et al (2017) 44 Shekelle (2013) 45 Twigg et al (2015) 46 , c Unruh (2008) 47	12	23 [17]	Moderate
9	Respiratory failure	Driscoll et al (2018) 17 Griffiths et al (2014) 39 Kane et al (2007) 6 Twigg et al (2015) 46 , c	4	9 [9]	Moderate
10	Central nervous system (CNS) complications	Driscoll et al (2018) 17 , c Griffith et al (2014) 39	2	1	Low
11	Deep venous thrombosis	Chin (2013) 37 Donaldson and Shapiro (2010) 38 Driscoll et al (2018) 17 Griffiths et al (2014) 39 Kane et al (2007) 6 , c Twigg et al (2015) 46 , c Unruh (2008) 47	7	10	Low
12	Emergency Department visit	Griffiths et al (2014) 39 Twigg et al (2015) 46	2	2	Low
13	Infection with multi‐resistant germs	Griffiths et al (2014) 39	1	1	Low
14	Missed discharge preparation	Griffiths et al (2014) 39 Twigg et al (2015) 46	2	2	Low
15	Patient falls	Bae and Fabry (2014) 36 Chin (2013) 37 Donaldson and Shapiro (2010) 38 Driscoll et al (2018) 17 Griffiths et al (2014) 39 Heslop (2014) 40 , c Kane et al (2007) 6 , c Min and Scott (2016) 42 Recio‐Saucedo et al (2017) 44 Shekelle (2013) 45 Stalpers et al (2015) 1 Twigg et al (2015) 46 , c Unruh (2008) 47	13	40	Low
16	Physiological / metabolic derangement	Driscoll et al (2018) 17 , c Griffith et al (2014) 39 Twigg et al (2015) 46 , c	3	2	Low
17	Pressure ulcer	Bae and Fabry (2014) 36 , c Chin (2013) 37 Donaldson and Shapiro (2010) 38 Driscoll et al (2018) 17 Griffiths et al (2014) 39 Heslop (2014) 40 , c Kane et al (2007) 6 , c Min and Scott (2016) 42 Recio‐Saucedo et al (2017) 44 Shekelle (2013) 45 Stalpers et al (2015) 1 Twigg et al (2015) 46 , c Unruh (2008) 47	13	55	Low
18	Sepsis	Bae and Fabry (2014) 36 Chin (2013) 37 Donaldson and Shapiro (2010) 38 Driscoll et al (2018) 17 Griffiths et al (2014) 39 Heslop (2014) 40 , c Kane et al (2007) 6 Recio‐Saucedo et al (2017) 44 Shekelle (2013) 45 Twigg et al (2015) 46 , c Unruh (2008) 47	11	23	Low
19	Shock or cardiac arrest	Bae and Fabry (2014) 36 Chin (2013) 37 Driscoll et al (2018) 17 Griffiths et al (2014) 39 Heslop (2014) 40 , c Kane et al (2007) 6 , c Twigg et al (2015) 46 , c Unruh (2008) 47	8	9	Low
20	(Surgical) wound infection	Chin (2013) 37 Driscoll et al (2018) 17 , c Griffiths et al (2014) 39 Kane et al (2007) 6 Twigg et al (2015) 46 , c Unruh (2008) 47	6	6	Low
21	Upper gastrointestinal bleeding	Bae and Fabry (2014) 36 Driscoll et al (2018) 17 Griffiths et al (2014) 39 Heslop (2014) 40 , c Kane et al (2007) 6 , c Twigg et al (2015) 46 , c Unruh (2008) 47	7	5	Low
22	Urinary tract infection	Bae and Fabry (2014) 36 Chin (2013) 37 Driscoll et al (2018) 17 Griffiths et al (2014) 39 Heslop (2014) 40 , c Kane et al (2007) 6 Recio‐Saucedo et al (2017) 44 Twigg et al (2015) 46 , c Unruh (2008) 47	9	19	Low

^aThe following outcome indicators were not included in the list: “hypoglycemic events,” “errors or near misses,” “pneumonia deaths,” “abdominal aortic aneurysm postoperative deaths,” “surgical bleeding,” “unplanned extubation,” “cost,” “malnutrition,” “patient safety,” “adverse events / outcomes,” “presence of complication,” “critical incident,” “acuity level,” “transfer to ICU,” “life years gained based on differences in FTR pre‐ and postintervention,” “restraint use,” “hospital‐acquired infections,” “congestive heart failure,” “adverse drug event,” “pain,” “vital sign status,” “self‐care ability,” “symptom resolution,” “symptom reduction,” “pulmonary compromise,” “delirium.”

^bAll 15 literature reviews combined cover a total number of 201 primary studies. Compare Appendix S6 in the supplement of this paper for a list of included primary studies for each literature review. Here we report the sum of primary studies considering each NSPO. The number covers only primary studies we used to calculate the strength of evidence. The number in parentheses represents the sum of unique primary studies (if more than one literature review reported results from the same primary study).

^cWe did not include the results of this literature review when calculating the strength of evidence because the primary studies considered for this particular NSPO did not meet our inclusion criteria (eg, they were focused on intensive or pediatric care settings) or because the authors did not provide the information required for the calculations.

Using our methodological framework, we classified four NSPOs as having a high strength of evidence (“length of stay,” “patient dissatisfaction,” “poor quality of nurse‐delivered care,” and “readmission”), five NSPOs as having a moderate strength of evidence (“failure to rescue,” “medication error,” “mortality,” “pneumonia,” and “respiratory failure”), and the remaining 13 NSPOs as having a low strength of evidence.

The NSPOs most frequently included in literature reviews and analyzed in the underlying primary studies were failure to rescue, mortality, pneumonia, patient falls, pressure ulcer, and sepsis (at least in 11 of the 15 literature reviews). The number of underlying primary studies was particularly high for mortality and pressure ulcer. The NSPOs with the highest strength of evidence relied on a medium to small amount of literature reviews and primary studies.

3.2. Expert interviews

In our interview data, we identified 76 patient‐related indicators that could conceivably be used to report the quality of nursing care (see Appendix S10). Most of these indicators reflected NSPOs already included in the list derived from the literature reviews (eg, pressure ulcer and medication error), while some of them reflected aspects of broader NSPOs (eg, lack of written information for the patient as aspect of missed discharge preparation).

Four of the indicators mentioned by the experts had not yet been identified based on the data from the literature reviews, namely, “poor discharge status,” “central venous catheter occlusion,” “infection in vascular access site,” and “mycosis.” Furthermore, we refined “poor quality of nursing care” into five separate dimensions according to the results from the expert interviews, that is, poor quality of nursing sensitivity with regard to (a) communication (eg, not using understandable language), (b) counseling and instruction of the patient (eg, regarding the usage of medical aids), (c) disease and condition related treatment (eg, missed recognition of delirium), (d) basic nursing care and personal hygiene (eg, bad skin condition), and (e) the involvement of significant others (eg, lack of preparation and training of family/friends for care).

4. DISCUSSION

In our umbrella review, we identified a large amount of evidence for the association between nurse staffing levels and NSPOs from a total of 15 literature reviews, which themselves included 201 unique primary studies. The low degree of overlap can be explained in part by differences in the screening periods of the literature reviews but might also be due to differences in their search strategies. This underscores the value of having conducted an umbrella review to synthesize prior evidence.

Interestingly, the strength of evidence for only two of the five NSPOs included most frequently in the 15 literature reviews were rated as moderate according to our criteria, while the strength of evidence for the other three was rated as being low. Hence, data availability and, potentially, path dependency appear to have played an important role in determining which NSPOs were studied in the primary literature.

To broaden the potentially limited perspective of previous literature on relevant NSPOs, we carried out expert interviews. This way, we identified four additional NSPOs that have not been considered in any of the literature reviews.

In total, we ranked the strength of evidence for nine of the 22 NSPOs included in the literature reviews as moderate or high, namely, length of stay, patient dissatisfaction, poor quality of nurse‐delivered care, readmission (high strength of evidence), failure to rescue, medication error, mortality, pneumonia, and respiratory failure (moderate strength of evidence). Of the four NSPOs with a strong evidence, poor quality of nursing care and patient dissatisfaction assumedly have an immediate and strong relationship with nurse staffing, that is, fewer nurses likely increase the probability that patients do not feel cared for with an adequate amount of time or attention. One explanation for the strong evidence for length of stay and readmission could be that nurses facing understaffing might be less likely to detect complications or new health problems at early stages. Such complications might increase length of stay or become apparent and worsen after discharge, ultimately necessitating readmission.

Regarding the NSPOs with a moderate strength of evidence, all seem strongly affected by nursing care—for example, it is primarily nurses who are responsible for administering drugs in hospitals and preventing pneumonia by ensuring adequate respiration among patients at risk, for example, by mobilizing them early and having them perform breathing exercises. At the same time, NSPOs in the moderate strength of evidence category are also determined by various factors that are not related to nursing care, such as disease severity or the medical treatment provided by physicians.

We ranked the strength of evidence for 13 of the 22 NSPOs as being low. A low strength of evidence might have several explanations. First, it could result from a low actual degree of nursing sensitivity, for example, if the outcomes are also heavily affected by the quality of care delivered by physicians. For instance, it seems that upper gastrointestinal bleeding is more strongly affected by physician care than by nursing. Moreover, some NSPOs might be highly sensitive to nursing care but only for specific groups of patients. The NSPO shock or cardiac arrest, for example, seems to be relevant only to patients with severe acute conditions such as heavy bleeding or severe fluid loss. Another reason for a low strength of evidence being assigned to NSPOs might be limited sample sizes in the primary studies or, more generally, a low number of empirical studies investigating the link between nurse staffing and the respective NSPO. For instance, we assigned a weak strength of evidence to the NSPO infection with multi‐resistant pathogens because only one of the primary studies investigated this outcome.

Furthermore, it has to be noted that the strength of evidence and its categorization might be limited by the empirical evidence suffering from endogeneity problems. There are many aspects influencing the impact of hospital staffing on patient outcomes, leading to challenges in drawing reliable statistical conclusions in empirical studies. One example for a patient‐related factor interacting with the staffing‐outcome relationship is the patient case mix. ³⁰ On the one hand, the patient case mix has a direct impact on the likelihood of adverse outcomes. On the other hand, staffing decisions are often made as a result of patient needs such that the patient case mix might also influence hospital staffing levels. Because an increase in the patient case mix is, in many cases, mainly recognized due to an increase in adverse outcomes, patient outcomes might also influence staffing levels leading to simultaneity. Along similar lines, there are various hospital and staff characteristics that may relate to staffing levels and also affect patient outcomes, for example, the hospitals’ technology level and the skill mix. ³¹ , ³² , ³³

Another endogeneity issue might be the presence of counteracting effects—that is, more resources might also increase the probability of detecting and reporting negative outcomes, leading to an underestimation of effects. One NSPO where this seems particularly probable and might explain its low strength of evidence categorization is pressure ulcer. Nursing standards define the prevention of pressure ulcers as a core task of nursing care. ³⁴ Hence, the direct link of this outcome and nurse staffing levels seems plausible, but its empirical identification might be distorted.

Furthermore, identifying the effect of nurse staffing on NSPOs in an international research context might be aggravated by the fact that both the educational levels of nurses in general as well as the classification of different nursing professions within one country according to their requisite training vary internationally. As the educational level might impact the effect of nurse staffing on patient outcomes, this inhomogeneity might limit the explanatory power of the aggregated results. Similarly, the allocation of responsibilities between the different medical professions varies across countries. Therefore, nurses’ contribution to selected outcomes might be high in countries where tasks influencing an outcome are mainly in the remit of nurses’ practice and vice versa.

The four NSPOs mentioned by our experts but not covered by the literature reviews were central venous catheter occlusion, infection of vascular access site, poor discharge status, and mycosis. Because the periodic cleaning of central and peripheral venous access sites, as well as related hygiene measures, are usually the responsibility of nurses, the expert assessment of the first two seems plausible. This undermines evidence relating infections in vascular access site to missed nursing care. ³⁵ Similar considerations apply to poor discharge status and mycosis: While nursing care is supposed to adopt a holistic approach and improve the general condition of the patient instead of focusing on single medical conditions, understaffing might force nurses to focus only on the most urgent priorities, negatively affecting patients’ general condition at discharge. Similarly, understaffing might also prevent mycosis risks from being managed adequately. Reasons why these four NSPOs were not covered by the evidence from the literature reviews might be data availability or path dependency. In addition, expert interviews differentiate different aspects of quality of nursing care, which partially overlap with the dimensions of missed nursing care. ³⁵

Although our study yields interesting insights, it has several important limitations, each of which offers avenues for further research. First, rating the strength of evidence existing for the association of each outcome with nurse staffing requires choices of thresholds, both for classifying the results of the included literature reviews as significant and for categorizing strength of evidence. However, we tested several variations of these thresholds and found that our ratings remained largely stable. Additionally, our assessment of the strength of evidence is based on primary studies that were heterogeneous with regard to the number and selection of considered NSPOs. To allow conclusions on the degree of NSPOs’ actual nursing sensitivity, a larger number of empirical studies covering all NSPOs would be needed.

Addressing the potential influence of path dependency and data availability on the set of NSPOs that have previously been considered in empirical studies, our expert interviews revealed valuable insights into further outcomes that might depend on nurse staffing.

However, due to the limited number of interviews we conducted with nursing experts, our list of NSPOs derived from expert opinion cannot be comprehensive or transferrable to other countries. Related to this, we made choices about combining certain patient‐related indicators into NSPOs that might require further classification, validation, and scale developments. To develop a full picture of relevant NSPOs, future studies should address these limitations. Furthermore, as we identified several endogeneity issues hampering the assessment of the NSPOs’ actual degrees of nursing sensitivity, an expert discussion on the causality of nurse staffing with respect to the NSPOs identified from literature would be a valuable further research endeavor.

5. CONCLUSION

In this study, we undertook an umbrella review of literature reviews to extract a list of 22 NSPOs. We ranked the NSPOs according to the strength of evidence existing for their empirically observed association with nurse staffing. Accounting for the potentially limited scope of recent literature due to data availability and path dependency, we additionally conducted expert interviews. Thereby, we identified four additional NSPOs that have not yet been considered in empirical studies.

Our results provide researchers with a guidance in selecting NSPOs they might wish to prioritize in future studies. In particular, it would be valuable to investigate rarely studied NSPOs, such as missed discharge preparation and infection with multi‐resistant pathogens, as well as NSPOs for which we did not find evidence in our umbrella review, that is, poor discharge status, central venous catheter occlusion, infection of vascular access site, and mycosis.

Furthermore, future research should focus on advanced estimation approaches to overcome challenges of isolating the effects of nurse staffing on NSPOs.

Supporting information

Supplementary Material

Appendix S1‐S10

Blume KS, Dietermann K, Kirchner‐Heklau U, et al. Staffing levels and nursing‐sensitive patient outcomes: Umbrella review and qualitative study. Health Serv Res. 2021;56:885–907. 10.1111/1475-6773.13647