Quality Outcomes of Hospital Supplemental Nurse Staffing

Abstract

Background

Use of supplemental registered nurses (SRNs) is common practice among U.S. hospitals to fill gaps in nurse staffing.

Objective

To examine the relationship between use of SRNs and patient outcomes.

Methods

Multilevel modeling was performed to analyze hospital administrative data from 19 hospital units in a large tertiary medical center for the years 2003–2006. Patient outcomes included in-hospital mortality, medication errors, falls, pressure ulcers, and patient satisfaction with nurses.

Results

SRN use ranged from 0–30.4% of total RN hours per unit quarter. Among 188 of the 304 unit quarters in which SRNs were used, the average SRN use was 9.8% in non-ICUs and 6.4% in ICUs. All observed effects of SRN use on patient outcomes were non-significant.

Conclusions

SRN use was substantial and varied widely by unit. No evidence was found that links SRN use to either adverse or positive patient outcomes.

Introduction

Hospitals use supplemental registered nurses (SRNs) hired from staffing agencies to fill vacant nursing positions temporarily. These SRNs work either as per diem or traveling nurses with multiple-month contracts. In 2001, 56% of U.S. hospitals used traveling nurses¹. In 2006, the community tracking study, monitoring 12 U.S. geographic markets, indicated that 75% of participating hospitals used agency nurses as supplemental nursing staff². Among hospitals using SRNs, the level of use varied across hospital unit types, with a range of between 0–33%³.

Despite the widespread use of SRNs, only a few empirical studies have investigated the effects of SRN use on patient outcomes such as mortality, medication errors, central line infections, catheter associated bloodstream infections, and falls in hospitals. The findings from these studies have been inconsistent. One possible reason is that some studies were not designed to examine the relationship between use of SRNs and patient outcomes but included SRN use as a covariate^4–5. The lack of a theoretical match could obscure the relationships between use of SRNs and patient outcomes. Another possible reason is that most studies did not adjust for the potential confounding effect of the nursing work environment (NWE) on the association between SRN use and patient outcomes^6–7. The NWE has been recognized as a key factor in the promotion of quality patient care. Empirical evidence has demonstrated that better NWEs are linked to higher nurse-to-patient ratios and better patient outcomes^8–9, and poorer NWEs are linked to poorer patient outcomes, as well as higher nurse burnout, job dissatisfaction and nurse turnover⁸. Nurses who work in less supportive work environments are more likely to leave their hospital⁸, resulting in vacant positions, creating the need for SRNs². One notable exception is the study by Aiken, Xue, Clarke, & Sloane³, in which poorer NWE with inadequate staff and not the use of SRNs were found to explain negative patient outcomes at the hospital level. This study demonstrates the critical need to consider the nature of NWEs when examining the relationship between use of SRNs and patient outcomes. Another limitation of previous studies is the absence of analysis involving SRN use and patient outcomes at the hospital unit level over time. Levels of SRN use, patient outcomes, and NWEs vary widely across units even in the same hospital¹⁰. Unit-level analysis would capture these variations and strengthen the estimation of the effect of SRN use on patient outcomes.

Building on the limited empirical evidence of the previous studies, the objective of this study was to examine the relationship between use of SRNs and patient outcomes using unit-level data, while controlling for features of the NWE and other covariates. Patient outcomes that were examined included in-hospital mortality, medication errors, falls, pressure ulcers (PU), and patient satisfaction with nurses. These outcomes have been recognized as key quality indicators^11–12. Our study was limited to SRNs who were hired through several staffing agencies and had 3-month renewable contracts because the study hospital from which data were collected exclusively employed this type of SRN.

Methods

Research Design

A retrospective multilevel longitudinal design was used to analyze hospital administrative data from 19 hospital units, including 15 adult medical, surgical or step down units and 4 intensive care units (ICUs) within a large tertiary medical center in the Northeast. Retrospective hospital data were collected for the years 2003–2006, the most current available data at the time of data collection. The hospital received Magnet designation in 2004. The study was approved by the university research subjects review board.

Data were collected quarterly from each of the 19 hospital units from 2003 to 2006, resulting in a total of 304 data collection points (19 units × 16 annual quarters per unit). Multilevel modeling was used to account for multiple observations over time by annual quarter nested within units. Our multilevel analysis thus consisted of 2 levels. The 1st level (level-1) consisted of multiple observations over time by annual quarter within units. Level-1 variables were those that changed across quarters within units and included patient characteristics, availability of rapid response team (RRT, used only with the mortality outcome model), nurse staffing characteristics and patient outcomes. The 2nd level (level-2) was composed of unit characteristics that were constant over time within each unit but varied across units. Level-2 variables included type of unit and NWE.

The multilevel modeling design enabled separation of between (level-2) and within (level-1) hospital unit variance¹³. Partitioning the between-unit differences from the remainder of the variance improved estimation of effects within units¹³ and may therefore reveal the effect of SRN use on patient outcomes, which might be less apparent in a study using only one level of data.

Variables and Measures

We obtained data on discharge patients, nurse staffing and unit characteristics. Variables, operational definitions, frequency of measurement, and level of data for multilevel modeling are listed in Table 1. All level-1 variables were measured as aggregate data by annual quarter by hospital unit between 2003 and 2006. The NWE was measured as a level-2 variable based on survey data collected in 2003 and 2005.

Table 1.

Variables and their operational definitions, frequency of measurement and level of data for multilevel model

Types of Variable	Variables	Operational Definitions	Frequency of Measurement	Level of Data for Multilevel Model
Outcome variables	In-hospital mortality	Number of in hospital deaths divided by number of discharges	Unit quarter	Level-1
	Medication errors1	Number of reported medication error events divided by inpatient days multiplied by 1,000	Unit quarter	Level-1
	Falls prevalence	Number of inpatient falls (unplanned descent to the floor) divided by inpatient days multiplied by 1,000 (NQF, 2004)2	Unit quarter	Level-1
	Pressure ulcers	Number of inpatients with national pressure ulcer advisory panel stage II or greater hospital-acquired pressure ulcer divided by number of inpatients in the prevalence study (NQF, 2004)	Unit quarter	Level-1
	Patient satisfaction with nurses	Mean score of the nurses section subscale of the Press Ganey satisfaction survey	Unit quarter	Level-1
Primary predictor	Use of SRNs	Proportion of total RN hours provide by SRNs	Unit quarter	Level-1
Covariates	Age	Mean age at admission	Unit quarter	Level-1
	Gender	Percentage of male and female
	Race	Percentage of white and non-white
	Primary insurance	Percentage of Medicare, private insurance, and others
	Principal diagnosis	Primary medical diagnosis at discharge and reclassified into 1 of 19 broad diagnosis categories according to ICD-9-CM codes in 2003–2006
	DRG-SIW	Service intensity weight for each DRG
	Comorbidity	Mean Charlson/Deyo comorbidity score
	Pressure ulcer risk	Mean score of Braden scale
Covariate	Availability of RRT2	Whether RRT was available in the unit	Unit quarter	Level-1
Covariates	Nurse educational level	Proportion of nurses who hold Baccalaureate degree or higher including SRNs	Unit quarter	Level-1
	Nursing experience	Mean years of nursing experience since first nursing degree including SRNs	Unit quarter	Level-1
	RN skill mix	Proportion of total nursing hours (RNs, LPNs, NAs) provided by RNs including SRNs	Unit quarter	Level-1
	Nursing hours per patient day	Divided by inpatient days, the number of productive hours worked by all nursing staff (RNs, LPNs, NAs, including SRNs) assigned to the unit who have direct patient care responsibilities for greater than 50% of their shift	Unit quarter	Level-1
Covariates	Type of unit	ICU vs. non-ICU	Constant by unit for the study period	Level-2
	Nursing work environment	Better, mixed or poorer work environment, categorized based on subscales of the NDNQI adapted index of work satisfaction	Constant by unit for the study period	Level-2

Note. NQF: national quality forum; SRNs: supplemental registered nurses; DRG-SIW: diagnosis related group service intensity weight; RRT: rapid response teams; ICU: intensive care unit; NDNQI: national database of nursing quality indicators.

¹A medication error event was defined as one of the following: dose given after the drug was discontinued or not ordered; dose omission (failure to administer an ordered dose to a patient before the next scheduled dose); improper dose; wrong drug, dosage form (e.g. tablet vs. liquid), duration, patient, route, or time (based on the best judgment of clinicians, usually following the criteria of one hour before or after the scheduled administration time).

²Availability of RRT is a level-1 covariate used only with the mortality outcome model.

Outcome Variables

In-hospital mortality, medication errors, and falls were obtained from a hospital office that maintained clinical and national comparative databases to support and facilitate outcomes research and local quality improvement efforts. Medication errors and falls were reported by clinicians through the hospital’s voluntary electronic reporting system, an approach that has been used to improve error reporting nationally¹⁴. Data on PUs were extracted from the hospital database reported to the National Database of Nursing Quality Indicators (NDNQI). Patient satisfaction with nurses was measured by the mean of the subscale of the nurses section of the Press Ganey patient satisfaction survey¹⁵. Satisfaction scores ranged from 0 to 100, with a higher score representing higher satisfaction.

Primary Predictor

The primary predictor was use of SRNs. All SRNs were registered nurses (RNs). SRNs did not float between units but worked for the same unit under a contract. We defined use of SRNs as the proportion of total RN hours provided by SRNs per unit per quarter; this proportion measure has been used in prior studies^{3, 16}.

Covariates

Covariates included patient characteristics, availability of rapid response teams (RRTs) (used only with the mortality outcome model), nurse staffing characteristics (other than use of SRNs) and unit characteristics.

Patient Characteristics Used As Level-1 Covariates

The set of covariates used for risk adjustment in each outcome model was selected based on theory, extant literature and availability of data. For the outcome of pressure ulcers, we adjusted for patient age, gender, and pressure ulcer risk as measured by the Braden scale¹⁷. For the outcome of patient satisfaction with nurses, we adjusted for patient age and gender. For the outcomes of in-hospital mortality, medication errors, and falls, we adjusted for patient age, gender, race/ethnicity, primary source of payment, principal diagnosis, diagnosis-related group service intensity weight, and comorbid conditions. Comorbidity was measured by the Charlson/Deyo comorbidity score¹⁸, which was calculated based on up to six secondary diagnoses.

Availability Of RRT Used Only As A Level-1 Covariate With The Mortality Outcome Model

During the study period, the hospital was in the process of implementing hospital-wide RRT on different units at different times. Use of RRT has been shown to reduce preventable deaths¹⁹. We therefore controlled for the availability of RRT for the mortality outcome. Since availability of RRT varied over time within each unit, it was treated as a level-1 covariate.

Nurse Staffing Characteristics (Other Than Use Of SRNs) Used As Level-1 Covariates

Prior research has shown that adequate hospital nurse staffing was associated with better patient outcomes^20–21. We controlled for nurse staffing level in the analysis, measured by total nursing hours per patient day. In addition, we also controlled for the effects of nurse educational level, nursing experience, and RN skill mix because prior studies have shown a link between these variables and patient outcomes²⁰. Since there is no evidence to indicate a difference in the effects of these attributes on patient outcomes between permanent nurses and SRNs, we modeled nurse educational level and nursing experience by pooling the data of both permanent nurses and SRNs.

Unit Characteristics Used As Level-2 Covariates

Unit characteristics included type of unit (ICU vs. non-ICU) and nursing work environment. The 19 units included 4 ICUs and 15 non-ICUs that consisted of 4 medical, 7 surgical and 4 step-down units. Palliative care units were not included. Dichotomization of unit type to ICU and non-ICU categories was based on preliminary analyses that showed no difference in outcomes among medical, surgical and step-down units (non-ICUs) compared to ICUs.

Nursing work environment was measured using the NDNQI adapted index of work satisfaction (IWS), which the hospital had selected to survey nurses. The psychometric properties of the adapted IWS have met standards for reliability and validity in a wide variety of hospital settings and RN populations. The range of internal consistency reliability has been reported as 0.63 to 0.91 for the 7 subscales²². Theoretically, according to Lake²³, the concept of NWE consists of 7 domains, 5 of which match subscales of the IWS. The IWS has been shown to correlate highly (r=0.68) with the National Quality Forum-endorsed practice environment scale²⁴, a frequently used measure of NWE, which covers 4 of the 7 domains.

In order to better differentiate NWE among units within the hospital, we used the modified IWS scores to create a 3-category variable indicating better, mixed, and poorer work environment. This classification has been widely used in prior research and has demonstrated good predictive validity⁸. Since preliminary analysis showed that measures of nursing work environment did not vary significantly over time in each unit, we coded NWEs as better, mixed, or poorer for each unit based on the average scores of years 2003 and 2005 and treated it as a level-2 covariate.

Statistical Analysis

Data were aggregated by annual quarter within each hospital unit and hierarchically structured into the 2 levels. The unit of analysis at level-1 was the unit quarter; data consisted of 16 annual quarters from 2003 to 2006 nested within each of 19 hospital units, for a total of 304 observations. The unit of analysis at level-2 was the hospital unit, of which, as noted, there were 19.

Prior to inferential analysis, we performed diagnostic tests to ensure no violations of the assumptions of multilevel modeling. Using 2-level multilevel modeling, we modeled intercepts as random effects in order to examine the effects of level-2 predictors²⁵. We used restricted maximum likelihood for estimating the statistical parameters for the random intercept models²⁵. Unconditional models were evaluated first for each outcome variable separately. The intraclass correlation coefficients (ICC) for all outcome variables were significant at alpha 0.05 with a range of 0.33 to 0.90, indicating the need for multilevel analysis. Next, we performed multilevel analyses by entering all level-1 and level-2 predictors to specify each of the outcome models. Bias-corrected bootstrap methods were used to generate model parameter estimates. These methods have been shown to produce more accurate estimates in multilevel modeling when level-2 sample size is small or distributions deviate from normal²⁶. All reported p-values were adjusted using the false discovery rate method for type I experiment-wise error for multiple comparisons²⁷. Analyses were performed using SAS version 9.2. All statistical tests were 2-sided, with alpha set at 0.05.

Results

Use of SRNs in the Hospital

Use of SRNs (proportion of total RN hours provided by SRNs) ranged from 0–30.4% per unit quarter. SRNs were used in 188 of the 304 unit quarters observed (61.8%). Among the 188 unit quarters in which SRNs were used, the average SRN use was 9.8% (SD=7.2%) in non-ICUs and 6.4% (SD=4.7%) in ICUs. In SDC #1, we present SRN use by unit among quarters when SRNs were used. One unit (#12) did not use SRNs at all and 1 unit (#19) used SRNs in all 16 quarters. Other units used SRNs between 3–15 quarters. Among the 18 hospital units that used SRNs, the range of SRN use was from 0.2 to 30.4%. Unit 7 had the widest range of use (1.5%–30.4%) and unit 8 had the narrowest range (0.5%–2.9%).

Use of SRNs and Patient Outcomes

In Table 2, we present the results of multilevel modeling on the effects of SRN use on patient outcomes after adjusting for patient characteristics, availability of RRT (used only with the mortality outcome model), and nurse staffing and unit characteristics. All observed effects of SRN use on patient outcomes were non-significant at alpha 0.05. In Table 2, we also present estimates and significance tests of two random effect parameters, unit (random intercept) and residuals for each outcome variable. The estimates of the random intercept variance components were non-significant for all outcomes except for the pressure ulcer outcome. These results indicate that the models adequately account for unit level variance for these outcomes (except PUs). All residual tests were significant, indicating the occurrence of unexplained (error) variance remaining at the unit quarter level on patient outcomes after adjusting for both level-1 and level-2 predictors.

Discussion

Use of SRNs varied greatly by unit in the study hospital over time. Some units used SRNs consistently over time, whereas other units used SRNs sporadically. There was also substantial variation in the level of SRN use at any given time across units. These variations indicate that some units have consistently more vacant positions compared to other units and some units had more vacant positions at a single time. These 2 patterns may be associated with different underlying mechanisms for nurse staffing vacancies and demonstrate the importance of longitudinal data at the unit level in examining the relationship between SRN use and patient outcomes. Understanding the different patterns of SRN use and the factors that account for these patterns may shed further light on the relationships among SRN use, nursing work environment, and patient outcomes.

Our results suggest that use of SRNs had no significant impact, positive or negative, on patient outcomes. Several explanations might be pertinent to this finding. First, our finding is consistent with those of an earlier study³. These are the only 2 studies in which investigators have controlled for NWE and other covariates while examining the relationship between SRN use and patient outcomes. Controlling for NWE is critical because this factor may be linked to both SRN use and patient outcomes, and thus serves to confound the relationship between the two. Although no study has been conducted to examine the relationship between use of SRNs and organizational characteristics in acute care settings, a study conducted in a nursing home showed that use of agency nurses was associated with characteristics of poorer quality facilities²⁸. Second, the current study examined a specific type of SRNs who were hired from several preferred staffing agencies and had 3 month renewable contracts. Experienced SRNs who had worked in similar hospital settings were preferred for hiring, and each SRN was provided a 3-day unit-level orientation. During the study period, the hospital did not use internal float nurses, so there was no contamination effect of float nurses on patient outcomes. The homogeneous sample of SRNs used in this study strengthened the internal validity of the findings and created an opportunity to distinguish and explore the relationship between this type of SRN and patient outcomes. This type of SRN might be different from per-diem SRNs because they have multiple months to adapt to the units. Therefore the effect of SRNs on patient outcomes might be different between per-diem SRNs and SRNs with multiple contracts. Future research is needed to understand the relationship between different types of SRNs and patient outcomes. Third, while our study findings might not generalize to other hospitals with different characteristics in different settings, our institution-based results are nonetheless informative, as Needleman et al. have recently demonstrated that a single site study can provide valid findings that are consistent with those of the multisite studies, due to the robust design, proper model specification and statistical controls, and unit level measures afforded by a single site study²¹. Future research with multisite study design is needed to replicate the findings.

In summary, the results of this study did not indicate a significant association between SRN use and patient outcomes. This finding has relevance for future nursing workforce development and policy. Although current levels of SRN use might be relatively low due to the economic downturn, future hospital use of SRNs is expected to rise with projected future nursing shortages. Therefore, the issue of the impact of SRN use on hospital quality of care will be more significant in the coming years. Future studies are needed to further explore the role of organizational and unit characteristics on SRN use and the relationship between the use of different types of SRNs on patient outcomes across a variety of hospital settings.