Patient Outcomes After the Introduction of Statewide Intensive Care Unit Nurse Staffing Regulations

Abstract

Objective

To assess whether Massachusetts legislation directed at intensive care unit (ICU) nurse staffing was associated with improvements in patient outcomes.

Design

Retrospective cohort study; difference-in-difference design to compare outcomes in Massachusetts with outcomes of other states (before and after the March 31, 2016 compliance deadline).

Setting

Administrative claims data collected from medical centers across the United States (Vizient, Inc).

Patients

Adults between ages 18-99 years old who were admitted to ICUs for ≥1 day.

Interventions

Massachusetts General Law c. 111, § 231, which established (1) maximum patient-to-nurse assignments of 2:1 in the ICU, and (2) that this determination should be based on a patient acuity tool and by the staff nurses in the unit.

Measurements and Main Results

Nurse staffing increased similarly in Massachusetts (N = 11 ICUs, Baseline patient-to-nurse ratio 1.38±0.16 to Post-mandate 1.28±0.15, p = 0.006) and other states (N = 88 ICUs, Baseline 1.35±0.19 to Post-mandate 1.31±0.17, p = 0.002); difference-in-difference p = 0.20. Massachusetts ICU nurse staffing regulations were not associated with changes in hospital mortality within Massachusetts (Baseline N=29754, standardized mortality ratio 1.20±0.04 to Post-mandate N=30058, 1.15±0.04, p=0.11) or when compared to changes in hospital mortality in other states (Baseline N=572,952, 1.15±0.01 to Post-mandate N=567,608,1.09±0.01; difference-in-difference p = 0.69). Complications (Massachusetts: Baseline 0.68% to Post-mandate 0.67%; other states: Baseline 0.72% to Post-mandate 0.72%; difference-in-difference p = 0.92) and do-not-resuscitate orders (Massachusetts: Baseline 13.5% to Post-mandate 15.4%; other states: Baseline 12.3% to Post-mandate 14.5%; difference-in-difference p = 0.07] also remained unchanged relative to secular trends. Results were similar in interrupted time series analysis, as well as in subgroups of community hospitals and workload intensive patients receiving mechanical ventilation.

Conclusions

State regulation of patient-to-nurse staffing with the aid of patient complexity scores in intensive care was not associated with either increased nurse staffing or changes in patient outcomes.

INTRODUCTION

Nurses are critical to ensuring the safe care of patients. However, studies investigating patient outcomes associated with nurse staffing have shown mixed results.(1–8) Whereas multiple observational studies have identified associations between lower patient-to-nurse ratios (higher nurse staffing) and lower rates of complications within intensive care units (ICU),(1–3) interventions to increase nurse staffing on general wards have not shown improvements in patient outcomes.(4–8) Observational findings from the ICU – where complex nursing needs often differ from general wards – have not been explored using experimental or quasi-experimental study designs.

In response to concerns that unregulated patient-to-nurse ratios in the ICU may lead to patient harm,(9) Massachusetts became the first state to mandate 1:1 or 2:1 patient-to-nurse staffing in the ICU guided by an acuity tool in March of 2016. The passage of the Massachusetts mandate created a natural experiment to evaluate the downstream effects of statewide policies applied to nurse staffing in the ICU. We hypothesized that Massachusetts ICU nurse staffing regulations would result in decreased complications and mortality for critically ill patients, as compared with patients admitted to ICUs across the country unaffected by Massachusetts regulations.

METHODS

On June 30, 2014, Massachusetts General Law c. 111, § 231, “An Act relative to patient limits in all hospital intensive care units” was signed into law.(10) The law established that (1) “in all intensive care units the patient assignment for the registered nurse shall be 1:1 or 1:2” and (2) that this determination should be based “on the stability of the patient as assessed by [an] acuity tool and by the staff nurses in the unit.” Regulations governing the implementation of the law were unanimously approved as 958 CMR 8.00(11) on June 10, 2015. Academic medical centers, as defined by the Center for Health Information and Analysis(12), were given a compliance date of March 31, 2016. All other hospitals were required to comply with mandate requirements by January 31, 2017. Hospitals were required to report on four ICU quality measures, including central line-associated bloodstream infections, catheter-associated urinary tract infections, hospital-acquired pressure ulcers, and patient falls with injury.

Data source and Cohort Definitions

We conducted a retrospective cohort study of patients aged 18-99 years admitted to ICUs at 246 medical centers and affiliated hospitals in the United States. Data were abstracted from Vizient, Inc. (Clinical Data Base/Resource Manager and Action-OI databases), a collection of administrative claims and billing information from the largest network of healthcare systems in the United States. We used difference-in-difference methods to compare outcomes in academic Massachusetts and non-Massachusetts comparator hospitals during three time periods relative to the nurse staffing compliance date for academic hospitals: a “Baseline” period: April 1, 2014 to March 31, 2015, a “Preparation” period: April 1, 2015 to March 31, 2016, and a “Post-mandate” period: April 1, 2016 to March 31, 2017 (Figure 1), (See Supplemental Methods).

Figure 1. Monthly risk-standardized mortality of patients receiving intensive care in the 24 months before and 12 months after Massachusetts legislation regulating intensive care unit nurse staffing.

Dates of legislation milestones (signing of MA General Law c.111, § 231, approval of 958 CMR 8.00 governing implementation, and the official compliance deadline) are shown against the x-axis timeline. The 36-month period was divided into three time periods for the purposes of study analysis and are shown above the plot. All difference-in-difference (DD) analyses compared two 12-month time periods: a “Baseline” period from April 1, 2014 – March 31, 2015 and a “Post-mandate” period from April 1, 2016 –March 31, 2017. Primary analyses excluded the year immediately preceding the compliance date from April 1, 2015 – March 31, 2016, which was a “Preparation” preparation period during which MA ICUs may have begun implementing changes to staffing. Interrupted time series (ITS) analyses included all time periods.

Outcomes

The primary outcome was the change in risk-standardized mortality ratio during the Baseline and Post-mandate periods for patients hospitalized within Massachusetts academic ICUs, as compared with patients hospitalized in non-Massachusetts academic ICUs. Risk-standardized hospital mortality was calculated by Vizient 2016 Mortality Risk Adjustment Models.(13) To contextualize changes in standardized mortality ratios, we also reported changes in the component observed and expected mortality rates.

We evaluated two clinical secondary outcomes: 1) a composite in-hospital complication rate composed of four complications for which reporting was mandated (central line-associated bloodstream infections, catheter-associated urinary tract infections, hospital-acquired pressure ulcers, and patient falls with injury); and 2) rates of do-not-resuscitate (DNR) orders. Prior studies showed associations between ICU staffing and DNR rates that may influence mortality.(14) We also assessed for changes in nurse staffing prior to and after the mandate compliance date.

Statistical analysis

Difference-in-difference approaches can be used to evaluate effects of policy interventions introduced at a specific time, and can achieve unbiased effect estimates when temporal trends are the same within the intervention and comparison groups (despite differences in group characteristics).(15) After establishing parallel baseline trends in mortality between Massachusetts and non-Massachusetts hospitals in the two years preceding the compliance date (p for difference in trends = 0.58), we investigated the effects of the nurse staffing mandate for risk-standardized mortality using generalized linear models with a Poisson distribution and the natural log of expected mortality as an offset variable.(16) Difference-in-difference estimates were evaluated using similar methods for unadjusted hospital mortality, expected mortality, complications, and DNR orders. Three Massachusetts hospitals (11 ICUs) and 33 hospitals (88 ICUs) outside of Massachusetts reported nurse staffing data during both the Baseline and Post-mandate period; we calculated the ratio of total patient-days to the total nurse-days worked by licensed nurses to yield patient-to-nurse ratios for the Baseline and Post-mandate period and compared the change in nurse staffing using t-tests.

Sensitivity and Subgroup analysis

We performed four sensitivity and subgroup analyses. First, we used a controlled interrupted time series approach(17, 18) to evaluate the change in unadjusted mortality after implementation of the nursing mandate among academic hospitals in Massachusetts as compared to outside of Massachusetts. The controlled interrupted time series design allowed an alternative approach to assess changes to mortality at the date of mandate implementation that did not depend on risk-adjustment using a standardized mortality ratio. We assessed the difference in monthly mortality rates for MA and non-MA hospitals in the two years before and one year after the legislation compliance date. Due to the presence of first order autocorrelation (Durbin-Watson test=1.5, p=0.026) we used an exact maximum likelihood to account for autocorrelated errors and seasonality. Models included independent variables for time (i.e., number of months from start of follow up), the onset of the policy intervention (i.e. pre- vs. post- implementation), and time after the policy intervention.(19) Polynomial models were also explored for best fit.

Second, we conducted a difference-in-difference analysis for risk-standardized and unadjusted mortality in non-academic community hospitals. Prior analyses have shown that nurse staffing can vary between academic and community hospitals,(20) and that state nursing mandates may affect hospitals differentially depending on pre-legislation staffing levels.(4, 5) Non-academic community hospitals were required to comply with mandate requirements by January 31, 2017 date; therefore, two time periods were compared: February 1, 2015 to June 30, 2015 and February 1, 2017 to June 30, 2017 for Massachusetts and non-Massachusetts comparator non-academic hospitals.

Third, we examined the effect of the nurse staffing mandate in the subgroup of mechanically ventilated patients (using ICD-9 and ICD-10 procedure codes).(21) The presence of mechanical ventilation accounts for a significant increase in acuity score on a tool such as the Therapeutic Intervention Scoring System-76 (TISS-76),(22) and therefore, patients requiring mechanical ventilation may be more likely to have nurse staffing increased as a result of the mandate.

Fourth, we performed a post hoc exploratory analysis to evaluate changes in patient outcomes within Massachusetts alone using interrupted time series analysis.(17, 18) This analysis was conducted to evaluate whether the changes in nurse staffing after implementation of state regulations were associated with changes in patient outcome, without relying on assumptions that Massachusetts was comparable to other states. Methods were similar to the controlled time series approach, with exact maximum likelihood used to account for autocorrelated errors (Durbin-Watson test=1.4, p=0.01) and seasonality, independent variables for time, the onset of the policy intervention, and time after the policy intervention(19) and polynomial models explored for best fit.

All statistical testing was through SAS 9.4 (Cary, NC), with two-tailed and performed with α= 0.05. A sample size of 7,000 was required to detect a 10% relative change in mortality with α = 0.05 and power of 0.90. A sample size of 10,000 was required to detect a 25% relative change in complications with α = 0.05 and power of 0.90. This study of de-identified composite hospital data was deemed exempt from review by the Boston University Medical Institutional Review Board.

RESULTS

Cohort characteristics

During the Baseline period, we identified 29,754 ICU admissions in the six academic hospitals in Massachusetts and 572,951 ICU admissions at 114 academic hospitals outside of Massachusetts. In the Preparation period between passage of the staffing law and mandated implementation date, we identified 29,553 ICU admissions in Massachusetts, and 580,837 ICU admission outside of Massachusetts. In the Post-mandate period, we identified 30,058 ICU admissions in Massachusetts teaching hospitals and 567,608 ICU admissions at AAMC teaching hospitals outside of Massachusetts. Characteristics of the six cohorts are shown in Table 1.

TABLE 1.

Characteristic of study cohorts during three study periods.

	Baseline Period, April 1, 2014 –March 31, 2015	Preparation Period, April 1, 2015 - May 31, 2016	Post-Mandate Period, April 1, 2016 - March 31, 2017
Massachusetts
Total, N	29,754	29,553	30,058
Age, N (%)
18-50 Years	7,060 (23.7)	6,829 (23.1)	7,057 (23.5)
51-64 Years	8,818 (29.6)	8,785 (29.7)	9,042 (30.1)
65-79 Years	9,398 (31.6)	9,529 (32.2)	9,807 (32.6)
≥80 Years	4,478 (15.1)	4,410 (14.9)	4,152 (13.8)
Female, N (%)a	12,706 (42.7)	12,715 (43.0)	12,838 (42.7)
Race, N (%)
White	22,305 (75.0)	21,980 (74.4)	22,293 (74.2)
Asian	958 (3.2)	1,004 (3.4)	957 (3.2)
Black	2,955 (9.9)	3,169 (10.7)	3,236 (10.8)
Other	2,364 (7.9)	2,101 (7.1)	2,210 (7.4)
Missing	1,172 (3.9)	1,299 (4.4)	1,362 (4.5)
Outside of Massachusetts
Total, N	572,951	580,837	567,608
Age, N (%)
18-50 Years	166,476 (29.1)	167,249 (28.8)	160,343 (28.2)
51-64 Years	178,919 (31.2)	181,205 (31.2)	175,557 (30.9)
65-79 Years	162,148 (28.3)	167,385 (28.8)	167,674 (29.5)
≥80 Years	65,408 (11.4)	64,998 (11.2)	64,034 (11.3)
Female, N (%)b	247,284 (43.2)	249,391 (42.9)	244,318 (43.0)
Race, N (%)
White	379,214 (66.2)	384,851 (66.3)	377,073 (66.4)
Asian	11,758 (2.1)	11,797 (2.0)	12,525 (2.2)
Black	116,404 (20.3)	120,979 (20.8)	116,154 (20.5)
Other	49,039 (8.6)	48,358 (8.3)	45,765 (8.1)
Missing	16,536 (2.9)	14,852 (2.6)	16,091 (2.8)

^aUnknown sex in 1 patient (0.0%) in Massachusetts during the Preparation period.

^bUnknown sex in 43 patients (0.0%) during the Baseline period, 53 patients (0.0%) during the Preparation period, 106 (0.0%) during the Post-mandate period outside of Massachusetts.

Nurse staffing changes

Patient-to-nurse ratios decreased in Massachusetts after nurse staffing legislation (Baseline: 1.38±0.16 to Post-mandate: 1.28±0.15, p = 0.006) and outside of Massachusetts in the same time periods (Baseline: 1.35±0.19 to Post-mandate: 1.31±0.17, p = 0.002), without significant difference-in-differences for change in nurse staffing [−0.05, (95% CI −0.03-0.13), p = 0.20].

Mortality, Complications and Advance Directives

After implementation of nurse staffing regulations, risk-standardized mortality remained stable in Massachusetts [1.20 (95% CI 1.16-1.24) to 1.15 (1.11-1.19) (p = 0.11)] but decreased outside of Massachusetts [1.15 (1.14-1.16) to 1.09 (1.08-1.10) (p <.001)], without a significant difference-in-difference (OR = 1.01, 95% CI 0.96-1.06, p = 0.69) (Figure 2). Observed mortality (OR 0.96, 95% CI 0.91-1.00, p = 0.09) and expected mortality (OR 0.95, 95% CI 0.89-1.00, p = 0.06) also did not change significantly in Massachusetts after nursing legislation, as compared with outside of Massachusetts (Figure 2).

Figure 2. Mortality in the Baseline period vs. Post-mandate periods, comparing intensive care units within Massachusetts and intensive care units outside of Massachusetts.

Panel A. The unadjusted mortality rate remained stable in Massachusetts (10.9% to 10.7%, p = 0.32) and rose outside of Massachusetts (9.2% to 9.4%, p =0.003). The difference-in-difference in unadjusted mortality for patients in intensive care units in Massachusetts as compared to outside of Massachusetts was not significantly different (p = 0.09).

Panel B. The expected mortality remained stable in Massachusetts (9.1% to 9.2%, p = 0.53) and increased outside of Massachusetts (8.0% to 8.6%, p<.001). The difference-in-difference in expected mortality for Massachusetts compared with non-Massachusetts hospitals was not significantly different (p=.06).

Panel C. The risk-standardized mortality ratio in Massachusetts was 1.20 (95% CI 1.16-1.24) in the Baseline period and 1.15 (1.11-1.19) (p = 0.11) in the Post-mandate period, while the mortality index outside of Massachusetts was 1.15 (1.14-1.16) in the Baseline period and 1.09 (1.08-1.10) in the Post-mandate control period (p <.001). The difference-in-difference estimate for risk-standardized mortality associated with implementation of the Massachusetts nursing staffing mandate was not significant (OR = 1.01, 95% CI 0.96-1.06, p = 0.69). Error bars represent 95% confidence intervals.

The rate of complications was stable between the Baseline period and the Post-mandate period in Massachusetts (0.68% to 0.67%, p = 0.88) as well as outside of Massachusetts (0.72% to 0.72%, p = 0.83), with no significant difference in trends (p = 0.92). The proportion of ICU patients with a DNR status increased in Massachusetts (13.5% to 15.4%, p <0.001) as well as outside of Massachusetts (12.3% to 14.5%, p<.001), without a significant difference-in-difference (p = 0.07).

Sensitivity analyses

Sensitivity analyses showed similar results to the primary analysis. Controlled interrupted time series analysis did not demonstrate a significant change in observed mortality in Massachusetts as compared with ICUs outside of Massachusetts following the Massachusetts policy implementation (β = −0.81, p = 0.23) (Supplemental Figure 1). Similarly, analysis of non-academic community hospitals did not show a significant difference-in-difference of risk-standardized mortality (OR 0.99, 95% CI 0.91-1.13, p = 0.89). Risk-standardized mortality in Massachusetts decreased from 1.29 (95% CI 1.21-1.37) to 1.14 (95% CI 1.07-1.22) (p = 0.01), while outside of Massachusetts, the mortality index decreased similarly from 1.10 (95% CI 1.11-1.19) to 0.96 (95% CI 0.93-0.99) (p <.001) among non-academic hospitals. Analysis of complication rates and DNR status revealed results similar to academic hospitals (Supplemental Results). Mechanically ventilated patients also did not show a significant difference-in-difference in risk-standardized mortality (OR = 1.02, 95% CI 0.96-1.08, p = 0.57), rate of complications (p = 0.42), or in rates of DNR orders (p = 0.31). Analysis of interrupted time series within Massachusetts alone was also not associated with a significant change in observed mortality in Massachusetts (adjusting for 1^st order autocorrelation, β for immediate change in mortality= −1.13, p = 0.18), though some quadratic models suggested increasing mortality trends after implementation (Supplemental Figure 2).

DISCUSSION

We performed an analysis of the effect of a Massachusetts law mandating the use of a risk score within ICUs to assign patient-to-nurse ratios of either 2:1 or 1:1, based upon patient acuity and anticipated care intensity. Increases in nurse staffing in Massachusetts after institution of statewide regulations were not significantly higher than staffing trends in states that did not pass ICU nurse staffing legislation. We did not identify improvements in patient outcomes associated with the Massachusetts nursing regulations. Results were robust to multiple sensitivity and subgroup analyses.

Our findings may be considered from two perspectives. First, the difference-in-difference analyses comparing changes in Massachusetts after the implementation of ICU nurse staffing regulations to changes in other states were analogous to an intention-to-treat analysis of the Massachusetts nurse staffing policy legislation. Our results suggest that the Massachusetts nursing regulations were not associated with changes in staffing or patient outcomes when compared to secular trends. Second, when examined apart from secular changes in other states, modest nurse staffing increases in Massachusetts (the change from 1.38 to 1.28 patients per nurse is approximately 1 extra nurse per 20-bed ICU per 12-hour shift) remained unassociated with changes in hospital mortality.

Few states have passed laws regulating nurse staffing. California previously mandated maximum patient-to-nurse ratios within acute care hospitals but did not use scoring systems to determine 1:1 staffing levels. Although California regulations were associated with an increase in nurse staffing, outcomes of hospitalized did not improve after the California law, findings similar to Massachusetts ICUs.(4, 6, 8, 23)

Prior observational studies of nurse staffing have identified associations between lower patient-to-nurse ratios and improved outcomes.(1, 2, 24, 25) However, these observational studies were unable to distinguish the causal effect of nurse staffing on patient outcomes – patients in hospitals with a higher commitment to nursing staffing may also benefit from higher overall hospital commitment to a quality of care and nurse retention, improved nurse-physician collaboration, or other predictors of patient safety.(1, 24, 26–29) In the absence of large randomized trials to determine the causal effect of nurse staffing on patient outcomes, studies of policy interventions provide natural experiments to evaluate the effectiveness of interventions. While baseline demographic differences between Massachusetts and non-Massachusetts hospitals reflect existing underlying demographic differences in the population, the difference-in-difference analysis compares within State changes and therefore is likely robust to baseline demographic differences. Multiple factors may have contributed to the minimal changes in nurse staffing and observed lack of improvement in patient outcomes associated with Massachusetts’ ICU nurse staffing regulations, including: (1) adequate nurse staffing and competent nurse assignment determination prior to the mandate, (2) significant leeway given to hospitals in both the determination of the acuity/severity tool to be used to guide nursing assignments and unclear implementation of the tool in guiding nursing assignments, (3) multidisciplinary staffing factors important for patient outcomes in addition to nursing care (pharmacists, respiratory therapists, among others), (4) secular trends of increased ICU nurse staffing outside of Massachusetts, and (5) a diluted effect on mortality due to unchanged or increased hospital-ward mortality as a result of earlier transfer from the ICU to the floor in the setting of ICU nurse staffing limitations. Hospitals used different tactics in designing or choosing acuity tools to determine nurse staffing, which may reflect uncertainty as to how to optimally distribute nursing resources.

Our findings should be interpreted in the context of limitations. First, although our analysis is based on de-identified individual patient data, risk-standardized mortality ratios were averaged across hospitals. However, interrupted time series analysis of observed mortality rates, which did not incorporate risk-adjustment, corroborated the risk-standardized mortality findings. Second, nursing regulations may have had heterogeneous effects within certain patient populations or larger effects at hospitals with lower pre-mandate staffing. However, results were similar in a subgroup of mechanically ventilated patients who were more likely to be affected by acuity score-based nurse staffing, as well as in non-academic medical centers with generally higher patient-to-nurse ratios.(5) Further studies may seek to evaluate for heterogeneity of policy effects across other patient populations, such as across type of ICU. Third, measured complications were rare (less than 1% of patients), but sample size calculations suggested sufficient power to detect differences in complication rates. The complications chosen for the composite analysis were the four complications for which reporting to the state was mandated, but given rarity, were likely not ideal measures of improvements in nurse-staffing-sensitive ICU morbidity. The low rates of complications may reflect that other external forces aside from state legislation (e.g. Center for Medicaid Services and the Joint Commission) have already maximally reduced occurrence rates. Further studies should evaluate other candidate measures of nurse-staffing-sensitive morbidity; for example, Kane et al identified unplanned extubations as a measure potentially sensitive to nurse staffing.(24) Unfortunately, few academic hospitals contributed data regarding nurse staffing and therefore power to detect changes in nurse staffing was limited – though effect estimates suggest small differences in staffing changes after the legislation. Fourth, Massachusetts instituted both maximum patient-to-nurse ratios and the use of patient severity scores in a bundled approach; we were unable to distinguish effects of the patient-to-nurse ratio mandate from the regulation of nurse staffing based upon patient severity scores. It is possible that our findings are the result of suboptimal nursing resource allocation as a result of the acuity tool alone, and that using other methods of nurse staffing distribution may yield different outcomes. Fifth, we did not have data on costs attributable to this legislation, although costs associated with development and implementation of severity score-based nurse staffing and reporting compliance in response to new mandates may be substantial.(30) Sixth, other outcomes important to patients (such as patient and family satisfaction, emotional harms, or post-intensive care syndrome) and nurses (job satisfaction or creation of a healthier work environment leading to higher nurse retention rates) may be sensitive to nurse staffing regulations and deserve further investigation. A longer follow-up comparison period may also reveal that state legislation similar to MA may prophylactically prevent decrease in nurse staffing or may more readily facilitate appropriate increases in nurse staffing in response to rising patient complexity. Conversely, unintended adverse consequences (longer emergency room wait times or floor-to-ICU transfer times due to limits on ICU staffing, reduction in staffing with more experienced nurses) should be investigated as well. Finally, results may differ among hospitals that chose not to submit information to the Vizient database. Participation in Vizient is voluntary in nature and may reflect a sample of improvement-focused hospitals; however, all MA academic centers submitted data to Vizient, decreasing the likelihood that our findings do not represent MA hospital academic medical center outcomes.

CONCLUSION

In conclusion, Massachusetts General Law c. 111, § 231, which regulates the staffing assignments of nurses to patients in the ICU based upon patient acuity tools, failed to demonstrate improvements in patient mortality or complication rates among critically ill patients, potentially due to small effects on nurse staffing. Given the excess costs of enacting and enforcing state-wide legislation to implement acuity-guided ICU nurse staffing, future efforts to regulate patient-to-nurse ratios should carefully first consider the extent of the effects of the mandate on nurse staffing levels. In line with prior studies outside of the ICU, our study of ICU nurse staffing adds further evidence suggesting that state-wide legislation to mandate nurse staffing strategies may not effectively improve patient outcomes.