An Examination of Hospital Nurse Staffing and Patient Experience with Care: Differences between Cross‐Sectional and Longitudinal Estimates

Abstract

Objective

To study the association between hospital nurse staffing and Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) scores.

Data Sources

State hospital financial and utilization reports, Healthcare Cost and Utilization Project State Inpatient Databases, HCAHPS survey, and American Hospital Association Annual Survey of Hospitals.

Study Design

Retrospective study using cross‐sectional and longitudinal models to estimate the effect of nurse staffing levels and skill mix on seven HCAHPS measures.

Data Collection/Extraction Methods

Hospital‐level data measuring nurse staffing, patient experience, and hospital characteristics from 2009 to 2011 for 341 hospitals (977 hospital years) in California, Maryland, and Nevada.

Principal Findings

Nurse staffing level (i.e., number of licensed practical nurses and registered nurses per 1,000 inpatient days) was significantly and positively associated with all seven HCAHPS measures in cross‐sectional models and three of seven measures in longitudinal models. Nursing skill mix (i.e., percentage of all staff who are registered nurses) was significantly and negatively associated with scores on one measure in cross‐sectional models and none in longitudinal models.

Conclusions

After controlling for unobserved hospital characteristics, the positive influences of increased nurse staffing levels and skill mix were relatively small in size and limited to a few measures of patients' inpatient experience.

Policy makers, clinicians, and hospital administrators are focusing on patient experience as an important domain to assess the quality of the U.S. health care system. Surveys such as the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) have been used to assess patients' experiences with inpatient care (Centers for Medicare & Medicaid Services 2014a). Hospitals are being held increasingly accountable for their performance on HCAHPS. For example, the results from HCAHPS are publicly available on the Hospital Compare website (Centers for Medicare & Medicaid Services 2014b). Beginning in 2016, HCAPHS scores will be included in the calculation of the Centers for Medicare & Medicaid Services (CMS) Value‐Based Payment Modifiers, which will be used to calculate payments to hospitals on the basis of their quality of care.

Nursing care is considered a key determinant of patient experience (Larrabee et al. 2004; Kutney‐Lee et al. 2009). The front‐line nursing staff is the first and most frequent point of contact for patients. Nurses can improve the patients' experience by communicating effectively, responding to requests, and managing pain. Prior studies demonstrate that nursing communication with patients is the greatest contributor to hospitals' overall HCAHPS scores (Elliott et al. 2009) and that higher scores on nurse‐related questions are associated with increased odds of receiving the most positive HCAHPS responses (Wolosin, Ayala, and Fulton 2012).

Staffing levels and skill mix are likely factors in nurses' abilities to improve the patients' experience. Nurses who work on units that are understaffed or do not have an adequate skill mix are apt to have less time to respond to patient requests, assess their pain levels, and provide education. The objective of this study was to determine the extent to which nurse staffing and skill mix are associated with hospitals' HCAHPS scores.

Conceptual Framework

The concept of patient experience refers to patients' perceptions of different dimensions of care that they receive from their providers. These patient perceptions are captured on the HCAHPS, a validated survey used by CMS to assess patients' experience of inpatient care (Giordano et al. 2010).

In Figure 1, we show that nurse staffing and skill mix may affect patients' experience, primarily through missed nursing care (Kalisch 2006; Kalisch, Landstrom, and Hinshaw 2009; Bittner et al. 2011). Missed nursing care refers to the extent to which nurses fail to deliver necessary nursing care such as performing assessments; documenting patient care; performing vital signs; delivering as‐needed medication; ambulating, feeding, and turning patients; responding to call bells; and performing patient education. Previous studies have shown that as staffing levels increase, nurses are less likely to miss care (Kalisch, Tschannen, and Lee 2011, 2012). Another study demonstrated that patients are able to perceive and account for missed care, which may result in lower patient experience scores (Kalisch, McLaughlin, and Dabney 2012). Because of a patient's perception of missed nursing care, nurse staffing levels and skill mix may be associated with reductions in a hospital's score on selected HCAHPS measures, particularly measures that are aligned most closely with necessary nursing care that might be missed.

Figure 1.

Conceptual Model

Figure 1 also shows that hospital organizational characteristics and patient case mix are likely to be correlated with nurse staffing characteristics and with patient experience. Previous studies found that nurse staffing and patient experience scores are correlated with readily observable hospital organizational characteristics such as size, location, specialty, safety net, and profit status (Jha et al. 2008; Isaac et al. 2010; Manary et al. 2015) as well as with less observable characteristics such as nursing work environment or providers' engagement in improvement efforts (Kutney‐Lee et al. 2009; Chatterjee et al. 2012; Siddiqui et al. 2014). Furthermore, nurse staffing and patient experience are likely to be correlated with many characteristics of a hospital's patient case mix, including age, sex, race, health status, and education (Elliott et al. 2010; Goldstein et al. 2010). As demonstrated in previous studies, these patient characteristics need to be taken into account when assessing the relationship between nurse staffing and patient outcomes (Mark, Harless, and McCue 2005; Harless and Mark 2010; Newhouse et al. 2010; Mark and Harless 2011; Mark et al. 2013; Spetz et al. 2013; Martsolf et al. 2014).

Study Aims and Hypotheses

We have found important gaps in the literature related to our proposed conceptual model. Although a number of studies have investigated the relationship between nurse staffing and patient perception of care (Bolton et al. 2003; Hall et al. 2003; Lang et al. 2004; Larrabee et al. 2004; Seago, Williamson, and Atwood 2006; Clark et al. 2007; Schubert et al. 2008; Ausserhofer et al. 2013), only a small number of studies have directly investigated the effect of nurse staffing on HCAHPS scores. Previous research found that nurse staffing levels and skill mix were positively associated with hospitals' HCAHPS scores (Jha et al. 2008; Kutney‐Lee et al. 2009; Kang and Hasnain‐Wynia 2013). These studies relied on cross‐sectional designs that used multivariate regression to control for observed hospital characteristics (i.e., those characteristics that can be measured and included in a multivariate regression model) that might bias the relationship between nurse staffing characteristics and HCAHPS scores. However, the associations between nurse staffing and HCAHPS scores also could be biased by unobserved hospital characteristics that cannot be accounted for in multivariate regressions.

We believe that the lack of robust longitudinal designs in previous studies may have led to an overestimation of the effect that nurse staffing and skill mix have on patient experience. Many of these unobserved hospital characteristics that cannot be accounted for in standard, clinically based risk‐adjustment models (such as hospitals' general levels of engagement in quality, unique patient populations, or physical environment) might be time invariant within the short window of our analysis. For example, a correlation between high nurse staffing levels and high HCAHPS scores observed in a cross‐sectional analysis actually may be due to other hospital characteristics that are associated with nurse staffing. We can control for unobserved, time‐invariant hospital characteristics that could bias the relationship between nurse staffing and patient experience by using a longitudinal design. This design controls for the hospitals' baseline levels of nurse staffing and HCAHPS scores and then identifies an effect from longitudinal changes in those two variables.

To address these gaps in the literature, we used longitudinal data from U.S. hospitals in multiple states and hospital fixed effects models to account for unobserved, time‐invariant hospital characteristics that were correlated with both nurse staffing and patient experience. To our knowledge, this study is the first to examine this relationship using a rigorous longitudinal design. Our study is designed to address the following research hypotheses:

• Aim 1: To determine the extent to which nurse staffing levels and skill mix are associated with patient experience in cross‐sectional models. We hypothesize that nurse staffing and skill mix will be positively correlated with patient experience scores in the cross‐sectional models.

• Aim 2: To determine the extent to which nurse staffing levels and skill mix are associated with patient experience in longitudinal models that control for unobserved time‐invariant confounders. We hypothesize that the changes in nurse staffing and skill mix will be positively correlated with changes in patient experience scores in the longitudinal models.

Methods

Data Sources

We collected secondary hospital‐level data for the years 2009–2011 from four sources: state hospital financial and utilization reports, Hospital Compare (Centers for Medicare & Medicaid Services 2014b), Healthcare Cost and Utilization Project (HCUP) State Inpatient Databases (SID), and the American Hospital Association (AHA) Annual Survey of Hospitals (American Hospital Association Health Forum 2014). We calculated measures of nurse staffing and skill mix from the state hospital financial and utilization reports for hospitals in California, Nevada, and Maryland. We focused on these three states because they had nursing data that could be linked to Hospital Compare and HCUP SID data. We used Hospital Compare data on patient experience (described in the measures section below) from the 510 hospitals that had at least 1 year of HCAHPS data. These data are reported annually by the hospitals and are publicly available on the CMS data download website (Centers for Medicare & Medicaid Services 2014c).

For the control variables, we used discharge‐level data from HCUP SID to construct hospital‐level variables that describe the demographic and clinical characteristics of the patient populations. We used linked data from the AHA to describe hospitals' characteristics (e.g., size, teaching status).

We omitted 96 hospitals that did not have at least two consecutive years of data that were children's or long‐term stay hospitals, or that had outlier values for nurse staffing. We defined outlier nurse staffing values as those that were less than 1 or greater than 100 nurses per 1,000 inpatient days or as those that more than doubled (i.e., greater than a 100 percent change) or less than halved (i.e., less than a negative 50 percent change) from 1 year to the next. We believe that such drastic values were likely to be data reporting errors.

We also eliminated 73 hospitals that had missing data in the regression models. The final sample size for this analysis included 341 individual hospitals and 977 individual hospital‐year combinations. Across the three states, 80.3 percent of hospitals were in California, 13.2 percent in Maryland, and 6.6 percent in Nevada.

Measures

The dependent variables listed below consisted of 7 of the 10 HCAHPS measures reported on Hospital Compare (Centers for Medicare & Medicaid Services 2013) that were most likely to be affected by nurse staffing and skill mix:

• Five HCAHPS composite measures: Communication with Nurses, Responsiveness of Hospital Staff, Pain Management, Communication about Medicines, and Discharge Information

• Two summary measures: Overall Hospital Rating and Recommend Hospital

Our analyses focused on the publicly reported scores that were calculated using the top‐box methodology (Centers for Medicare & Medicaid Services 2014a). CMS calculates top‐box scores using three steps. First, the percentage of patients who scored each item with the most positive response is calculated: always for the five HCAHPS composite measures, yes for the “discharge information” and definitely for the “recommend hospital” categories, and 9 or 10 for “overall hospital rating.” Second, survey mode adjustment and hospital‐level, patient‐mix adjustment for such factors as education, self‐rated health, emergency department admission, and service line are applied to these unadjusted top‐box scores before they are made available through the CMS website. Third, publicly reported HCAHPS scores are created using the average of the adjusted scores from four consecutive quarters. Higher hospital top‐box scores represent higher ranks among participating hospitals.

The explanatory variables consist of two nurse staffing variables for each hospital‐year combination. One variable, the number of licensed nurses (registered nurses [RNs] and licensed practical nurses [LPNs]) per 1,000 inpatient days, reflected the level of nurse staffing at each institution. A second variable, the proportion of all nursing staff (RNs, LPNs, and aides) who were RNs, reflected the skill mix of each hospital. We constructed the number of licensed nurses per 1,000 inpatient days for each hospital using inpatient days reported in the HCUP SID. To account for the fact that many hospitals do not distinguish between inpatient and outpatient nurses, we calculated adjusted patient days by multiplying the number of patient days by the ratio of inpatient‐to‐outpatient revenue (Spetz et al. 2008).

The control variables consisted of a number of hospital‐level covariates reflecting changes over time in patient characteristics in a given hospital that we expected would correlate with the outcomes of interest. We included only those covariates that were not included in the patient‐mix adjustment that CMS applies to the HCAHPS data before they are released. We included the distribution of hospitals' patients based on age, sex, primary payer, racial/ethnic minority status, and income quartile for patients' ZIP Code of residence. We also included the proportion of the hospitals' patients who had been diagnosed with each of 21 comorbidities, such as metastatic cancer and renal failure, contained in Agency for Healthcare Research and Quality (AHRQ) comorbidity software (Healthcare Cost and Utilization Project 2013). Finally, we included the average Medicare Severity‐Diagnosis Related Group (MS‐DRG) weight (United States Department of Veterans Affairs 2013) in the hospital. We added DRG weight to account for the change in patients' case mix characteristics reflected by the combination of DRGs over time in a given hospital. DRG weights were constructed to estimate the relative resource use required to care for patients within each DRG. Although not designed for the purposes of our study, we believe that using DRG weights is a reasonable approach to account for variation over time in the DRG mix without having to add separate variables for each of the more than 700 individual DRGs, which would absorb the available degrees of freedom in our models.

In cross‐sectional models, we included specific hospital‐level characteristics from the AHA, including size based on the number of beds (small, medium, large), ownership status (public, not‐for‐profit, for‐profit), teaching status (teaching, nonteaching), and urban/nonurban location (located in large and small metro areas, not located in large or small metro areas). Size (number of beds) was based on a standardized variable within the Healthcare Cost and Utilization Project that varies by location (urban, rural) and teaching status (Healthcare Cost and Utilization Project 2008). We analyzed these variables and found them to be largely time invariant. Therefore, we did not include them in the models that included hospital fixed effects, which already controlled for hospitals' time‐invariant characteristics.

Analysis

First, we performed a pooled cross‐sectional regression analysis that controlled for the hospital‐level, time‐variant covariates, including the distribution of the hospitals' patients by age, primary payer, income quartile, comorbidities, race/ethnicity, and the average DRG weight. We also included the hospitals' size based on the number of beds, ownership, teaching status, and urban/rural location. This model controlled for observable hospital and patient characteristics that might account for differences in HCAHPS scores across hospitals. We included time fixed effects to account for temporal trends in HCAHPS scores and nurse staffing that affect all hospitals. For each regression model, we used a robust sandwich estimator to adjust for autocorrelation associated with repeated observations on the same hospitals. We weighted each regression by the number of discharges at each hospital to take into account the size of each hospital. We used linear ordinary least squares (OLS) regression models. Specifically, we estimated the following regression equation:

$$Y_{ht}={\mathit{\beta }}_0+{\mathit{\beta }}_{1}Staf{f}_{ht}+{\mathit{\beta }}_{2}Mi{x}_{ht}+X_{ht}{\mathit{\beta }}_3+{\mathit{\delta }}_t+{\mathit{ϵ}}_{ht}$$ (1)

where Y represented each of the seven different HCAHPS measures, Staff represented staffing levels, Mix represented skill mix, X represented a vector of hospital characteristics that included both patient characteristics and hospital structural attributes, δ represented year fixed effects, and ε represented the stochastic error term. The variables were indexed by hospital (h) and time (t).

Next, we analyzed models that were similar to those described in equation (1) but included hospital‐level fixed effects. These fixed effects accounted for the hospitals' unobserved time‐invariant characteristics that were correlated with both nurse staffing and HCAHPS scores. In such a specification, the model becomes longitudinal in nature by identifying the effect estimate via longitudinal changes in nurse staffing and HCAHPS scores. Specifically, we estimated the following regression equation:

$$Y_{ht}={\mathit{\beta }}_0+{\mathit{\beta }}_{1}Staf{f}_{ht}+{\mathit{\beta }}_{2}Mi{x}_{ht}+X_{ht}{\mathit{\beta }}_3+{\mathit{\delta }}_t+{\mathit{\gamma }}_h+{\mathit{ϵ}}_{ht}$$ (2)

The primary difference between equations (1) and (2) was the addition of a hospital‐level fixed effect (γ). By estimating a unique intercept for each hospital, the hospital fixed effect accounted for any unobserved, time‐invariant hospital characteristics that were correlated with both nurse staffing and patient experience. Including the hospital fixed effect in the longitudinal models also allowed us to test how changes in nurse staffing within these sampled hospitals were associated with changes in HCAHPS scores over the study period.

We performed a number of sensitivity analyses. First, we calculated unadjusted nurse staffing levels. Second, we reestimated the models after limiting the sample to hospitals that had at least 100 and 300 patients responding to the HCAHPS survey, respectively. We performed this reestimation because HCAHPS scores are measured with higher levels of measurement error for hospitals with fewer respondents, and measurement error tends to attenuate effects toward zero (Wooldridge 2012). Third, we estimated the regression models without weighting. Finally, we ran the models separately for each state to account for differences in staffing environments across the states (e.g., staffing mandates in California).

Results

Table 1 shows that the hospitals in this study, on average, staffed 4.5 licensed nurses per 1,000 adjusted inpatient days. RNs represented 74.1 percent of all nursing staff. The mean HCAHPS scores for the sample of hospital‐year combinations ranged from 56.7 (staff explaining medications) to 79.7 (staff provided discharge information). The hospitals were well distributed by size, with 21.4 percent in the small, 33.7 percent in the medium, and 44.9 percent in the large category. Most hospitals were not‐for‐profit (62.0 percent) and almost all hospitals were urban (91.8 percent).

Table 1.

Percentages and Standard Deviations of the HCAHPS Scores, Organizational Characteristics, and Patient Case Mix Characteristics

Variables	Mean	SD
Nurse staffing: adjusted total number of nurses per 1,000 inpatient days	4.5	1.4
Skill mix: % of nursing staff (LPN, RN, aides) that are RNs	74.1	12.4
HCAHPS top‐box scoresa,b
Nurses always communicated well	71.1	6.6
Patients always received help as soon as they wanted	57.3	8.7
Pain was always well controlled	66.2	6.0
Staff always explained (medications)	56.7	6.5
Yes, staff did give patients this (discharge) information	79.7	5.1
Patients who gave a rating of 9 or 10 for overall hospital rating	63.8	9.7
Yes, patients would definitely recommend the hospital	67.1	11.0
	Percent
Organizational characteristics
Size (number of beds)c
Small	21.4
Medium	33.7
Large	44.9
Ownership status
Not‐for‐profit	62.0
For‐profit	21.7
Public	16.3
Urban/nonurban location
Urban	91.8
Nonurban	8.2
Patient case mix characteristics
Age distribution of discharges, years
18–34	20.7	10.0
35–44	10.5	3.1
45–54	13.7	4.1
55–64	15.1	3.5
65–74	14.8	3.9
75+	25.1	10.0
Female sex	60.6	7.2
Primary payer
Private	26.3	13.5
Medicare	42.6	12.7
Medicaid	20.9	14.0
Other	10.1	7.9
Hospital admissions originating in the emergency department	58.6	19.1
Patient zipcode income quartile
Quartile 1	18.3	23.0
Quartile 2	20.0	19.1
Quartile 3	30.0	21.4
Quartile 4	31.7	27.9
Patient race/ethnicity status
Black	11.0	14.9
Hispanic	22.0	20.0
Comorbiditiesd
Rheumatoid arthritis/collagen vascular diseases	2.2	0.8
Congestive heart failure	7.8	3.1
Hypertension (combine uncomplicated and complicated)	45.2	9.2
Hypothyroidism	10.1	3.2
Liver disease	3.3	1.5
Fluid and electrolyte disorders	22.0	8.5
Metastatic cancer	1.8	0.9
Other neurological disorders	7.3	2.8
Paralysis	2.6	1.3
Peripheral vascular disorders	4.7	2.2
Renal failure	10.7	4.5
Solid tumor without metastasis	1.7	0.7
Valvular disease	3.4	2.1
Acquired immune deficiency syndrome/lymphoma/peptic ulcer disease excluding bleeding	0.9	0.6
Deficiency anemias/chronic blood loss anemia/coagulopathy	22.9	7.1
Chronic pulmonary disease/pulmonary circulation disorders	17.8	5.4
Diabetes, uncomplicated/diabetes with chronic complications	21.9	5.6
Alcohol abuse/drug abuse	8.5	5.7
Obesity/weight loss	14.8	6.8
Depression/psychoses	12.8	4.9

^aTop‐box scores are adjusted scores calculated using a CMS methodology; the scores reflect the percentage of patients who scored each HCAHPS measurement with the most positive response category (Centers for Medicare & Medicaid Services 2014a).

^bPercentages are specific to each item and therefore total more than 100%.

^cSize categories (number of beds) are assigned to hospitals on the basis of the hospitals' region, urban/rural location and teaching status. More information is available at https://www.hcup-us.ahrq.gov/db/vars/hosp_bedsize/nisnote.jsp.

^dThe comorbidities used in this study are those contained in Agency for Healthcare Research and Quality (AHRQ) comorbidity software (Healthcare Cost and Utilization Project 2013).

HCAHPS, Hospital Consumer Assessment of Healthcare Providers and Systems; LPN, licensed practical nurse; RN, registered nurse.

Patient ages were well distributed across the age categories, ranging from 10.5 percent in the 35–44 years age group to 25.1 percent in the 75 years and older age group. Female patients accounted for a greater percentage of discharges (60.6 percent) than male patients. Medicare was the primary payer for 42.6 percent of the hospitals' discharges, followed by private insurance (26.3 percent), Medicaid (20.9 percent), and other sources (10.1 percent, including self‐pay and no charge). More than half (58.6 percent) of patients who were admitted originated in the emergency department. The top three comorbidities were hypertension (45.2 percent), deficiency anemias/chronic blood loss anemia/coagulopathy (22.9 percent), and fluid and electrolyte disorders (22.0 percent).

Table 2 shows the average values of nurse staffing and HCAPS scores in 2009 and 2011 as well as the average absolute change per hospital. We found relatively little difference between 2009 and 2011 in average nurse staffing (4.44 and 4.53 nurses per 1,000 inpatient days, respectively) and skill mix (73.1 and 74.50 for RNs as a percent of all staff). However, the average, absolute within‐hospital change was 0.46 nurses per 1,000 inpatient days and 3.09 percent for the proportion of all staff who were nurses. The average absolute within‐hospital change in the HCAHPS measures ranged from 3.32 to 4.34 points.

Table 2.

Mean Scores and Standard Errors in Nurse Staffing and HCAHPS Scores in 2009 and 2011 and Average Change across These Years

Measures	2009		2011		Average Absolute Change per Hospital
	Mean	SD	Mean	SD	Mean	SE
Nurse staffing
Nurse staffing: adjusted total number of nurses per 1,000 inpatient days	4.44	1.50	4.53	1.32	0.46	0.49
Skill mix: % of nursing staff (LPN, RN, aides) that are RNs	73.31	13.15	74.50	12.36	3.09	3.47
HCAHPS scores (top‐box)a
Communication with nurses: nurses always communicated well	69.80	7.02	72.17	6.16	3.42	3.01
Responsiveness of hospital staff: patients always received help as soon as they wanted	56.25	9.16	58.54	8.50	4.23	3.72
Pain management: pain was always well controlled	65.40	6.60	66.86	5.77	3.41	3.57
Communication about medications: staff always explained	55.40	6.95	58.05	6.11	4.30	4.35
Discharge information: yes, staff did give patients this information	78.28	5.89	81.12	4.18	3.32	3.42
Overall hospital rating: patients who gave a rating of 9 or 10 (high)	62.29	10.45	65.22	9.10	4.34	3.93
Recommend hospital: yes, patients would definitely recommend the hospital	66.08	11.69	67.87	10.41	3.99	3.88

^aTop‐box scores are adjusted scores calculated using a CMS methodology; the scores reflect the percentage of patients who scored each HCAHPS measurement with the most positive response category (Centers for Medicare & Medicaid Services 2014a).

HCAHPS, Hospital Consumer Assessment of Healthcare Providers and Systems; LPN, licensed practical nurse; RN, registered nurse; SD, standard deviation; SE, standard error.

The results of the regression analyses for the seven HCAHPS measures are shown in Table 3. We present two sets of results for the effect of nurse staffing levels and skill mix on each HCAHPS measure; the first set of results represents pooled cross‐sectional regression models that included only the covariates, and the second set represents the longitudinal regression models that included hospital fixed effects.

Table 3.

Associations between Nurse Staffing and Skill Mix and Hospital Consumer Assessment of Health Care Providers and Systems Measures in Pooled Cross‐Sectional and Longitudinal Regression Models

Variable	Cross‐Sectional				Longitudinal
	Staffing Levela		Skill Mixb		Staffing Levela		Skill Mixb
	β	SE	β	SE	β	SE	β	SE
Communication with nurses	.541d	.168	−.030	.021	.457	.327	−.03	.029
Responsiveness of hospital staff	.842d	.244	−.041	.031	.461	.472	−.074	.042
Pain management	.439d	.157	−.019	.02	−.031	.345	−.065	.04
Communication about medicines	.559d	.180	−.061c	.025	.721c	.364	−.016	.039
Discharge information	.378c	.147	−.01	.018	.734c	.264	−.009	.026
Overall hospital rating	.555d	.218	−.014	.03	.646	.34	−.012	.045
Recommend hospital	.641d	.223	.019	.032	1.169d	.355	−.024	.055

^aNurse staffing level was based on the adjusted total number of nurses per 1,000 inpatient days.

^bSkill mix was based on the percentage of all staff (RN, LPN, and aides) that are RNs.

^c.01 ≤ p < .05.

.001 ≤ p < .01.

^d p < .001.

β, Beta coefficient; SE, standard error.

In the pooled cross‐sectional models that included only the covariates, nurse staffing levels were significantly associated with each of the HCAHPS measures. For example, a difference of one licensed nurse per 1,000 inpatient days was associated with a 0.842 percentage point difference in HCAHPS scores for the “responsiveness of hospital staff” measure. Skill mix was negatively significantly associated with the “communication about medications” (β = 0.061).

In the models that included hospital fixed effects (i.e., longitudinal), many of these relationships were no longer statistically significant. Nurse staffing levels were significantly associated with the “communications about medications” measure (β = 0.721), “discharge information” measure (β = 0.734), and the “recommend hospital” measure (β = 1.169). Skill mix was not associated with any of the measures.

We found that all of our estimates were robust to different specifications. However, we also found substantial differences across states, with the results for California being much smaller than those for Nevada and Maryland.

Discussion

This study examines the effect of nurse staffing levels and skill mix on patient‐reported experience. We found that staffing levels were positively associated with improvements in all of the HCAHPS scores in our pooled cross‐sectional analyses. These results suggest that in a cross‐sectional design, hospitals with higher nurse staffing levels appear to achieve higher scores in all key domains of patient experience as measured by the HCAPHS. However, in hospital fixed effect longitudinal analyses that control for unobserved time‐invariant differences in hospital characteristics, many of these associations were no longer statistically significant. Nurse staffing levels remained positively and significantly associated only with the discharge information, communication about medications, and recommend hospital measures.

Our findings largely confirm previous cross‐sectional analyses that identified a statistically significant association between nurse staffing and HCAHPS scores. Specifically, Jha et al. (2008) found that hospitals in the highest quartile of nurse staffing ratios had significantly better patient experience on all seven HCAHPS measures, with the differences ranging from 0.9 to 4.2 percentage points. However, we found that many of our cross‐sectional relationships were no longer statistically significant in the fixed effect models. This change in statistical significance likely can be explained by two mechanisms that are specific to the longitudinal fixed effect models.

First, these longitudinal models only use variation within individual hospitals across years to estimate effects. Using such an approach leads to larger standard errors than those obtained from pooled cross‐sectional models, especially if the within‐hospital variation over time is relatively small. Inflation of the standard errors may lead to the elimination of statistical significance. For one of the measures (overall hospital rating), the coefficient increased but the standard errors also increased substantially. For another, the coefficient decreased slightly but the standard error nearly doubled (communication with nurses). These findings suggest that inflation of the standard errors likely contributed to the elimination of statistical significance in these two cases.

The second potential mechanism for explaining the difference in statistical significance between the pooled cross‐sectional and longitudinal models involves the correction of bias based on the addition of the hospital fixed effect. The fixed effect estimator may account for bias caused by unobserved hospital characteristics that were not accounted for in the cross‐sectional models. In this case, adding the fixed effect would lead to a change in the coefficients. Although it is possible for bias correction to lead to an increase in the coefficients, in this case, the bias correction that would lead to loss of statistical significance is through smaller coefficients as all of the coefficients in the cross‐sectional model are greater than zero. For two measures (“responsiveness of staff” and “pain management”), the introduction of the hospital fixed effect led to moderate to large reductions in the coefficients. This finding suggests that the fixed effect addressed upward bias in the coefficient caused by unobservable hospital characteristics. The reduction in the coefficients likely contributed to elimination of statistical significance, although standard errors also increased. Therefore, we cannot exclusively attribute loss of statistical significance to bias correction alone.

To the extent that bias correction led to loss of statistical significance, we suspect that one key unobserved variable may be the nursing work environment. Our results are very similar to those found by Kutney‐Lee et al. (2009) that demonstrated a correlation between nurse staffing and improvements in the recommend hospital and the provision of discharge information measures. These referenced analyses were cross‐sectional, but they included key covariates for nursing work environment—a measure of nursing leadership capacity, nurse's participation in hospital affairs, and nurse–physician relationships. This result suggests that nursing work environment may be an important unobserved and largely time‐invariant hospital characteristic that is correlated with nurse staffing and may be driving observed cross‐sectional associations between nurse staffing and HCAHPS scores.

Our finding that nurse staffing levels are positively associated with patients' experiences with the discharge information measure is of particular interest. As hospitals are being held increasingly accountable for readmission rates, the effective transition of patients from hospital to home or other settings is growing in importance. Nursing discharge education will become a progressively more important part of that transition. Our study demonstrates that when a hospital has more nursing staff, nurses are able to spend more time providing discharge education, which is a vitally important nursing function. Educating patients and families (i.e., providing discharge instructions) is a frequently omitted nursing care activity—an omission that would increase when staffing levels are low (Ausserhofer et al. 2014). These findings are consistent with prior literature finding a positive correlation between inpatient nurse staffing and readmissions (Weiss, Yakusheva, and Bobay 2011; McHugh, Berez, and Small 2013; McHugh and Ma 2013; Ma, McHugh, and Aiken 2015).

It is important to note that the effects we identified were relatively small. An increase in nurse staffing of one nurse per 1,000 inpatient days (roughly equivalent to the mean absolute change in staffing levels) would lead to 0.721 percentage point increase in HCAHPS scores related to the delivery of discharge information and 1.169 percentage point increase for recommendation of the hospital. In addition to the methodological considerations discussed above, these modest results may be driven by the fact that patient experience may be influenced less by overall staffing levels and more by the qualities of the nursing staff such as educational levels and interpersonal skills (Lang et al. 2004), neither of which we were able to measure.

Our study has a number of limitations. First, although the sample includes multiple states, 80 percent of the observations are from California. The observed results are largely being driven by observations from that state, which may have particular implications for the nurse skill mix variable. In 2004, the state of California implemented legislation for the minimum nurse‐to‐patient staffing requirements (California HealthCare Foundation 2009). However, we found that California's staffing levels and within‐hospital change in staffing were similar to the other states in our sample. Second, our results are based on hospital‐level analyses. Therefore, we are unable to tie the nurse staffing levels directly to the outcomes of any individual patient. Previous studies have attempted to link nurse staffing more closely to patient‐level experience by using unit‐level staffing and patient perception measures (Seago, Williamson, and Atwood 2006; Ausserhofer et al. 2013). No studies to date, however, have attempted to investigate patient‐level care experience outcomes in relation to nursing staffing levels and skill mix. Recently, studies emerged linking other patient‐level outcomes (e.g., readmissions and length of stay) to patient‐level nurse staffing or other characteristics of nurses (e.g., educational levels) (Yakusheva, Lindrooth, and Weiss 2014a,b). Extending these patient‐level analyses to patient experience would require significant primary data collection efforts, although such work would be a valuable contribution to the literature. Third, our fixed effect model was unable to control for any unobservable time‐varying hospital characteristics that might be correlated with both temporal changes in patient experience and nurse staffing. We controlled for a large number of observable time‐varying case mix characteristics, but we were unable to control for other potentially important unobserved hospital characteristics that might have changed over time, such as magnet status.

In summary, our study shows that nurse staffing levels are significantly associated with scores on a number of HCAHPS measures. However, given the relatively small effect sizes, hospitals may need to increase their nurse staffing substantially to achieve meaningful improvements on patient experience scores. As hospitals are increasingly subject to pay‐for‐performance and public reporting of patient experience scores, understanding the determinants of improvements in patient experience can assist policy makers with evaluating current measures and help hospitals improve their care.

Supporting information

Appendix SA1: Author Matrix.