The Relationship Between Reported Staffing and Expenditures in Nursing Homes

Abstract

Dramatic improvements in reported nursing home quality, including staffing ratios, have come under increased scrutiny in recent years because they are based on data self-reported by nursing homes. In contrast to other domains, the key mechanism for real improvement in the staffing ratios domain is clearer: to improve scores, nursing homes should increase staffing expenditures. We analyze the relationship between changes in expenditures and reported staffing quality pre– versus post the 5-star rating system. Our results show that the relationship between expenditures and licensed practical nurse staffing is weaker in the post-5-star period, overall, and across subgroups; furthermore, there is a weaker relationship between expenditures and registered nurse staffing among for-profit facilities with a high share of Medicaid residents in the post-5-star period. The weaker relationship between staffing expenditures and staffing scores in the post-5-star era underscores the potential for gaming of the self-reported staffing scores and the need for more reliable sources.

Introduction

Nursing home costs and quality are important to Medicaid and Medicare. Approximately 1.4 million residents reside in about 15,000 nursing homes in the United States (National Center for Health Statistics, 2013) at a substantial cost to Medicare and Medicaid. Medicare spent approximately $28.7 billion on post–acute care in skilled nursing facilities in 2013 (Medicare Payment Advisory Commission, 2015), and Medicaid paid about $50 billion to nursing facilities in 2014 for long-term care services (Kaiser Family Foundation, 2014). Since Medicaid and Medicare cover a large portion of expenditures for nursing home residents, these programs have an incentive to promote increased efficiency and quality of care in nursing homes.

One potential way to improve quality is to publicly disseminate quality information. The role of information in well-functioning markets dates back to influential articles published in the 1960 to 1970s (Akerlof, 1970; Arrow, 1963). In recognition of the importance of quality information for nursing homes, the Centers for Medicare & Medicaid Services (CMS) started publicizing information on nursing home quality in 1998 through the Nursing Home Compare website. The publication of quality information was aimed at promoting quality of care through consumer and provider response. Several early studies suggested that nursing homes responded to the publication of nursing home quality information by improving on at least some quality measures. For example, there were improvements in reported quality for pain and physical restraints but not on pressure ulcers following the release of quality information (Mukamel, Weimer, Spector, Ladd, & Zinn, 2008). In a study focused on post–acute care quality measures, there were improvements in quality measures related to pain and walking but not delirium (Konetzka & Werner, 2009).

Despite these results, there was speculation that consumers had a difficult time using the vast amount of reported quality information in a meaningful way. Accordingly, in late 2008, CMS simplified the reporting system by assigning star ratings to nursing homes (1 to 5 stars, with 5 stars indicating the highest level of quality). These included both overall ratings and ratings specific to regulatory deficiencies, clinical quality, and staffing ratios. The calculation of the overall rating starts with the deficiency measure, considered the most objective because it is based on inspections by state surveyors but allows for the addition or subtraction of stars based on performance on staffing and clinical quality. For example, during the time period of our study, a nursing home with a 4-star or 5-star rating in staffing could shift up its overall rating by as much as one star. Staffing measures include separate measures for registered nurse (RN) staffing and total direct-care staffing (RNs, licensed practical nurses [LPNs], and nurse aides). Staffing is often considered a key marker of nursing home quality, given that nurses and nurse aides provide the bulk of direct care.

Based on these star levels, average reported nursing home quality started improving dramatically after the launch of the 5-star system. Specifically, the number of facilities rated as 5 stars increased from 12% to 15% from 2008 to 2010, while those rated as 1 star decreased from 23% to 16% during the same period (Williams, White, Muma, & Hadden, 2014). These trends were driven largely by improvements in domains based on data self-reported by nursing homes, that is, staffing ratios and clinical quality. For instance, there was a considerable increase in the number of facilities with a 4-star rating for staffing from 31% to 38% between 2008 and 2010 while those with a 1-star rating for staffing decreased dramatically from 23% to 15% during the same period. The number of facilities with a 5-star rating for staffing increased modestly from 7% in 2008 to 9% in 2010 (Abt Associates, 2014).

The rapid improvements in reported quality following the release of 5-star system, particularly in the domains that are based on self-reported data, have raised some concerns as to whether the quality improvements are real. An article published in The New York Times suggested that nursing homes are not only able to manipulate self-reported quality measures but that they also often anticipate and can better prepare themselves for inspections, potentially temporarily adding staff (Thomas, 2014). A recent study found that some people did not trust the quality information on Nursing Home Compare, a CMS website that allows people to access quality ratings for nursing homes (Konetzka & Perraillon, 2016); the reasons for distrust could be lack of awareness and/or negative publicity in the media.

When consumers use the 5-star system, the overall star rating tends to dominate (Konetzka & Perraillon, 2016). A closer look at how 5-star rating for staffing affects the overall 5-star rating for a facility allows us to understand the incentives nursing homes might have to game the system. Nursing home inspections for health deficiencies are carried out by state agencies once every 12 to 15 months and an aggregate deficiency score is obtained based on the number and severity of deficiencies. A weighted average is calculated from the past three inspections for each facility. Five-star ratings for deficiencies are then calculated for each facility based on the distribution of these weighted averages within each state. In addition, staffing and quality 5-star ratings are calculated separately based on self-reported data from nursing homes. If the staffing rating is 4 or 5 stars, the overall rating is increased by 1 star so long as the staffing ratings exceed the deficiency ratings. For instance, if the rating based on deficiency scores is 3 stars, and staffing rating is 4 stars, the overall rating becomes 4 star; alternatively, if the staffing rating is 1 star, then the overall rating is decreased to 2 stars. Thus, staffing ratings can not only increase but also decrease the overall rating. Similarly, quality ratings can also increase or decrease the overall rating by 1 star.

The ability to shift the overall rating gives nursing homes strong incentive to achieve a high rating on staffing (and some consumers may look at the staffing ratings directly). However, while inspections are carried out by independent inspectors, data for staffing is self-reported by nursing homes based on their staffing levels during the 14 days prior to the health inspections. As suggested in The New York Times article (Thomas, 2014), nursing homes are often able to anticipate the inspection dates and thus, increase staffing during the 14 days prior to inspections. Furthermore, they could misreport their staffing levels, although evidence on this has not been documented. Using the self-reported staffing data in OSCAR (now CASPER), case-mix adjusted RN hours per resident day and total staffing hours per resident day are obtained. Then, using fixed thresholds and percentiles on a national level, each facility gets a separate 5-star rating for RN staffing and total staffing. The star rating for staffing is obtained by giving equal weights to separate star ratings for RN and total staffing (RN + LPN + Nurse aides). For instance, to get a 5-star rating in overall staffing, a facility needs to obtain 5-star ratings for both RN staffing and total staffing (Abt Associates, 2014).

A fundamental challenge facing both policymakers and consumers is that little is known about exactly how facilities improve their performance in any of the domains. When the processes for improvement are uncertain or unknown, it is difficult to assess when we should applaud dramatic improvement and when we should be skeptical that improvement is too dramatic. In this paper, we begin to address that challenge by assessing one main pathway to improved scores: increasing spending on staffing. Unlike the other domains, it is difficult to imagine genuine and sizable improvements in staffing ratios without a corresponding increase in spending. Minor improvements may be achieved through the hiring of cheaper staff, for example, either by hiring less experienced nurses of the same type or substituting among nurse types (using more LPN hours in place of RN hours). There is some evidence for factor substitution in nursing homes from nurses to nonlabor materials when nursing wages increase (Cawley, Grabowski, & Hirth, 2006) or from nonclinical staff to clinical staff when states initiate minimum clinical staffing requirements (Bowblis & Hyer, 2013). Although we cannot rule out all these other pathways, a high correlation between improvement and additional expenditures would provide some assurance that reported improvements in staffing ratios may be genuine. Our approach is to link the raw scores for staffing measures used by CMS to Medicare cost reports as well as nursing home level information available in Online Survey, Certification, and Reporting (OSCAR) data. We use facility fixed-effects regression to assess whether better reported staffing quality scores are associated with higher nursing home expenditures, and whether this relationship is different in the post-5-star era.

New Contribution

Nursing home reported quality for staffing improved rapidly since the CMS started assigning star ratings to nursing homes in 2008. However, to our knowledge, no study has evaluated whether the improvements in reported staffing ratios in Nursing Home Compare are corroborated by additional spending on staffing. Our study has three key contributions. First, we employ a new approach to test gaming in quality improvement efforts. Our approach to testing gaming relies on the assumption that if there is no gaming, then the relationship between quality and expenditures should be the same before and after a policy change. This approach not only applies to staffing quality in Nursing Home Compare but could be utilized in other settings including hospitals and home health agencies as long as the quality measure in question can be tied conceptually to expenditures. Second, we assess whether gaming actually occurred with respect to different types of staffing in nursing homes. We identify the relationship for each type of staff—RN, LPN, and nurse aides, allowing us to explore the staffing types that are more likely to be the subject of gaming. Assuming that the reported staffing quality measures identified by CMS are associated with improved patient outcomes, if nursing homes are spending more to improve reported staffing ratios, it provides us with additional confidence that improvements in these reported quality measures may be real and are likely to benefit residents. If, however, the relationship between expenditures and staffing scores is weaker in the post-5-star era, it is possible that nursing homes may be engaged in strategies that do not reflect true changes in staffing, such as bending the definition of which staff should be included in the ratios or “staffing up” before an inspection. Finally, we explore the heterogeneity across different types of facilities including profit status, Medicaid share, initial quality ratings, and cross-stratified by profit status and Medicaid share to identify the types of facilities that are more likely to engage in gaming. As CMS continues to refine the 5-star system, filling this gap provides essential evidence about the validity of the measures and the underlying data as well as insights into provider response and the potential need to monitor certain types of staffing and/or facilities.

Conceptual Framework

Nursing homes are predominantly for-profit firms, and like other firms have incentives to maximize profits by pursuing quality improvement as efficiently as possible and to the extent that it is profitable for them. Even not-for-profit nursing homes (the minority), while they may have objectives other than profits, need to balance their investment in quality improvement against the returns on their investment. Prior research has shown that improving reported quality can improve financial performance of nursing homes (Park, Konetzka, & Werner, 2011). An increase in reported quality can increase profits if the additional costs of improving quality are lower than the increased revenue that results. Improved quality can attract high-margin patients (i.e., Medicare or private-pay residents), and also allow for higher prices to private-pay residents to reflect the increased quality; either could boost profits for nursing homes.

Despite the theoretical motivations for improving quality, whether a nursing home needs to invest more resources to improve reported quality is not clear and varies by domain. To improve staffing scores, nursing homes theoretically have to hire more staff for a given patient case-mix, and in most instances, such additional hires will increase expenditures. However, in reality, they could also achieve the goal of improving staffing scores without increased expenditures through changes to documentation or by temporarily hiring staff before an inspection (when the data are collected). Alternative routes to a better score on staffing might include the substitution of more expensive staff with cheaper staff. There is some evidence that nursing homes substitute from nonclinical staff to clinical staff in response to the state requirements on minimum clinical staffing in nursing homes (Bowblis & Hyer, 2013). While the 5-star system did not mandate minimum clinical staffing, it is possible that nursing homes may prioritize the type of staffing that would boost their staffing ratings.

The decision to hire more staff to improve scores versus less costly mechanisms to improve scores may depend on baseline quality, Medicaid share, and profit status. On one hand, there is evidence for diminishing returns to additional staffing on mortality (Mark, Harless, McCue, & Xu, 2004) and adverse events (Blegen & Vaughn, 1998) and for diminishing returns in quality improvement more generally (Kesteloot & Voet, 1998). Thus, nursing homes that start with low staffing scores may have the most to gain from hiring new staff. On the other hand, nursing homes that start with low quality or those with a higher share of Medicaid patients are also the most likely to face resource constraints that may impede the desire to hire more staff, and may also have the most to gain from increasing reported quality scores even in the absence of real improvement. Similarly, for-profit facilities have a strong incentive to maximize profits and thus, may pursue less costly paths to improve staffing scores. Consequently, for-profit facilities, low-quality facilities or those facilities with higher Medicaid-share may look for multiple mechanisms to improve their star ratings for staffing in the post5-star period including reallocation of staff, misreporting the staffing numbers, and hiring more staff temporarily prior to inspection.

Finally, if a nursing home decides to hire more clinical staff in order to improve quality and/or reported quality ratings, there remains the decision of which types of staff to hire. A large body of literature supports the importance of RN staffing as a key driver of nursing home quality, over and above the role of other types of staff (Cohen & Spector, 1996; Konetzka, Stearns, & Park, 2008; Weech-Maldonado, Meret-Hanke, Neff, & Mor, 2004); thus, nursing homes may choose to prioritize RN staffing. This is reinforced through the star rating system, as the overall staffing rating formula gives equal weight to RN staffing and total staffing, effectively double-counting RN hours. However, because RNs are paid more than other nurses, facilities that are facing severe financial constraints or facilities with strong profit incentives might be motivated to game the RN hours. A secondary strategy to improve overall staffing while fulfilling some RN functions (at lesser cost) would be to increase LPN hours.

Method

Data

We use three main data sources: Medicare Cost Reports (2007–2010), OSCAR (2007–2010), and raw Nursing Home Compare rating scores (2007–2010).

We primarily use Medicare Cost Reports to obtain nursing-home-level information on several expenditure measures. Medicare-certified nursing homes are required to file annual Medicare Cost Report (also known as Health Care Cost Report Information System or HCRIS-CMS Form 2540–96) containing information on facility characteristics and expenditures. A small fraction of nursing homes report costs for less than 365 days and when that happens, we combine multiple reports to ensure that the facilities included in our analysis have costs reported for a full 12 months. It is a common practice to require a 12-month cost report in analysis (Bowblis & Brunt, 2014). We also follow the literature to exclude outliers for expenditures and staffing ratios that fall outside of 1st and 99th percentiles (Doyle, Graves, & Gruber, 2017) and this ensures that we have reasonable, positive costs.

Total expenditures in Medicare cost reports can be divided into several components including clinical expenditures, capital expenditures, and administrative expenditures. These expenditures can be further divided into staffing and nonstaffing components. For our analysis, we focus on clinical staffing expenditures (CMS Form 2540–96, Worksheet A, lines 9, 16–20). While we do not have salary information specific to RNs, LPNs, and CNAs, we believe that clinical staffing expenditures are the best available proxy for staffing expenditures related to RNs, LPNs, and CNAs in a nursing home setting. In our robustness checks, we also use a broader measure of clinical expenditures identified in prior research (Mukamel, Spector, Zinn, Weimer, & Ahn, 2010) and include staffing expenditures for skilled nursing, nursing administration, lab services, therapy services, and so on to account for potential misclassification of expenditures.

We obtain nursing home characteristics from OSCAR. OSCAR is a data set compiled by CMS based on inspections of Medicare-certified nursing homes that occur at least once every 12 to 15 months. It has nursing-home level information on facility characteristics including resident census, ownership type, chain membership, size, urban/rural location, staffing, and data used to calculate several case-mix indicators.

Finally, we use the raw rating scores used by CMS to assign star ratings to nursing homes (the underlying continuous scores, not simply the star categories). These raw scores were not available publicly for the time period in our data but were obtained through a special request to CMS. Specifically, we use the adjusted RN, adjusted LPN, and adjusted nurse aide scores. CMS did not release the new 5-star rating for nursing homes until December of 2008 but the data have monthly raw scores from the beginning of 2008. For the years 2008 to 2010, we use the staffing scores at the end of each year. We use the raw scores from January of 2008 to proxy for the staffing scores for the year 2007; since most of the staffing scores in January of 2008 reflect staffing data from health inspections that occurred sometime in 2007, it is a reasonable proxy. We also conduct a sensitivity analysis using data from 2008 onwards.

Since we are using different data sources for staffing scores and expenditures, it is important to align the data between the different sources as closely as possible. After requiring all facilities to have 12 months of expenditure data, we classify facility expenditure reports into different years based on when the fiscal year ends. A majority of facilities report expenditures for the calendar year (70%) but some facilities have fiscal years different from calendar years. A further 9% have a fiscal year that ends in September, and another 14% have a fiscal year that ends in July. Since staffing inspections occur throughout the year, there is not a single optimal approach to combining the two data sources. To best match the timing of expenditures to the timing of the staffing rating, we use the most recent staffing score collected by the end of each calendar year for our main analysis and conduct a sensitivity analysis that requires a new inspection within each fiscal year.

Study Sample

This study is limited to Medicare-certified, freestanding nursing homes. We exclude hospital-based nursing homes because their cost structures are different from freestanding nursing homes. In addition, because our data sets were not collected for research and include erroneous values, we take several steps to avoid potential data errors and outliers influencing our regression estimates. First, we adopt the CMS guidelines and follow the literature (Park & Stearns, 2009) to exclude facilities that (a) report more residents than beds, (b) have zero RN hours but have 60 or more beds, (c) have total staffing hours per resident day that are less than 0.5 or more than 12, and (d) report zero residents. These criteria exclude about 5% of our sample.

Furthermore, we limit our analytical data to those observations between the 1st and 99th percentiles on the key dependent variable (clinical staffing expenditures per resident day), and on independent variables (adjusted staffing quality scores for RNs, LPNs, and nurse aide). We exclude the entire facility if any observation for a given facility falls into the outlier category for either the expenditures or the staffing measures. In each regression, we exclude approximately 15% of the observations based on these exclusion criteria. We conduct sensitivity analysis using a different definition for outliers as described in the Robustness section.

Dependent Variables

Our key dependent variable of interest is clinical staffing expenditures per resident day. Clinical staffing expenditures per resident day are obtained by dividing the total clinical staffing expenditures in a year by the number of inpatient days during that year. Factor substitutions in a nursing home can lead to an increase in staffing scores without increasing overall staffing expenditures. In order to address this issue, we should either limit our expenditures to include spending related to only the clinical staff or we should incorporate other staffing measures in the regression model as additional controls. As described earlier in the data section, clinical staffing expenditures comprise staffing expenditures that are more likely to be related to RNs, LPNs, and nurse aides; this clinical staffing expenditure measure excludes staffing expenditures on certain clinical activities like therapists and lab technicians as well as non-clinical activities like housekeeping, dietary, laundry, and other general services. When we restrict our expenditure measures to include only clinical staff, staffing substitutions should have limited impact on the relationship between expenditures and clinical staffing ratios. Since expenditures could be misclassified at times, as a robustness check, we use a broader measure of clinical expenditures on staffing, as described in the robustness section. Expenditures are normalized to 2011 U.S. dollars using the medical care component of the consumer price index.

Key Independent Variables

We have three continuous staffing variables of interest based on self-reported data—RN staffing, LPN staffing, and nurse aide staffing. These staffing measures are taken directly from CMS calculations for Nursing Home Compare. RN staffing represents registered nurses per resident per day and is based on self-reported data in the 2-week period prior to state inspection. Similarly, LPN staffing includes hours for licensed practical nurses and licensed vocational nurses per resident per day. Finally, staffing scores for nurse aides include hours for nurse aides in training and medication aides per resident per day. Adjustments for resident case-mix are made for all three staffing variables as per the CMS measure definitions; case-mix differences based on Resource Utilization Groups (RUG-III) are used to adjust the staffing ratios for RN, LPN, and nurse aides to account for resident differences in health status and care need (Abt Associates, 2014). Substitutions that occur between clinical staffing types can make it difficult to interpret the findings on the relationship between expenditures and clinical staffing. Since we explicitly include the three separate staffing variables in one regression model, it allows us to examine the contribution of each while controlling for the others.

Covariates

In all regressions, we control for several variables that might confound the relationship between expenditures and staffing ratios. Most important, we include additional adjustments for case-mix to control for the possibility that the CMS measures do not adequately capture this confounder. Specifically, we control for resident resource needs using the Activities of Daily Living (ADL) index and Special Care Index (SCI; Cowles, 2010). The ADL index is derived from the proportion of residents needing assistance for bathing, dressing, toileting, transferring, and eating while the SCI accounts for proportions of residents with need for respiratory care, suctioning, IV therapy, tracheostomy, and parenteral feeding (Cowles, 2010). In addition, we control for payer mix (the percent of residents whose stay is paid by Medicare and Medicaid). To control for market-level influences, we include an indicator for nursing home market concentration, as the additional gains from improving quality may not be substantial for nursing homes in markets without competition. We estimate nursing home market concentration at the county level using the Herfindahl–Hirschman Index (HHI) defined as the sum of the squares of market shares of all nursing homes in terms of inpatient days (Rhoades, 1993). A recent study explored alternative definitions of HHI and found that a more robust definition of HHI that accounts for chains is 0.2 points higher than a traditional definition of HHI used in our study (Hirth et al., 2017). Since HHI is not our primary variable of interest, our definition of HHI should be adequate as a control measure. We do include chain ownership as an additional control variable as well. Finally, we control for facility characteristics that may change over time including total bed size, and profit status.

Statistical Analysis

First, we summarize key facility characteristics across all facilities. We also summarize the reported staffing quality scores and clinical staffing expenditures pre- (2007–2008) and post-5-star era (2009–2010).

Next, we use a series of facility and year fixed-effects regressions to estimate the relationship between changes in spending and changes in staffing scores within facilities over time. Facility fixed-effects regressions allow us to control for unobserved but plausibly time-invariant confounding variables at the facility level. Similarly, year fixed effects will capture common time trends across all facilities. Subgroup analyses include facilities stratified by profit status, baseline Medicaid share, and baseline quality levels. A facility is classified as having a high Medicaid share if the share of Medicaid residents as a percentage of total beds (Medicaid residents × 100/total beds) in the given facility is above the median (54.2) for all facilities at baseline, low Medicaid share otherwise. We classify a facility as a low-quality facility if the facility received an overall quality rating of 1 to 2 stars at baseline; we defined a facility as a high-quality facility if the overall quality rating was 4 to 5 stars at baseline. For all regressions, standard errors are clustered at the facility level to account for dependence among observations from the same facility.

We use the following model to analyze the relationship between nursing home expenditures and staffing scores:

$$Y_{\mathrm{it}}={\text{β}}_0+{\text{β}}_{1j}Staffin{g}_{jit}+{\text{β}}_{2}Pos{t}_t+{\text{β}}_{3\text{j}}Staffin{g}_{jit}\times Pos{t}_t+{\text{β}}_4{X}_{it}+{\text{β}}_{5}Yea{r}_t+{\text{β}}_{6}Facilit{y}_i+{\text{ε}}_{it}$$

In this model, Y_it represents clinical staffing expenditures for facility i at time t; Staffing_jit represents the three adjusted staffing variables (RN, LPN, and nurse aides) for facility i at time t; Post equals 1 if the year is following the 5-star release (i.e., Year = 2009 or 2010), 0 otherwise; X represents a vector of exogenous controls described earlier, Year represents time fixed effects, and Facility represents facility fixed effects. ε is a random error term. The coefficients of interest in this regression are β_1j and β_3j. If β_1j is positive and significant, it implies that higher staffing scores are associated with higher clinical staffing expenditures in the pre-5-star era. Similarly, if β_3j is negative and significant, it suggests that the relationship between expenditures and staffing scores is weaker in the post-5-star era.

Robustness Checks

We conduct a number of robustness checks to see if our findings are sensitive to our definition of clinical staffing expenditures as well as other data cleaning decisions made in our analyses. First, state inspections are generally carried out within every 12 months on average but sometimes these inspections can be up to 15 months apart. This implies that we may not have a new state inspection every year for some facilities. In cases where there is no new state inspection, the staffing scores largely reflect the staffing levels reported in the last inspection, although adjustment for patient case-mix can change the actual staffing scores. We exclude observations without a new state inspection within the fiscal year and re-estimate the main models. Second, to allow for nursing homes to respond to the release of 5-star ratings, we only use the first and last year of observations and re-estimate our main model. Third, our decision to exclude outliers that fall outside of 1st and 99th percentile for each measure is somewhat arbitrary even though based on common practice (Doyle et al., 2017). To address this issue, we consider two other approaches to excluding outliers. First, we exclude observations with clinical staffing expenditures in top or bottom 2.5% instead of 1%. Second, we exclude outliers only for the clinical staffing expenditures (while the outliers for staffing are partly addressed by our initial exclusion criteria that excludes facilities that report total staffing hours per resident day below 0.5 and above 12; it does not exclude outliers by specific staffing type) and re-estimate our main models.

Fourth, in our main analysis, we proxy the 2007 staffing scores using the staffing scores in January of 2008. While a small fraction of facilities has inspections each month, and most of the staffing scores in January of 2008 represent inspections that occurred in 2007, these scores in January of 2008 are still a proxy for 2007 scores and are subject to error. Thus, we use data only from 2008 onwards and re-estimate our main models.

Fifth, staffing expenditures are potentially subject to misclassification and thus, our approach to identify clinical staffing expenditures may not be ideal. As a sensitivity analysis, we use a broader measure of clinical expenditures for staffing as the dependent variable of interest. These expenditures not only include clinical staffing expenditures related to nurses and nurse aides but also include staffing expenditures on therapy, laboratory, nursing administration and others identified as clinical in nature in a recent study (Mukamel et al., 2010).

Results

Descriptive characteristics of our sample are shown in Table 1. We have a total of 37,095 observations for 11,091 facilities; mostly for-profit (74.0%); with total occupancy averaging 84.4% (±13.7) while Medicaid occupancy averaging 52.4% (±17.9). The average case-mix in terms of the ADL index is 10.3, and the SCI index is 0.22. Average clinical salary expenditures per person per day are similar in the pre-5-star period and post-5-star period ($68.29 vs. $68.24). Average staffing quality scores are lower in the pre-5-star period than post-5-star period: RN staffing scores (0.32 vs. 0.34), LPN staffing scores (0.95 vs. 0.97), and nurse aide staffing scores (2.34 vs. 2.37).

Table 1.

Summary Statistics for the Sample of Nursing Homes (2007–2010).

Characteristics	Mean (SD)
Herfindahl-Hirschman Index	0.237 (0.265)
Special care index	0.223 (0.157)
ADL index	10.34 (1.137)
Facility total beds	116.9 (63.37)
Total occupancy %	84.39 (13.70)
Medicaid occupancy %	52.35 (17.87)
Medicare occupancy %	12.12 (8.971)
RN staffing score: Pre-5-star	0.320 (0.130)
RN staffing score: Post-5-star	0.342 (0.131)
LPN staffing score: Pre-5-star	0.951 (0.312)
LPN staffing score: Post-5-star	0.971 (0.314)
Nurse aide staffing score: Pre-5-star	2.339 (0.481)
Nurse aide staffing score: Post-5-star	2.367 (0.477)
Clinical staffing expenditures: Pre-5-star ($)	68.29 (18.93)
Clinical staffing expenditures: Post-5-star ($)	68.24 (18.06)
	N (%)
For-profit	27,443 (74.0)
Not-for-profit	8,353 (22.5)
Government-owned	1,299 (3.50)
Multi-facility organization	21,713 (58.5)
Number of facilities	11,091
Observations	37,095

Note. RN = registered nurse; LPN = licensed practical nurse. Facilities that have staffing scores and expenditures below 1st and above 99th percentiles are excluded from the sample. Expenditures are measured per person per day. Staffing quality measures are adjusted for case-mix.

Table 2 shows the results from the facility fixed effects regressions for the overall group as well as groups stratified by profit status, Medicaid share, and baseline quality. Regression results show that improved staffing scores are significantly associated with higher expenditures for all staffing types (p < .01) in the baseline period, as expected. In the overall sample, a 1-hour increase in RN staffing per resident day is associated with a $6.56 increase in clinical staffing expenditures per resident day in the pre-5-star period. The effect was lower during the post-5-star period, indicating a weakening of the relationship between staffing and expenditures. Similarly, an hour increase in LPN staffing is associated with a $1.6 increase in clinical staffing expenditures in the pre5-star era and this relationship is again weaker in the post-5-star period. There is no difference in the relationship between nurse aide staffing scores and expenditures in the pre- versus post-5-star period. These results are broadly consistent across the different subgroups except that the differential relationship between RN staffing and expenditures appears to be largely driven by facilities that are for-profit and/or those with a high share of Medicaid residents.

Table 2.

Association Between Changes in Clinical Staffing Expenditures and Changes in Staffing Scores: Fixed Effects Regressions.

	All	Not-for-profit	For-profit	Low Medicaid	High Medicaid	High quality	Low quality
Post	0.56*** (0.15)	0.96*** (0.36)	0.41** (0.17)	1.31*** (0.22)	−0.12 (0.21)	0.47* (0.27)	0.35 (0.24)
Adjusted RN	6.56*** (0.56)	5.78*** (1.35)	7.16*** (0.64)	6.46*** (0.84)	6.81*** (0.74)	6.85*** (0.99)	6.66*** (0.90)
Adjusted LPN	1.60*** (0.23)	1.90*** (0.56)	1.53*** (0.25)	1.53*** (0.33)	1.63*** (0.30)	1.86*** (0.43)	1.67*** (0.33)
Adjusted nurse aide	1.32*** (0.16)	2.00*** (0.41)	1.17*** (0.17)	1.43** (0.23)	1.15*** (0.23)	1.20*** (0.29)	1.17*** (0.24)
Post × Adjusted RN	−1.23** (0.55)	0.0018 (1.20)	−2.31*** (0.66)	−1.30 (0.80)	−1.81** (0.74)	−1.34 (0.91)	−1.48 (0.96)
Post × Adjusted LPN	−0.99*** (0.22)	−0.94* (0.53)	−1.04*** (0.25)	−1.48*** (0.32)	−0.49 (0.31)	−0.95** (0.38)	−0.82** (0.36)
Post × Adjusted nurse aide	0.22 (0.14)	−0.041 (0.32)	0.015 (0.15)	0.31 (0.20)	0.070 (0.18)	0.42* (0.23)	0.16 (0.21)
Observations	37,095	7,620	26,828	17,907	19,188	11,381	17,629

Note. RN = registered nurse; LPN = licensed practical nurse. Facilities that have staffing scores and expenditures below 1st and above 99th percentiles are excluded from the sample. Results are obtained using facility and time fixed effects controlling for other covariates. Standard errors in parentheses are clustered at the facility level.

^*p < .1.

^**p < .05.

^***p < .01.

Table 3 shows results for groups cross-stratified by profit status and Medicaid share. The results are largely similar to our main results except that the for-profit facilities have a weaker relationship between RN and LPN staffing and expenditures in the post-5-star period irrespective of whether they have high or low share of Medicaid residents. However, the magnitude of the coefficient on the interaction term for RN staffing in the post-5-star period is much higher and highly significant only for for-profit, high-Medicaid facilities. Among not-for-profit facilities, there is a significantly weaker relationship between staffing and expenditures in the post-5-star period only for LPN staffing and only among those with low Medicaid share. The sample sizes for not-for-profit facilities stratified by Medicaid share are much smaller.

Table 3.

Association Between Changes in Clinical Staffing Expenditures and Changes in Staffing Scores: Fixed Effects Regressions.

	Not-for-profit and low Medicaid	Not-for-profit and high Medicaid	For-profit and low Medicaid	For-profit and high Medicaid
Post	1.76*** (0.48)	−0.18 (0.56)	1.12*** (0.26)	−0.20 (0.23)
Adjusted RN	6.70*** (1.89)	4.12** (1.88)	6.56*** (0.96)	7.86*** (0.85)
Adjusted LPN	2.09*** (0.77)	1.53** (0.75)	1.33*** (0.38)	1.74*** (0.34)
Adjusted nurse aide	2.17*** (0.59)	1.49*** (0.51)	1.20*** (0.24)	1.10*** (0.26)
Post × Adjusted RN	−0.36 (1.73)	0.17 (1.48)	−1.66* (0.92)	−3.60*** (0.94)
Post × Adjusted LPN	−1.51** (0.69)	−0.022 (0.84)	−1.52*** (0.36)	−0.63* (0.34)
Post × Adjusted nurse aide	−0.062 (0.44)	−0.075 (0.46)	0.26 (0.24)	−0.26 (0.20)
Observations	4,444	3,176	12,349	14,479

Note. RN = registered nurse; LPN = licensed practical nurse. Facilities that have staffing scores and expenditures below 1st and above 99th percentiles are excluded from the sample. Results are obtained using facility and time fixed effects controlling for other covariates. Standard errors in parentheses are clustered at the facility level.

^*p < .1.

^**p < .05.

^***p < .01.

Results from our robustness checks are presented in a series of tables (Tables 4 to 13) and are largely consistent with our main findings. When we excluded observations without a new state inspection during the fiscal year to ensure that the expenditures represent staffing during that year, increased staffing continues to be associated with higher expenditures but the association is weaker between LPN staffing and expenditures in the post-5-star era; a weaker relationship between RN staffing and expenditures in the post-5-star period is observed only among for-profit facilities and among for-profit facilities with high Medicaid share (Tables 4 and 5). When we used only the first and last year of observations to allow sufficient time for nursing homes to respond to the release of 5-star ratings, the results are essentially similar to our main findings (Tables 6 and 7). Tables 8 and 9 present the findings from our analysis that included data only from 2008 to 2010 and the results again largely confirm our main findings. The results for the main analysis are similar when we excluded observations using alternative exclusion criteria that excluded only the facilities with clinical expenditures outside the 1st and 99th percentile (Tables 10 and 11). When we excluded observations with clinical expenditures in the top or bottom 2.5% instead of 1%, the results are largely similar (results available on request).

Table 4.

Association Between Changes in Clinical Staffing Expenditures and Changes in Staffing Scores: Fixed Effects Regressions (Sensitivity Analysis—New Survey Within the Fiscal Year).

	All	Not-for-profit	For-profit	Low Medicaid	High Medicaid	High quality	Low quality
Post	0.53*** (0.18)	1.21*** (0.41)	0.37* (0.20)	1.32*** (0.26)	−0.24 (0.23)	0.38 (0.32)	0.31 (0.26)
Adjusted RN	6.43*** (0.64)	5.11*** (1.50)	7.27*** (0.74)	6.61*** (0.95)	6.35*** (0.82)	7.01*** (1.14)	6.16*** (0.99)
Adjusted LPN	1.68*** (0.26)	1.61** (0.65)	1.68*** (0.29)	1.78*** (0.38)	1.47*** (0.35)	1.92*** (0.49)	1.70*** (0.38)
Adjusted nurse aide	1.53*** (0.18)	2.51*** (0.48)	1.30*** (0.20)	1.71*** (0.25)	1.28*** (0.27)	1.49*** (0.33)	1.35*** (0.27)
Post × Adjusted RN	−0.59 (0.64)	−0.042 (1.41)	−1.84** (0.77)	−0.68 (0.94)	−1.06 (0.84)	−1.01 (1.08)	−0.49 (1.08)
Post × Adjusted LPN	−1.04*** (0.26)	−1.20** (0.58)	−1.06*** (0.30)	−1.66*** (0.38)	−0.36 (0.33)	−0.93** (0.45)	−0.82** (0.40)
Post × Adjusted nurse aide	0.18 (0.16)	−0.083 (0.38)	−0.0059 (0.17)	0.26 (0.23)	0.025 (0.21)	0.39 (0.27)	0.15 (0.25)
Observations	29,185	5,959	21,147	14,072	15,113	8,929	13,824

Note. RN = registered nurse; LPN = licensed practical nurse. Facilities that have staffing scores and expenditures below 1st and above 99th percentiles are excluded from the sample. Results are obtained using facility and time fixed effects controlling for other covariates. Standard errors in parentheses are clustered at the facility level.

^*p < .1.

^**p < .05.

^***p < .01.

Table 13.

Association Between Changes in Clinical Staffing Expenditures and Changes in Staffing Scores: Fixed Effects Regressions (Sensitivity Analysis—Broader Measure of Clinical Staffing Expenditure).

	Not-for-profit and low Medicaid	Not-for-profit and high Medicaid	For-profit and low Medicaid	For-profit and high Medicaid
Post	2.17*** (0.59)	−0.21 (0.61)	2.54*** (0.33)	0.38 (0.27)
Adjusted RN	6.81*** (2.24)	4.30** (2.16)	6.18*** (1.21)	8.05*** (0.92)
Adjusted LPN	2.64*** (0.90)	1.42* (0.85)	1.13** (0.48)	1.81*** (0.39)
Adjusted nurse aide	1.98*** (0.68)	1.67*** (0.60)	1.32*** (0.30)	1.12*** (0.29)
Post × Adjusted RN	1.38 (2.24)	0.59 (1.77)	0.67 (1.20)	−3.39*** (0.97)
Post × Adjusted LPN	−1.89** (0.83)	0.60 (0.91)	−1.40*** (0.47)	−0.49 (0.39)
Post × Adjusted nurse aide	0.59 (0.51)	0.050 (0.56)	−0.078 (0.28)	−0.28 (0.24)
Observations	4,426	3,160	12,302	14,514

Note. RN = registered nurse; LPN = licensed practical nurse. Facilities that have staffing scores and expenditures below 1st and above 99th percentiles are excluded from the sample. Results are obtained using facility and time fixed effects controlling for other covariates. Standard errors in parentheses are clustered at the facility level.

^*p < .1.

^**p < .05.

^***p < .01.

Table 5.

Association Between Changes in Clinical Staffing Expenditures and Changes in Staffing Scores: Fixed Effects Regressions (Sensitivity Analysis—New Survey Within the Fiscal Year).

	Not-for-profit and low Medicaid	Not-for-profit and high Medicaid	For-profit and low Medicaid	For-profit and high Medicaid
Post	1.76*** (0.55)	0.30 (0.60)	1.23*** (0.31)	−0.43* (0.26)
Adjusted RN	6.25*** (2.19)	2.71 (1.97)	7.07*** (1.12)	7.65*** (0.95)
Adjusted LPN	1.58* (0.88)	1.52* (0.89)	1.80*** (0.44)	1.55*** (0.39)
Adjusted nurse aide	3.09*** (0.70)	1.42** (0.56)	1.32*** (0.26)	1.26*** (0.31)
Post × Adjusted RN	−0.65 (2.12)	0.89 (1.57)	−1.06 (1.10)	−3.27*** (1.05)
Post × Adjusted LPN	−1.60** (0.80)	−0.39 (0.73)	−1.74*** (0.46)	−0.42 (0.38)
Post × Adjusted nurse aide	−0.25 (0.53)	0.085 (0.50)	0.27 (0.25)	−0.34 (0.24)
Observations	3,476	2,483	9,726	1,1421

Note. RN = registered nurse; LPN = licensed practical nurse. Facilities that have staffing scores and expenditures below 1st and above 99th percentiles are excluded from the sample. Results are obtained using facility and time fixed effects controlling for other covariates. Standard errors in parentheses are clustered at the facility level.

^*p < .1.

^**p < .05.

^***p < .01.

Table 6.

Association Between Changes in Clinical Staffing Expenditures and Changes in Staffing Scores: Fixed Effects Regressions (Sensitivity Analysis—First and Last Year of Observation).

	All	Not-for-profit	For-profit	Low Medicaid	High Medicaid	High quality	Low quality
Post	0.73*** (0.22)	0.78 (0.49)	0.74*** (0.26)	1.66*** (0.32)	−0.063 (0.30)	0.89** (0.40)	0.30 (0.34)
Adjusted RN	9.00*** (0.89)	9.14*** (2.29)	9.48*** (1.00)	8.36*** (1.32)	10.0*** (1.17)	9.81*** (1.57)	9.14*** (1.45)
Adjusted LPN	2.28*** (0.37)	3.46*** (0.99)	2.06*** (0.41)	2.23*** (0.55)	2.32*** (0.49)	2.95*** (0.70)	2.30*** (0.54)
Adjusted nurse aide	1.59*** (0.24)	1.85*** (0.66)	1.61*** (0.27)	1.73*** (0.34)	1.40*** (0.35)	1.32*** (0.46)	1.56*** (0.36)
Post × Adjusted RN	−1.49* (0.80)	0.25 (1.87)	−3.23*** (0.96)	−1.16 (1.20)	−2.53** (1.04)	−2.34* (1.40)	−1.55 (1.35)
Post × Adjusted LPN	−1.46*** (0.34)	−0.95 (0.75)	−1.78*** (0.40)	−2.01*** (0.49)	−0.90* (0.46)	−1.62*** (0.59)	−1.11** (0.54)
Post × Adjusted nurse aide	0.25 (0.19)	0.18 (0.44)	−0.031 (0.21)	0.25 (0.27)	0.16 (0.26)	0.52 (0.32)	0.063 (0.30)
Observations	21,502	4,380	15,610	10,445	11,057	6,600	10,205

Note. RN = registered nurse; LPN = licensed practical nurse. Facilities that have staffing scores and expenditures below 1st and above 99th percentiles are excluded from the sample. Results are obtained using facility and time fixed effects controlling for other covariates. Standard errors in parentheses are clustered at the facility level.

^*p < .1.

^**p < .05.

^***p < .01.

Table 7.

Association Between Changes in Clinical Staffing Expenditures and Changes in Staffing Scores: Fixed Effects Regressions (Sensitivity Analysis—First and Last Year of Observation).

	Not-for-profit and low Medicaid	Not-for-profit and high Medicaid	For-profit and low Medicaid	For-profit and high Medicaid
Post	1.62** (0.67)	−0.37 (0.73)	1.75*** (0.39)	−0.077 (0.34)
Adjusted RN	9.89*** (3.22)	8.03*** (2.99)	7.98*** (1.48)	11.2*** (1.35)
Adjusted LPN	3.94*** (1.37)	2.40* (1.36)	1.81*** (0.63)	2.40*** (0.53)
Adjusted nurse aide	2.29*** (0.86)	0.94 (1.00)	1.66*** (0.39)	1.52*** (0.37)
Post × Adjusted RN	−0.44 (2.79)	0.42 (1.99)	−2.08 (1.38)	−4.93*** (1.33)
Post × Adjusted LPN	−1.39 (0.98)	0.054 (1.15)	−2.49*** (0.59)	−1.18** (0.52)
Post × Adjusted nurse aide	0.0043 (0.58)	0.22 (0.67)	0.095 (0.32)	−0.19 (0.28)
Observations	2,554	1,826	7,240	8,370

Note. RN = registered nurse; LPN = licensed practical nurse. Facilities that have staffing scores and expenditures below 1st and above 99th percentiles are excluded from the sample. Results are obtained using facility and time fixed effects controlling for other covariates. Standard errors in parentheses are clustered at the facility level.

^*p < .1.

^**p < .05.

^***p < .01

Table 8.

Association Between Changes in Clinical Staffing Expenditures and Changes in Staffing Scores: Fixed Effects Regressions (Sensitivity Analysis—Using Only 2008–2010 Data).

	All	Not-for-profit	For-profit	Low Medicaid	High Medicaid	High quality	Low quality
Post	−0.12 (0.15)	0.24 (0.37)	−0.30* (0.17)	0.36* (0.22)	−0.57*** (0.22)	−0.24 (0.28)	−0.24 (0.23)
Adjusted RN	5.76*** (0.65)	5.23*** (1.53)	6.58*** (0.76)	5.80*** (0.98)	5.85*** (0.85)	5.12*** (1.16)	6.36*** (1.05)
Adjusted LPN	1.41*** (0.26)	1.38** (0.61)	1.43*** (0.29)	1.25*** (0.38)	1.51*** (0.34)	1.52*** (0.45)	1.58*** (0.37)
Adjusted nurse aide	0.98*** (0.18)	1.71*** (0.46)	0.86*** (0.19)	0.93*** (0.26)	0.99*** (0.25)	0.77** (0.32)	0.82*** (0.26)
Post × Adjusted RN	−1.73*** (0.56)	−1.24 (1.19)	−2.63*** (0.69)	−2.16*** (0.81)	−1.78** (0.79)	−1.37 (0.91)	−2.31** (1.00)
Post × Adjusted LPN	−0.98*** (0.23)	−1.08** (0.54)	−0.95*** (0.25)	−1.35*** (0.31)	−0.62* (0.33)	−0.90** (0.38)	−0.85** (0.36)
Post × Adjusted nurse aide	0.32** (0.14)	0.030 (0.36)	0.12 (0.15)	0.52** (0.21)	0.067 (0.19)	0.33 (0.24)	0.42* (0.22)
Observations	28,475	5,841	20,592	13,808	14,667	8,754	13,524

Note. RN = registered nurse; LPN = licensed practical nurse. Facilities that have staffing scores and expenditures below 1st and above 99th percentiles are excluded from the sample. Results are obtained using facility and time fixed effects controlling for other covariates. Standard errors in parentheses are clustered at the facility level.

^*p < .1.

^**p < .05.

^***p < .01.

Table 9.

Association Between Changes in Clinical Staffing Expenditures and Changes in Staffing Scores: Fixed Effects Regressions (Sensitivity Analysis—Using Only 2008–2010 Data).

	Not-for-profit and low Medicaid	Not-for-profit and high Medicaid	For-profit and low Medicaid	For-profit and high Medicaid
Post	0.70 (0.49)	−0.50 (0.61)	0.16 (0.24)	−0.71*** (0.24)
Adjusted RN	6.97*** (2.15)	2.30 (2.14)	6.08*** (1.15)	7.24*** (0.99)
Adjusted LPN	1.41* (0.83)	1.28 (0.86)	1.24*** (0.43)	1.58*** (0.37)
Adjusted nurse aide	1.76*** (0.68)	1.40*** (0.53)	0.62** (0.26)	1.04*** (0.28)
Post × Adjusted RN	−2.05 (1.69)	−0.24 (1.61)	−2.19** (0.96)	−3.62*** (0.99)
Post × Adjusted LPN	−1.65** (0.67)	−0.11 (0.92)	−1.17*** (0.35)	−0.76** (0.36)
Post × Adjusted nurse aide	0.13 (0.50)	−0.13 (0.46)	0.50** (0.24)	−0.29 (0.20)
Observations	3,430	2,411	9,505	11,087

Note. RN = registered nurse; LPN = licensed practical nurse. Facilities that have staffing scores and expenditures below 1st and above 99th percentiles are excluded from the sample. Results are obtained using facility and time fixed effects controlling for other covariates. Standard errors in parentheses are clustered at the facility level.

^*p < .1.

^**p < .05.

^***p < .01.

Table 10.

Association Between Changes in Clinical Staffing Expenditures and Changes in Staffing Scores: Fixed Effects Regressions (Sensitivity Analysis—Alternative Exclusion Criteria for Outliers).

	All	Not-for-profit	For-profit	Low Medicaid	High Medicaid	High quality	Low quality
Post	0.54*** (0.14)	1.10*** (0.35)	0.32** (0.15)	1.28*** (0.21)	−0.13 (0.19)	0.51* (0.26)	0.34 (0.21)
Adjusted RN	6.55*** (0.54)	6.75*** (1.33)	6.77*** (0.61)	6.44*** (0.80)	6.80*** (0.71)	7.19*** (0.96)	6.55*** (0.85)
Adjusted LPN	1.51*** (0.19)	2.31*** (0.53)	1.33*** (0.20)	1.51*** (0.28)	1.46*** (0.25)	1.83*** (0.40)	1.55*** (0.26)
Adjusted nurse aide	1.25*** (0.15)	1.82*** (0.39)	1.12*** (0.16)	1.29*** (0.21)	1.16*** (0.21)	1.12*** (0.28)	1.16*** (0.22)
Post × Adjusted RN	−1.29** (0.53)	−0.33 (1.20)	−2.11*** (0.62)	−1.32* (0.77)	−1.91*** (0.72)	−1.58* (0.90)	−1.51* (0.91)
Post × Adjusted LPN	−0.98*** (0.20)	−1.27** (0.51)	−0.89*** (0.22)	−1.47*** (0.30)	−0.49* (0.28)	−1.01*** (0.37)	−0.82*** (0.31)
Post × Adjusted nurse aide	0.20 (0.13)	0.024 (0.32)	−0.025 (0.15)	0.34* (0.19)	0.016 (0.18)	0.43* (0.22)	0.13 (0.20)
Observations	38,563	7,857	27,973	18,718	19,845	11,877	18,201

Note. RN = registered nurse; LPN = licensed practical nurse. Facilities that have staffing scores and expenditures below 1st and above 99th percentiles are excluded from the sample. Results are obtained using facility and time fixed effects controlling for other covariates. Standard errors in parentheses are clustered at the facility level.

^*p < .1.

^**p < .05.

^***p < .01.

Table 11.

Association Between Changes in Clinical Staffing Expenditures and Changes in Staffing Scores: Fixed Effects Regressions (Sensitivity Analysis—Alternative Exclusion Criteria for Outliers).

	Not-for-profit and low Medicaid	Not-for-profit and high Medicaid	For-profit and low Medicaid	For-profit and high Medicaid
Post	1.89*** (0.47)	−0.051 (0.51)	1.00*** (0.23)	−0.26 (0.21)
Adjusted RN	7.65*** (1.78)	5.00** (1.97)	6.10*** (0.92)	7.54*** (0.80)
Adjusted LPN	2.49*** (0.69)	1.89** (0.81)	1.26*** (0.30)	1.42*** (0.26)
Adjusted nurse aide	2.00*** (0.55)	1.29** (0.52)	1.01*** (0.22)	1.20*** (0.24)
Post × Adjusted RN	−0.92 (1.69)	0.24 (1.49)	−1.22 (0.87)	−3.60*** (0.89)
Post × Adjusted LPN	−1.85*** (0.67)	−0.29 (0.77)	−1.32*** (0.32)	−0.53* (0.30)
Post × Adjusted nurse aide	−0.0089 (0.42)	0.015 (0.46)	0.27 (0.22)	−0.33* (0.19)
Observations	4,633	3,224	12,924	15,049

Note. RN = registered nurse; LPN = licensed practical nurse. Facilities that have staffing scores and expenditures below 1st and above 99th percentiles are excluded from the sample. Results are obtained using facility and time fixed effects controlling for other covariates. Standard errors in parentheses are clustered at the facility level.

^*p < .1.

^**p < .05.

^***p < .01.

Finally, when we repeated the analyses with a broader measure of clinical staffing expenditures as the dependent variable, the results largely corroborate our main findings (Tables 12 and 13).

Table 12.

Association Between Changes in Clinical Staffing Expenditures and Changes in Staffing Scores: Fixed Effects Regressions (Sensitivity Analysis—Broader Measure of Clinical Staffing Expenditure).

	All	Not-for-profit	For-profit	Low Medicaid	High Medicaid	High quality	Low quality
Post	1.36*** (0.18)	1.28*** (0.43)	1.35*** (0.21)	2.49*** (0.28)	0.34 (0.24)	1.20*** (0.33)	1.26*** (0.29)
Adjusted RN	6.80*** (0.67)	5.98*** (1.60)	7.03*** (0.76)	6.73*** (1.02)	7.06*** (0.83)	7.20*** (1.17)	5.94*** (1.02)
Adjusted LPN	1.58*** (0.27)	2.17*** (0.65)	1.45*** (0.31)	1.41*** (0.40)	1.70*** (0.34)	2.29*** (0.50)	1.60*** (0.39)
Adjusted nurse aide	1.38*** (0.19)	1.99*** (0.48)	1.23*** (0.20)	1.53*** (0.27)	1.16*** (0.26)	1.17*** (0.34)	1.26*** (0.28)
Post × Adjusted RN	−0.32 (0.66)	1.02 (1.53)	−0.83 (0.78)	0.15 (1.02)	−1.80** (0.80)	−0.45 (1.11)	0.86 (1.10)
Post × Adjusted LPN	−0.91*** (0.27)	−0.96 (0.62)	−0.90*** (0.31)	−1.56*** (0.40)	−0.26 (0.34)	−1.15** (0.46)	−0.57 (0.42)
Post × Adjusted nurse aide	0.16 (0.16)	0.4 (0.38)	−0.14 (0.18)	0.2 (0.24)	0.048 (0.22)	0.52* (0.27)	0.099 (0.26)
Observations	37,053	7,586	26,816	17,846	19,207	11,367	17,618

Note. RN = registered nurse; LPN = licensed practical nurse. Facilities that have staffing scores and expenditures below 1st and above 99th percentiles are excluded from the sample. Results are obtained using facility and time fixed effects controlling for other covariates. Standard errors in parentheses are clustered at the facility level.

^*p < .1.

^**p < .05.

^***p < .01.

Discussion

In this study, we analyze the relationship between nursing home expenditures and reported staffing ratios in the pre- versus post-5-star period. We find that higher reported staffing scores are significantly associated with higher expenditures but the relationship between expenditures and LPN (and RN for for-profit and high-Medicaid facilities) staffing quality scores is weaker following the release of the 5-star system. In terms of magnitudes, an additional hour of RN and LPN staffing cost a facility $6.6 and $1.6, respectively, in the pre-5-star period. However, in the post-5-star period, an additional hour of RN and LPN cost a facility only $5.33 (a decrease of $1.23), and $0.61 (a decrease of $0.99), respectively. The baseline magnitudes of the estimates appear small relative to typical wages but there could be several reasons for this. First, the staffing measures are case-mix adjusted and it is difficult to interpret such estimates. Second, while we hypothesized that gaming of reported staffing exacerbated substantially after the 5-star system was implemented, this does not mean that no gaming existed before. In fact, the idea of nursing homes “staffing up” before a survey has been a well-known concern for decades. Thus, we would not expect incremental costs of an hour of staffing to be equal to an hourly wage rate even in the pre-5-star period. In order to verify if the baseline magnitudes of our estimates are lower because of our control variables, we estimated fixed-effects regressions where our expenditures were a function of only staffing ratios, the post variable, and an interaction between staffing ratios and the post variable. The results from these fixed-effects regressions without other controls yield similar magnitudes and significance for the coefficients. Accordingly, our focus in this analysis is not so much in the magnitudes but rather on the relationship between these staffing scores and expenditures pre- versus post-5-star period. Our findings suggest potential gaming by nursing homes with respect to reported LPN and RN staffing.

The results from this study are subject to several limitations. In particular, our study is based on pre-post differences in expenditures and staffing scores rather than a more robust difference-in-difference study design. Since all nursing homes were subject to 5-star rating system, we lack a control group for a difference-in-difference design. However, we have minimized the bias to the extent possible using nursing home and year fixed effects. While the nursing home fixed effects model accounts for timeinvariant unobserved covariates and year fixed effects adjust for common trends across all nursing homes, and we include a number of potential time-varying confounding variables in the regression, other time-varying confounding may remain. In addition, there are some limitations to the quality and expenditure measures used in our study. We are generally less concerned about the reliability of total expenditures reported by facilities since these expenditures are reported based on facilities’ accounting systems, but the allocation mechanism of expenditures into different categories (clinical/staffing, etc.) may not be uniform across different facilities. However, as long as these differences are constant over time, they should not create bias in our fixed-effects approach. We attempted to address this issue of misclassification by using different measures for staffing expenditures and the results were similar. We draw further confidence from the fact that the estimated clinical expenditures per resident per day using staffing ratios from our data and the 2010 wage data from the Bureau of Labor Statistics for RNs ($28.84), LPNs ($20.5) and nurse aides ($11.7) employed in nursing care facilities is approximately $57—a measure close to the $68 we observe in our data (Bureau of Labor Statistics, 2010).

Although there are anecdotal reports and speculation about the reliability of nursing home staffing quality information, our study is the first to add new evidence on whether nursing homes increased expenditures to improve staffing scores in Nursing Home Compare. Our finding that higher staffing ratios are associated with higher clinical staffing expenditures on average is not surprising, given that hiring new staff will generally require additional expenditures. The lack of pre- versus post-5-star difference in the relationship between expenditures and staffing scores for nurse aides is also unsurprising since nurse aides are cheaper than LPNs and RNs. However, the weaker association between expenditures and LPN and RN staffing scores in the post5-star era is revealing. The staffing measures are calculated as ratios per resident-day, so a nursing home might improve scores by hiring cheaper staff with less experience, but at some level higher staffing ratios should incur higher expenditures. The increase in staffing scores without a corresponding increase in spending supports the current skepticism about the validity of these improvements and the underlying data.

The weakening of the relationship between reported staffing and expenditures after the 5-star was more pronounced and more consistent overall for LPN staffing than for RN staffing, which is somewhat surprising given the underlying incentives. RN staffing is a better target for gaming if a facility wants to improve its staffing ratings since RN staffing counts twice in the overall staffing rating. At the same time, increasing reported LPN hours contributes to higher overall staffing ratings and the increases may be harder to verify; given that RN hours are tightly constrained, a substantial increase in reported hours may attract attention by surveyors. “Staffing up” LPN hours before a survey is also cheaper than staffing up on RNs.

Importantly, for-profit facilities with a high Medicaid census exhibit a weakening relationship between staffing and expenditures for both LPN and RN hours, consistent with incentives. The returns to achieving a higher quality rating may be attractive enough for these facilities to attempt different ways to improve their scores in the post-5-star period. As we hypothesized in the conceptual framework, for-profit facilities with a higher share of Medicaid residents may not have the financial means to hire new RN staff and thus may have resorted to inflating their RN staffing prior to inspections or may have simply misreported their RN staffing to improve their staffing ratings.

The weakening relationship between expenditures and staffing scores in the post-5-star period raises several possible alternative explanations for what could have happened following the release of 5-star rating system. First, the OSCAR-based measure used by CMS is based on the 2-week period prior to an inspection, and this period may not be representative of staffing levels throughout the year. Nursing homes may anticipate the timing of their inspection and may “staff up” during this period. Second, the self-reported staffing levels during the 2-week period may be subject to manipulation. For example, nursing homes may count staffs that are not actually providing resident care in the staffing ratios. While our analysis cannot distinguish among these explanations and cannot completely rule out alternative pathways such as shifting to less experienced, lower cost staff, our findings suggest that the self-reported staffing data from OSCAR surveys may not be reliable and that CMS’s ongoing development of a system to collect staffing information from payroll data is well advised. When these data become available and are incorporated into Nursing Home Compare, our findings suggest that scores in the staffing domain can become more meaningful. At the same time, those new data will require ongoing monitoring for other ways in which the data might be manipulated.