The effect of state regulatory stringency on nursing home quality

Abstract

Objective

To test the hypothesis that more stringent quality regulations contribute to better quality nursing home care and to assess their cost-effectiveness.

Data Sources/Setting

Primary and secondary data from all states and U.S. nursing homes in 2005–2006.

Study Design

We estimated seven models, regressing quality measures on the Harrington Regulation Stringency Index and control variables. To account for endogeneity between regulation and quality, we used instrumental variables techniques. Quality was measured by staffing hours by type per case-mix adjusted day, hotel expenditures, and risk-adjusted decline in activities of daily living, high-risk pressure sores, and urinary incontinence.

Data Collection

All states' licensing and certification offices were surveyed to obtain data about deficiencies. Secondary data included the Minimum Data Set, Medicare Cost Reports, and the Economic Freedom Index.

Principal Findings

Regulatory stringency was significantly associated with better quality for four out of the seven measures studied. The cost-effectiveness for the ADL measure was estimated at about $72,000 in 2011 $/QALY.

Conclusions

Quality regulations lead to better quality in nursing homes along some dimensions, but not all. Our estimates of cost-effectiveness suggest that increased regulatory stringency is in the ballpark of other acceptable cost-effective practices.

INTRODUCTION

Quality of care has been an important component of the health policy agenda for several decades. The recent passage of the Affordable Care Act (Act HR3590 2010) followed by Secretary Sebelius' National Strategy for Quality Improvement in Health Care (The National Quality Strategy, HealthCare.gov 2011) continue to emphasize the development and implementation of policies and practices designed to enhance the quality of the U.S. health care system. One of these policy tools is regulation.

The economic rationale for the regulation of quality is the presence of market failures that lead to inefficiencies. It has long been recognized (Arrow 1963) that information asymmetries favoring health care providers and incentives for over-provision and over-consumption created by third-party payment (Pauly 1968) often prevent competition from delivering the highest possible quality at any given price. Regulations, which seek to improve these market outcomes, take a variety of forms, addressing one or more of the three dimensions of quality measurement (Donabedian 1988): structure (e.g. minimum staffing standards), processes (e.g. assistance with walking), and outcomes (e.g. rates of high-risk patients with pressure sores).

Regulations impose several types of costs in achieving these benefits. Administrative costs accrue to both the regulatory body and the regulated entity. Rules that affect production processes, either directly through minimum requirements for inputs or indirectly through output or outcome standards, raise the cost of production, potentially creating inefficiencies in production as well. Furthermore, sociologists (Merton 1957) and economists (McKie 1970) have long recognized that when products or services have multiple dimensions of quality, the typical case in health care, regulation of some quality dimensions may result in reductions in the unregulated quality dimensions, imposing offsetting costs on consumers. The U.S. Congress expressed concerns about the relative costs and benefits of regulation in its 2001 Regulatory Right-to-Know-Act, which instructed the Director of the Office of Management and Budget (OMB) to submit an annual cost-benefit report on the federal regulatory programs (Public Law No. 106-554 2001).

To our knowledge there have not been rigorous, comprehensive evaluations of the cost-effectiveness of the regulation of quality of the nursing home industry in the U.S. The only cost-effectiveness study we found was published by the OMB in its 2009 Report to Congress. The analysis focused on fire safety requirements (sprinkler systems). It estimated the costs to be $45–56 million (in 2001 dollars) compared with $53–56 million in estimated benefits (OMB 2009). Despite this lack of empirical evidence about the cost-effectiveness of the regulations, and despite the wide spread perception among over 1000 long-term care experts that the current regulatory system is ineffective, the majority of these experts expressed their belief that enhanced and more stringent regulations should be implemented (Mor, Miller, and Clark 2010).

In this paper we have two objectives: First, we assess the impact of the stringency of regulations on a number of dimensions of nursing homes' quality. Second, we provide a rough assessment of the cost-effectiveness of the stringency of the regulation of quality of nursing homes by combining the findings from this study with findings from a previous study, in which the cost of regulation of quality of nursing homes in the U.S. was estimated (Mukamel et al. 2011a).

REGULATION OF NURSING HOMES' QUALITY

Institutional Framework

Currently, the Centers for Medicare and Medicaid Services (CMS) sets minimum quality and safety standards that all nursing homes wishing to receive Medicare or Medicaid funding have to meet (over 90% of nursing homes participate in Medicare and Medicaid). There are over 180 such quality standards (CMS 2012). States have the option of adding their own standards to those promulgated by CMS. Thus, the standards vary across states.

CMS also delegates to the states the responsibility for monitoring compliance with the standards. States are charged with issuing sanctions, including citations (or deficiencies), financial penalties, and administrative actions (such as management change or facility closures) when nursing homes do not meet the standards (CMS 2012). State licensing and certification offices inspect nursing homes every 9 to 15 months. This is referred to as the “annual survey.” In addition, states are required to investigate any serious complaints about quality and may conduct more frequent surveys if they deem those necessary (Harrington, Mullan, and Carrillo 2004).

States are known to vary substantially in their investment in the survey process, as well as in their interpretation and application of the standards. For example, in 2000 state budgets funding the survey process ranged from $94 to $770 per nursing home bed (Walshe and Harrington 2002). In addition to the state specific standards, other theoretical reasons and empirical evidence have been offered in the literature to explain variation in regulations across states. Those include a style typical of decentralized behavioral regulations (Brown 1992; Kagan 1994), variation in states' social, economic, and political environments (Harrington et al. 2004; Kagan 1994), and the degree of capture of the regulators by the industry (Jost 1985).

The state regulatory stringency – i.e. the additional standards imposed by the state and the harshness of its survey and sanctions process – are likely to be endogenously associated with the quality level offered by the industry in the state. While one would expect more stringent regulations to lead to better quality, it is also possible that states will choose more stringent regulations if they are faced with lax industry practices that lead to low-quality (Weissert and Silberman 2002). Thus, because of reverse causality, empirical estimates of the effect of regulatory stringency on quality ignoring endogeneity will be negatively biased. Estimation with instrumental variables (IVs) can address endogenity, and will likely show that more stringent regulation leads to better quality.

In addition to the minimum quality standards and the annual survey standards, the federal government and some states have established specific staffing standards. The federal standards, which are uniform across the country, require nursing homes to staff up to a minimum level of licensed nurses. State standards apply either to licensed or unlicensed nurse staff, or both, and vary across states (Harrington et al. 2012).

Nursing homes' response to quality regulation

For both for-profit and nonprofit nursing homes the optimal quality choice increases with the benefits derived from providing more quality and decreases with costs. The benefits derive from increased revenues through increases in the number of prospective patients, both private pay and Medicaid, and through the ability to charge higher prices to private pay patients. Therefore, the demand nursing homes face, the competitiveness of their markets, and if they are located in an excess demand market the Medicaid pay rate, are important factors contributing to their choices of optimal quality levels (Grabowski 2001; Nyman 1985, 1989). The cost of producing quality depends on the technology nursing homes employ, economies of scale, and the cost of inputs.

Regulations enter the choice process only if the quality level nursing homes would choose in the absence of regulation is below the level imposed by the regulation, i.e. if the quality standards are binding. For those facilities that find the standards to be binding, the choice is between 1) making the investment required to meet the standards, thereby benefiting from demand for higher quality (assuming that the standards confer higher observed quality); and 2) ignoring the standards and thus facing the costs of non-compliance, such as deficiencies, fines, or lower rating in quality report cards. National statistics show that only about 7% of nursing homes meet all standards, and on average nursing homes receive over 7 deficiencies in each inspection, indicating that for many the optimal quality choice is below their state standards (CMS 2009).

The impact of regulation on quality of care provided by nursing homes has only been recently addressed. Bowblis et al. (2012; 2011) in two recent studies assessed the impact of the number of federal citations, a measure of one dimension of regulatory stringency, on quality. These studies found an impact of specific types of citations on specific quality outcomes. Furthermore, they showed that there is potentially a spillover effect across targeted dimensions of quality, depending on clinical complementarity and substitutability. These studies did not, however, address the possibility that regulatory stringency, even when lagged, is endogenous with quality.

The impact of minimum staffing standard has also been studied. Several studies (Bowblis 2011; Harrington, Swan, and Carrillo 2007) concluded that minimum staffing standards lead to higher staffing levels, although the effect might be small (Park and Stearns 2009). These studies also found a positive impact on several quality outcomes, but did not account for the potential contribution of other quality regulations, which may change overtime as the political and economic climate changes.

METHODS

Study sample

The study included all 16,352 Medicare and Medicaid certified nursing homes nationally in 2005 and 2006 and all their residents. All other data were also obtained for 2005 and 2006 unless noted otherwise.

Variables and data sources

Stringency of regulation

We measured the stringency of regulations in each state using the Harrington Regulation Stringency Index (HRSI) (Harrington et al. 2004). The HRSI is defined as the sum of five standardized components describing the regulatory process, as follows: 1) the average number of deficiencies per facility, 2) the percent of facilities with any deficiency, 3) the percent of facilities with a deficiency at G level or higher (actual harm or serious jeopardy to residents), 4) the percent of facilities with substandard care, and 5) the average number of state and federal Civil Monetary Penalties (CMPs) issued per facility. Each component was converted to a Z score based on its national distribution (i.e. $Z_{i,j}=\frac{X_{i,j}-{\overline{X}}_i}{{\sigma }_i}$ where X_i,j denotes component i=1…5 for state j). This measure has been shown in previous studies to have face validity (Harrington et al. 2004; Li et al. 2010) and it was used in measuring the cost of regulation (Mukamel et al. 2011a). We, therefore, use it here. A higher value of the HRSI indicates a more stringent regulatory regime.

Data to calculate the HRSI were obtained from a survey of all Directors of State Licensing and Certification Offices that we conducted. We obtained information about 2005–2006 deficiencies and CMPs, both state and federal. These data are preferred to the Online Certification and Reporting (OSCAR) data because they include information about state deficiencies and CMPs in addition to information about the federal sanctions, thus capturing more accurately variation in state regulatory stringency.

Measures of quality

As different aspects of quality are typically not correlated it is important to assess quality along more than one dimension.

Structural measures of quality

We defined three structural measures based on staffing levels: Certified Nurse Assistants (CNAs), Licensed Practical Nurses (LPNs), and Registered Nurses (RNs), all per case-mix-adjusted day. FTE data by type were obtained from OSCAR and included both salaried and agency staff. We excluded facilities that reported zero staffing or were at the top 1 percentile of the staffing distribution. Case-mix-adjusted days were calculated based on the Minimum Data Set (MDS) information. We calculated a length-of-stay weighted RUGs III case-mix for each facility by weighting each resident's RUGs score by the number of days the resident was reported to experience a particular level of RUGs. Higher staffing levels have been shown in prior studies to be associated with better quality (Bostick et al. 2006; Castle 2008; CMS 2001; Kim et al. 2009; Spilsbury et al. 2011).

Process measure of quality

We included a proxy measure for hotel quality based on resource use because there are no direct measures of hotel quality available for all nursing homes nationally. Following prior studies (Mukamel, Bajorska, and Spector 2005; Mukamel et al. 2010) this measure was defined as the sum of expenditures for cost centers providing hotel services, such as linens and laundry, housekeeping, and maintenance. These data were obtained from the Medicare Cost Reports. We assume that higher levels of expenditures on hotel activities are associated with better quality in this domain (Mukamel et al. 2010).

Outcome measures of quality

Long-term-care risk-adjusted health outcomes were measured using MDS data following methods described previously (Berlowitz et al. 2005; Mukamel et al. 2008). Three outcome measures that are of particular importance to nursing home patients and have been the focus of previous studies (Berlowitz et al. 2001; Mor et al. 2003; Rosen et al. 2000) were calculated: decline in activities-of-daily-living (ADLs), prevalence of pressure sores among high risk residents, and prevalence of urinary incontinence. Outcome definitions were the same as those used by CMS in its Nursing Home Compare report card (Abt Associates Inc. 2004). Risk-adjustment was applied at the individual level using logistic models. The individual risk-adjusted predicted outcome probabilities were summed to obtain the facility-level risk-adjusted expected outcome rates. The risk-adjusted outcome measure was then defined as the ratio of the facility observed outcome rate to the facility expected (risk-adjusted) outcome rate. Nursing homes with outcome measures exceeding 1 have quality worse than the average and nursing homes with outcome measures below 1 have better than average quality.

Other control variables

We also included additional covariates likely to influence provision of quality, including competition (Zwanziger, Mukamel, and Indridason 2002), market median income (U.S. Census Bureau 2000), hospital wage index used by CMS for reimbursement (CMS 2012), state Medicaid rates for 2004 (Brown University), state staffing standards for both licensed and direct care nurses combined into an index constructed similarly to the HRSI(Harrington 2008), total beds, ownership, hospital and chain affiliation (OSCAR).

Endogeneity and instrumental variable

To address endogenity we used the Area 2 of the Economic Freedom Index of North America of 2010: “Takings and Discriminatory Taxation” as an IV. It is calculated from four components: (1) total tax revenue as a percentage of Gross Domestic Product (GDP); (2) top marginal income tax rate at the income threshold at which it applies; (3) indirect tax revenue as a percentage of GDP; and (4) sales taxes collected as a percentage of GDP (Ashby et al. 2010). It is defined such that higher values indicate more economic freedom. We hypothesized that states with higher values, i.e. more economic freedom, will have less stringent regulations. We tested this hypothesis empirically.

A proper IV has to meet two criteria. First, it has to be highly correlated with the instrumented, endogenous variable, i.e. the HRSI. Based on both Staiger-Stock (1997) and Stock-Yogo (2005) criteria we rejected the hypothesis of weak instrument. The incremental F and minimum eigenvalue statistic for the IV is 11.81, exceeding the critical values of 10 for the Staiger-Stock criterion and 8.96 for a 5% Wald test of size 0.15 for the Stock-Yogo criterion. Second, the instrument must be uncorrelated with the error in the second stage equation. This criterion cannot be tested empirically. However, we have no reason to believe that this instrument would directly affect nursing home quality. Further, as the model controls for all the obviously important factors that affect quality, it is unlikely that there is endogeneity induced by omitted variables.

Utility and cost estimates for cost-effectiveness calculations

We performed a literature search to obtain empirical estimates for the utility gain due to incremental increases in our quality measures. We found information only for utility loss due to decline in ADLs. We used the utility loss associated with an ADL loss in mid-range: from inability to do some essential activities to partial dependence in all essential activities (Spicer and Miller 2010). The costs associated with regulatory stringency were obtained from Mukamel et al. (2011a), which also measured stringency using the HRSI.

Analysis

Estimation of the impact of regulation on quality

We analyzed the relationship between quality and regulation separately for each one of the seven quality measures: three staffing measures, a hotel expenditures measures, and three risk-adjusted patient health-outcomes measures. For each we estimated two-stage least square models. The first-stage equation was the same for all and predicted the state regulatory stringency as a function of the IV, the Area 2 of the Economic Freedom Index, and all other variables. Each of the seven second-stage equations had as dependent variable the log of one of the quality measures. The independent variables included the predicted HRSI and all the control variables. Because the HRSI is a state level variable, the first stage equation was estimated at the state level, with all exogenous variables averaged at the state level for the year. The second stage equation was estimated at the nursing home level with all variables at the facility level, except for the predicted HRSI. For comparison we also present OLS estimates.

Standard errors for the second stage equations were estimated using bootstrapping that took into account the uncertainty in the predicted values of the instrumented variable as well as sampling error, as follows: Step 1) We drew 500 random vectors of the predicted HRSIs (each consisting of 100 HRSI values for all 50 states in each of two years) from a normal distribution with means equal to the predicted HRSIs and variance-covariance matrix of the predicted HRSIs in the first stage equation. This step accounted for the variation in the predicted HRSI, including clustering by state. Step 2) For each of the 500 HRSI vectors generated in step 1 (i.e. a given set of predicted HRSIs), we drew 200 random samples (with replacements) of nursing homes. Thus, standard errors for each estimated parameter are based on 100,000 replications (500 draws of state-level predictions of HRSI times 200 draws of samples of individual nursing homes.)

We used the Durbin-Wu-Hausman test statistic to test the hypothesis that the HRSI is endogenous with the quality measures.

To make the estimated coefficients for the HRSI more meaningful we used the estimated models to predict the incremental change in quality (the dependent variables) for a one standard deviation increase in the HRSI for the average nursing home at the average level of quality.

Sensitivity Analyses

We performed several sensitivity analyses. We estimated models assuming that the staffing standards are also endogenous, including them in the HRSI as well, and using the same IV. To further test for potential endogeneity of other variables we estimated a model without the staffing standard and Medicaid payment rates, as well as a model without wages. Finally, we also repeated the analyses on a smaller sample, which included the same facilities in all models, to test for the possibility that the different finding for hotel expenditures were driven by differences in sample.

Cost-effectiveness estimates

To estimate the cost-effectiveness of the regulation of quality we estimated the costs and impact associated with an increase of 1 standard deviation in the HRSI for the average facility of 100 beds. The costs were reported in Mukamel et al. (2011). To calculate the impact in terms of Quality Adjusted Life Years (QALYs) we multiplied the gain predicted in ADLs by our estimated model (after adjusting for the smearing effect) by the increase in utility per person (measured in QALYs) of 0.19 QALY/ADL from Spicer and Miller (2010) and by 100 residents. The resulting cost-effectiveness ratio is in terms of dollars/QALY and does not depend on our assumptions regarding the increase of 1 standard deviation in the HRSI. To gauge the sensitivity of the ratio to the assumption about QALY/ADL, we also calculated it for two other values reported by Spicer and Miller (2010): 0.09 and 0.27. These values reflect changes in utility for higher and lower ADL starting points.

RESULTS

Table 1 presents descriptive statistics for the study sample. Sample size varied depending on the variable. The HRSI and the IV were at the state level, with an N=100 (50 states in each of two years). All other outcomes and control variables were at the facility level with sample sizes around 15,000 in each year. The exception was hotel expenditures, for which the sample was about 10,000 per year because the source for this was the Medicare Cost Reports, which exclude hospital-based nursing homes. The HRSI averaged close to zero but exhibited substantial variation across states, with a standard deviation of 2.88. The quality measures also exhibited substantial variation, with the coefficient of variation ranging between 30% and 100%.

Table 1.

Descriptive Statistics

Variables	Sample Size	Mean	SD
Harrington Regulatory Stringency Index (HRSI)	100	−0.21	2.88
Economic Freedom Index – Area 2	100	6.89	0.79
Minimum staffing requirement index-2007	100	−0.02	1.65
Medicaid per diem-2004 (in hundreds of dollars)	100	1.36	0.37
CNA hours per case-mix adjusted day	30,350	2.34	0.82
LPN hours per case-mix adjusted day	30,224	0.77	0.39
RN hours per case-mix adjusted day	30,246	0.33	0.35
Hotel expenditures (in hundreds of thousands of dollars)	20,216	20.66	20.28
Risk-adjusted ADL decline	29,678	1.12	0.74
Risk-adjusted high-risk pressure sores	29,574	1.03	0.77
Risk-adjusted urinary incontinence	29,774	1.00	0.60
Competition (1-HHI)	30,992	0.68	0.21
Median Income for the nursing home market (in tens of thousands of dollars)	30,992	3.98	1.18
Hospital Wage Index	30,992	0.93	0.12
Total number of beds	30,992	108.93	70.18
Non profit (Yes=1, no=0)	30,992	0.28	0.45
Government owned (Yes=1, no=0)	30,992	0.06	0.24
Chain (Yes=1, no=0)	30,992	0.53	0.50
Hospital based (Yes=1, no=0)	30,992	0.08	0.27
2006 (= 1 if year = 2006, = 0 if year=2005)	30,992	0.5	0.5

Table 2 presents the OLS estimates for all seven quality measures. The HRSI significantly increases only quality related to CNA staffing and risk adjusted pressure sores. It is not significantly associated with LPN staffing and risk-adjusted urinary incontinence and it decreases quality related to RN hours, hotel expenditures and risk adjusted ADLs.

Table 2.

Ordinary Least Squares Models

	CNA hours per case-mix adjusted day	LPN hours per case-mix adjusted day	RN hours per case-mix adjusted day	Hotel expenditures (in hundreds of thousands of $)	Risk-adjusted ADL decline	Risk-adjusted high-risk pressure sores	Risk-adjusted urinary incontinence
Regulation Index(HRSI)	0.005***	0.000	−0.002***	−0.011***	0.010***	−0.002**	0.000
Minimum staffing standards index	0.009***	−0.002*	0.004***	0.012***	0.006***	0.009***	0.003***
Medicaid per diem (hundreds of dollars)	−0.056***	0.013***	0.062***	0.578***	0.061***	0.024***	−0.033***
Competition	−0.012	0.143***	0.062***	0.452***	−0.017*	0.122***	0.000
Market income (ten of thousands of dollars)	−0.017***	−0.025***	0.026***	0.122***	−0.002	0.009***	0.003***
Market wage	0.149***	−0.133***	0.118***	−0.288***	0.394***	−0.155***	0.076***
Log (beds)	−0.006**	−0.012***	−0.067***	0.749***	0.016***	0.097***	0.020***
Log (bed-average bed)2	0.000	−0.002***	0.010***	0.021***	−0.005***	−0.002**	−0.001
Non-profit	0.050***	−0.009***	0.046***	0.317***	−0.008*	−0.036***	0.009***
Government-owned	0.103***	0.015***	0.049***	0.136***	−0.044***	−0.073***	0.026***
Chain	−0.074***	−0.007***	−0.009***	−0.037***	0.057***	−0.002	−0.001
Hospital-based	−0.027***	0.104***	0.277***	−0.400***	0.041***	0.067***	−0.025***
2006	−0.015***	0.007***	−0.007***	0.047***	−0.011***	−0.012***	−0.005***
Constant	1.242***	0.717***	0.138***	9.271***	0.210***	0.228***	0.553***
N	30,350	30,224	30,246	20,216	29,678	29,574	29,774
R2	0.063	0.059	0.316	0.609	0.049	0.042	0.013

^*0.05 ≤ p <0.1

^**0.01 ≤ p < 0.05

^***p < 0.01

Table 3 presents the regression coefficients for the first-stage equation and for all seven second-stage equations. The first-stage equation, shown in column 2, predicts the HRSI based on the IV. The IV is highly significant (p<0.000) and higher values of the Economic Freedom Index are associated with less stringent regulations as expected.

Table 3.

Two Stage Least Squares Models Estimating Regulation Stringency in the 1^st Stage and Quality in the 2^nd Stage

	1st Stage Equation:	2nd Stage Equations
	Regulation Stringency Index (HRSI)	CNA hours per case-mix adjusted day	LPN hours per case-mix adjusted day	RN hours per case-mix adjusted day	Hotel expenditures (in hundreds of thousands of $)	Risk-adjusted ADL decline	Risk-adjusted high-risk pressure sores	Risk-adjusted urinary incontinence
Economic Freedom IV≠	−1.48***	—	—	—	—	—	—	—
Regulation Index (HRSI) – Predicted	—	0.013***	0.013***	−0.007***	−0.003	−0.011≠	−0.003	−0.008***
Minimum staffing standard index	0.573***	0.005*	−0.008***	0.008***	0.010**	0.016***	0.010***	0.008***
Medicaid Per Diem (hundreds of dollars)	0.041	−0.068***	0.014**	0.065***	0.597***	0.035***	0.028***	−0.034***
Competition	−25.17***	−0.008	0.151***	0.058***	0.456***	−0.033***	0.122***	−0.006
Market income (tens of thousands of dollars)	1.94**	−0.017***	−0.024***	0.026***	0.122***	−0.002	0.009***	0.003***
Market wage	−1.53	0.158***	−0.138***	0.117***	−0.315***	0.424***	−0.158***	0.079***
Log (beds)	7.00***	−0.006*	−0.013***	−0.067***	0.748***	0.018***	0.097***	0.020***
Log (bed-average bed)2	−1.65*	0.001	−0.002***	0.009***	0.021***	−0.005***	−0.002**	−0.001**
Non-profit	−7.77***	0.052***	−0.006**	0.045***	0.320***	−0.012***	−0.036***	0.007***
Government-owned	−4.45	0.102***	0.015**	0.049***	0.137***	−0.044***	−0.073***	0.027***
Chain	5.73*	−0.076***	−0.008***	−0.008***	−0.037***	0.057***	−0.002	−0.001
Hospital based	−1.03	−0.027***	0.103***	0.277***	−0.403***	0.043***	0.066***	−0.025***
2006	−0.16	−0.015***	0.008***	−0.007***	0.049***	−0.013***	−0.012***	−0.005**
Constant	−0.61	1.245***	0.718***	0.137***	9.273***	0.213***	0.227***	0.552***
N	100	30,350	30,224	30,246	20,216	29,678	29,574	29,774
R2	0.400	0.064	0.064	0.317	0.607	0.045	0.042	0.016
95% CIboot for R2		[0.057,0.072]	[0.056,0.074]	[0.303,0.331]	[0.595,0.620]	[0.040,0.05]	[0.037,0.048]	[0.012,0.020]
DWH test of endogeneity: coefficient for residuals from 1st stage added to OLS		−0.010*	−0.016***	0.008**	−0.010	0.026***	0.001	0.011***

Incremental F and minimum eigenvalue statistic for the IV is 11.81 (p<0.01)

^*0.05 ≤ p < 0.1

^**0.01 ≤ p ≤ 0.05

^***p < 0.01

^≠0.05 ≤ p <0.1 for the one tailed hypothesis: β<0

The following columns in Table 3 show the results of the second-stage equations. Compared with the OLS, all models, except RN staffing, show a change in the HRSI coefficients in the direction of improved quality, indicating that the OLS estimates are biased as expected. The relationship between predicted HRSI and quality is significant and in the expected direction —i.e. more stringent regulation leads to better quality for four measures: CNA and LPN staffing and risk-adjusted urinary incontinence (all at the 0.01 level) and ADL decline. The latter was significant at the 0.1 level only for the one tailed test, but not for the two tailed test. However, in the model estimated on the smaller sample, excluding hospital based facilities, this result was significant at the two tailed 0.1 level as well. The relationship with the HRSI is not significant for hotel expenditures and risk-adjusted high risk pressure sores. It is also significant (at the 0.01 level) for RN staffing but in the opposite direction—i.e. more stringent regulation leads to less RNs.

Figure 1 presents the estimated incremental effect on quality for the average facility due to an increase of one standard deviation in the HRSI. The effect is presented as percent of the national mean value for each quality measure. The largest significant increase in quality is for LPNs at 8.7% of its national mean and the smallest is for urinary incontinence at 4.6% of its national mean. The effect on RN staffing is a decrease of 8.7% of its mean. To provide a sense of what a difference of 1 standard deviation in the HRSI means we present several examples in Table 4. Note that because the HRSI is defined as the sum of its components, a facility with a higher HRSI does not necessarily have higher values for all its components.

Figure 1.

Impact on quality of a 1 standard deviation increase in strigency of regulation index

Table 4.

Comparison of Nursing Homes with Different HRSI values

	Mean HRSI minus 1 standard deviation	Mean HRSI	Mean HRSI plus 1 standard deviation
2006 Example
State	Iowa	Nevada	Maine
HRSI	−3.13	−0.14	2.75
Average deficiencies per nursing home	6.09	12.98	11.17
% nursing homes with any deficiencies	88.99	95.74	100.00
% nursing homes cited for harm or jeopardy	25.62	23.40	24.32
% nursing homes cited for substandard care	3.60	4.26	13.51
Average CMPs per nursing home	0.31	0.15	0.35
2005 Example
State	Pennsylvania	Wisconsin	Colorado
HRSI	−2.94	−0.14	2.80
Average deficiencies per nursing home	6.53	5.76	10.76
% nursing homes with any deficiencies	94.37	85.60	96.68
% nursing homes cited for harm or jeopardy	20.70	30.08	46.65
% nursing homes cited for substandard care	1.83	9.77	7.11
Average CMPs per nursing home	0.12	0.98	0.17

All sensitivity analyses led to qualitatively similar results with respect to the impact of the HRSI on the quality measures. We do not present them due to space limitations.

Our cost-effectiveness assessment is based on estimates of incremental costs from Mukamel et al. (2011a) and the ADL increase from this study. These two complementary studies suggest that at about 1.1% of the annual operating costs of a nursing home, $87,600 in 2011 dollars per year for the average facility, quality can be improved in at least several areas as shown in figure 1, with improvements ranging from 4.6% to 8.7%. Our rough estimate of the cost-effectiveness ratio was about $72,000 in 2011 $/QALY with a plausible range of 51,000 to 152,000 $/QALY reflecting the range in assumptions about the ADLs to QALY conversion.

DISCUSSION

We examined seven measures of quality and found that in four of them there is a significant relationship between stronger regulation and better quality. We also found that for RN staffing there is a significant relationship but in the opposite direction, and in two areas, there is no statistically significant relationship. Previous studies (Bowblis and Lucas 2012; Bowblis et al. 2011), using panel data but not accounting for endogeneity, and assessing the impact of specific elements of regulation on quality, had similar findings.

The counterintuitive finding with respect to RN staffing might be due to substitution between expensive RN labor with less expensive LPN and particularly CNA labor, when nursing homes respond to more stringent regulations. RNs are the most expensive nursing labor component with wages three times those of CNAs (Mukamel, Schnelle, and Chang 2011b). Furthermore, federal regulations with respect to RN staffing are minimal, requiring nursing homes to employ only one RN per 24 hours. If nursing homes view LPNs and CNAs as more effective in meeting those quality aspects monitored by states, they are likely to increase investment in these two nursing types at the expense of RNs. Indeed, on average, the total net increase in nursing per case-mix-adjusted day is positive, at 5.4% of the mean. The finding of substitution is similar to results obtained by Feng et al. (2008) when investigating the impact of payment on staffing and Mueller et al. (2006) when examining the impact of staffing standards.

The lack of significant relationship between stringency of regulation and hotel expenditures is more difficult to explain. Regulations do address areas that can be considered as hotel quality, including the physical condition of the facility and the food it serves. Perhaps this finding reflects the regulators' primary focus on the medical aspects of care, leaving the quality of hotel services to be regulated by the market. In fact, as the second-stage equation estimates suggest, hotel expenditures are positively associated with both competition and market income, as one would expect if nursing homes were responding to demand for hotel quality, a dimension of quality that is much easier for consumers to assess (Mukamel et al. 2010). It is also possible that our measure of quality for this dimension, which unlike the others is a resource based proxy, is not sufficiently sensitive to capture the impact of regulations, a limitation that future studies should address.

We also note that our assessment of the impact of the regulations on quality is not comprehensive. We examined seven specific measures of quality, ranging from the more patient-focused, outcome-based measures to the more global staffing and expenditures based measures. Yet, these quality measures are not correlated. It is, therefore, likely that regulations might have a positive impact on other aspects of care that were not included in this analysis. If, as we would expect, they are also not correlated with the measures we studied, then this study underestimates the impact of regulations on quality. This limitation could also be addressed in future studies.

The quantitative estimates we present allow us to assess, however roughly, the quality regulation of nursing homes in the context of comparative effectiveness. This study demonstrates that quality regulation of nursing homes improves the quality of care they provide. In the spirit of comparative effectiveness we should ask whether this is the best policy tool for achieving this degree of quality improvement (Institute of Medicine 2009). The two alternative policies that have been used to date to achieve this objective are increased “degree” of competition, through increase in the number of nursing homes in the market and “improved” competition, through publication of quality report cards and informing consumers about quality.

Our analysis shows that staffing levels and hotel expenditures are more sensitive to competition then to the regulation index. In our analysis a one standard deviation increase in competition results in 11.9% net increase in total staffing. A similar one standard deviation increase in the HRSI results in a 5.4% net increase in total staffing. Similarly, a one standard deviation in competition increases hotel expenditures significantly by 9.9%, while regulation does not. On the other hand, for the two risk adjusted outcomes, regulation seems to have more of an impact than competition.

It is more difficult to assess the comparative effectiveness of the CMS report card because we were unable to include it explicitly in our analyses. However, prior studies suggest that the effect of report cards in bringing about quality improvement is small and tends to be limited to a few outcome measures (Werner, Konetzka, and Kruse 2009a; Werner et al. 2009b). It seems to be stronger in those areas that are more competitive as might be expected (Grabowski and Town 2011), and by shifting the emphasis from hotel quality to clinical quality may have actually led to a relative reduction in hotel quality (Mukamel et al. 2010).

A particular limitation of the comparative effectiveness assessment is that each of the two policies – i.e. competition and public quality reporting – is fundamentally different and hence measured in different units. Even if we were able to place them on equal footing by comparing the impact of a one standard deviation of each, the comparison would not be meaningful or intuitively understandable. An alternative is cost-effectiveness analysis, in which we compare the costs per Quality Adjusted Life Year (QALY) to an accepted benchmark.

We estimated the cost-effectiveness of a one standard deviation increase in regulatory stringency, assuming that regulation only affects ADL loss, at about $72 thousand in 2011 $/QALY, with a range of about 51,000 to 151,000$/QALY. As a benchmark for comparison, the Food and Drug Administration currently uses $100,000 per QALY in its regulatory analyses, a number double the commonly used rule-of-thumb but less than what would be consistent with estimates of people's willingness to pay to avoid increases in mortality risks (Evans, Tavakoli, and Crawford 2004). Treatments that have cost-effectiveness ratios below this benchmark are considered to be worthwhile adopting. This suggests that the cost-effectiveness of the quality regulations are in the ball park of accepted health care practices.

We note that we did not have data to calculate similar cost-effectiveness ratios for the improvement obtained in other domains. Furthermore, some of the domains, in particular the staffing domains, likely encompass the health outcomes domain, and it is unclear how one would separate their contribution to quality. At the same time, our cost estimates are comprehensive. Therefore, the cost effectiveness we calculated for ADL improvement is likely an underestimate of the true cost-effectiveness of the regulation.

In summary, we find that quality regulation of nursing homes leads to better quality, at least in some domains, it seems to be less effective than competition but more effective than report cards, and comparable to the acceptable benchmark for adopting interventions based on cost-effectiveness ratios. As competition is not an option that is available in most markets (Mukamel and Spector 2002), regulations seem to be an important and effective policy tool for quality improvement for nursing homes.