Examination of the New Short-Stay Nursing Home Quality Measures: Rehospitalizations, Emergency Department Visits, and Successful Returns to the Community

Abstract

In 2016, the Centers for Medicare & Medicaid Services (CMS) introduced 3 new quality measures (QMs) to its report card, Nursing Home Compare (NHC). These measures—rehospitalizations, emergency department visits, and successful discharges to the community—focus on short-stay residents. We offer a first analysis of nursing homes’ performance in terms of these new measures. We examined their properties and distribution across nursing homes using descriptive statistics and regression models. We found that, similar to other QMs, performance varies across the country, and that there is very minimal correlation between these 3 new QMs as well as between these QMs and other NHC QMs. Regression models reveal that better performance on these QMs tends to be associated with fewer deficiencies, higher staffing and more skilled staffing, nonprofit ownership, and lower proportion of Medicaid residents. Other characteristics are associated with better performance for some but not all 3 QMs. We also found improvement in all 3 QMs in the second year of publication. This study contributes to the validity of these measures by demonstrating their relationship to these structural QMs. It also suggests that these QMs are important by demonstrating their large variation across the country, suggesting substantial room for improvement, and finding that nursing homes are already responding to the incentives created by publication of these QMs.

• What do we already know about this topic?

• This is the first article to examine the properties of the new claims–based short-stay quality measures for nursing homes introduced by the Centers for Medicare & Medicaid Services.

• How does your research contribute to the field?

• This study provides information about the properties of the claims based short-stay quality measures, including their distributional properties across nursing homes, correlations with other quality measures, and structural measures of quality.

• What are your research’s implications toward theory, practice, or policy?

• The findings of this study contribute information toward the validity of these measures. Furthermore, it makes the case for including these measures in Nursing Home Compare and the Five-Star composite measure by demonstrating that they offer additional information about quality, not included in other quality measures.

Introduction

Overview

Nursing Home Compare (NHC)¹ is the oldest quality report card that the Centers for Medicare & Medicaid Services (CMS) publishes. It is intended to provide consumers with information that is useful in selecting and monitoring nursing homes’ (NHs) quality while simultaneously encouraging providers to improve performance. Since its first publication of quality measures (QMs) in 2002, NHC has reported separate QMs related to the care of short-stay and long-stay NH patients, commonly referred to in this context as residents. Short-stay residents are typically those admitted from an acute care hospital for postacute or rehabilitation care, often for stays of days or a few weeks. Long-stay residents are those who remain in the NH for longer periods, often until death, because of need for assistance with physical or mental limitations and a lack of corresponding community resources.

In 2016, CMS added 6 new QMs to NHC. Four of these were specifically designed to assess the quality of care provided to the short-term residents, and as noted in the CMS’ press release at the time,² this doubled the number of short-stay QMs, which until then were significantly fewer than the long-stay QMs. These new QMs greatly increased the value of NHC to the short-stay population, which represents an increasing proportion of NH admissions.³ Furthermore, 3 of the new QMs assess a new care domain not addressed by the prior QMs—resident discharge and transfer status. These 3 QMs are NH rehospitalization rates, NH rates of emergency department (ED) visits, and NH rates of successful discharge to the community.⁴ In addition, these QMs are unique in NHC because, unlike all other QMs, they rely primarily on hospital claims data, which are not self-reported by NHs.² Thus, as these 3 QMs are fundamentally different from all the other QMs, they are of particular interest and the focus of the analyses presented here.

Description of QMs and Study Questions

These 3 QMs are important markers of NH quality. Stakeholders have identified the rate of discharge from NHs to the community as an important indicator of person-centered care.⁵ Conversely, rehospitalizations and ED transfers from NHs increase health care expenditures, expose residents to potentially adverse health events in the hospital, and are considered potential markers of inappropriate or inadequate NH care.^6-9

The data sources for these QMs include both the Medicare hospital claims and the Minimum Data Set (MDS). The MDS¹⁰ is a standardized assessment of all residents in the over 95% of NHs nationally that are Medicare or Medicaid certified. MDS assessments are made by NH staff at the time of admission, quarterly thereafter, and at other prescribed intervals during the resident’s stay. Hospital claims provide information for the QMs about the outcomes and some of the risk factors, and the MDS provides information for other risk factors. This hybrid specification differs from the other NHC QMs that are based solely on MDS data.

This hybrid approach presents both advantages and disadvantages relative to the QMs that rely solely on MDS data. A major advantage is that the addition of the hospital claims data offers diagnoses and procedural information from the hospitalization preceding the NH admission, thus potentially improving the risk adjustment.¹¹ In addition, as NHs are not required to complete information for all ED visits and hospitalizations, the frequency of ED visits and hospitalizations is likely to be more accurate if based on hospital and ED claims which are used for reimbursement purposes. Furthermore, because the hospital data are not controlled by the NHs, these data could not be subject to manipulation or gaming by NHs. This consideration is particularly compelling given CMS’ plans to implement a value-based purchasing system in 2019 that would include a rehospitalization measure.¹²

A disadvantage of this hybrid approach is that it is based on data for only fee-for-service residents because comparable hospital Medicare claims are not available for Medicare Advantage (MA) enrollees. This means that, for the first time, publicly reported NH QMs do not include all residents who meet the specification for the measure. This could potentially bias these QMs, especially for NHs that serve predominantly the MA population. The exact magnitude of this bias is difficult to ascertain because data about the percent of MA residents in NHs are not generally available. A study by Meyers et al¹³ found that in 2012-2014 in a sample of about 4.5 million Medicare NH residents, including both short-term and long-term residents, 27% were MA enrollees. This number closely parallels the 27% to 30% of MA enrollees in the general Medicare population. Enrollment in MA has continued to grow since that study was conducted. In 2017, 33% of Medicare beneficiaries were enrolled in MA and the percent enrollment in MA is highly variable across states.¹⁴ This suggests that the potential bias introduced by these data limitations would affect different states and possibly different NHs differentially.

Another disadvantage resulting from the use of claims data is that these measures, unlike the MDS-based QMs, currently are not updated quarterly. They are updated less frequently, thus offering NHs less frequent data points for quality improvement efforts and providing less timely information for consumers. This disadvantage, however, could be overcome by CMS, for example, if the measures were calculated as an annual rolling average with more frequent updates.

As these new QMs are different in their construction from prior QMs, little is known about their performance in actual use and their relationship to other QMs. In this study, we examine 3 of their properties.

1. Correlations between the 3 new QMs: Unlike most other QMs, these 3 QMs are more global in nature and should capture the quality of care provided by NHs in terms of prevention of decline, early detection of change that would avert more serious acute decline, and once serious decline occurs, providing appropriate evaluation and services in the NH (eg, physical exam, obtaining laboratory testing, appropriate antibiotics) to manage the acute change and avoid transfer.⁷ Because of that, we expect risk-adjusted measures of rehospitalization and ED visits to be positively correlated with each other and inversely correlated with successful community discharge. It should be noted, however, that return to the community, while influenced by these medical care practices in the NH, is also influenced by the quality of rehabilitation services, quality of discharge planning, and knowledge about as well as availability of community-based support services that are available to meet the resident’s needs after discharge. Hence, the magnitude of the negative correlation with the rehospitalization and ED QMs might be tempered.

2. Correlations between the new QMs and all other QMs: It is unclear a priori whether the new QMs should be correlated with all other MDS-based QMs, either the short-stay or the long-stay QMs, which tend to be very condition-specific (eg, percent of residents with pain, percent of residents who received antipsychotic medications). On the one hand, they are more global in nature and may, therefore, capture health outcomes related to the more condition-specific QMs. For example, inappropriate care leading to acute chronic heart failure and rehospitalization may also lead to loss of activities of daily living, which is measured by one of the long-term QMs. This would result in a positive correlation. Furthermore, the new short-stay QMs are measured over the short-term residents while a large number of the old QMs are measured over the long-term resident population. Despite this difference in population over which these QMs are measured, one might expect that management policy vis-à-vis quality will affect both “product lines” the same way, resulting in similar levels of quality and a positive correlation between these new short-stay QMs and the older long-term QMs.¹⁵ On the other hand, as has been observed in prior studies of correlations of various QMs for NHs¹⁶ as well as other care settings,^17-19 QMs measuring different domains of quality tend to be uncorrelated.

3. Correlations with organizational and market characteristics of the NH: If these new QMs behave as expected, then NHs that perform well on these measures would have structural characteristics commonly associated with better quality of care, such as higher levels of staffing.

Methods

Sample

Our sample included all NHs included in the CMS published report card, NHC, between April 2016 and October 2017. At the time we performed the study, QMs were available for 2 partially overlapping time periods: period 1, which included data for all of 2015, and period 2, which included data for the third and fourth quarter of 2015 and first and second quarters of 2016. There were 15 652 NHs in period 1 and 15 660 in period 2, resulting in a total of 31 312 facility/period observations. We downloaded the 3 new QMs for each facility for each period from the CMS web site.²⁰ Because NHC does not report QMs if the facility had fewer than 20 eligible residents for the QM for the reporting period,⁴ 12.7% of the facility/period rehospitalizations and admissions to outpatient ED QMs observations were deleted as were 26.9% of the successful discharges to the community QM facility/period observations (see Table 1). These data were then merged with the 2015 data available about NHs’ characteristics in the Long Term Care focus (LTCfocus) database²¹ and urban/rural status data from the Census.²² LTCfocus is a web-based interactive database maintained by Brown University that offers data about all certified US NHs including data on average resident characteristics, ownership, financial, and market characteristics. Observations with missing NH characteristics were excluded, accounting for 2.7% of rehospitalizations and admissions to outpatient ED QMs facility/period observations and 2.4% of successful discharge to the community QM facility/period observations. The final sample included 26 497 (84.6%) observations for the rehospitalizations and admissions to outpatient ED QMs and 22 133 (70.7%) observations for the successful discharges to the community QM.

Table 1.

Nursing Homes’ Descriptive Statistics—Mean (SD in Parentheses).

	Full NHC samplea	Study sample for rehospitalized and ED visits QMsb	Study sample for discharged to community QMb
Sample size (no. of facilities/periods)	31, 312	27, 332	22, 896
	Mean (SD)
Dependent variables—QMs
% short-stay residents rehospitalized after admission	NA	21.83 (6.04)	NA
% short-stay residents with outpatient ED visit	NA	11.96 (5.38)	NA
% short-stay residents discharged to the community	NA	NA	56.98 (10.13)
Independent variables
Total admissions per beds	2.36 (2.51)	2.57 (2.57)	2.88 (2.72)
Health deficiencies	7.33 (6.55)	7.45 (6.64)	7.44† (6.65)
Direct care staff hours per resident day	3.74 (1.19)	3.73† (1.12)	3.81 (1.12)
RN/(RN + LPN)	0.36 (0.20)	0.36† (0.20)	0.37 (0.20)
Beds	107.39 (61.02)	113.47 (59.80)	117.59 (60.84)
Beds’ distribution: 25th; 75th; and 99th percentile bed size	76; 134; 320	76; 134; 320	81; 140; 321
% Medicaid residents	59.06 (23.12)	58.02 (22.80)	55.90 (22.75)
% Medicare residents	15.09 (14.59)	16.49 (14.58)	18.15 (15.13)
HHIc	0.21 (0.25)	0.19 (0.23)	0.18 (0.22)
	%
Year (2016 = 1)	50.01	50.51	51.10
Urban/rural (urban = 1)	69.74	72.47	75.66
Hospital based (yes = 1)	5.21	3.73	4.08
Part of a chain (yes = 1)	57.01	59.44	60.50
Nonprofit (yes = 1)	30.68	28.09	28.32

Note. ED = emergency department; QMs = quality measures; NA = not applicable.

^aNHC = Nursing Home Compare sample before exclusions for missing QMs due to denominator less than 20.

^bSample used in analyses after exclusion for missing QM.

^cRN=Registered Nurse, LPN=Licensed Practical Nurse.

^dHHI is the Herfindahl-Hirschman Index measuring competition. It is defined as the sum of the squares of the market shares of nursing homes in the county and ranges between 0 = infinite competition and 1 = monopoly.

Unless noted with a^†, all variables were significantly different at the .05 level when compared with the initial sample with all QMs.

Variables

The dependent variables of interest, rehospitalization rates, outpatient visits to the ED, and rates of successful discharge to the community were calculated by CMS from Medicare claims data augmented with MDS data for all Medicare fee-for-service residents. These measures are risk-adjusted, using both claims and MDS data as described in the CMS Technical Specifications.⁴ They are defined by CMS as follows: Rehospitalization is measured as the percentage of all new NH admissions or readmissions from a hospital where the resident was then readmitted to a hospital for an unplanned inpatient or observation stay within 30 days of the NH entry or reentry. Outpatient ED visits is measured as the percentage of all new admissions or readmissions to a NH where the resident had an outpatient ED visit (ie, an ED visit not resulting in an inpatient hospital admission) within 30 days of entry or reentry to the NH. Successful discharge to the community is measured as percentage of all new NH admissions from a hospital where the resident was discharged to the community within 100 calendar days of NH entry, and for the 30 days after discharge to community, did not die, did not have an unplanned inpatient stay, or get readmitted to a NH. The first 2 QMs should decrease with quality and the third QM should increase with quality.

Independent variables included NH organizational and market characteristics that were shown in previous studies to be associated with quality of care in NHs. Direct care staffing per resident day and higher ratio of registered nurses to total licensed nurses have been shown to be associated with better quality.^23-25 NHs that are hospital based, nonprofit,²⁶ and not part of a chain have also been shown to provide higher quality.²⁷ Similarly, lower Medicaid census and higher percentages of private pay and Medicare residents have been associated with better quality.^28,29 The relationship of the number of beds with quality varies depending on the quality domain studied.^30-32 Urban location³³ and competition also are associated with quality.^34,35 In addition, we considered deficiency citations. These are issued based on a surveyor visiting the NH and determining that the facility does not meet specific state and federal quality standards. Although often dependent on state policies, once controlling for state fixed effects (as in the models we estimate), they can be interpreted as an indicator of the quality of care the NH provides.

All variables were defined as categorical variables. Those variables that were originally continuous, percent Medicaid residents, percent Medicare residents, and number of beds, were redefined into 3 dichotomous variables, indicating if the facility had fewer than 25%, between 25% and 75%, or more than 75% to allow for nonlinear relationships between the variable and the QM. For the competition measure, the Herfindahl-Hirschman Index (HHI), defined as the sum of squares of market shares of all NHs in the market divided by 10,000, we used the Department of Justice’s general definitions in effect during the time interval under study: HHI < 0.11—unconcentrated, ie, competitive markets; 0.11 < HHI < 0.18 moderately concentrated; HHI > 0.18 highly concentrated, ie, not competitive markets.³⁶ To examine change between the first and second period of QM data, we also included a year variable with the value 1 for period 2, 0 for period 1. To control for cross-sectional differences between NHs in their tendency to serve mostly postacute or long-stay residents, we included the number of total admissions per NH bed, assuming that NHs that tend to serve short-stay residents preferentially would have more admissions per bed.

Analyses

The analyses address the 3 questions we presented in the introduction. First, we calculated the correlations between each of the 3 new QMs between the two periods and among the 3 QMs in the same period. Second, we calculated, for each time period, the correlation between each of the 3 new QMs and all the old QMs, both the short-stay and the long-stay QMs. This tested the hypothesis that these new QMs offer new information and perspective on quality and do not repeat information already carried by previous QMs. We also calculated and present the average QMS for each state and the national average. We examined the states’ averages relative to the national mean and identified those that are statistical outliers based on lying above or below 2 standard deviations (SDs) from the national mean.

To address the third question, we estimated 3 separate regression models, one for each of the new QMs as a dependent variable and the independent variables described above. The models were estimated as ordinary least squares models with state fixed effects and random NH effects to account for potential correlation between observations for the same facility over the 2 time periods.

Results

Table 1 presents descriptive statistics for the full NHC sample and the 2 analytical samples created after excluding observations with missing QMs. A comparison of the facility characteristics between the samples reveals statistical differences for all but 3 characteristics, likely due to the large sample size, as the size of the differences was not considered meaningful. The 2 analytical samples have slightly higher admissions per bed, larger facilities (beds), more urban, more chain membership, and higher percent of Medicare residents, lower percent Medicaid resident, and fewer nonprofits. The differences tend to be somewhat larger in the return to community data set.

Distributional Properties of the New QMs

Each of the 3 QMs shows strong positive correlations between period 1 and period 2 ranging from 0.64 for rehospitalization to 0.70 for ED visits. However, the correlations among the measures are much weaker. The correlation between the ED and the rehospitalizations measures was approximately 0.25, between discharge to community and rehospitalization it was approximately −0.3, and between ED visits and discharge to the community it was only approximately −0.05. The correlations to the older QMs were also very weak. Most of the correlations between the new QMs and the old QMs were below 0.1 and none exceeded 0.19.

Figure 1 presents the geographic distribution of the QMs across states (averaged over the 2 periods). These graphs show the national average as a heavy horizontal line, with 4 dashed lines showing 1 and 2 SDs above and below the national mean. States are ordered based on the sorting of the rehospitalization QM from lowest to highest and are kept in the same order for all 3 panels. The range of values on the y-axis for all the 3 panels is the same to make the visual comparison valid. There is substantial variation across states in all QMs, with the highest variation for ED visits, with a facility-level coefficient of variation of 0.45 and the least variation for discharge to the community, with a facility-level coefficient of variation of 0.18. Despite that, the number of statistical outliers lying outside the 2 SDs for any of the QMs is minimal. In fact, for rehospitalization, only 2 states are statistical outliers performing beyond 2 SDs of the mean: Alaska performs better than average and Arkansas performs worse than the average. For outpatient ED visit, Hawaii is the only state significantly better than the national average with Oklahoma and Arkansas being slightly but significantly worse than the national average. Similarly, for successful discharges to the community, 2 outliers are better than average (Alaska and Hawaii) and one (Louisiana) is worse than average.

Figure 1.

Geographic distribution of the new short-stay QMs.

Note. QMs = quality measures.

Table 2 reports the results of the regression analyses. Note that due to the definition of the QMs, positive coefficients mean increased quality for the discharges to the community QM and decrease in quality for the 2 other QMs. Because all are measured as a percentage, the interpretation of the magnitude of the coefficients is similar, although the relative magnitude is different as the average level is different for each QM (see Table 1).

Table 2.

Short-Stay QMs as Functions of NH Characteristics.

	Short-stay residents rehospitalized after admissiona	Short-stay residents who had outpatient emergency department visita	Short-stay residents successfully discharged to the communityb
Period (2015 = 0;2015/2016 = 1)	−1.462***	−0.223***	0.368***
Number of admission/beds	0.278***	0.030	0.291***
Total number of health deficiencies	0.022***	0.036***	−0.081***
Direct care staff hours per resident day	−0.133**	−0.113*	0.243**
RN/(RN + LPN)	−2.110***	−1.792***	4.008***
Hospital based (yes = 1)	−1.569***	−0.353	0.175
Part of a chain (yes = 1)	−0.222*	0.267**	0.137
Nonprofit (yes = 1)	−0.446***	−0.287**	2.209***
Number of beds
0-25th percentile	Reference category	Reference category	Reference category
25th-75th percentile	0.074	0.137	0.526*
75th-100th percentile	0.048	−0.393**	0.843**
% Medicaid residents
Less than 25%	Reference category	Reference category	Reference category
25%-75%	0.359**	0.482***	−0.271
More than 75%	0.480**	0.884***	−1.830***
% Medicare residents
Less than 25%	Reference category	Reference category	Reference category
25%-75%	0.095	0.003	−0.223
More than 75%	0.038	0.140	−0.179
HHIc
<0.11	Reference category	Reference category	Reference category
0.11 < HHI < 0.18	−0.375*	1.243***	0.434
>0.18	−0.448**	1.243***	0.663*
Urban location (yes = 1)	0.716***	−1.401***	−0.385
Constant	23.359***	11.253***	56.708***
Sample size—NH/periods	26 497	26 497	22 133
Sample size—Unique NH observations	13 516	13 516	11 515
R2 between	0.116	0.147	0.141
R2 overall	0.107	0.130	0.120

Note. Models estimated with fixed state effects and random facility effects, not shown. QMs = quality measures; HHI = Herfindahl-Hirschman Index; NH = nursing home, RN = registered nurse, LPN = licensed practical nurse.

^aNegative coefficient means increasing the variable is associated with increasing quality.

^bPositive coefficient means increasing the variable is associated with increasing quality.

^cHHI is the Herfindahl-Hirschman Index measuring competition. It is defined as the sum of the squares of the market shares of NHs in the county and ranges between 0 = infinite competition and 1 = monopoly.

^*P < .05. **P < .01. ***P < .001.

The coefficient for period shows that all 3 QMs improved in period 2 compared with period 1, with the largest improvement observed for rehospitalization. As we expected, both the number of rehospitalizations and discharges to the community of short-stay residents increased with their increased proportion in the facility (higher percent of total admissions per bed). Surprisingly though, this variable was not significantly associated with the ED visits variable.

Of the variables hypothesized to be associated with higher quality, our findings are mixed. Six of the 11 regression coefficients shown in Table 2 are associated with quality as expected for all 3 QMs. They include the number of health deficiencies, staffing, both total hours per resident day and the ratio of RNs to licensed nurses, nonprofit status, percent Medicaid but not percent Medicare. The other five show mixed results. Being hospital-based significantly increases quality as measured by readmissions. While the direction of the coefficients for the other 2 measures is also toward higher quality, these are not statistically significant. Of the other characteristics/QM dyads, only the following were associated significantly with quality as expected: larger NHs performed better in successful discharge to the community and lower ED visits, and competition and urban location improved performance on the ED QM.

Discussion

In 2016, CMS introduced 3 new QMs for short-stay residents to NHC—rehospitalizations, ED visits, and successful discharges to the community. The objectives of this study were to examine their properties and distribution across NHs using descriptive statistics and regression models. The study found that the ED and the rehospitalization measures are only slightly correlated with each other. This correlation does not stem from overlapping definitions, as the ED QM excludes ED visits that are followed up by a hospital admission. The correlation between them probably reflects the fact that both are conceptually related to a NH’s ability to avert and manage acute events. The correlation, however, is relatively low because the ED measure likely captures more stable transfers where the ED is effectively substituting for evaluation and treatment in the NH or an outpatient office setting, while the rehospitalization measure reflects transfers of more clinically complex and resource-intensive residents that the NH cannot care for in the facility. Neither of these QMs is correlated with the successful return to community QM, which covers a longer time window and excludes death as well as hospital admissions. Thus, these 3 QMs offer different, nonoverlapping information and perspective on NHs’ quality, in a new domain not previously available, namely, resident discharge and transfer status.

These new QMs are uncorrelated with the older NHC QMs, suggesting that the new QMs offer new information beyond that available from the existing QMs and, as such, are an important addition to NHC. Thus, their inclusion in the NHC Five-Star composite measure, which summarizes several of the QMs to give an overall ranking, makes the Five-Star composite more informative.

A lack of correlation between the older QMs is a well-documented phenomenon. Previous studies have found weak or no correlation among the old NH QMs,¹⁶ among hospital QMs,^17,18 and QMs in primary care.¹⁹ The absence of correlations can be attributed to several potential causes, some related to the production of quality and others related to its measurement. The production of quality in all these settings can create competition for resources. For example, both the prevention of pressure ulcers and nonpharmacological management of rejection of care behaviors to avoid antipsychotics are labor-intensive, requiring nurse assistant times to reposition the resident frequently to avoid pressure ulcers or to identify and address unmet needs to decrease rejection of care behaviors.³⁷ These tasks compete for labor resources in a resource-limited environment, which is common for health care organizations. Hence, excellence in one QM can translate into average, or even less than average, performance in another QM. This results in uncorrelated QMs, suggesting that quality assessment should be based on consideration of multiple measures.

Furthermore, the lack of correlation between the new and the old QMs may also reflect differential ability of NHs to adjust to the new measures. In addition, measurement-related issues could contribute to the lack of correlation between the old and the new QMs due to reliance on different data sources. The old QMs are based on MDS data only. The new QMs are also based on claims data. Aside from potential data quality differences between the two, they are also calculated for and reflect care of different residents. The old QMs reflect care provided to all residents while the new QMs include only fee-for-service Medicare residents and do not include MA enrollees. As the penetration of MA is highly variable across states¹⁴ and the medical management and health of MA residents may differ in unobserved ways,³⁸ this may also lead to measurement-related differences between the old and the new QMs and, hence, lack of correlation.

A large number of NHs did not have a sufficient number of eligible residents to have their QMs published in NHC. During our study period, 2015/2016, 12.7% of NHs did not have their rehospitalizations and admissions to outpatient ED QMs reported and 26.9% did not have their successful discharges to the community QM reported. CMS excluded these NHs from reporting because of valid concerns about the statistical accuracy of QMs when the sample has fewer than 20 residents. These NHs, for the most part, were similar to the majority of NHs that did have all QMs reported, although, particularly with respect to the NHs missing the return to community QM, these NHs tended to be slightly smaller, with fewer admissions per bed, lower percent Medicare residents, and higher percent Medicaid residents. This suggests that these NHs care for a resident mix that is slightly more skewed toward the long stay, rather than the postacute, short-stay residents, consistent with the fact that they do not have a large enough sample of eligible short-stay residents for calculating these QMs. Despite that, these NHs are a large percent of all NHs nationally and they do admit and care for short-stay residents. Further studies to understand the implications of missing QM information about 25% of NHs on both consumers’ search and providers’ behavior are warranted.

The variation both at the facility level and at the state level is substantial. At the facility level, the calculated rehospitalizations QM ranges from a low of 0% to a high of 56%, the ED QM from 0% to 55%, and the discharges to the community from 0% to 100%. And while there are very few states that are statistical outliers when compared with the national mean, the range of values for the state means across the 3 QMs is between 8 and 15 percentage points. This variation suggests that there is substantial room for improvement. It also suggests that future studies should explore sources of state variation such as funding for home- and community-based services and Medicaid reimbursement for medical providers.

The regression findings suggest that improvement has already been achieved in all 3 QMs between the first period of data, 2015, which was published in early 2017, and the second period, which included data for the second half of 2015 and the first half of 2016. The magnitude of average improvement was not large. The relative improvement between the 2 periods was the largest for rehospitalization at 6.7%, followed by ED visits at 1.9% and return to the community at 0.6%. The larger improvement in rehospitalization is not surprising, given an ongoing trend toward decreasing rehospitalizations from any location, in part associated with hospitals facing financial penalties for rehospitalizations.³⁹ It remains to be seen if NHs will continue to improve in the coming years and at what levels these QMs stabilize. The fact that improvement has already begun speaks to the importance of these QMs. This conclusion might be tempered if NHs, rather than actually improving quality along these 3 new dimensions, have found other ways to improve their scores. While some of the data, the QMs and some of the risk factors, originate in hospital claims and cannot be manipulated by a NH, the risk factors originating in the MDS could be. In addition, NHs might be paying more attention to correctly documenting relevant information supporting the QMs or engage in adverse patient selection, such as limiting the admission of individuals with high risk of readmission that is not fully captured by the risk adjustment. If the improvement we observe results from “severity creep” or unobserved adverse selection, we would expect it to reach a steady state within a few years and stabilize, at which time only true care improvement would lead to improvement in the QMs.

The regression analyses we present explore the construct validity of these QMs, showing the association between them and NHs’ structural characteristics that are known to be associated with better quality care and resident health outcomes. For the majority of the characteristics explored and for all 3 QMs, relationships were as anticipated. For example, higher staffing level, as well as staffing mix skewed toward registered nurses, have been shown to lead to better resident outcomes, which is particularly important for short-stay residents and residents with complex conditions. Consistent with other studies of quality in this setting, nonprofit status and percent Medicaid were also associated with QM performance. Another important finding is the relationship to deficiencies issued by state regulators during their annual inspections. The regression analyses show that all 3 QMs indicate higher quality in NHs that received fewer citations, as one would expect.

We note that not all NH characteristics were associated with all 3 QMs in the direction expected. For example, higher competition was associated with better performance on the ED QM, as one might expect. However, both the rehospitalizations and returns to the community QMs exhibited worse performance with higher competition. Similarly, a larger numbers of beds was associated with improvement in the community discharge and ED QMs but was not associated with the rehospitalizations QM. This phenomenon, whereby NHs respond to incentives to improve along some but not all dimensions measured in the quality report card, likely reflects resource constraints they face and the need to make strategic choices about investment in quality initiatives when new information about performance is released. Earlier studies of NHs reactions to publication of report cards similarly found that improvement was limited to a subset of the QMs and did not address all areas the report card covers.^40,41

We note some limitations of the study. First, because the period 2 QMs overlapped with period 1 QMs for half of the period, it is likely that our findings underestimate the improvement between the 2 periods if the trend we observed continues. Second, the regression models we estimated are likely more noisy than they would have been if all data had been time-consistent. This noise might result in inflated standard errors and failure to identify statistically significant associations. It is unlikely, however, to introduce a bias, as for the majority of NHs the changes in characteristics from year to year are likely to be minimal. Finally, the literature suggests additional organizational characteristics that are associated with quality, such as staff communication,⁴² team cohesion, and consistent assignment.⁴³ We were unable to include these in our analyses because data were not available. This might have introduced missing variable bias to the degree that such variables are correlated with the included variables. Future studies will have to address this shortcoming.

Understanding the behavior of these QMs, showing that they contribute unique information about facilities, and are associated with structural factors that are associated with quality care suggest that these QMs make unique and valuable contributions in NHC¹ and in the Five-Star composite measure.⁴⁴ High variation across facilities and states suggests that opportunities for improvement exist. At the same time, the mixed construct validity points to the need to better understand NHs’ behaviors and characteristics that will reliably improve performance across these measures. Understanding the performance of these measures is particularly important given CMS’ plans to include the rehospitalization measure in the Nursing Home Value–Based Purchasing program slated to begin in 2019.¹²