Have Nursing Home Compare quality measure scores changed over time in response to competition?

Abstract

Background

Currently, the Centers for Medicare and Medicaid Services report on 15 Quality Measures (QMs) on the Nursing Home Compare (NHC) website. It is assumed that nursing homes are able to make improvements on these QMs, and in doing so they will attract more residents. In this investigation, we examine changes in QM scores, and whether competition and/or excess demand have influenced these change scores over a period of 1 year.

Methods

Data come from NHC and the On‐line Survey Certification And Recording (OSCAR) system. QM change scores are calculated using values from January 2003 to January 2004. A series of regression analyses are used to examine the association of competition and excess demand on QM scores.

Results

Eight QMs show an average decrease in scores (ie, better quality) and six QMs show an average increase in scores (ie, worse quality). However, for 13 of the 14 QMs these average changes averaged less than 1%. The regression analyses show an association between higher competition and improving QM scores and an association between lower occupancy and improving QM scores.

Conclusion

As would be predicted based on the market‐driven mechanism underlying quality improvements using report cards, we show that it is in the most competitive markets and those with the lowest average occupancy rates that improvements in the QM scores are more likely.

The former Health and Human Services Secretary, Tommy Thompson, recently announced that many nursing homes have improved quality scores on the web‐based Nursing Home Compare (NHC) report card.¹ NHC reports information about the number of beds, types of ownership, staffing and quality (given in the form of Quality Measures (QMs)) for every Medicare/Medicaid‐certified nursing home in the USA.² (Almost all US nursing homes participate in this certification, in order to be eligible for government payments.) By using this information, consumers select a nursing home that best meets their preferences in these areas. However, the intention of the NHC report card was not only to provide consumers with information, but also to promote quality change in the nursing home industry. Therefore, this statement by Tommy Thompson was noteworthy.

The mechanism behind this change in the industry rests on consumers' migrating towards higher‐quality facilities, and nursing homes' competing to improve their quality, in order to attract potential residents. The use of NHC by consumers can be either real or imaginary—that is, the threat of potentially losing future residents may be incentive enough for nursing homes to make improvements on the quality measures reported in NHC. They do not necessarily have to experience any loss of business. However, these changes in the marketplace rest on the assumption that nursing homes compete for residents.

In the nursing home industry, some markets experience limited competition between facilities. For example, the 2004 On‐line Survey Certification And Recording (OSCAR) system data (which include almost all nursing homes in the USA) show that 18% of markets in the USA have only one or two nursing homes. Many nursing homes also experience high demand (most notably observed as very high occupancy rates and even waiting lists). The 2004 OSCAR shows that 20% of markets in the USA have facilities with an average occupancy rate of 95%. Thus, the reported average changes in QM scores may mask the more notable changes in some markets. We propose, following the mechanism described above, that nursing homes in markets with more competition or excess supply will respond to NHC by improving their quality.

Understanding whether competition and excess supply influence QM scores may be important. First, this represents the linchpin to the consumer empowerment policies currently being pursued in healthcare by the Bush Administration. (This approach seeks to give consumers information and enable them to choose providers.) Little evidence exists that this represents a tenable approach to complementing the regulatory approach of prior administrations. Second, if the QM scores are sensitive to competition and occupancy (ie, excess supply), then this will probably represent a harbinger of future gains. As policies such as those promoting increased competition in the long‐term care market proceed (such as Home and Community‐based Waivers), and other providers expand (such as assisted living providers), we will probably see changes in many more markets towards increased competition and lower occupancy. In this investigation, we seek to determine whether competition and/or excess supply influence QM scores reported on NHC over a period of 1 year.

Data and methods

Source of data

Data used in this analysis primarily came from the NHC website. The NHC data were downloaded from the website in January 2003 and again in January 2004. This included information from 14 QMs (which included all of the QMs reported during this timeframe). In addition, aggregate resident characteristics and facility characteristics from the OSCAR data were used. The OSCAR is collected yearly for most facilities in the USA as part of the Medicare/Medicaid certification process.³ We used OSCAR information that was collected during 2003, thus matching the time in which the NHC information was collected.

Model specification

The dependent variables used in the analyses were the 14 QMs. These are described in further detail in box 1. For each QM, for each facility, the score in 2004 was subtracted from that in 2003. In all cases for these change scores, negative numbers indicate an increase in quality. These change scores were used in the multivariate analyses.

Box 1 Quality measures used in Nursing Home Compare during 2004

Quality measures

Long‐stay residents

• Percentage of residents whose need for help with daily activities has increased

• Percentage of residents with moderate to severe pain*

• Percentage of low‐risk residents who have pressure sores†

• Percentage of residents who are physically restrained

• Percentage of residents who are more depressed or anxious

• Percentage of low‐risk residents who lose control over their bowels or bladder

• Percentage of residents who have/had a catheter inserted and left in their bladder

• Percentage of residents who spend most of their time in bed or in a chair

• Percentage of residents whose ability to move about in and around their room has become worse

• Percentage of residents with a urinary tract infection

• Percentage of residents who have lost too much weight

Short‐stay residents

• Percentage of short‐stay residents with delirium*

• Percentage of short‐stay residents with moderate to severe pain

• Percentage of short‐stay residents with pressure sores

Some quality measures are risk adjusted to take into account resident and facility characteristics. Risk adjustments include the adjustments in resident level* and facility level†.

SOURCE: http://www.medicare.gov/NHCompare/Static/Related/DataCollection.asp

The independent variables of interest were competition and excess supply, which came from the OSCAR data. The Herfindahl index was used as a measure of competition. The county was the market area used in this analysis, and the Herfindahl index was calculated by taking each nursing home's squared percentage share of beds in the county and summing for all nursing homes in the county. The resulting scores varied from 0 to 1, with higher values representing less competition. This index was used in several previous nursing home studies (eg, Intrator et al.⁴). The average occupancy rate in the county was used as a measure of excess supply. The occupancy rate represents the number of residents divided by the number of beds (multiplied by 100), giving occupancy values from 0 to 100.

In addition, we included the interaction between occupancy and competition, because markets with both low occupancy and high competition may respond differently from those with just one of these market conditions. We converted the occupancy and competition variables to Z scores, and then multiplied these, to form the interaction score.

In all cases, these independent variables of interest (competition, occupancy and interaction) were recoded as ordinal categorical variables, using the quartile distributions. The coding was performed so that the fourth quartile for competition represented the highest levels of competition and the first quartile for occupancy represented the highest occupancy levels.

When examining quality using the raw rates of indicators, it is a common practice to use risk adjustment (eg, Mukamel et al,⁵ Mor et al⁶). This is because quality indicators may be a function of both actual quality and resident case‐mix and facility characteristics. Risk adjustment controls for resident case‐mix and facility characteristics in the analyses. This may also be the case with the QMs—that is, resident case‐mix and facility characteristics may need to be included in the estimating equations. However, the QMs were either reported with some risk adjustment or were recommended for use with no such adjustment. Nevertheless, in this analysis, we used additional risk adjustment. The rationale behind this is that (1) no gold standard for risk‐adjusted nursing home quality indicators exists⁵; (2) the risk adjustment of the QMs has been described as limited⁷; and (3) the QMs are publicly reported, which necessitates use of simple methodologies. As such, the QMs may still be underadjusted. For our analyses, it thus seemed prudent to risk adjust the QMs further. However, we acknowledge that this is probably a conservative approach, and may overcontrol for the impact of competition and occupancy on the QMs.

Aggregate resident factors used as independent variables for risk adjustment include activities of daily living (ADLs), bladder incontinence, bowel incontinence and mental health problems. The ADL score is a facility score (ranging from 0 to 1) based on six OSCAR questions (difficulty with bathing, dressing, toileting, transferring, feeding or walking). Increasing scores indicate a greater average ADL impairment within the facility. The number of residents with bladder and bowel incontinence was used to calculate proportions. The proportions of residents with mental retardation, dementia and psychiatric diagnoses were used as mental health variables.

We controlled for staffing levels within facilities because increased staffing levels will enable individual staff to increase the time they spend in direct resident care, and in turn will benefit residents.⁸ We included full‐time equivalents per 100 residents of nurse aides, licensed practical nurses and registered nurses, including full‐time, part‐time and temporary staff. We know from other nursing home studies that facility factors (in addition to staffing levels) have a strong impact on quality indicators. Therefore, chain membership, ownership, size and Medicaid census were included as facility level variables.⁹ Members of a chain were coded as 1 (with non‐chains as 0), for‐profit facilities were coded as 1 (with not‐for‐profit facilities as 0), and Medicaid census was operationalised as a proportion.

Market areas are defined using geographical data in the OSCAR system such that facilities within a given county represent a market. Many studies have used the county as a proxy for the nursing home market (eg, Cohen et al).¹⁰ As noted by Banaszak‐Holl et al,¹¹ the county may be a reasonable approximation of the market for nursing home care, given patterns of funding and resident origin. For example, federal block grant funds for long‐term care services are distributed at the county level. Furthermore, Gertler¹² found that 75% of residents in New York facilities had previously lived in the county where the home was located. Similarly, Nyman¹³ found that 80% of residents in Wisconsin facilities chose a nursing home located in the county in which they had resided before entering the home.

Analyses

We first combined the data from both years, using the facility identification number, and then subtracted each 2004 QM score from the 2003 QM score, in order to explore the changes between the 2 years. We present descriptive analyses including the mean QM scores in each year and the percentage of facilities with either increasing or decreasing scores for each of the 14 QMs. We also include the relative change for each measure. These relative change scores are calculated because they more appropriately account for the changes seen in the narrow range of scores for most QMs.

Second, logistic regression models were used to examine the influence of competition and excess supply on the QM scores. In these models, the odds (adjusted) of improving QM scores is estimated relative to the odds of showing worsening or unchanged QM scores, controlling for competition, occupancy, and aggregate resident factors and facility characteristics. Thus, the QM scores in these analyses were recoded as binary variables, with a negative change score (meaning improved quality) coded as 1 and positive or no change scores coded as 0 (meaning worsening or unchanged quality).

Results

Table 1 shows the mean scores for each of the QMs in 2003 and 2004. This also showed the average change in these scores. Eight measures showed an average decrease in scores (ie, better quality) and six measures showed an average increase in scores (ie, worse quality). However, for 13 of the 14 QMs, these changes averaged <1%. For example, the difference between the 2004 and 2003 scores for the percentage of residents who lost control over their bowel or bladder was −0.11% whereas the difference between the 2004 and 2003 scores for the percentage of low‐risk residents who had pressure sores was 0.79%.

Table 1 Means and differences of all facilities on each quality measure in 2003 and 2004.

Quality measure	n	Mean in 2003	Mean in 2004	Average difference (2004 score – 2003 score)	Relative change (improvements in italic), %
1. Percentage of residents whose need for help with daily activities has increased	8446	13.76	13.39	−0.37	2.7
2. Percentage of residents with moderate to severe pain	6105	2.78	2.63	−0.15	5.4
3. Percentage of low‐risk residents who have pressure sores	9999	6.77	7.56	0.79	1.7
4. Percentage of residents who are physically restrained	11 885	6.72	6.34	−0.38	5.7
5. Percentage of residents who are more depressed or anxious	11 901	5.67	5.89	0.22	3.9
6. Percentage of low‐risk residents who have lost control over their bowel or bladder	12 005	4.35	4.24	−0.11	2.5
7. Percentage of residents who have/had a catheter inserted and left in their bladder	12 019	7.83	7.29	−0.54	6.9
8. Percentage of residents who spend most of their time in bed or in a chair	11 942	14.67	14.61	−0.06	0.4
9. Percentage of residents whose ability to move about in and around their room has become worse	10 068	12.06	12.11	0.05	0.4
10. Percentage of residents with a urinary tract infection	11 451	15.25	15.32	0.07	0.5
11. Percentage of residents who have lost too much weight	12 024	8.36	8.59	0.23	2.8
12. Percentage of short‐stay residents with delirium	8596	23.11	23.25	0.14	0.6
13. Percentage of short‐stay residents with moderate to severe pain	8513	3.36	3.02	−0.34	10.1
14. Percentage of short‐stay residents with pressure sores	7731	20.47	19.15	−1.32	6.5

The total number of facilities was 14 673 in 2003 and 14 724 in 2004. In all, 14 554 facilities were identified as operating in both 2003 and 2004; however, for each of the 14 quality measures, we include only those facilities with scores available in both years for calculating the difference. This is the n given in the first column of the figures.

The percentage of facilities with decreasing QM scores (ie, better quality) are, for the most part, greater than those with an increase in scores (ie, worse quality). This is shown in table 2. For example, 38% of facilities had decreasing scores for the percentage of residents with moderate to severe pain, whereas 35% of facilities had increasing scores. Indeed, facilities decreasing their QM scores did so by approximately 5%, on average.

Table 2 Facilities with increasing and decreasing scores between 2003 and 2004 on each quality measure.

Quality measure	n	Facilities with increasing score n (%)	Average Increase (worse quality) Mean (SD)	Relative increase	Facilities with decreasing score n (%)	Average decrease (better quality) Mean (SD)	Relative decrease	No change n (%)
1. Percentage of residents whose need for help with daily activities has increased	8446	3740 (44.3)	5.65 (4.29)	41.1%	4161 (49.3)	−5.99 (4.57)	43.5%	545 (6.4)
2. Percentage of residents with moderate to severe pain	6105	2158 (35.4)	3.21 (2.19)	115.5%	2346 (38.4)	−3.29 (2.29)	118.3%	1601 (26.2)
3. Percentage of low‐risk residents who have pressure sores	9999	5048 (50.5)	7.67 (5.89)	16.4%	4423 (44.2)	−7.02 (5.34)	15.0%	528 (5.3)
4. Percentage of residents who are physically restrained	11 885	4842 (40.7)	3.95 (3.55)	58.8%	5398 (45.4)	−4.36 (4.30)	64.9%	1645 (13.9)
5. Percentage of residents who are more depressed or anxious	11 901	5199 (43.7)	3.04 (2.24)	53.6%	4720 (39.7)	−2.79 (2.02)	49.2%	1982 (16.6)
6. Percentage of low‐risk residents who have lost control over their bowel or bladder	12 005	4263 (35.5)	3.31 (2.90)	76.1%	4642 (38.7)	−3.28 (3.01)	75.4%	3100 (25.8)
7. Percentage of residents who have/had a catheter inserted and left in their bladder	12 019	4265 (35.5)	4.32 (3.82)	55.2%	5110 (42.5)	−4.79 (4.48)	61.2%	2644 (22.0)
8. Percentage of residents who spend most of their time in bed or in a chair	11 942	5332 (44.6)	7.08 (6.52)	48.3%	5878 (49.2)	−6.58 (5.44)	44.9%	737 (6.2)
9. Percentage of residents whose ability to move about in and around their room has become worse	10 068	4698 (46.6)	6.02 (5.18)	49.9%	4689 (46.6)	−5.93 (4.96)	49.2%	681 (6.8)
10. Percentage of residents with a urinary tract infection	11 451	5349 (46.7)	7.18 (6.20)	47.1%	5442 (47.5)	−6.88 (5.73)	45.1%	660 (5.8)
11. Percentage of residents who have lost too much weight	12 024	5639 (46.9)	4.58 (3.52)	54.8%	5166 (43.0)	−4.42 (3.45)	52.9%	1219 (10.1)
12. Percentage of short‐stay residents with delirium	8596	4249 (49.4)	9.90 (7.98)	42.8%	3999 (46.5)	−9.74 (8.13)	42.1%	348 (4.1)
13. Percentage of short‐stay residents with moderate to severe pain	8513	2803 (32.9)	3.82 (3.86)	113.7%	3251 (38.2)	−4.16 (3.88)	123.8%	2459 (28.9)
14. Percentage of short‐stay residents with pressure sores	7731	3313 (42.9)	7.50 (5.94)	36.6%	4078 (52.7)	−8.28 (6.26)	40.5%	340 (4.4)
Overall change in quality measures	12 974	6164 (47.5)	2.22 (2.08)	52%	6602 (50.9)	−2.25 (2.10)	59%	208 (1.6)

The total number of facilities was 14 673 in 2003 and 14 724 in 2004. In all, 14 554 facilities were identified as operating in both 2003 and 2004; however, for each of the 14 quality measures, we include only those facilities with scores available in both years for calculating the difference. This is the n given in the first column of figures.

Table 3 shows the results from the 14 logistic regression models used to examine the influence of competition and occupancy on the QM scores. The results show the high‐competition quartile compared with the low‐competition quartile; the high‐occupancy quartile compared with the low‐occupancy quartile; and, the high interaction scores (representing high competition×low occupancy) quartile compared with the low interaction scores quartiles. The competition quartiles are at values of 0.04, 0.13 and 0.31, and the occupancy quartiles are at values of 77, 89 and 95.

Table 3 Multivariate logistic model results examining the influence of competition and occupancy on quality measure scores.

Quality measures	n	Competition*1	Occupancy*2	Competition × occupancy interaction*3
		AOR (95% CI)	AOR (95% CI)	AOR (95% CI)
1. Percentage of residents whose need for help with daily activities has increased	8446	1.18 (1.03 to 1.27*)	0.85 (0.64 to 0.96**)	0.92 (0.76 to 1.05)
2. Percentage of residents with moderate to severe pain	6105	1.10 (0.98 to 1.32)	0.97 (0.61 to 1.04)	0.99 (0.67 to 1.10)
3. Percentage of low‐risk residents who have pressure sores	9999	1.14 (1.01 to 1.26*)	0.86 (0.70 to 0.97*)	0.88 (0.71 to 0.97*)
4. Percentage of residents who are physically restrained	11 885	0.81 (0.76 to 1.03)	1.11 (0.90 to 1.32)	0.91 (0.86 to 1.12)
5. Percentage of residents who are more depressed or anxious	11 901	0.95 (0.80 to 1.02)	0.96 (0.80 to 1.12)	0.97 (0.82 to 1.15)
6. Percentage of low‐risk residents who have lost control over their bowels or bladder	12 005	0.92 (0.81 to 1.08)	0.98 (0.81 to 1.22)	0.93 (0.78 to 1.19)
7. Percentage of residents who have/had a catheter inserted and left in their bladder	12 019	1.07 (0.89 to 1.10)	0.89 (0.77 to 1.00*)	0.90 (0.79 to 0.98*)
8. Percentage of residents who spend most of their time in bed or in a chair	11 942	0.93 (0.81 to 1.05)	0.90 (0.72 to 1.15)	0.95 (0.77 to 1.12)
9. Percentage of residents whose ability to move about in and around their room has become worse	10 068	0.96 (0.86 to 1.11)	0.93 (0.88 to 0.99*)	0.95 (0.82 to 0.99*)
10. Percentage of residents with a urinary tract infection	11 451	0.95 (0.86 to 1.05)	1.03 (0.93 to 1.12)	0.93 (0.86 to 1.10)
11. Percentage of residents who have lost too much weight	12 024	0.87 (0.79 to 1.08)	0.91 (0.88 to 1.13)	0.97 (0.89 to 1.10)
12. Percentage of short‐stay residents with delirium	8596	1.25 (1.04 to 1.29*)	0.85 (0.69 to 0.99*)	0.88 (0.70 to 0.97*)
13. Percentage of short‐stay residents with moderate to severe pain	8513	1.21 (1.07 to 1.33*)	0.73 (0.61 to 0.95*)	0.75 (0.68 to 0.98*)
14. Percentage of short‐stay residents with pressure sores	7731	1.15 (1.03 to 1.27*)	0.84 (0.78 to 0.97*)	0.99 (0.70 to 1.17)
Overall quality measures' difference	12 974	1.12 (1.03 to 1.16*)	0.89 (0.76 to 0.98*)	0.93 (0.79 to 0.97*)

AOR, adjusted odds ratio.

The total number of facilities is 14 554. However, for each of the 14 quality measures, we include only those facilities with scores available for both years for calculating the difference. In all cases, variables included in the estimating equations include aggregate resident factors (ie, dementia, activities of daily living) and facility factors (ie, ownership, chain membership, bed size, staffing levels).

*AOR results reported for the highest quartile compared with the lowest quartile; (1) The analyses are coded such that AOR >1 indicates that greater competition is associated with improved QM scores. (2) The analyses are coded such that AOR >1 indicate that high occupancy is associated with improved QM scores. (3) The analyses are coded such that AOR >1 indicates that lower competition and high occupancy are associated with improved QM scores.

In markets with high competition, five QMs showed significantly higher adjusted odds ratios (AORs), indicating an association between high competition and improving QM scores. Some of these AORs identified were quite high. For example, facilities in highly competitive markets (compared with those in low‐competitive markets) were 21% more likely to improve their QM scores for the percentage of short‐stay residents who had moderate to severe pain.

In markets with lower occupancy rates, seven QMs showed significantly lower AORs, indicating an association between low occupancy and improving QM scores. Again, some of these AORs identified were quite high. For example, facilities in low‐occupancy markets (compared with those in high‐occupancy markets) are 27% more likely to improve their QM scores for the percentage of short‐stay residents who had moderate to severe pain.

In markets with low occupancy rates and high competition, five QMs showed significantly lower AORs, indicating an association between occupancy × competition (ie, interaction) and improving QM scores. Following the previous findings, some of these AORs identified were quite high. For example, facilities in low‐occupancy high‐competition markets (compared with those in high‐occupancy low‐competition markets) were 25% more likely to improve their QM scores for the percentage of short‐stay residents who had moderate to severe pain.

Discussion

Nursing home regulations probably serve a necessary function in helping promote quality in nursing homes. However, the regulatory system set up to promote quality has many well‐publicised faults,¹⁴ and the regulatory approach can facilitate the creation of a quality floor rather than promoting quality improvement. Nevertheless, quality improvement is badly needed in many facilities, as quality problems in many nursing homes are chronic issues. It is clear that, for at least some providers, the current regulatory system may be a necessary but not sufficient mechanism for quality improvement. Report cards (such as NHC) are a relatively new market‐based supplement to the regulatory model. However, we know very little about nursing home report cards and the impact they have on consumers and providers.

With the exception of a recent General Accounting Office report² describing the federal NHC initiative and four recent descriptive articles,¹⁵,¹⁶,¹⁷,¹⁸ we could not identify any research on nursing home report cards. This descriptive literature would suggest that nursing home administrators are aware of NHC, and that a majority of nursing homes are implementing quality improvement efforts as a direct response to NHC.¹⁵ What has received less attention in the research literature, so far, is whether nursing homes are making improvements in QMs, and whether factors such as competition and occupancy rates are associated with any improvements.

The only information we could identify addressing these issues comes from the Centers for Medicare and Medicaid Services (CMS). (CMS is the Federal agency that oversees nursing home inspections and the NHC website.) CMS maintains that progress has been made in improving the quality of nursing homes in that some QM scores have declined over time.¹⁹ However, the details provided by CMS are incomplete, and it is not clear whether the results reported are representative of all nursing homes.

In this investigation, our results are mixed. On the one hand, we clearly found that many nursing homes have improved their scores on the NHC QMs. For example, 44% of facilities were able to reduce the aggregate incidence of pressure sores by about 7%, but, because the initial average pressure sore rate in these facilities was 16%, this represents a large relative improvement of about 40%. Thus, some of these changes were considerable. On the other hand, we also found a considerable number of facilities increasing their QM scores (ie, getting worse). Thus, overall, the changes seen in NHC QMs were positive but relatively small.

The aggregate mechanism underlying the potential outcomes expected when report cards are used is described in the consumer choice model. In the consumer choice model, consumers are given relevant information, so that they can make informed choices.²⁰ Most often, this information consists of cost and quality. In turn, the market should respond by competing on both cost and quality, with probable decreases in costs and/or increases in quality. However, it would seem reasonable to assume that the results of such market‐based initiatives depend on the markets in which they operate (in this case, more effects should be identified in the most hostile markets)—hence our hypothesis that nursing homes in markets with more competition or excess supply will respond to NHC by improving their quality. Our results clearly show that this is indeed the case. Competition, occupancy and the interaction between the two would seem to promote improvements in QM scores.

Moreover, the pattern of our findings is also of interest. We found consistently significant findings for the short‐stay QMs. This is of interest because short‐stay residents are most often paid for by Medicare. (Medicare most often pays for rehabilitation, whereas Medicaid pays for longer term care.) In general, Medicare rates are higher, and thus more lucrative, than Medicaid rates, which primarily pay for long‐stay residents. Thus, nursing homes tend to compete for these Medicare residents. Our finding that nursing homes in high‐competition and low‐occupancy areas are improving their short‐stay QM scores would thus also be expected as a market reaction. Moreover, facilities may be most able to make gains in these areas because of the high resident turnover. Facilities can implement quality improvement initiatives, see whether they work, and refine the process on the next cohort of short‐stay residents.

Changes in the nursing home market over the past 10 years have included increasing competition from providers, such as assisted living and lower occupancy rates.²¹ Moreover, these trends have been actively fostered by both state and federal governments, eager to reduce their nursing home bills. For example, in 1981, the Home and Community‐based Services 1915c Waiver was passed by Congress, in order for “States to have the flexibility to develop and implement creative alternatives to placing Medicaid‐eligible individuals in hospitals, nursing facilities or intermediate care facilities” (www.hcfa.org/medicaid/hpg4). Although this waiver was originally designed for those with mental retardation, newer state initiatives have incorporated language into waiver proposals, in order to steer older people away from nursing homes. Moreover, more generic “ageing‐in‐place” initiatives also promote the use of home services or services such as residential care, and attempt to delay the use of nursing homes.²²

There are about 600 000 beds in assisted living facilities, most of which are occupied. The number of assisted living facilities is around 12 000 nationwide, and they generate about US$15 billion per year in revenue.²³ The number of facilities has grown rapidly over the past 10 years, with more than half of the facilities in business for <10 years, and one‐third in business for <5 years. Assisted living tends to cater to private‐pay residents, which only serves to heighten the competitive pressures nursing homes face for attracting other residents. That is, if assisted living facilities attract private‐pay residents from nursing homes, then nursing homes are likely to compete for the remaining private‐pay residents and Medicare residents. Assisted living is almost certainly partly responsible for the drop in occupancy rates seen in nursing homes.

These market pressures (Home and Community‐based Services, ageing‐in‐place and assisted living) will probably continue to expand. As they do so, our results would suggest that we will probably see further improvements in the QM scores on NHC.

Limitations

We know that nursing home quality is a multidimensional construct, with a wide array of available quality measures. NHC includes only a small subset of these measures; thus nursing homes could have increasing or decreasing quality in many other areas, and this would not be reflected on the website.

In table 2, we show that the matching success rate between 2003 QMs and the same QMs in 2004 varies from 6105 to 12 024. These differences occur because rates are needed for both years to calculate the difference scores. NHC does not report values for facilities with low observed numbers of each QM (ie, <30 cases for the long‐stay QMs and <20 cases for the short‐stay QMs). This might have caused many missing observations in NHC and accounts for our matching rate. In addition, some facilities could have closed during the period of observation. However, recent research on the closure of nursing homes suggests that only 0.7% of facilities close each year.²⁴ Thus, this probably represents a minor impact on our match rate.

Additionally, the changes we see in the NHC QMs may not necessarily be attributable to the publishing of the report card. Many facilities could have changed their scores, irrespective of NHC. Many other initiatives to improve quality exist, such as those developed by Quality Improvement Organizations and by facilities themselves. These may also influence the QMs.

With approximately as many facilities with increasing QM scores as there are facilities with decreasing QM scores, it is possible that we are simply seeing regression to the mean in the descriptive analyses. However, in the multivariate analysis, our significant findings would seem to indicate that such random variation is not responsible for all of the changes seen.

The OSCAR data used may also have some limitations. For example, a small number of facilities (n = 640) had occupancy rates of <30%. These may represent errors in the data, or recently opened facilities. Therefore, they were removed from our analyses. However, this approach of determining whether low‐occupancy rates represent errors in the data, or not, represents a best guess on our part. Nevertheless, the number of facilities involved is small and the reported results are unaffected by these exclusions.

In our estimation approach, we use logistic regression. In doing so, we created a dichotomous dependent variable. The advantage of this approach is that it provides readily interpretable odds ratios (ORs). The disadvantage of this approach is that information is lost in the transformation of the dependent variable. Nevertheless, in preliminary analyses with least squares regression, similar significant findings were obtained—thus, our choice to report OR. Our independent variables of interest (competition, occupancy and interaction) were specified as ordinal categorical variables. This approach enables an easier interpretation of the OR. However, we also acknowledge that the results presented are for the highest and lowest quartiles, which maximises the comparisons, and are the most significant findings.

There are a number of alternative approaches to defining market areas; however, these are mostly based on migration data, which are not available a priori. An example is Makuc's market definitions of hospital service areas. This previous work has defined healthcare service areas on the basis of Medicare data on travel patterns between counties for routine hospital care.²⁵ Also, other studies have used variable‐radius measures of local competition to define markets.²⁶ Similar approaches may be especially pertinent to our study, given the recent research by Zwanziger et al²⁷ identifying some measurement error in nursing home studies using counties as market areas. However, the fact remains that national data on patient flows for nursing homes do not yet exist.

Despite the fact that the Herfindahl index is the most commonly used measure of competition in nursing home research, it may prove instructive to examine other indexes of competition. Schramm and Renn²⁸ describe that the Herfindahl index was developed for industrial and retail settings, and thus may not be as sound for healthcare facilities.

We are sensitive to these risk‐adjustment issues inherent to NHC. The QMs were initially created for use with various resident and facility risk adjustments. This approach was criticised, and was changed slightly to include only some risk adjustment for some QMs.² For our analyses, it is therefore questionable whether risk adjustment should be used in the estimating equations. Preliminary analyses using models with no risk adjustment, resident adjustment and facility adjustment did influence the AORs, but did not greatly influence the significance levels (results not shown). Because we cannot ignore the possibility that our results depend on the risk‐adjustment method used, we report analyses using risk‐adjustment models that include resident and facility characteristics, which provide the most conservative effects for the variables of interest.

Finally, nursing homes may also be simply getting better at completing the minimum data set (MDS) that is used to construct the NHC QMs. So reporting may have changed; but, actual levels of quality may not have changed. Several authors have noted that the MDS is generally reliable,²⁹ and others have used these data for research and quality improvement purposes.³⁰,³¹ Nevertheless, it has also been reported that these data may have reliability issues.²⁹,³⁰ This potential bias is clearly important in examining NHC QMs. Still, in this investigation, it is unclear how this bias, if it exists, would have influenced our results. This bias would tend to reduce the change scores observed, and may be more pronounced for some QMs and not others. But, if this bias in the MDS is random, and not associated with either competition or occupancy rates, then our regression results would probably not be affected. However, the fact remains that we know little about bias coming from this data source.

Key points

• Nursing homes are somewhat responsive to public reporting of quality indicators—that is, over time, they seek to improve their quality scores. However, this does vary, based on the market in which the nursing homes operate.

Conclusion

The NHC report card was introduced with some trepidation by the nursing home industry. However, some recent research has indicated acceptance by nursing home administrators, to the degree that a majority thought they would use the information for quality improvement purposes.¹⁷ We found that this has resulted in improvement in quality in some nursing homes, albeit small. However, as could be predicted on the basis of the market‐driven mechanism underlying quality improvements using report cards, we showed that improvements in the QM scores are more likely in the most competitive markets and in those with the lowest average occupancy rates.

Abbreviations

ADL - activities of daily living

AOR - adjusted odds ratio

CMS - Centers for Medicare and Medicaid Services

NHC - Nursing Home Compare

OSCAR - On‐line Survey Certification And Recording

QM - quality measure