Use Of Nursing Home Compare Website Appears Limited By Lack Of Awareness And Initial Mistrust Of The Data

Abstract

In December 2008 the Centers for Medicare and Medicaid Services (CMS) launched a five-star rating system of nursing homes as part of Nursing Home Compare, a web-based report card detailing quality of care at all CMS-certified nursing homes. Questions remain, however, as to how well consumers use this rating system as well as other sources of information in choosing nursing home placement. We used a qualitative assessment of how consumers select nursing homes and of the role of information about quality, using semistructured interviews of people who recently placed a family member or friend in a nursing home. We found that consumers were receptive to using Internet-based information about quality as one source of information but that choice was limited by the need for specialized services, proximity to family or health care providers, and availability of Medicaid beds. Consumers had a positive reaction when shown Nursing Home Compare; however, its use appeared to be limited by lack of awareness and, to some extent, initial lack of trust of the data. Our findings suggest that efforts to expand the use of Nursing Home Compare should focus on awareness and trust. Useful additions to Nursing Home Compare might include measures of the availability of activities, information about cost, and consumer satisfaction.

It has long been argued that consumers have limited ability to judge the quality of health care and to distinguish among providers of different quality levels.¹ In response, public reporting of health care quality has become a prominent part of the quality improvement landscape. In the nursing home setting, the evidence that consumers use publicly reported information to choose higher-quality providers is mixed.^2,3 Consumers’ responses also varied by socioeconomic status.^3,4

Nursing homes are funded largely by public payers, with approximately two-thirds of residents in a typical facility on Medicaid. Prior research indicates that people of low socioeconomic status are more likely than others to be admitted to the worst nursing homes,⁵ which have lower nurse staffing ratios, have more serious regulatory deficiencies, and are more likely to be terminated from the Medicaid program compared to higher-rated nursing homes. These disparities have been directly linked to payment source, with high-Medicaid population nursing homes being of lower quality.^6-8

Why consumers (or their agents, such as discharge planners or family members) do not respond more strongly and consistently to reported information about quality and avoid low-quality providers is still a matter of speculation. Response could break down in multiple ways: Consumers may not know about the public reports, have access to them, understand them, or trust them; they may not value the information relative to other considerations; or they may face obstacles to accessing the high-quality providers they would prefer. However, most studies of consumer response to health care report cards have been purely quantitative, with no ability to incorporate consumer-reported experience into interpretation of results or explore what information consumers use when selecting a nursing home. This limits the ability to distinguish the most effective ways in which public reporting could be improved.

More generally, little is known about how consumers choose nursing homes and the role of information about quality in that choice. One quantitative study found that before nursing home quality was publicly reported, consumers chose nursing homes based on shorter distance from home, nonprofit status (potentially as a signal for higher quality), and nonclinical amenities, but not on clinical quality.⁹ Two key prior studies used qualitative methods to explore nursing home choice. Nicholas Castle interviewed nursing home residents and family members in two states about their decision-making process.¹⁰ Lisa Shugarman and Julie Brown conducted focus groups in four states with prior short-stay nursing home residents, family members, hospital discharge planners, and community-based case managers.¹¹ These studies found that residents themselves played little role in the decision; that respondents had little or no access to the Internet and did not use it to help with selection of nursing home; and that proximity to home was important, while clinical quality was not a prime consideration.

These studies may have limited applicability to current nursing home decisions because of their age. The Castle interviews were conducted in the late 1990s, and the Shugarman and Brown focus groups were conducted in the early 2000s. Several key contextual factors have changed since then. First, use of the Internet has become much more commonplace. Second, the main vehicle for public reporting of nursing home quality (Nursing Home Compare) was greatly simplified in 2008, with the expectation that it would be easier for consumers to use. Finally, policy makers have increasingly focused attention on the need to improve health care quality in consumer-centered ways across the health care spectrum, such that quality may be a more important consideration in consumers’ decisions than it used to be.

In this study we used qualitative methods to examine consumer choice of nursing homes directly, updating prior studies. Our specific goals were to assess how consumers chose a nursing home; how they defined nursing home quality; the role of quality in deciding on a nursing home; obstacles faced in choosing a high-quality home, especially the role of Medicaid; and to what extent consumers used the Internet, and specifically Nursing Home Compare, in making a decision. To meet these goals, we conducted in-depth, semistructured interviews of sixty-three individuals involved in a recent decision to admit a family member or friend to a nursing home in the greater Chicago metropolitan area, including both urban and suburban areas. The overarching goal was to assess the role of quality information in consumers’ choice of nursing homes for long-term care and to use that information to identify potential improvements to Nursing Home Compare to enhance consumers’ decision making.

The Nursing Home Setting

Broadly, nursing homes serve two populations: short-stay (postacute) and long-stay (chronic care) residents. Long-stay residents often spend the remainder of their lives in a nursing home, receiving nonskilled care for functional or cognitive impairment. Medicaid is the dominant payer for nonaffluent long-stay residents, although many pay out of pocket initially. Most nursing homes are also certified by Medicare to provide short-term rehabilitative care to individuals following an acute care hospital episode. However, in many cases, a postacute patient becomes a long-stay resident if his or her health deteriorates. In this study our focus was mainly on long-stay nursing home residents, but we included those whose long-term stay began with a short-term stay.

Nursing Home Compare

In December 2008 the Centers for Medicare and Medicaid Services (CMS) upgraded its Nursing Home Compare system by launching a five-star rating system of nursing homes, which is a web-based report card detailing quality of care at all CMS-certified nursing homes. Nursing Home Compare had existed for more than a decade but included numerous individual measures of staffing, regulatory compliance, and clinical outcomes without an easy way for consumers to aggregate the information. In contrast to the prior system, the star rating system was intended to provide highly simplified information in a form that consumers would find familiar. The stars are based on three domains of quality, jointly across short-stay and long-stay services—staffing ratios, clinical outcomes, and results of regulatory inspections—with the overall rating combining the three domains. A primary goal of the system is to enable consumers to select high-quality providers.

Study Data And Methods

INTERVIEWS

We conducted in-depth, semistructured interviews with people who had placed a family member or a friend in a nursing home to assess their process for choosing a nursing home. Respondents were recruited from the greater Chicago metropolitan area (both urban and suburban areas) through flyers placed in nursing homes, geriatrics clinics, and senior centers and through online advertisements. Potential interviewees were screened by telephone for eligibility (having been involved in placing a family member or friend in a nursing home in the prior six months, able to complete an in-person interview, and variation in location and Medicaid status). Our recruitment materials targeted people who had helped place a family member or friend in a nursing home “for long-term care,” which intentionally skewed our sample toward respondents associated with long-stay residents. Potential long-stay residents were our focus because we specifically wanted to explore obstacles to choosing a high-quality provider, such as distance and Medicaid bed availability—issues that are most salient to the long-stay population. We nonetheless included respondents who said that the admission in question was initially intended to be short stay, because it is often difficult to know length-of-stay at the time of admission and because many expected short stays turn into long stays. In addition, Nursing Home Compare does not differentiate between quality for short- and long-term stays; it provides common star ratings across the two.

Respondents were asked, but were not required, to identify the nursing home in which the resident was placed. In cases in which the nursing home was identified, we recorded the Nursing Home Compare star rating. We stopped after sixty-three interviews because no new, relevant information seemed to be emerging from additional interviews—that is, we had reached “theoretical saturation” in the terminology of qualitative methods.^12,13

A research assistant conducted interviews in semiprivate places, such as a quiet corner of a restaurant. All interviews were conducted between June and September 2014. Interviewees provided oral consent and were offered a $75 gift card in return for an approximately one-hour interview. The research assistant was trained by the authors to use a semistructured interview protocol that listed basic questions for all participants and optional follow-up questions to use to probe further when particular issues were raised.

First, participants were asked to describe their decision-making process about nursing home placement. After being asked about the role of quality information, they were asked about Nursing Home Compare. If they were unfamiliar with Nursing Home Compare, interviewees were shown screenshots of the Nursing Home Compare website and were asked for their impressions.

All interviews were audiorecorded and transcribed. Based on reading of the transcripts, the authors developed a list of nodes to frame the data coding, which was performed by a research assistant. To accommodate new or unanticipated concepts, additional nodes were identified during the coding process and through checks by the authors for consistency and accuracy.¹⁴ Minor discrepancies were resolved through discussion among the authors and the research assistant until agreement was reached. Analysis of the nodes resulted in the emergence of themes and conclusions agreed upon by the investigators.We used NVivo version 10 software to facilitate coding and analysis.

LIMITATIONS

We acknowledge that there were several limitations with our study. First, our findings were based on a voluntary convenience sample in the Chicago metropolitan area and may not be generalizable to rural areas or to consumers nationwide. For example, different state policies could affect the search process. Illinois does not have a state nursing home report card, which in some states may provide an alternative source for quality information, but does have certificate-of-need regulation that may constrain bed availability in some areas. However, many of our questions were not obviously location specific. Also, the convenience sample might not be representative of caregivers and nursing home residents in the United States. For example, most nursing home residents are female,¹⁶ while we interviewed more respondents involved in placing a male in a nursing home. However, we did not see important differences in our results by gender.

Second, we interviewed respondents not at the time that the decision was made but up to six months later, so respondents’ opinions may incorporate modifications as a result of experience with the nursing home in the interim. Finally, our sample was intentionally skewed toward long-stay placements (including those that started with a rehabilitation stay), and results might not generalize to those seeking short stays strictly for postacute rehabilitation.

Study Results

We conducted sixty-three interviews with family members or friends of patients recently admitted to a nursing home. The majority (57 percent) of the interviewees were adult children of the nursing home resident. Of the sixty-three recently admitted nursing home residents whose placement the respondents described, there was a distribution across demographics, insurance type, location, and intended length-of-stay. The nursing homes they were admitted to spanned the range of Nursing Home Compare ratings. Exhibit 1 presents characteristics of the nursing home residents who were the subjects of our interviews.

EXHIbIT 1.

Characteristics of a convenience sample of recently admitted nursing home residents, June–September 2014

Characteristic	Mean	Standard deviation
Mean age (years)	76	9
	Number of cases (N = 63)	Percent
SEX
Male	45	71
Female	18	29
RACE
White	41	65
Black	18	28
Hispanic	3	5
Asian	1	1
INSURANCE TYPE FOR initial AdMISSION
Medicare	33	54
Medicaid	27	42
Veterans Affairs	3	5
INITIAL ADMISSION INTENDED TO BE:
Long term	55	87
Short term (postacute)	8	13
NURSING HOME COMPARE OVERALL STAR RATING
1	3	5
2	16	25
3	8	12
4	11	17
5	8	12
Unknown	17	27
LOCATION
Urban	37	58
Suburban	26	42
RELATIONSHIP OF INTERVIEWEE TO RESIDENT
Adult child	36	57
Grandchild	8	13
Nephew	6	9
Spouse	3	5
Niece	3	5
Friend	3	5
Sibling	2	3
Son-in-law	1	1
Daughter-in-law	1	1

SOURCE Authors’ calculations.

We summarized our analysis into five key themes: the process of choosing a nursing home, defining quality, competing considerations, the role of Medicaid, and awareness and trust of Nursing Home Compare. We describe each of these in more detail, with supporting representative quotations from the interviews in Exhibit 2.

EXHIBIT 2.

Representative quotations from interview respondents, June–September 2014

Theme/subtheme	Quotation
THEME 1: CHOOSING A NURSING HOME
Recommendations from health care providers	“The hospital suggested it. ‘Cause the doctor is affiliated with that nursing home.”
	“The doctor pretty much, uh, recommended that that would be one that would be…suitable for him, one they’ve used in the past.”
Use of Internet	“I did a little research, just on Google. I even did reviews on Yelp, to see what other people think, and then reviews all over.”
Visiting the nursing home	“[Consumers should] go more than one time, because if they know you [are] making an appointment to come in, they gonna be on their best behavior. You want to do a surprise visit.”
	“Without seeing the place—and I mean, like I had told you before, really seeing the place—you’re not gonna know.”
THEME 2: DEFINING QUALITY
Cleanliness	“The first thing I looked for, does it smell like urine?”
	“Cleanliness I would tie into the, uh, competency of the staff.‘Cause, you know, if they’re not taking care of the place, what else aren’t they doing?”
Staffing	“They don’t have enough CNAs [certified nursing assistants]. They need one or two more, because they putting all this work on one girl.”
	”Very attentive and, and they seem to develop like a friendship with the patients as opposed to just ‘Hey, I gotta do this to this patient.’”
	“Basically whether or not the staff is at the very least content to be there. Because in some nursing homes, the staff is just not motivated by their management or not motivated by their surroundings.”
Activities	“[Important that they] have a level of interaction that is self-fulfilling and enriching. And respect.”
	“You might not think that’s important, but to let a person lay there and die, it’s not humane. You gotta make sure they have activities.”
THEME 3: COMPETING CONSIDERATIONS
Specialized services	“Due to the situation and the equipment that he needed, the social worker picked [the nursing home].”
Proximity	“[Distance was a factor because] obviously I wasn’t going to put him 50 miles away.
	“It’s a pretty badly run place. But it was at least in the neighborhood so that we could get there and keep an eye on him.”
Bed availability	“There were some places that had a waiting list, but they were nicer.”
	“Most of the times…you can’t get a room by yourself as a new patient, if they even have a bed at all for you at some of these places. You sometimes have to wait a while.”
THEME 4: ROLE OF MEDICAID
Different treatment	“Some nursing homes right out and told me, ‘No, you know, we don’take Medicaid. Sorry to waste your time.’”
	“Once they hear Medicaid, you know, they give you a cold shoulder.”
Delays	“We ran into a brick wall, because right now his, his status with Medicaid is pending. And until it becomes official, we had a lot of places say, until it’s locked in, you know, we can’t accept it.”
THEME 5: AWARENESS AND TRUST OF NURSING HOME COMPARE
Usefulness	“I think this is fantastic. This saves a lot of time. You know, I could just look right on here, ‘cause I know if it has a certain rating I’m not gonna want it.”
	“The hospital should tell you, “Here, there’s a website you should go on, and there’s the choice.”
Vehicle for communication	“It definitely would have been something I could have brought with me and said, ‘Oh, can you explain to me about this?’”
	“I would go to the administrator and ‘I need you to see this’. …You know? ‘Because if you’re below average, you need to come up a little bit. You need to step up your game.’”
Trust	“Anybody can say anything on the Internet.”
	“I’m sure it’s very tempting for the people to change things that don’t quite put in the best light.”
	“I would be suspect on these, because, first of all, I would like to know if they accept advertising.”
	“But if it’s a company from Medicare, it’s different. I would trust them more.”
Improvements	“You know what else this should show? What activities they offer. You know, in a day, do they take the residents, you know, or different games? What kinda arts and crafts?”
	“[I would like to know] the efficiency of it. Like even though…it might be a facility that costs more, or, you know, some kinda rating where they would show what I was getting, they actually giving me what I’m paying for.”

SOURCE Authors’ analysis.

PROCESS OF CHOOSING A NURSING HOME

The process of gathering information to make a choice of nursing home was described similarly by many respondents, starting with recommendations from a physician or social worker, gathering opinions from friends and relatives, researching information online, and then visiting one or more nursing homes before deciding. Recommendations from the doctor or hospital seemed to be weighted heavily.

Many respondents said that they “Googled” places online and used the Internet to obtain information about nursing homes. Very few respondents said that they did not have access to the Internet. However, most could not recall the particular websites used.

Visiting the nursing home prior to a decision was particularly important, even after gathering information from other sources. Respondents paid close attention to cleanliness, smells, and staff friendliness and demeanor.

DEFINING QUALITY

The interviews revealed fairly consistent perceptions of what a high-quality nursing home should look like and how one might judge quality. These perceptions focused on cleanliness, availability of nursing and medical staff, and availability of activities.

▸ CLEANLINESS:

For many respondents, a clear signal of nursing home quality is the smell. Smell and general cleanliness are viewed as indicators for other conditions in the nursing home, such as clinical competence.

▸ STAFFING:

Respondents overwhelmingly pointed to the facility’s having adequate staffing levels as an important condition for high quality. However, respondents also expressed preferences about more nuanced aspects of staffing, such as whether staff were caring, friendly, and experienced. One even mentioned a connection between high quality and whether staff were satisfied with their jobs.

▸ ACTIVITIES:

Many respondents highlighted the importance of meaningful and entertaining activities for residents, such as exercise classes, trips to lakes or restaurants, bingo, conversation areas, or a television room. Conversely, a lack of meaningful activities was seen as an important signal of poor quality.

Notably, there was almost no discussion of clinical quality outcomes such as pain, pressure sores, infections, restraints, and falls—areas that constitute one of the star domains on Nursing Home Compare. Even the few respondents who reported using Nursing Home Compare did not appear to look at that domain, and respondents generally did not raise this issue when discussing what they were looking for in a nursing home.

COMPETING CONSIDERATIONS

For many respondents, the potential role of quality in making a choice was limited by other practical considerations, including the need for specialized services, distance to family or other health care providers, or bed availability.

SPECIALIZED SERVICES:

For some respondents, the need for condition-specific services (such as respiratory care), apparently not available at most nursing homes, trumped any consideration of overall quality.

▸ PROXIMITY:

Proximity of the nursing home to family was a primary consideration so that family members could visit and monitor care. A typical comment was that some nursing homes could not be considered because they were not close enough. Health care providers appear to facilitate the desire for proximity by recommending facilities by distance. It was also clear that proximity often overshadowed considerations of quality.

▸ BED AVAILABILITY:

When asked whether bed availability was a problem, several respondents indicated that many nursing homes have waiting lists. Some nursing homes have a limited number of postacute beds and few Medicaid beds. Thus, consumers may face a trade-off between finding the most appealing nursing home and finding a faster placement. The lack of available private rooms was also mentioned by several respondents.

After narrowing down options by the availability of needed services, geographic area, bed availability, and payment source accepted, most respondents indicated that they had choices when selecting nursing homes—that is, they could choose from at least two. This was true of people who ended up in either highly rated or poorly rated homes. However, contingent on the above factors, the choice set did not always include nursing homes on the high end of the quality range.

THE ROLE OF MEDICAID

Reliance on Medicaid payment appeared to exacerbate the challenges associated with both bed availability and proximity as considerations that compete with quality. Many respondents noted either that nursing homes did not accept Medicaid or that they treated potential residents who would be Medicaid recipients differently, such as accepting the application but with less enthusiasm. Some respondents found that even in facilities that accepted Medicaid, the wait time might be much longer for a Medicaid bed. When one applies to nursing homes that readily accept Medicaid, the process of obtaining Medicaid coverage can lead to additional delays in securing a bed.

For many people relying on Medicaid, who by definition have fewer financial resources, the concept of proximity was affected by the need for family and friends to use public transportation to visit the nursing home. Thus, the set of nursing homes within a reasonable geographic distance may be further narrowed depending on convenience to public transportation stops.

AWARENESS AND TRUST OF NURSING HOME COMPARE

Despite widespread use of the Internet, most of the respondents were not aware of Nursing Home Compare or were not able to recollect if the information that they found online was from Nursing Home Compare.When shown examples of the ratings, they did not recognize the site. However, most seemed enthusiastic and indicated that they would have used the information had they known that it was available or if the hospital had mentioned it, at least in terms of an efficient way to gather basic information about multiple nursing homes. Many stressed that the ratings would be a place to start and that they would definitely use them, but a visit would still be necessary, and recommendations from health care providers or friends and family might still be more important in making a choice. The Nursing Home Compare ratings were mentioned as a potential tool in those visits, particularly to request additional information on problems reported in the ratings (or, later, to pressure administrators to make improvements).

Despite enthusiasm by many when shown the Nursing Home Compare example, a number of respondents were skeptical about the source and accuracy of the data and thought that the nursing homes themselves might have control over the site. There were numerous comments about the unreliability of information on the Internet. However, some respondents who understood that Nursing Home Compare was a website associated with Medicare were more trusting.

When asked what could be improved about the Nursing Home Compare site or what important information was missing, common answers included information about the availability of activities, information about costs, customer satisfaction ratings, and testimonials from current residents. Many respondents mentioned pricing or costs as important, and a few indicated that having information about value (not only price but what one gets for that price) would be useful.

DIFFERENCES BY INITIAL EXPECTED LENGTH-OF-STAY

We observed some differences across the first few themes in how consumers weighed factors based on the initial expected length-of-stay. Respondents expecting a short-term stay mentioned that proximity to their usual place of care or physician was important in addition to proximity to family. The existence of appropriate rehabilitation services and equipment were more salient and sometimes limiting factors, as was the influence of physicians, discharge planners, and social workers in choosing a nursing home based on current clinical needs for the initial postacute care. On the other hand, for long-term stays, distance to family and friends; the existence of meaningful and entertaining activities; and, more broadly, the level of comfort with the environment appeared to receive more consideration, although respondents also valued recommendations from health care providers. Other than these differences, results across themes were similar for those consumers who originally expected a short stay versus a long stay in the nursing home.

Discussion

In this study we set out to explore qualitatively how consumers choose nursing homes, the role of information about quality, and information drawn from Nursing Home Compare in particular. We found that consumers weighed heavily the recommendations of physicians, social workers, and other health care providers as well as friends and family; that they used the Internet; and that they valued in-person visits before making a decision. The choice of nursing homes was often limited by the desire for proximity to family and friends and sometimes by the need for specialized services. Especially for those who rely on Medicaid, the choice of nursing homes was also limited by the availability of Medicaid beds and public transportation.

There appears to be potential for Nursing Home Compare to play a greater role in these decisions. Most respondents seemed receptive to Nursing Home Compare when shown an example, and many said that theywould have used it in their decision-making process. However, most were not aware of it, and some expressed an initial lack of trust of the data, which in some cases was alleviated when the respondent was told that the site was associated with Medicare. More persistent distrust may be magnified by press reports questioning the validity of the ratings.¹⁵ In addition, we found some discrepancies among domains of quality that consumers seemed to find most important (cleanliness, staffing, and activities) and those that are reported on Nursing Home Compare (inspection results, staffing, and clinical quality outcomes).

Our findings are consistent with some results from prior studies: that consumers value recommendations from health care providers and visits to nursing homes before choosing, that smell serves as a signal for quality, that little consideration is given to clinical quality, and that proximity to family is important.^10,11 However, our findings are substantially different in several key ways. First, whereas prior studies found little use of the Internet when choosing a nursing home,^10,11 we found that almost all respondents had access to the Internet and used it in their decision-making process in some way. Second, while these same prior studies found that there was often no time to make the desired nursing home visits before choosing, we found that most consumers were able to visit one or more homes. This may be because of our focus on the long-stay population, where imminent discharge from the hospital does not always drive the timeline, or it may be because of use of the Internet to make the process more efficient. Third, these prior studies found that cost and source of payment were not key factors in the decision-making process. On the contrary, we found that concerns about cost, and the barriers to bed availability and access to high-quality nursing homes associated with reliance on Medicaid, were key factors in limiting choice for some consumers.

These findings lead directly to several policy implications. First, if the use and potential benefit from Nursing Home Compare are to improve, consumers need to be more aware of it. Although it was heavily advertised in multiple newspapers at its launch, ongoing advertising and outreach may be required. Because the need for nursing home care is usually an infrequent event and consumers may not pay attention to information before the need arises, dissemination through word of mouth might not suffice to increase awareness over time.

Second, additional “branding” may be useful to highlight that Nursing Home Compare is a website maintained by CMS and not controlled by nursing home providers. This may help consumers differentiate between Nursing Home Compare and other sources that may come up in their Internet searches, such as websites of individual nursing homes or chains.

Third, our findings point to ways in which Nursing Home Compare might be improved to increase utility to consumers. These improvements involve adding information. We found that respondents highly valued two aspects of quality that are not included in the Nursing Home Compare measures: cleanliness/smells and availability of activities. An objective measure of cleanliness and smells is unlikely to be readily available, but a measure of activities—in terms of staff hours devoted to activities or a measure of residents’ time spent in activities—seems feasible given current data sources. Participants in our study also noted that a measure of resident experience or satisfaction would be helpful, an addition that would require new data collection but would provide a perspective on quality that consumers appear to value. Finally, respondents noted the practical difficulties of finding information about cost or pricing and the availability of specialized services, additions that would almost certainly increase consumers’ interest in using the system.

Conclusion

Our results can be used in conjunction with a growing body of quantitative evidence to improve consumers’ ability to obtain high-quality nursing home care. Some avenues for improvement, such as improvements to Nursing Home Compare, have fairly straightforward policy solutions. Other barriers to obtaining high-quality care, such as the constraints associated with Medicaid payment, remain bigger problems to solve.