Nursing Home Response to Nursing Home Compare: The Provider Perspective

Abstract

Nursing Home Compare (NHC) publishes composite quality ratings of nursing homes based on a five-star rating system, a system that has been subject to controversy about its validity. Using in-depth interviews, we assess the views of nursing home administrators and staff on NHC and unearth strategies used to improve ratings. Respondents revealed conflicting goals and strategies. Although nursing home managers monitor the ratings and expend effort to improve scores, competing goals of revenue maximization and avoidance of litigation often overshadow desire to score well on NHC. Some of the improvement strategies simply involve coding changes that have no effect on resident outcomes. Many respondents doubted the validity of the self-reported staffing data and stated that lack of risk adjustment biases ratings. Policy makers should consider nursing home incentives when refining the system, aiming to improve the validity of the self-reported domains to provide incentives for broader quality improvement.

Introduction

A distinctive feature of the health care sector is consumers’ limited ability to assess the quality of services and thus choose providers of high quality. Policy makers have responded by publicly reporting information about health care quality with the additional goal of motivating providers to improve and compete on quality. In the nursing home sector, the Centers for Medicare & Medicaid Services (CMS) publishes online quality indicators for CMS-certified nursing homes on a website called Nursing Home Compare (NHC), which was updated in 2008 to include a five-star composite rating. The ratings combine information on three quality domains: health inspection violations, clinical quality measures, and staffing levels (Centers for Medicare and Medicaid Services, 2012; Williams et al., 2010).

Numerous quantitative studies show that nursing homes strategically respond to public reporting in intended and unintended ways. Nursing homes appear to increase quality on some reported indicators but not all (Feng Lu, 2012; Mukamel, Weimer, Spector, Ladd, & Zinn, 2008). A survey of nursing homes conducted soon after the 2002 launch of NHC found that the majority of facilities checked their scores and took action to try to improve them, especially those with low scores (Mukamel et al., 2007). More recently, the ratings have shown substantial improvement under the five-star system, but validity of the ratings has been called into question because of concerns that nursing homes manipulate the self-reported domains of staffing and clinical quality (Thomas, 2014).

In addition to the fundamental questions of whether and how providers respond to NHC in terms of quality improvement, researchers, and policy makers have been concerned about unintended consequences of public reporting, such as avoidance of sicker patients in favor of healthier ones, or “cream-skimming.” Some evidence has emerged to support unintended consequences: Studies found that high-quality nursing homes reduced admissions of less profitable Medicaid residents in response to NHC and selectively discharge postacute patients before they affect their quality measures (He & Konetzka, 2015; Konetzka, Polsky, & Werner, 2012). However, there is little evidence supporting the concern that nursing homes select healthier residents on admission (Konetzka, Brauner, Perraillon, & Werner, 2015; Mukamel, Ladd, Weimer, Spector, & Zinn, 2009; Werner et al., 2011).

Conceptual Framework

We adopt Donabedian’s (2005) general framework of structure, process, and outcomes to conceptualize the improvement in performance of quality measures in nursing homes. Briefly, structure refers to the setting in which health care takes place, such as the physical environment, staffing, and management qualifications. Process refers to the interactions between nursing home staff and residents. Outcomes refer to health-related endpoints. NHC reports information concerning all three aspects of the Donabedian framework. For example, the level of staffing reported in NHC is a measure of structure. Some of the quality measures are process measures, such as the use of restraints or catheters, while others, such as the percentage of residents who develop pressure sores, are outcome measures. Finally, the information based on health inspections of nursing homes encompasses some aspects of all three categories of Donabedian’s framework.

As a consequence, nursing homes seeking to improve performance on composite ratings of quality need to consider many aspects influencing the quality of care. Nursing homes, however, have limited resources and must balance competing priorities. A nursing home’s choice of quality improvement approach, focusing on structure or process mechanisms or both, depends on (a) characteristics of the nursing home, including the level of resources available to invest in quality improvement; (b) characteristics of the quality measure itself, including whether it is more objectively or more subjectively measured and how easy or difficult it is to improve; and (c) an assessment of the relatively importance of good ratings compared with other priorities. Viewed within this framework, we hypothesize that strategies developed to improve specific, limited aspects of reported quality are likely to be different than strategies developed to improve five-star ratings that cover many different dimensions of structure and process (Peters, Dieckmann, Dixon, Hibbard, & Mertz, 2007). For example, a nursing home may choose to pursue improvement on a limited set of outcome measures by optimizing coding strategies, limiting admissions of high-risk residents, or choosing the “low-hanging fruit” where improvement is easiest, or may instead focus on broad-based quality improvement through structural changes. Although the five-star system may provide more incentives for nursing homes to pursue the broad-based approach because of its composite nature, nursing homes may favor the more limited strategy depending on resource constraints and competing priorities. Specifically, we hypothesize that lower quality nursing homes, which generally face tighter resource constraints, would be more likely to pursue the limited strategies.

New Contribution

Several prior studies have targeted provider response to NHC through provider surveys. One surveyed nursing home administrators shortly after the 2002 version of NHC was launched, asking whether administrators checked scores, took action to improve scores, and what types of action (Mukamel et al., 2007). A follow-up study by the same team used a second provider survey to examine response specifically to the inclusion of the clinical quality indicators on NHC and whether nursing homes made investments to improve their scores on these measures (Zinn, Weimer, Spector, & Mukamel, 2010). Although these studies provide interesting insights, they leave several key gaps in knowledge for current policy making. First, they were conducted prior to the launch of the five-star system, after which response to NHC became stronger and more salient to nursing home administrators. Second, these studies use primarily administrative and survey data to infer provider behavior, leaving an important gap in knowledge about provider perspectives and their motivations. Although surveys can enable large sample sizes and efficient analysis, they are limited in the extent to which rich and nuanced detail can be uncovered, including unanticipated responses.

In this study, we fill these major gaps in evidence through a qualitative analysis of the provider perspective on NHC in the current context of the five-star system, including specific strategies used to improve ratings. Using in-depth, semi-structured interviews, we are able to probe the context and nuances of provider response to NHC under the five-star system, including the elicitation of detailed, spontaneous descriptions of efforts to improve scores. In addition, our approach allows for candid conversations with nursing home staff, who are not easily reachable through other methods. This type of evidence is essential to incorporating and anticipating provider response as NHC continues to be refined.

Method

Study Design

We conducted in-depth, semistructured interviews during onsite visits to nursing homes in a large metropolitan area in the Midwest, with particular interest in (a) the perceived importance of performing well on NHC relative to other priorities of a nursing home; (b) understanding strategies to improve ratings, especially in the self-reported domains; (c) exploring the possibility of selectively admitting healthier patients to improve ratings; and (d) understanding whether and how strategies vary according to nursing home attributes. We developed three interviews guides, one for administrators and directors of nursing; one for nurses who code the minimum data set (MDS), the source of data for the clinical quality measures; and one for nurse aides (see Appendix A in the Supplementary Material). We used purposeful sampling (maximum variation) to identify nursing homes and a snowball sampling approach within each facility (Coyne, 1997; Patton, 1990). We approached nursing homes that vary on attributes that may affect their quality improvement process and the type of residents they admit, such as profit status, chain affiliation, size, payer mix, and geographic location (urban or suburban). Once a nursing home administrator agreed to the interview, we requested to interview the director(s) of nursing (DON), one or more MDS nurses or coders, and a nurse aide. After visits to four facilities, we discontinued interviews with nurse aides because these interviews were not revealing information relevant to our study goals. We continued to conduct additional visits until no new concepts emerged from the interviews (theoretical saturation; Guest et al., 2006). Our final sample consisted of 10 nursing homes and 30 individuals.

We conducted on-site interviews in teams of two researchers with one respondent at a time. In three cases, on request of participants, we interviewed two people jointly. However, our request to conduct interviews with MDS nurses without the presence of an administrator was always honored. In one case, our sampling approach led us to interview the director of clinical services at the management level for an entire chain, in addition to the administrator and DON for a specific nursing home within that chain. Interviews were audio-taped after obtaining signed informed consent from all participants (no respondent declined to be taped) and transcribed. Interviews lasted from 30 to 90 minutes each. At the end of the interviews, we offered a $125 gift card to the nursing home administrator with the recommendation that it be used for a staff raffle, party, or otherwise distributed to participants as a token of appreciation for their time. The research protocol and consent materials were approved by an institutional review board.

Data Analysis

Transcripts were reviewed independently by two of the authors and then together to code and identify a list of recurrent concepts and statements. The code structure was developed inductively based on concepts as they emerged from a reading of the transcripts, although some codes were determined a priori based on the interview guide (Bradley, Curry, & Devers, 2007; Bradley et al., 2010). Discrepancies were resolved through discussion among the authors until agreement was reached on themes and representative quotes. We used NVivo version 11 software for coding and analysis.

Results

Characteristics of the Sample

The 10 nursing homes (Table 1) varied in terms of ownership, profit status, and five-star ratings for different domains at the time of the interviews. We interviewed 10 administrators, 7 DONs, 8 MDS nurses/coders, 2 nurse aides, 1 chain-level director of clinical services, and 1 chief executive officer (Table 2). The interviewees represented the staff directly involved in the setting of quality goals and day-to-day administration and operations.

Table 1.

Characteristics of Nursing Homes (N = 10) and Ratings at the Time of the Interview.

Characteristics	n (%)
Chain	9 (90.0)
For-profit	9 (90.0)
Star rating
5	1 (10.0)
4	1 (10.0)
3	4 (40.0)
2	3 (30.0)
1	1 (10.0)
Star rating—Inspections
5	1 (10.0)
4	1 (10.0)
3	4 (40.0)
2	3 (30.0)
1	1 (10.0)
Star rating—Staffing
5	1 (10.0)
4	0 (0)
3	3 (30.0)
2	4 (40.0)
1	2 (20.0)
Star rating—Quality
5	3 (30.0)
4	6 (60.0)
3	0 (0)
2	0 (0)
1	1 (10.0)

Table 2.

Characteristics of the Respondents (N = 30).

Position	n (%)
Administrator	10 (32.3)
DON	7 (22.6)
Assistant DON	1 (3.2)
Director clinical services	1 (3.2)
MDS coder	8 (25.8)
Nurse aide	2 (6.5)
Chief executive officer	1 (3.2)
Director of business	1 (3.2)

Note. DON = director of nursing; MDS = minimum data set.

Themes

Overall, interviewees were forthcoming, willing to answer questions about their operations and to express opinions about NHC. We group our main findings into five themes, described in detail below.

Nursing Homes Care About NHC Ratings but Some Care More Than Others.

The extent to which facilities pay attention to their NHC ratings depends to some degree on administrators’ perceptions of whether consumers pay attention to the ratings. Nursing homes in more affluent areas, better rated, with fewer Medicaid beds and a larger percentage of private payers mentioned that consumers, hospital partners, and even creditors pay attention to their ratings. However, consistent with previous research about consumers’ perceptions of nursing home quality (Konetzka & Perraillon, 2016), many respondents expressed belief that NHC ratings are not heavily weighted and that other attributes of nursing homes are more important for consumers:

The ones that are more savvy, which is maybe 40 percent, will have done their homework, gone to Medicare.gov … or thought to ask for your survey process. But you’d be surprised the majority that do not. I think the majority of people [focus on] on the aesthetics when they walk in. The smell. How the staff is interacting. How they’re being treated when they walk in. What the patients look like … I tend to think they go more on that than statistical data. (Administrator)

On the other hand, some administrators thought that ratings are not important for potential residents and families and, consequently, for them. This was particularly true in nursing homes with a higher percentage of Medicaid beds operating at full capacity and located in lower income areas:

5 percent of consumers maybe [check ratings]. And of course, our first response is, pardon the French, “Screw the ratings!” Talk to the residents. I have a wait list. People want to be here. People want to come here … I’m full. What extra business do I want? I’m full. (Administrator)

Maybe 5 percent of the people are aware of the star ratings, but they don’t understand them and they’re not asking as much. (DON)

As the last quote suggests, several respondents mentioned that consumers do not understand the different rating domains and the detailed information provided on NHC. Instead, respondents stated that consumers who are aware of the ratings mostly pay attention to the number of overall stars.

Avoiding Litigation Plays a Major Role in Quality Improvement Decisions.

Avoiding litigation was mentioned by many respondents when asked about current areas of focus, and it was clear that avoiding litigation has a large influence on day-to-day decisions, including the time allocated for proper documentation and resident admissions. Particular areas of litigation concern were falls, pressure ulcers, infections, and weight loss. Litigation concern was a pervasive preoccupation regardless of facility rating and type (chain, independent, for-profit):

[Avoiding lawsuits] is just as important as it is taking care of the patients and having quality outcomes. We have to successfully manage it on all levels … Falls is a high risk and so is pressure ulcers. Typically a stage 4 pressure ulcer, never, ever wins in court. Most juries are quite sympathetic and they ultimately rule in the individual who is suing. Falls are also critical. (Administrator)

[Fear of litigation] affects everything. I’d say the average nurse spends 50–60% of her time, maybe more, documenting; it seems excessive. They document to prevent lawsuits. But that’s an industry wide problem. (Administrator)

In contrast, respondents rarely mentioned that NHC is used for quality improvement decisions. Some administrators stated that the main reason for not using the information on NHC is because the information is not current. Moreover, most nursing homes have other systems to track quality-related events. Several administrators mentioned that quality improvement is guided by a “root-cause analysis” (Wilson, 1993) of events:

[NHC] is just not my basis for my actions today. Because I would be living in a dream if I was going to rely on what they tell me three months ago. Because then I would not have my finger on what’s going on in my building. (Administrator)

Several respondents mentioned that their goal is to always be prepared for the health inspection visit, or, as one administrator put it, to be “survey-ready every day.” Nursing homes are aware of the general time window for their yearly health inspection, but some inspections are triggered by complaints and can happen on any day. As one respondent stated, consumer complaints are also tied to litigation:

[A complaint is] largely driven [by litigation] because it is the precursor to a lawsuit. (Administrator)

Nursing Homes Employ Multiple Specific Strategies to Improve Ratings.

Although some administrators do not consider ratings to be a major factor affecting their practice, they expressed a desire to have good ratings, sometimes because of pressure from corporate management. Some mentioned that performance bonuses are tied to quality metrics, but only a few said that the metrics are directly linked to NHC data. We asked respondents to discuss their strategies for improving each rating domain. We were particularly interested in understanding improvement strategies for domains that have been criticized because they are based on self-reported data.

Coding of the MDS.

Most administrators and MDS nurses, who code the information into the MDS, confirmed and emphasized that the MDS data are collected through actual observation, chart review, and conversations with residents and family members:

You have to see the person. You cannot not see the person. You have to know … we go back to [the] nurses’ notes and … see what the nurses are charting. That way we can come to a decision on what we’re going to code. (MDS nurse)

MDS nurses indicated that they were not pressured to code the MDS in unethical ways. Most strongly expressed concerns about not following the appropriate rules, in particular because health inspectors verify that the coded information is backed up by documentation:

[My administrators] haven’t really said, well, we gotta be careful of how we code this. It’s pretty much it is what it is. If the patient says they’re in pain, they’re in pain … We can’t make up something. (MDS nurse)

We have lawyers that come in that talk to us about integrity and about being truthful on the MDS. And I’ve never been approached about [upgrading] the coding or anything like that. (MDS nurse)

However, several administrators were candid about specific MDS-related strategies they used to improve ratings. The pain measure, for example, appears to be sensitive to the particular time at which pain is assessed. Without changing anything with respect to pain treatment, nursing homes are able to improve pain scores by asking residents about pain after pain medication has been taken and avoiding times when the last dose may be wearing off:

Don’t have them being in severe pain … Don’t ask them at the end of the day… so when you do the actual interview has an impact too … there are things that we do that are manipulating the process a bit. But within the [parameters] of what we can do. (DON)

One facility with mostly Medicare postacute and private-pay residents had five stars in all rating domains, up from initially receiving three stars in quality measures. The administrator explained that careful use of the MDS rules was the main reason for improvement. This nursing home found ways to make potentially low-cost improvements by using the quality measure coding windows:

Only certain [Assessment Reference Dates, ARD] are used for the quality scores … the ARD dates have a window … they have a look-back period. So if an MDS nurse and [Director of Nursing] are savvy at looking at the [ARD] relative to the section of the MDS assessment that triggers each quality measure, then they can choose an ARD date that makes the most sense for the quality measures in question … Since the goals have been set up that way I think it’s totally within appropriate realm of practice to look at your ARD dates in that way. (Administrator)

In other words, the dates that the MDS assessments are filled out can be chosen strategically within the legal window such that the look-back period does not include an adverse event. This strategy would result in a better score without any actual change in care or resident outcomes. However, other examples were given that do affect outcomes. For example, if residents are at risk of triggering the weight loss measure, the nursing home may use the ARD window to set a deadline for addressing the weight loss.

Employing some of these strategies requires management resources to develop and implement them across multiple measures. The administrator of the facility using the assessment windows believes that the ratings are fundamental for the facility’s future and that the investment is worth it:

The more we pay for and understand the system … the better they’re able to structure the way they do assessments to hit the five-star rating. So we talk about it all the time. (Administrator)

However, strategies to code the MDS to improve ratings must be weighed against possible revenue losses. Information in the MDS is the basis for CMS reimbursement, with sicker patients potentially reimbursed at a higher rate. Administrators are conscious about the competing goals of maximizing revenue versus maximizing ratings:

The biggest issue with [the NHC quality measures] that people have [is] that the reimbursement system and the federally five-star system are in two different pages … Medicaid reimburses the facility higher, more money for a more complex patient that isn’t improving as much. But yet, CMS grades the facility by their improvement of that patient. And so I think it’s a struggle for operators to figure out what that balance, that ethical and moral balance is, by how we evaluate that patient and to what kind of score we’re going to [get]. (Administrator)

A nursing home with a low percentage of Medicaid residents stated that in their calculation improving ratings is more important than coding to maximize reimbursement:

Manipulating the ARD … most of, all of the MDS coordinators [do] that … There are two reasons… One is for reimbursement and the other one is to have your quality measures … For us here, we don’t care much of our reimbursements because most of our patients here are either paying privately or they are Medicare patients. (Administrator)

This is in sharp contrast to most other respondents who view reimbursement as a more pressing goal and thus make coding decisions to maximize reimbursement rather than ratings. As an administrator stated:

All the nursing homes have been consulted by the people of higher learning and so they say … Two weeks before their assessment reference date for their MDS, you’re going to look at them, you’re going to evaluate them and see if they need therapy.’ Now you score therapy. Therapy, you know, pays you at 45 dollars … adds 45 dollars to your nursing component … all within the rules … That’s the game that they play. (Administrator)

Reporting of staffing.

Facilities self-report their staffing levels based on the 2 weeks prior to health inspections. At the time of our interviews, CMS did not systematically verify this information. Some respondents expressed concerns about other nursing homes misreporting their staffing levels or increasing staffing in anticipation of the inspection without a commensurate increase throughout the year, although no respondent admitted to misreporting:

Some facilities are able to beef up their staffing [because they know the inspection date] so they look good when the surveyors come in. (Administrator)

There are people who try to get away with the minimum amount of staffing … there’s nothing to stop you from inflating some of the numbers so you could look better … nobody validates that information. (Administrator)

On the other hand, some administrators, especially in facilities with higher ratings, see the staffing ratings as a straightforward formula, where the number of stars is a function of the number of nurses hired. Thus, the staffing domain is a business decision driven by the ability to afford more staff and by an acceptable return on investment:

Owners make staffing decisions … staffing is 70% of your budget … So owners are making conscious decisions whether they want to be three, four, or five star rated facility … owners make a decision based on what they feel an acceptable return on investment is for the community and what an acceptable star rating is. (Administrator)

Preparing for health inspections.

Most respondents indicated that planning for health inspections is difficult and requires always being aware of the possibility of a surprise visit. Although the health inspection rating is considered the most reliable, there was agreement among respondents that inspections are heavily dependent on inspectors’ impression that sometimes appear biased and that inspectors are often confrontational:

The hardest is the survey … the rest of it, you put the systems in place … the survey stuff is every administrator’s worst nightmare … You never know what you’re going to get when a surveyor walks in your building. (Administrator)

The survey process is a very biased process … it’s left up to the opinion and the judgment of whoever is surveying you at that time. (Administrator)

A respondent expressed concerns that facilities with fewer resources tend to be judged more harshly:

I’ve worked in really low-income nursing homes and I’ve worked in really high-income nursing homes … when surveyors come in I see a totally different reaction how the high-income nursing homes are surveyed versus the low-income nursing homes and I think that there’s a bias … low-income nursing homes get significantly more deficiencies based on the physical appearance of their nursing home. (Administrator)

Most respondents stated that they are aware of the approximate window in which their next inspection will occur, but one respondent stated that some facilities know the exact date:

Everybody in the industry knows that. How prevalent, I can’t tell you. But all you need is one employee who is related or has a friend with the state and there, typically you’ll get to know when [the inspectors are] coming in. (Administrator)

Patient selection.

Nursing homes could improve ratings by not admitting residents that negatively affect quality measures (“cream-skimming”). To explore cream-skimming, we presented respondents with four hypothetical scenarios. Two of the scenarios related to long-term admissions and two to postacute admissions. We told interviewees to assume that they only had space to admit one patient under each scenario. The scenarios, along with the most common choices and responses, are summarized in Table 3 (see Appendix B in Supplementary Material for vignettes).

Table 3.

Patient Admission Scenarios and Representative Answers and Reasons.

Scenario	Common respondent choice	Reasons
Two long-stay potential admits. One patient with pain that is difficult to control; the other with agitation but not delirious.	Patient with pain	Pain can be managed. A patient with agitation is not only more difficult to manage but it also takes resources away from other patients and has a higher risk of falls and litigation
Two long-stay potential admits both with dementia. One patient with weight loss and declining functioning; the other at high risk for falls	Respondents were ambivalent but tended to choose the patient with weight loss	Both types of patients are difficult to manage but a patient at high risk of falls also has a higher risk of litigation
Two postacute potential admits. One patient with pressure sores; the other with a surgical nonhealing abdominal wound	Patient with surgical nonhealing wound	Respondents were concerned about the litigation risk of a patient with pressure sores. They felt that the nursing home would not be legally liable for a nonhealing surgical wound
Two postacute potential admits who are recovering from pneumonia. One patient with delirium; the other very functionally impaired.	Respondents were ambivalent but tended to prefer the functionally impaired patient	Respondents were concerned about not being able to care for a person with delirium and the risk of litigation. The patient with delirium due to pneumonia may need to remain in the hospital

Several patterns emerged from their responses to the scenarios. First, not a single respondent mentioned NHC ratings as a factor in admissions. Instead, most respondents based their preferences on whether the nursing home staff would be able to provide adequate care or whether the patient would be a burden to the staff:

[We ask ourselves] can we meet your needs? How are we going to satisfy and meet your needs? We want to make sure that the staffing is properly trained in all areas … And we want to make sure that we are able to provide the service that you are actually seeking. (DON)

[If a patient] is noncompliant and is, has a big history of noncompliance or, to the point of almost self-harm … that they’re not eating and they won’t let you turn and reposition them and they won’t take their medicine … That does play a part, because … we’re not inclined to take those patients that are extremely difficult like that, that won’t follow the plan of care. (Director clinical services)

Second, avoiding litigation was a major concern in patient selection under every scenario:

We do look at [potential litigation] factors as well [when admitting patients] … If you have someone who’s a repeat—we call them “nursing home jumpers.” And what that individual does is goes from nursing home to nursing home to nursing home to nursing home, and those are the type of red flags that goes up … because those are the ones that go for the lawsuit. (DON)

The one with the weight loss … because … safety is to me the highest liability on any level. (Administrator)

Finally, responses about admissions were consistent across nursing homes, regardless of ownership type, main source of payment or other characteristics. Nursing homes do not appear to consider NHC ratings when making admission decisions.

Nursing Home Managers Feel That Lack of Risk Adjustment and Timing Biases NHC.

When asked about the validity of NHC, respondents were particularly concerned about inadequate risk adjustment of quality measures. Most respondents thought that they were unfairly penalized for the type of residents they admit and not because of the care they provide, although some facilities with less medically complex patients recognized that they benefit from minimal risk adjustment:

We would always score high for pain. And if you look on NHC, that’s considered bad. But we have a lot of [orthopedics] patients on our Medicare side. We do strictly knees and hip replacements. 95 percent of that is what we get for our Medicare side. (Administrator)

I come from a Medicaid nursing home. That was my background and I think it would have been impossible to be rated at five stars … I think that a lot of what [the ratings] are reflecting is the income for the area … and also the type of residents that you’re admitting. (Administrator)

Another concern about the quality measures is that they do not take into account the natural progression of the disease and that nursing homes have no way to provide justification for the factors that are outside their control:

[A] Parkinson [resident] is going to lose some of their ADL ability. It’s just going to happen … And dementia is huge. No matter what measures you put into place, they’re still going to decline. So it is, it doesn’t, like, say, what is the reason for the decline? (DON)

Several respondents complained that health inspections ratings use survey results that are three or more years old. NHC assigns more weight to the most recent inspection but it uses the three most recent inspection surveys, even if management has changed or quality has changed dramatically. Thus, even if the most recent survey reflects improvements, it may take several years for the ratings to match current quality.

Suggested Improvements to NHC.

We asked respondents about their broader views on NHC and suggestions for improvement. Most respondents had mixed opinions. On the one hand, there was agreement that providing information to consumers is useful and should be done for the sake of consumers and nursing homes alike:

I think that staffing and inspection scores are the best measures that we have. So and I think it’s great that they’re accessible to families, at least it gives us all something to strive for. (Administrator)

On the other hand, most respondents thought that some of the quality indicators were not under their control and did not reflect the care provided. Even though most respondents could not articulate with precision how to provide this information, they expressed interest in giving consumers information about resident satisfaction, the day-to-day life of their residents, and the quality of the staff, including staff turnover:

We want either some way of informing the consumer or letting them know there’s other ways they could also get a better feel of the building. Whether they tour or ask questions, talk to the staff. (Administrator)

I don’t quite know how to do this. But I, I really think that there has to be some consideration for what you’re doing with the patient. So, this patient has a Foley catheter. Is there a reason why? (Director, Clinical Services)

Discussion

This qualitative study revealed nuances about nursing homes’ perspectives on NHC and their strategies to improve ratings. In particular, we were able to uncover competing priorities, such as prioritizing litigation avoidance over improving ratings when considering admissions. Some administrators also consider an inherent trade-off between maximizing revenue and maximizing ratings, as complex patients mean higher payments but nursing homes score better if they have less complex patients. Overall, our interviews reveal a provider community that monitors their ratings and, to varying degrees, puts forth effort to improve ratings despite mixed feelings about the validity and importance of many of the measures. They also reveal an array of strategies to improve ratings, including some that have little or no effect on actual outcomes, even though the strategies appear to follow MDS rules. Finally, substantial heterogeneity exists in the incentives and motivation for facilities to improve ratings, which to large extent depend on assessments of the return on investment of quality improvements. These findings are consistent with expectations stemming from our conceptual framework, in which we posited that nursing homes will choose broad-based or more limited strategies to improve performance depending on resource constraints and competing priorities. Specifically, we hypothesized that lower quality nursing homes, which generally face tighter resource constraints, would be more likely to pursue the limited strategies, and we generally found this to be true.

Our findings should be viewed in light of several limitations. First, like other qualitative studies, we trade depth of information for generalizability. Our sample of 30 individuals in 10 nursing homes may not generalize to nursing homes across the United States or to other metropolitan areas. For example, competing concerns about liability may not be as prevalent in states with low rates of litigation against nursing homes. However, the differences we found by most facility attributes—most important, by quality rating and payer mix—are not obviously confounded by location in a particular state and may well have broader generalizability. Second, our ability to recruit facilities that differ on all aspects of potential interest was limited, and our sample was skewed toward for-profit, chain facilities of medium or low reported quality. Conclusions about nonprofit and independent facilities may be more tenuous.

The implications of this study are important for both consumers and policy makers. For consumers, our findings indicate that asking about NHC may provide an impetus for facilities to improve quality. At the same time, consumers and regulators should be skeptical about the staffing and clinical quality measures in their current form, and should be aware that some reporting domains may not reflect current quality levels. Given the varied strategies employed to improve these ratings, it is not clear that improved ratings reflect improved quality of care over time. For policy makers, understanding the provider perspective offers an opportunity to improve NHC in ways that are consistent with nursing home incentives and strategies. In particular, it is important to create incentives for real improvement in clinical outcomes beyond coding changes, and to be mindful that nursing homes may find ways of making coding changes that potentially do not correlate with patient outcomes. Additional risk-adjustment of all measures may improve the perception of validity and fairness. Finally, the call for measures reflecting resident experience is one that has been expressed by both providers and consumers (Konetzka & Perraillon, 2016) and should be explored for feasibility.

As a qualitative study, our findings also reveal the need for further research in key areas, research that could inform policy by expanding our findings to a broader population and/or exploring some of them in more detail. First, respondents were consistent in their belief that lack of comprehensive risk adjustment biases the quality measures reported. However, other than one study (Mukamel, Glance, et al., 2008), which was conducted prior to the five-star system, there is a dearth of research attempting to refine risk adjustment of NHC measures to include more clinical or other factors. For example, there is an active debate in research and policy circles about whether to adjust hospital quality measures for socioeconomic status (Jha, 2014), but this debate has not extended to nursing home quality measures, despite the fact that nursing homes often blame low ratings on the challenges of serving a low-income population. Without compelling research attempting to separate true differences in quality from those that nursing homes cannot control and should potentially be adjusted for, CMS is unlikely to be interested in improving the current risk-adjustment scheme. Such studies would naturally be quantitative, extending prior data used to study Nursing Home Compare by merging in additional data on health attributes or socioeconomic status.

Given the minimal risk adjustment of most quality measures, a related concern has been the possibility that nursing homes could admit healthier patients as a way to improve their ratings (Mukamel et al., 2009; Werner et al., 2011). However, we found no evidence that nursing homes admit healthier patients to improve quality scores. Our study shows that lack of cream-skimming is likely due to nursing homes’ competing priorities when making admission decisions. Administrators are more concerned about evaluating admission based on their potential for litigation, the burden that a patient will impose on their staff, and the profitability of patients. However, these alternative priorities are important in and of themselves and deserving of more research. Litigation, in particular, is understudied in nursing homes relative to other sectors, especially its role in quality. Several studies have examined the cross-sectional relationship between poor quality and the probability of a lawsuit and found that relationship to be weak (Johnson, Dobalian, Burkhard, Hedgecock, & Harman, 2004a, 2004b; Studdert, Spittal, Mello, O’Malley, & Stevenson, 2011), while several other studies looked for a deterrence effect and found that nursing homes do not increase quality to a meaningful extent when faced with increasing malpractice pressure (Konetzka, Park, Ellis, & Abbo, 2013; Stevenson, Spittal, & Studdert, 2013). Qualitative studies that further probe the role of litigation across organizational decisions, and quantitative studies that examine the intended and unintended effects of potential solutions, such as tort reform, would be helpful.

An additional concern expressed by administrators about the validity of the ratings was the potential for dishonest manipulation of some measures. Although no respondent admitted to misreporting data, administrators suspect that other nursing homes may engage in dishonest manipulation, particularly in the staffing domain, which, at the time of our interviews, was based on self-reported data with no objective verification by CMS. However, partly due to negative press reports, CMS has announced substantial changes in the reporting of staffing levels. The new directive states that “data submitted shall be the number of hours direct care staff work each day and is based on payroll and other verifiable information” (Centers for Medicare and Medicaid Services, 2017). This new requirement also opens the possibility that tenure and staff turnover will be reported on NHC, which several of our respondents consider to be a better indicator of job satisfaction and quality of care. At the time of this writing, CMS already made submission of the new payroll-based data mandatory and was testing new measures derived from the data. These data open new channels for future research, which could help to refine NHC. Research will be needed to assess the extent to which these data solve the problem of potential manipulation of the Nursing Home Compare staffing measures and whether any new, unintended consequences arise from the new measures. For example, it will be useful to investigate whether the new system engenders more hiring of staff compared to the old system and whether new coding loopholes emerge. In addition, as prior data were quite limited in the extent to which turnover, tenure, and nonnursing staff could be studied, these new data will offer an unprecedented opportunity to study many aspects of the effectiveness of nursing home staffing on a national scale.

Although manipulation of MDS data, the basis for the quality measures domain reported on NHC, has also been a concern, our results shed light on conflicting goals that cast doubt on widespread manipulation of MDS coding. However, our results also show that some nursing homes, especially those that view ratings as key to their long-term success, find ways to use MDS coding rules to improve their quality measures. These results lead to several key areas for future research. First, qualitative and/or quantitative research exploring the extent to which these and other loopholes are used across the nation is needed. Second, it would be interesting to examine the extent to which reimbursement incentives and quality reporting incentives conflict or complement each other, examining each measure separately. For those that conflict, policy makers may want to consider changes to either reimbursement or the quality measure such that the incentive to improve quality is not dampened.

In addition to issues of risk adjustment and manipulation of data, administrators were concerned about the use of data that is up to 3 years old, even though the most current health inspection is given more weight. This problem is most relevant to the health inspection domain, since the quality measures use more up-to-date MDS information and staffing levels are at most 1 year old. Administrators reported that they are unlikely to use 3-year-old data (or even 1-month-old data) for quality improvement efforts. At the same time, currently, consumers do not have enough information to ascertain if the current health inspection rating reflects the most recent inspection or if it is affected by positive or negative inspections that could be 2 or 3 years old. The main argument for using three survey cycles is that the combined data will be more reliable and that a history of problems is meaningful even if the latest survey is good. Further research could examine longitudinal trends in survey results to assess the extent to which the inclusion of the older data affects NHC rankings, and the extent to which the trajectory of survey performance s helpful to consumers over and above current performance.

A final implication for further research is in the area of chain ownership and the extent to which organizational structure matters. Although we were limited by sample size in the extent to which we could compare responses across multiple organizational characteristics, we found that chain management played a substantial role in response to NHC for some facilities. For example, corporate management often employs one or more quality improvement staff who are responsible for monitoring and providing strategies for improvement on reported quality scores. The ultimate decision on how many direct-care staff to hire also often rests with corporate management. In addition, some of the strategies that were followed by nursing homes in our study—like extensive monitoring of MDS assessment windows—clearly require resources. These are resources that are more likely to exist in a chain organization that can take advantage of economies of scale across multiple nursing homes. Although most prior quantitative studies of NHC have included controls for chain status, it would be helpful to see a much more in-depth examination of the strategies that chains employ, including whether these strategies should affect our interpretation of quality scores for chains versus independent nursing homes.

Overall, our qualitative study fills an important gap in the existing research on NHC by eliciting the provider perspective and specific strategies used to improve reported scores. Our findings lend explanation to prior quantitative studies and suggest potential policy modifications. They also point to promising directions for future quantitative and qualitative research that can improve the reporting of nursing home quality and ultimately the care that nursing home residents receive.