Associations between family ratings on experience with care and clinical quality-of-care measures for nursing home residents

Abstract

Several states are currently collecting and publicly reporting nursing home resident and/or family member ratings of experience with care in an attempt to improve person-centered care in nursing homes. Using the 2008 Maryland nursing home family survey reports and other data, this study performed both facility- and resident-level analyses, and estimated the relationships between family ratings of care and several long-term care quality measures (pressure ulcers, overall and potentially-avoidable hospitalizations, and mortality) after adjustment for resident characteristics. We found that better family evaluations of overall and specific aspects of care may be associated with reduced rates of risk-adjusted measures at the facility level (range of correlation coefficients: −0.01 to −0.31). Associations of overall experience ratings tended to persist after further adjustment for common nursing home characteristics such as nurse staffing levels. We conclude that family ratings of nursing home care complement other types of performance measures such as risk-adjusted outcomes.

INTRODUCTION

Nursing homes are an important long-term care option for older and disabled Americans who are too frail to be supported in home- and community-based settings. In 2011, over 1.3 million patients received nursing home care with total healthcare expenditures of $149 billion (Hartman, Martin, Benson, and Catlin, 2013; State Health Facts 2014).

The quality of nursing home care, however, has been a concern for decades (Wunderlich and Kohler, 2001), with recent reports suggesting that thousands of nursing homes nationally continue to deliver care of serious deficiencies (USGAO, 2007; USGAO, 2012). Ongoing quality and safety problems span diverse clinical and nursing care areas such as avoidable high-stage pressure ulcers (Li, Yin, Cai, Temkin-Greener, and Mukamel, 2011) and high rate of hospitalizations (most of which being potentially avoidable) (Ouslander and Berenson, 2011; Ouslander, Lamb, Perloe, Givens, Kluge, et al., 2010; Spector, Limcangco, Williams, Rhodes, and Hurd, 2013; Walsh, Wiener, Haber, Bragg, Freiman, et al., 2012). In response, various efforts have been made to address these issues including improved state inspections and regulations on care (Harrington, Olney, Carrillo, and Kang, 2012; Li, Harrington, Spector, and Mukamel, 2010), public reporting (Mukamel, Weimer, Spector, Ladd, and Zinn, 2008) and, most recently, healthcare and payment reforms built in the 2010 Affordable Care Act to reduce hospital transfers from nursing facilities and to improve other aspects of care as well (Ouslander and Berenson, 2011; Ouslander, et al. 2010).

Meanwhile, federal, state, and local programs have been developed to promote patient-centered care in nursing homes (Rantz and Flesner, 2003; Sangl, Buchanan, Cosenza, Bernard, and Keller, et al. 2007; Koren, 2010; Frentzel, Sangl, Evensen, Cosenza, Brown, 2012; Li, Cai, Ye, Glance, and Harrington, et al., 2013) that emphasizes not only the quality of clinical and residential care, but other, more subjective aspects of care such as resident autonomy, resident-staff relationships, collaborative care decision making, and resident and family member experience with care. The Institute of Medicine, in its landmark “quality chasm” report (Institute of Medicine, 2001), defines patient-centered care as “health care that establishes a partnership among practitioners, patients, and their families (when appropriate) to ensure that decisions respect patients' wants, needs, and preferences and that patients have the education and support they need to make decisions and participate in their own care.” This widely accepted definition of patient centeredness emphasizes the engagement of patients as well as family members (in appropriate situations such as in many long-term care practices (White, Corazzini, Twersky, Buhr, and McConnell et al., 2012)) during healthcare provision.

The principles and concepts of patient-centered care in nursing homes may date back to the Omnibus Budget Reconciliation Act (OBRA) of 1987 which, for the first time, emphasized quality assessments based on resident-centered outcome measures and explicitly endorsed that delivering care in a resident-centered way is an essential element of high-quality care in nursing homes (Institute of Medicine, 1986). As a result of the 1987 OBRA, the Health Care Financing Administration (now Centers for Medicare and Medicaid Services, or CMS) implemented standardized, comprehensive Resident Assessment Instrument/Minimum Data Set (RAI/MDS) for all certified nursing homes; the RAI/MDS allows for resident-centered care planning and routine collection of resident-centered outcome information of multiple domains.

Since then, innovative care models, such as the Eden Alternative (Thomas, 1996), the Wellspring Program (Stone, Reinhard, Bowers, Zimmerman, and Philips et al., 2002), and the Green House Project (Kane, Lum, Cutler, Degenholtz, and Yu, 2007) have been developed and have helped introduce person-centered care practices into nursing homes. These initiatives were generally designed to foster homelike environment in nursing homes, residents’ quality of life, individualized care, and engagement of residents and families, all embraced as essential components of the “Cultural Change” movement in nursing homes (Koren, 2010; White et al., 2012; Sullivan, Shwartz, Burgess, Pekoz, and Christiansen et al., 2013).

More recently, several states have been conducting routine resident and/or family member surveys to track and assess nursing home care from consumers’ perspective, with ratings of experience with care being published on state websites (Li et al., 2013). In addition, the MDS has been revised considerably to incorporate more person-centered assessment items and direct resident surveys about individual care preferences and experiences beyond traditional assessment items such as those for functional status and medical conditions (Saliba and Buchanan, 2012); this new version of MDS (MDS 3.0) was implemented in October 2010. Finally, the CMS and the Agency for Healthcare Research and Quality together are developing standardized consumer assessment instruments for nursing homes that can be potentially used for national implementation and public reporting in the near future (Sangl, Buchanan, Cosenza, Bernard, and Keller et al., 2007; Frentzel et al., 2012). Taken together, these continued efforts of developing, collecting and disseminating performance data from nursing home consumers’ perspective will likely further promote person-centered care in nursing homes.

Measuring nursing home care based on consumer-reported information also presents challenges (Harris-Kojetin and Stone, 2007; Frentzel et al., 2012; Manary, Boulding, Staelin, and Glickman, 2013). In particular, despite the increasing appeal of consumer-reported measures to policy-makers and researchers, for at least the following reasons consensus has not been well established regarding the legitimacy of these measures in nursing home quality assessment. First, many nursing home residents suffer from severe psychiatric, cognitive, and physical impairments that prevent them from responding reliably to consumer surveys; although family members can be an alternative source of consumer evaluations (Li et al., 2013), evidence suggests that experience of care scores reported by family members were generally higher than ratings reported by residents (although the two sets of ratings tended to be highly correlated) (Gasquet, Dehe, Gaudebout, and Falissard, 2003; Castle, 2006).

Second, it is possible that both residents and family members fear retribution from their healthcare providers for reporting negative evaluations of care. Thus, ratings from both residents and their family members may tend to overstate their actual care experiences. This concern may be especially relevant for long-term custodial residents (compared to post-acute short-stay residents) and their family members due to the extended stays of these residents in nursing homes.

Finally, questions remain about whether consumer-reported evaluations of care are highly correlated with well-accepted measures of clinical process of care (e.g. rate of hospital transfers) or clinical outcomes (e.g. pressure ulcer rate). Consumer ratings of care may tend to reflect interpersonal care experiences (e.g. communication between nursing staff and family members) that represent a distinct dimension of quality not well captured by clinically-oriented indicators. However, the interpersonal and clinical aspects of care should be highly correlated in the sense that improved interpersonal relationships between patients and care practitioners help achieve optimal processes and outcomes of care, and vice versa (Donabedian, 1988). Thus, nursing homes with higher levels of consumer ratings may also be more likely to demonstrate improved care practices and to produce better health outcomes. Nevertheless, direct evidence about the linkage between experiences with care and clinical quality indicators does not exist in the nursing home literature.

New Contributions

To our knowledge, no study has been conducted to examine the direct relationship between consumer-reported evaluations of care and more “objective” risk-adjusted outcome measures of quality in the nursing home setting. Consequently, this study took the first step to determine the direct relationships between nursing home consumer ratings and several risk-adjusted measures for long-term residents (pressure ulcer rate, overall and potentially-avoidable hospitalization rates, and mortality rate). Using the consumer survey reports of Maryland nursing homes, we further determined whether consumer ratings continue to be associated with these clinically-oriented measures after controlling for common nursing home and market characteristics.

METHODS

Data Sources

This study used data from multiple sources that included (1) Maryland nursing home experience with care reports of 2008; (2) 2007 and 2008 MDS files to define Maryland nursing home long-term resident cohorts; (3) 2007 and 2008 Medicare Provider Analysis and Review (MedPAR) files to define hospitalizations; (4) 2007 and 2008 Medicare beneficiary summary and crosswalk files to link MDS and MedPAR files; (5) 2008 Online Survey, Certification, and Reporting (OSCAR) file to obtain information on nursing home characteristics; and (6) 2008 Area Resource File (ARF) to obtain county (market) variables. Data of 2008 were used since multiple sources of data for more recent years were not available when this study was conducted.

Maryland nursing home family member survey of 2008

Starting from 2007 and using a pilot-tested instrument, the Maryland Health Care Commission (MHCC) conducted annual mail surveys of designated responsible parties of all long-term care residents (i.e. with length of stay≥90 days) in Maryland nursing homes, and published facility rating scores on its website (Kozlowski and Christmyer, 2008; Calikoglu, Christmyer, and Kozlowski, 2012; Maryland Health Care Commission and Market Decisions, LLC., March 2009). Responsible parties were mostly family members (e.g. 83% were adult children or spouses of the residents in 2007 (Kozlowski and Christmyer, 2008), although could be non-relatives such as friends (because of this and for ease of understanding, we follow the MHCC and describe nursing home ratings as derived from family member surveys, rather than from surveys of responsible parties, in this study). During each year’s survey, roughly two-thirds of the respondents visited the nursing home ≥20 times, and 80% visited the nursing home ≥10 times, within 6 months before the survey (Kozlowski and Christmyer, 2008; Calikoglu, Christmyer, and Kozlowski, 2012; Maryland Health Care Commission and Market Decisions, LLC., March 2009).

The 2008 survey was conducted between September 12^th, 2008 and January 13^th, 2009 and responses reflected family member evaluations on care provided in 2008 (Maryland Health Care Commission and Market Decisions, LLC., March 2009).. All Maryland nursing homes serving long-term residents (n=223 facilities) participated in this year’s survey. The Maryland Health Care Commission identified 17,057 responsible parties from facility lists and mailed survey questionnaires to them (according to the MHCC, facility lists of responsible parties were maintained for all residents in Maryland nursing homes; <1% responsible parties with missing or erroneous contact information were not surveyed). Responsible parties who did not respond initially were further contacted through repeated mails, follow-up calls, and reminder postcards at pre-defined time points. A total of 9,645 completed surveys were received finally, resulting in an overall response rate of 57%. In addition, a minimum of 50% response rate was achieved for individual facilities (Kozlowski and Christmyer, 2008; Calikoglu, Christmyer, and Kozlowski, 2012; Maryland Health Care Commission and Market Decisions, LLC., March 2009).

The 2008 survey asked 25 questions to assess 5 domains of residents’ life and care including (1) staff and administration, (2) care provided to residents, (3) food & meals, (4) autonomy & resident rights, and (5) physical aspects of the facility. Each domain typically contains several questions that in the majority of cases rate reported experiences with care on a scale of 1 to 4 (1=never, 2=sometimes, 3=usually, 4=always). An example of such questions is “in the last 6 months, if you asked for information about the resident, how often did you get the information within 48 hours?” (for the staff and administration domain). There are several other questions with possible responses being yes and no (e.g. “in the last 6 months, did you have issues or concerns with the care the resident received in the nursing home?” for care provided to residents). The rating of each domain is calculated by transforming and averaging the scores of all questions within the domain; the domain score ranges between 1 (worst experience with care) and 4 (best experience with care) (Maryland Health Care Commission and Market Decisions, LLC., March 2009). The survey also asked two additional questions about (1) overall experience with care in the nursing home on a rating from 1 (worst possible care) to 10 (best possible care); and (2) whether the respondent would recommend the facility to someone he/she knows who need nursing home care (yes/no). See the Appendix for more details about the survey questions and scoring method.

Most of the domain-specific questions as well as the two questions for overall care experience and recommendation were adopted from the nursing home Consumer Assessment of Healthcare Providers and System surveys that the CMS and the Agency for Healthcare Research and Quality developed and tested (Sangl et al. 2007; Frentzel et al. 2012). A recent comprehensive report (Frentzel et al. 2012) shows at least good interval consistency (e.g. Cronbach α>0.75), reliability (e.g. inter-unit reliability>0.70) and other psychometric attributes for individual items and composite scores. In a longitudinal analysis of the Maryland reports, another study (Li, Ye, Glance, and Temkin-Greener, 2014) revealed high correlations between domain composites and the two ratings on overall care experience and recommendation, suggesting their high concurrent validity.

Clinical quality of care measures

We analyzed several nursing home quality measures that have been previously validated, are central to the well-being of residents, and, to various degrees, are amenable to appropriate interventions. Pressure ulcers are a significant health problem among long-term residents that increases morbidity, mortality, and costs of care, and reduces quality of life (Berlowitz and Wilking, 1993; Li, Yin, Cai, Temkin-Greener, and Mukamel, 2011). Improving the prevention and treatment of pressure ulcers in nursing homes thus continues to be the focus of various quality improvement efforts (Berlowitz and Wilking, 1993; Rollow, Lied, McGann, Poyer, and LaVoie et al., 2006; Olsho, Spector, Williams, Rhodes, and Fink et al., 2014). High acute care hospital admissions of frail residents tend to suggest problematic care in nursing homes because these admissions oftentimes are deemed clinically inappropriate or preventable, cause additional physical and psychological suffering, and expose residents to iatrogenic problems such as falls with injuries and hospital acquired infections (Ouslander and Berenson, 2011; Ouslander et al., 2010; Spector, Limcangco, Williams, Rhodes, and Hurd, 2013; Walsh et al., 2012). The lack of coordination between Medicare and state Medicaid programs may also create financial disincentives for nursing homes to reduce hospital transfers, especially those that are largely avoidable (Ouslander and Berenson, 2011; Ouslander et al., 2010). Thus, in this study we focused on both overall hospitalizations and those due to conditions developed at nursing homes that are potentially avoidable (PAHs). Finally, we estimated risk-adjusted 1-year mortality rate of long-term residents. Although the primary goal of nursing home care is to maintain quality of life and is not to prolong life, high mortality rate in a nursing home may be the result of many other intermediate adverse outcomes such as serious functional declines of residents, and thus be indicative of inferior care (Flacker and Kiely, 2003).

We defined pressure ulcers for high-risk residents using previously developed methods (Abt Associates 2004; Li, Yin, Cai, Temkin-Greener, and Mukamel, 2011). We first identified Maryland long-term residents using MDS annual and significant-change assessments of 2008. Among them, high-risk residents were further identified as those who required extensive assistance or were totally dependent on staff assistance for bed mobility or moving between surfaces, were in a coma, or had malnutrition (ICD-9-CM codes 260–262, 263.0–263.2, 263.8, and 263.9). Pressure ulcers of stage 2 or higher were determined by nurse assessment or physician diagnosis (ICD-9-CM 707.22–707.24) in MDS.

To define other measures (hospitalizations and mortality), we first identified Maryland long-term residents using the quarterly, annual or significant-change assessments in the last quarter of 2007. Using encrypted beneficiary ID, we then linked residents’ most recent assessment to the Medicare beneficiary summary files to identify their insurance type (Medicare fee-for service versus Medicare HMO) and mortality status within a year of baseline assessment; and to the MedPAR files to identify 1-year all-cause hospitalizations and PAHs based on the ICD-9-CM (International Classification of Diseases, 9^th revision, Clinical Modification) diagnostic codes available in the inpatient claims. Our analyses of hospitalizations and mortality excluded residents younger than 65 years, enrolled in Medicare HMO, with a do-not-hospitalize order, or who were in a coma. PAHs were identified using the methods recently developed by a CMS project (Walsh et al., 2010; Walsh et al., 2012) and included hospitalizations due to anemia, congestive heart failure, hypertension, hypotension, diabetes mellitus with ketoacidosis or hyperosmolar coma, acute renal failure, constipation, diarrhea, Clostridium difficile, gastroenteritis with nausea and vomiting, cellulitis, skin ulcers, pneumonia bronchitis, urinary tract infection, falls and trauma, altered mental status/acute confusion/delirium, psychosis, severe agitation, organic brain syndrome, COPD, weight loss or nutritional deficiency, and seizures.

Resident, nursing home, and county covariates

In order to calculate risk-adjusted quality measures and perform multivariable analyses, we first identified resident characteristics that were potentially associated with these measures based on previous literature (Flacker and Kiely, 2003; Li, Yin, Cai, Temkin-Greener, and Mukamel, 2011; Ouslander et al., 2010; Spector, Limcangco, Williams, Rhodes, and Hurd, 2013; Walsh et al., 2012). They included age, gender, race (non-Hispanic White versus otherwise), difficulties in the activities of daily living (ADLs), the Cognitive Performance Scale (CPS), do-not-resuscitate order, and the presence or absence of dementia (Alzheimer disease or other types of dementia), stroke, diabetes or other endocrine disease, cardiovascular disease, musculoskeletal disease, cancer, incontinence (frequent or complete bowel or bladder incontinence), antipsychotic drug use, hip fracture in last 180 days, or being at the end stage of life (≤6 months to live). ADLs included bed mobility, transferring, dressing, eating, toilet use, personal hygiene, and bathing; each ADL component was coded into 5 categories from 0 (independence) to 4 (total dependence), resulting in a total range of the aggregate ADL score of 0 to 28. The CPS was defined using a validated MDS algorithm developed by Morris, Fries, Mehr, Hawes, and Phillips et al., 2007, and had a range of 0 (cognitively intact) to 6 (very severely impaired in cognition).

Nursing home characteristics from the 2008 OSCAR included total number of beds, profit status (for-profit or not), chain affiliation (yes or no), occupancy rate, presence of physician extenders (nurse practitioners or physician assistants), and nursing staff levels (hours per resident day) for registered nurses (RNs), licensed practical/vocational nurses (LPNs/LVNs), and certified nurse assistants (CNAs). Finally, the 2008 ARF was used to obtain several county variables including the percentage of elderly population (≥65 years), nursing home competition measured using the Herfindahl index, median household income, and urban versus rural location.

Statistical analysis

We performed univariate analyses on overall and domain-specific family rating scores of nursing homes, risk-adjusted rates of nursing home quality measures (see below), other nursing home variables and county covariates.

We also performed separate univariate analyses on residents in the cohort used for defining pressure ulcer rates and residents in the 2^nd cohort for defining other clinical measures. We estimated 4 separate resident-level logistic regression models for resident process/outcome measures in order to calculate risk-adjusted rates for all nursing homes. The dependent variables were 4 separate binary variables (1/0) for whether the resident had stage 2 or higher pressure ulcers, whether had at least one acute care admission, whether had at least one PAH, and whether died during the 1 year follow-up period. All models used facility-level random effects to control for the clustering of residents within facilities (Li, Cai, Glance, Spector, and Mukamel, 2009). Each random-effects model adjusted for resident characteristics described above and model estimates were used to predict risk scores for all residents associated with a measure. Predicted scores were then aggregated to the nursing home level to calculate average expected process/outcome rates for all nursing homes, which in turn were used to calculate the final risk-adjusted process/outcome rates using the observed-to-expected comparison approach described in detail in a previous methodological study (Li, Cai, Glance, Spector, and Mukamel, 2009).

After facility-level risk-adjusted rates were calculated, we ran bivariate Pearson correlation analyses among risk-adjusted measures and all types of consumer ratings. In addition, we ranked nursing homes by their overall experience-with-care scores and categorized nursing homes into quartile groups, that is, those of low overall ratings (scores<7.78), medium overall ratings (7.78<scores≤8.25), medium-high overall ratings (8.25<scores≤8.75), and high overall ratings (scores>8.75). We then graphically presented all risk-adjusted measures and their 95% confidence intervals (95% CIs) by quartile groups.

We finally performed 4 sets of resident-level multivariate regression analyses in order to better understand the independent associations of facility’s overall ratings and resident process of care or outcome indicators. The independent variables in all regressions were 3 dummies for the quartile groups of overall family ratings (the 1^st quartile group being the reference group). For pressure ulcers≥ stage 2, the dependent variable was binary and we estimated random-effects logistic models. For all-cause hospitalizations or PAHs, the dependent variable was time to first acute-care hospital admission for any reason or for potentially-avoidable conditions. In both cases, we estimated competing risk proportional hazard models with death serving as a competing risk to the outcome. Residents were treated as right censored if no event occurred before the 1-year follow-up ended. For the last measure of mortality, the dependent variable was time to death and we estimated Cox proportional hazard models where residents were right censored if they did not die in 1 year. In both the competing risk and the Cox proportional hazard models we estimated robust sandwich estimators for standard errors to account for intra-facility correlation of residents.

In multivariate analyses for each outcome, we sequentially adjusted for resident characteristics as used in previous risk adjustment analyses (Model 1); common nursing home characteristics including bed size, profit status, chain affiliation, and occupancy rate (Model 2); staffing patterns including presence of physician extenders, and nursing staff levels for RNs, LPNs/LVNs, and CNAs (Model 3); and county characteristics described before (Model 4). All analyses were performed in Stata version 13 (StataCorp, College Station, Texas).

RESULTS

Our study included 12464 residents used for estimating risk-adjusted pressure ulcer rates and 14013 residents used for estimating other measures after excluding 22 facilities (and their residents) with less than 30 eligible residents in the facility. Included residents lived in 201 Maryland nursing homes (Table 1). Residents in both cohorts were similarly old (average ages>80 years), were female in over 70% cases, and were nonwhites in roughly one-third cases, but resident in the 1^st cohort at high risk for pressure ulcers tended to be more physically disabled (average ADLs 23 vs. 18) and cognitively impaired (average CPS 3.3 vs. 2.9; see Appendix 2 for details).

Table 1.

Maryland nursing home (n=201) and county characteristics, 2008

	Prevalence (%) or Mean±SD
Nursing home characteristic
Overall rating (1–10)	8.25±0.65
Rating on (1–4)
Staff & administration	3.65±0.15
Care provided	3.47±0.19
Food & meals	3.49±0.21
Autonomy	3.48±0.24
Physical environment	3.37±0.21
Percentage of recommendation (0–100)	89.27±8.81
Risk-adjusted rate of
Pressure ulcers (≥stage 2)	0.12±0.07
All-cause hospitalizations	0.35±0.11
Potentially-avoidable hospitalizations	0.18±0.08
Mortality	0.30±0.08
Total number of beds	131.43±66.28
For-profit ownership	68.06%
Chain affiliation	57.87%
Percentage of occupied beds	87.29±10.06
Presence of physician extenders	37.50%
Nursing staffing (hrs per resident day) for
Registered nurses	0.32±0.24
Licensed practical/vocational nurses	0.95±0.43
Certified nursing assistants	2.12±1.32
County characteristic
Competition for nursing home care	0.88±0.15
Percentage of elderly population (≥65 yrs)	12.87±2.75
Median household income, $	67149.97±18667.99
Rural location	14.81%

Table 1 shows that overall family ratings on a 1-to-10 scale were 8.3 on average (SD=0.7) for Maryland nursing homes in 2008. In addition, average ratings on individual aspects of care ranged from 3.4 for physical environment and 3.7 for staff & administration (on a 1-to-4 scale), and overall 89% of family members would recommend the nursing home to somebody who needs nursing home care. The relatively high consumer ratings were similar to those found for Massachusetts nursing homes (Li et al., 2013). Risk-adjusted rates of clinical quality measures were 0.12 (SD=0.07) for pressure ulcers, 0.35 (SD=0.11) for all-cause 1-year hospitalizations, 0.18 (SD=0.08) for hospitalizations due to potential avoidable conditions, and 0.30 (SD=0.08) for 1-year mortality (see Appendix 2 for details of risk adjustment models). Table 1 also presents summary statistics of other facility and county characteristics.

The Pearson correlation analyses (Table 2) suggested that higher overall family ratings were correlated with lower pressure ulcer rate (ρ=−0.21, p<0.01), lower all-cause hospitalization rate (ρ=−0.14, p<0.10), lower mortality rate (ρ=−0.24, p<0.01), but not lower PAH rate (ρ=−0.05, p>0.10). Scores for other individual care domains and family recommendation were also negatively associated with several but not all risk-adjusted quality measures. Figure 1 shows general trends of reduced risk-adjusted rates by facility quartiles groups of overall family ratings.

Table 2.

Correlation coefficients between experience-with-care ratings and risk adjusted quality of care measures for Maryland nursing homes, 2008

Risk-adjusted quality measures	Overall rating	Rating on					Recommendation rate
		Staff & administration	Care provided	Food & meals	Autonomy	Physical environment
Pressure ulcer (≥ stage 2) rate among high-risk residents	−0.21***	−0.21***	−0.20**	−0.17**	−0.06	−0.15*	−0.12
1-year all-cause hospitalization rate	−0.14*	−0.08	−0.09	−0.09	−0.21***	−0.15**	−0.12
1-year potentially-avoidable hospitalization rate	−0.05	−0.02	−0.01	−0.09	−0.16**	−0.12	−0.05
1-year mortality rate	−0.24***	−0.15**	−0.31***	−0.08	−0.27***	−0.23***	−0.30***

^*p<0.10;

^**p<0.05;

^***p<0.01

Figure 1.

Risk-adjusted quality of care measures by quartile groups of overall care experience ratings among Maryland nursing homes (error bars indicate 95% confidence intervals)

Table 3 shows that higher overall family ratings of the facility was associated with residents’ reduced likelihoods for pressure ulcers, hospital transfers, and mortality after controlling for resident characteristics. In addition, these associations were partially explained by nursing home common characteristics including staffing patterns, and county covariates. For example, compared to residents in facilities with low overall rating scores (1^st quartile), residents in facilities with high overall ratings (4^th quartile) had almost 40% reduced adjusted odds of having ≥stage 2 pressure ulcers (odds ratio [OR]=0.61, 95% confidence interval [CI] 0.47–0.81, p<0.01); this association was tempered after nursing home and county covariates were further adjusted for (OR=0.76, 95% CI 0.55–1.05, p<0.10).

Table 3.

Associations of facility overall experience-with-care ratings with risks for pressure ulcers (≥stage 2), all-cause hospitalizations, potentially avoidable hospitalizations and mortality among Maryland nursing home long-term residents

Overall rating (low ratings as the reference group)	Pressure ulcer		All-cause hospitalization		Potentially-avoidable hospitalization		Mortality
	Odds ratio	95% CI	Sub-hazard risk ratio	95% CI	Sub-hazard risk ratio	95% CI	Hazard ratio	95% CI
Model 1
Medium ratings	0.92	(0.73–1.17)	0.86**	(0.74–0.98)	0.82**	(0.69–0.97)	0.96	(0.85–1.10)
Medium high ratings	0.85	(0.66–1.08)	0.91	(0.79–1.04)	0.97	(0.83–1.14)	0.94	(0.83–1.07)
High ratings	0.61***	(0.47–0.81)	0.81**	(0.69–0.95)	0.84*	(0.70–1.01)	0.80***	(0.70–0.92)
Model 2
Medium ratings	0.94	(0.74–1.19)	0.86**	(0.75–0.99)	0.83**	(0.70–0.98)	0.96	(0.84–1.09)
Medium high ratings	0.88	(0.68–1.14)	0.93	(0.81–1.07)	0.99	(0.84–1.17)	0.93	(0.82–1.06)
High ratings	0.69**	(0.50–0.93)	0.83**	(0.69–0.99)	0.85	(0.69–1.04)	0.80***	(0.68–0.93)
Model 3
Medium ratings	0.91	(0.72–1.14)	0.88*	(0.77–1.01)	0.85*	(0.72–1.01)	0.95	(0.83–1.09)
Medium high ratings	0.93	(0.72–1.19)	0.93	(0.80–1.08)	1.00	(0.84–1.19)	0.95	(0.83–1.08)
High ratings	0.76*	(0.55–1.04)	0.84*	(0.69–1.01)	0.86	(0.70–1.06)	0.82**	(0.69–0.97)
Model 4
Medium ratings	0.91	(0.71–1.14)	0.88*	(0.77–1.00)	0.84**	(0.72–0.99)	0.94	(0.82–1.07)
Medium high ratings	0.94	(0.72–1.21)	0.96	(0.82–1.12)	1.03	(0.86–1.23)	0.96	(0.84–1.09)
High ratings	0.76*	(0.55–1.05)	0.86	(0.71–1.05)	0.91	(0.73–1.13)	0.83**	(0.70–0.98)

Model 1 adjusted for resident risk factors including age, gender, race, difficulties in activities of daily living, cognitive performance scale, do-not-resuscitate order, and the presence or absence of dementia, stroke, diabetes or other endocrine disease, cardiovascular disease, musculoskeletal disease, cancer, bowel or bladder incontinence, antipsychotic drug use, hip fracture in last 180 days, and being at the end stage of life.

Model 2 further adjusted for common facility characteristics including bed size, profit status, chain affiliation, and occupancy rate.

Model 3 further adjusted for nursing home staffing patterns including presence of physician extenders, and nursing staff levels for registered nurses, licensed practical/vocational nurses, and certified nurse assistants.

Model 4 further adjusted for county variables including the percentage of elderly population, market competition, median household income, and urban versus rural location.

^*p<0.10;

^**p<0.05;

^***p<0.01.

95% CI: 95% confidence interval.

DISCUSSION

Our analyses on Maryland nursing home data revealed that higher family ratings on overall and specific aspects of care (e.g. staff performance, resident autonomy) tended to be associated with several risk-adjusted quality measures including lower rate of pressure ulcers, lower hospital admission rate (all cause or for potentially avoidable conditions), and lower mortality rate for long-term residents. Variations in nursing home common characteristics, staffing patterns, and county characteristics may explain a portion of these associations, but the associations between overall ratings and improved quality measures tended to persist after these variables were adjusted for.

Do residents in nursing homes with better reported experiences with care receive better quality of care and have better outcomes? This study tried to answer this question by determining the relationships between experience-of-care ratings (reported by family members) and several clinically-oriented quality measures. Findings of this study confirmed our expectations that the 2 sets of measures, although evaluating care from distinct perspectives, tended to be correlated although the correlations were not very strong (ρ<0.30 in absolute values in general) and were evident only for a subset of possible combinations between the 2 sets of measures.

These findings lend some credence to the increasing use of consumer experience-of-care reports as person-centered measures of care in nursing homes. In general, measures based on consumer feedbacks offer unique insights into care quality and residents’ quality of life because, by construction, they may largely reflect the varied aspects of interpersonal care experiences of residents and their family members, which are likely not well captured by clinical measures such as risk-adjusted mortality rate. The fact that the two sets of measures (consumer-reported versus clinically-oriented) emphasize different aspects of care (interpersonal versus clinical) may explain in part the lack of very strong and across-the-board associations between them. Nevertheless, to the degree that synergies exist between patient-provider relationship and the “technical” aspects of care delivery (e.g. better interpersonal relationship fosters improved clinical and personal care), higher family ratings on nursing home care should be associated with better care process and resident outcomes. Our correlational analyses showed that except for family evaluations on food & meals provided by the facility and overall recommendation of the facility, these publicly-reported care experience ratings were significantly correlated with at least 2 of the 4 risk-adjusted quality measures examined in this study. In addition, higher overall evaluation of care by family members tended to be associated with better resident care and outcomes in all cases even after important factors such as staffing levels were accounted for.

Findings in this study are consistent with the results of a limited number of previous studies that examined the associations between consumer reports and nursing home quality indictors or proxies (but not risk-adjusted resident outcome or care process measures) (Calikoglu, Christmyer, and Kozlowski, 2012; Li et al., 2013; Lucas, Levin, Lowe, Robertson, and Akincigil, 2007). A recent study of Massachusetts nursing home satisfaction surveys showed that family ratings of care varied across facilities in the state and that these variations were associated with variations in nurse staffing levels and deficiency citations issued to the facility due to violations of minimum federal/state quality standards (Li et al., 2013); in addition, for-profit ownership of nursing homes, a commonly used proxy for reduced quality of care compared to nonprofit ownership, was associated with lower scores of family ratings. Two other studies also found that improved nurse staffing in nursing homes was associated with higher overall experience ratings (Calikoglu, Christmyer, and Kozlowski, 2012; Lucas, Levin, Lowe, Robertson, and Akincigil, 2007). Our study extends these findings by showing that consumer experiences with care have direct and independent implications for resident outcomes and care process (e.g. hospital transfers) even after common quality indicators and proxies are accounted for.

Our findings also parallel findings of a growing body of literature on patient experiences with hospital care and their relationships with quality of care and patient outcomes (Jha, Orav, Zheng, and Epstein, 2008; Glickman, Boulding, Manary, Staelin, and Roe et al., 2010; Boulding, Glickman, Manary, Schulman, and Staelin, 2011). For example, Jha, Orav, Zheng, and Epstein, 2008 analyzed the first wave of the HCAHPS (Hospital Consumer Assessment of Healthcare Providers and Systems) survey data and found that hospitals with higher patient ratings of care also showed better performance scores of evidence-based process of care measures for several conditions such as acute myocardial infarction (AMI) and for the prevention of complications from surgery as well. Another study by Boulding et al. 2011 further reported that higher HCAHPS scores of the hospital were independently associated with reduced risk-adjusted 30-day readmission rates for patients who were initially admitted the hospital for AMI, heart failure, or pneumonia.

Results of this present study have important implications for future efforts of evaluating and improving person-centered care in nursing homes. Our results support broader use of consumer (residents and their family members) ratings on care in the future because they complement existing quality indicators (e.g. staffing levels currently available on the “Nursing Home Compare” website) and offer insights into the more subjective or interpersonal aspects of care. Meanwhile, improved person-centeredness and resident-staff relationship in nursing homes, as demonstrated in this study, may have the opportunity to create synergies for improved clinical care delivery and resident outcomes, or at least would not divert resources or staff attentions from clinical care given the nonexistence of associations between improved consumer ratings and worsened clinical measures in our analyses.

This study has several limitations. First, our analyses were limited to nursing homes in Maryland and thus results may or may not be generalized to nursing homes in other states. Maryland is among the few states that currently are collecting and publishing consumer evaluations of nursing home care (Li et al., 2013); national public reporting of such data (Frentzel et al., 2012), which is currently planned and would be implemented in the future, would allow researchers to confirm our results at the national level.

Second, our analyses were cross-sectional and therefore could determine associations but not causal effects between consumer ratings of care and risk-adjusted quality measures. The estimated associations could be biased due to imperfect adjustment for important variables such as patient frailties and functional impairments. Imperfect risk adjustment may lead to underestimation of patients’ expected outcome rates in the patient-level risk-adjusted analyses of quality measures, and overestimation of the risk-adjusted quality measures (e.g. mortality rate) for nursing facilities with more highly-impaired residents relative to other facilities. Meanwhile, evidence suggests that worse health status of patients tends to be negatively associated with consumer ratings of care (Elliott, Swartz, Adams, Spritzer, and Hays, 2001; Lucas et al. 2007). Thus, incomplete risk adjustment for resident frailties may also cause an issue of endogeneity between consumer ratings and risk-adjusted measures, which would bias their estimated associations towards a more negative direction. In addition, the extent to which family members were actually involved in resident care was unmeasured, but it could affect both care process and outcomes (e.g., more family involvement may lead to better care and lower likelihood of developing pressure ulcers), and reported care experience ratings (e.g. better engaged family members may report higher evaluations of the facility and its staff or, alternatively, are more likely to identify staff and care issues in the facility and report lower rating scores). This could bias the estimated associations to a more or less negative direction. The overall direction of bias due to omitted variables from the analyses may not be easily determined; the different sources of bias may cancel out each other leading to relatively unbiased estimates of the association between consumer ratings and risk-adjusted quality measures.

Third, although the Maryland nursing home experience-of-care survey had a response rate of 57%, nonresponse may bias the reported consumer ratings. Different types of responsible parties of residents, such as resident’s sibling versus resident’s adult child, may have different likelihoods to respond to the survey and when they do respond, may tend to rate nursing homes differently. However, we did not have access to the demographic data of surveyed responsible parties, and had no way to test potentially different response patterns by demographic groups or test the impact of other factors (such as geographic locations) on response rates or reported scores. However, the relatively high response rate overall and for individual nursing homes (at least >50%) should help minimize such response biases. Finally, nursing home care is multidimensional and in this study we focused on several important clinical measures and were, of course, unable to cover all measures. Additional work is necessary to determine the associations of consumer evaluations of care with other quality indicators such as those for resident physical and mental health functioning, and resident quality of life indicators available in the current version of MDS.

Despite these limitations, our study suggested that better family ratings on experience with care tended to associated with lower rates of pressure ulcers, hospitalizations for any and potentially avoidable conditions, and mortality among long-term residents in Maryland nursing homes. Data on consumer evaluations of care complement other types of performance measures such as nurse staffing levels and clinical outcomes, and broader use of these consumer-reported data (e.g. in public reporting, pay for performance) will help promote both residents’ quality of life and quality of care in nursing homes.

Appendix 1: 2008 Maryland Nursing Home Family Member Experience With Care Survey Domains and Items a

Satisfaction with Overall Experience

1. Using any number from 1 to 10, where 10 is the best care possible and 1 is the worst care possible, what number would you use to rate the care at this nursing home? (1–10 scale)

Recommendation

1. If someone needed nursing home care, would you recommend this nursing home to them? (yes/no)

Domain 1: Staff and Administration of the Nursing Home

1. In the last 6 months, if you asked for information about the resident, how often did you get the information within 48 hours?

2. In the last 6 months, how often did the nurses and nursing assistants treat you with courtesy and respect?

3. In the last 6 months, did the nurses and nursing assistants treat the resident with courtesy and respect?

4. In the last 6 months, did the nurses or nursing assistants ever discourage you from asking questions about the resident? (yes/no)

Domain 2: Care Provided to Residents

1. Were you invited to participate in a care conference in the last 6 months? (yes/no)

2. In the last 6 months, how often were you involved as much as you wanted in care decisions?

3. In the last 6 months, during any of your visits, did you help the resident with toileting? (yes/no)

4. In the last 6 months, how often, if at all, did you help with toileting because the nurses or nursing assistants either were not available or made him or her wait too long?

5. In the last 6 months, did the resident look and smell clean?

6. In the last 6 months, did the resident use the nursing home's laundry service for his or her clothes? (yes/no)

7. In the last 6 months, how often were you satisfied with the laundry service the resident received?

8. In the last 6 months, did you see any resident, including this resident, behave in a way that made it hard for nurses or nursing assistants to provide care? (yes/no)

9. How often did nurses/nursing aides handle the situation in a way that was acceptable to you?

10. In the last 6 months, did you have issues or concerns with the care the resident received in the nursing home? (yes/no)

11. In the last 6 months, did you discuss any issues or concerns with nursing home staff? (yes/no)

12. In the last 6 months, were you satisfied with the way the nursing home staff handled issues or concerns that you brought to their attention?

13. In the last 6 months, did you ever stop yourself from talking to any nursing home staff about your concerns because you thought they might take it out on the resident? (yes/no)

Domain 3: Food and Meals

1. If you helped the resident with eating or drinking during any of your visits, how often did you help with eating or drinking because the nurses or nursing assistants were not available to help or made him or her wait too long?

Domain 4: Autonomy & Resident Rights

1. If the resident desires private space for visits such as with clergy or family, is private space provided?

2. In the last 6 months, did you observe that the resident’s or other residents’ privacy was protected when the resident was dressing, showering, bathing, or in a public area?

Domain 5: Physical Aspects of the Nursing Home

1. In the last 6 months, did the public areas of the nursing home look and smell clean?

2. In the last 6 months, when you visited, how often did the resident's room look and smell clean?

3. In the last 6 months, when you visited, was the noise level around the resident’s room acceptable to you?

Note: unless otherwise noted in the end of each question, the answer to each survey question is based the 4 point Likert scale: 1=never, 2=sometimes, 3=usually, 4=always.

To calculate a nursing home’s rating of each domain, the percentage of survey respondents answering “yes” for each yes/no question under the domain can be calculated, and then linearly transformed to a 1–4 possible range. For other questions under the domain, the item score of a nursing home can be calculated by averaging responses across all respondents of the facility.

Then the domain score for each facility is calculated as the average of the scores of all questions within the domain. The domain score ranges between 1 and 4 with higher value indicating better reported experience with care.

^a Excerpted from the 2008 Maryland nursing facility family survey: Statewide report, by Maryland Health Care Commission and Market Decisions, LLC, March 2009.

Appendix 2

Table 1.

Random-effects logistic risk adjustment model for the likelihood of pressure ulcers (≥stage 2) among high-risk residents (n=12464) in Maryland nursing homes, 2008

Resident risk factors	Prevalence (%) or Mean±SD	Estimates of random-effects logistic model
		Odds ratio	P-value
Age in years	80.28±13.60	0.99	0.003
Female gender	71.92	0.69	0.000
Nonwhite	35.23	1.02	0.829
Activities of daily living	22.88±4.34	1.20	0.000
Cognitive Performance Scale	3.27±1.91	0.90	0.000
Dementia	56.57	0.86	0.041
Stroke	26.15	0.76	0.000
Diabetes	34.48	1.37	0.000
Other endocrine disease	19.66	1.09	0.282
Cardiovascular disease	83.26	1.20	0.032
Musculoskeletal disease	46.38	1.15	0.036
Cancer	6.76	1.00	0.989
Bowel or bladder incontinence	89.13	0.77	0.019
Antipsychotic drug use	21.14	0.85	0.037
Hip fracture in last 180 days	1.44	2.89	0.000
End stage of life	2.65	1.99	0.000
C-statistic		0.76

Table 2.

Random-effects logistic risk adjustment models for the likelihoods of 1-year all-cause hospitalizations, 1-year potentially avoidable hospitalizations, and 1-year mortality among Maryland nursing home long-term residents (n=14013), 2008

Resident risk factors	Prevalence (%) or Mean±SD	All-cause hospitalizations		Potentially avoidable hospitalizations		Mortality
		Odds ratio	P-value	Odds ratio	P-value	Odds ratio	P-value
Age in years	83.93±8.21	1.00	0.478	1.00	0.273	1.04	0.000
Female gender	72.80	0.81	0.000	0.86	0.005	0.57	0.000
Nonwhite	28.72	1.06	0.265	0.96	0.473	0.74	0.000
Activities of daily living	18.24±7.54	1.03	0.000	1.02	0.000	1.06	0.000
Cognitive Performance Scale	2.92±1.80	0.90	0.000	0.93	0.000	1.03	0.061
Do-not-resuscitate order	53.39	0.69	0.000	0.72	0.000	1.18	0.000
Dementia	57.18	0.88	0.004	0.91	0.070	0.96	0.355
Stroke	20.99	1.03	0.557	1.01	0.927	0.87	0.004
Diabetes or other endocrine Disease	45.09	1.18	0.000	1.11	0.025	1.09	0.027
Cardiovascular disease	84.75	1.28	0.000	1.33	0.000	1.19	0.002
Cancer	7.54	1.04	0.568	1.09	0.321	1.42	0.000
Bladder incontinence	64.09	0.79	0.000	0.93	0.254	0.74	0.000
Bowel incontinence	54.17	1.07	0.247	0.95	0.408	1.25	0.000
Antipsychotic drug use	22.71	1.02	0.611	0.98	0.672	1.07	0.150
Hip fracture in last 180 days	5.10	0.96	0.624	0.93	0.502	0.84	0.051
End stage of life	1.33	0.44	0.000	0.43	0.003	2.56	0.000
C-statistic		0.66	0.63	0.73