Patient Satisfaction Associates with Dialysis Facility Quality and Star Ratings

Abstract

The Dialysis Facility Compare (DFC) Star Rating and the Quality Incentive Program (QIP) generate separate performance scores from clinical measures, and the In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH-CAHPS) survey evaluates patient satisfaction across 6 separate domains related to nephrologists, dialysis facility, and information transmission.

We examined the relationship of the three measures for US dialysis clinics -- modeling the 6 ICH-CAHPS as independent variables, and QIP and star ratings as dependent variables.

Among 3176 dialysis clinics, domains assessing dialysis facility and information transmission had a consistently stronger relationship with QIP and star ratings than the domains assessing nephrologists: QIP, β (95% CI) = 1.62 (1.26–1.97) for dialysis facility staff rating, 0.70 (0.35–1.05) for nephrologists; star rating, odd ratio (95% CI) = 1.38 (1.29–1.49) for dialysis facility staff rating, 1.17 (1.09–1.25) for nephrologists.

Patient satisfaction associates with dialysis care quality, with surprising differences among nephrologists, and dialysis facilities.

INTRODUCTION

Health care may grow to account for 20% of the gross domestic product by 2020 in the United States (US). Yet many countries that spend less than the US outperform the US in care quality and other outcomes. In recognition of these trends, payment reforms and quality improvement have become leading foci of national health policy. The end-stage renal disease (ESRD) program has been at the forefront of innovation in payment design and quality assessment because of its high cost, coverage of a well-defined population, and limited progress in improving survival.

The Prospective Payment System for dialysis debuted in 2011¹, followed in 2012 by the Quality Incentive Program (QIP)², the first mandatory federal pay-for-performance system. In 2014, the Centers for Medicare and Medicaid Services (CMS) introduced a Five-Star Quality Rating System to help inform patients and caregivers in choosing a dialysis clinic in the US³. Most recently, the In-Center Hemodialysis Consumer Assessment of Healthcare Provider and Systems (ICH-CAHPS)⁴ program directs each dialysis center to survey patients about the quality of their care experience within discrete domains that include nephrologists, dialysis facility, and transmission of information.

While these programs have helped us understand differing aspects of dialysis care quality, there is no evidence as to how the metrics are related. Particularly, it is not clear if the patients’ care experiences have any bearing on dialysis facilities’ clinical performance. Patients’ willingness to engage with staff and clinicians on treatment recommendations for vascular access, dialysis adequacy, and dietary compliance may be influenced by their perception of care provided by members of the care team. Ideally, patients’ adoption of recommendations leads to outcomes used to calculate a facility’s QIP score or star rating. Understanding the relationship of the ICH-CAHPS surveys to star ratings and QIP scores may help providers, organizations and policymakers develop strategies to improve dialysis care delivery.

We examined the relationship of ICH-CAHPS survey results with the QIP scores and star ratings for dialysis clinics in the US for calendar year 2016 in the primary analysis, and repeated the analyses using 2015 data to check consistency and robustness of the findings. We hypothesized that patient satisfaction, assessed through ICH-CAHPS, would be positively associated with QIP scores and star ratings. However, we did not know a priori whether one ICH-CAHPS domain would demonstrate a greater strength of association with star ratings or QIP than another.

METHODS

Data Source and Description

Data for the ICH-CAHPS, QIP, and star ratings are publically available through Dialysis Facility Compare (DFC), a website developed and maintained by CMS³. We used the latest versions of these datasets, year 2015 and 2016 data, uploaded on CMS website on July 12, 2017.

The ICH-CAHPS⁴ survey asks adults with ESRD receiving in-center hemodialysis about their care through a series of 40 questions about nephrologists, dialysis facility, and transmission of information that can be grouped into 6 discrete domains: 1) nephrologists communication and caring; 2) quality of dialysis center care and operations; 3) providing information to patients; 4) rating of nephrologists; 5) rating of dialysis center staff; and 6) rating of dialysis center. The aggregate data for individual facilities are presented as percentage in the top, middle, and lower box (i.e., trinomial) for all except domain 3, which shows percentage in top box and lower box (i.e., binomial).

The star ratings consist of eight individual clinical measures of dialysis quality³: 1) standardized mortality ratio; 2) standardized transfusion ratio; 3) standardized hospitalization ratio; 4) adequate waste removal for hemodialysis; 5) adequate waste removal for peritoneal dialysis; 6) hypercalcemia; 7) percentage fistula; and 8) percentage catheters used >90 days. A composite score, determined by the values of clinical measures of a clinic, is then used to calculate an ordinal star rating, with 1 indicating the lowest quality and 5 the highest.

The QIP has evolved and is currently a composite metric of a total of 11 measures – 8 clinical and 3 reporting.⁵ The eight clinical measures of dialysis quality are: 1) waste removal for adult hemodialysis; 2) waste removal for adult peritoneal dialysis; 3) waste removal for pediatric dialysis; 4) percentage of fistula usage; 5) percentage of catheters present greater than 90 days; 6) hypercalcemia; 7) rate of blood stream infections; and 8) percentage of patients with hemoglobin >12 g/dl. The three reporting measures are: 1) measures of anemia; 2) measures of bone and mineral metabolism; and 3) percentage of ICH-CAHPS survey completion.

The three datasets were merged and linked by a unique dialysis facility identifier. From a total of 7198 dialysis facilities around the country, 6004 clinics had star ratings, 6229 facilities had QIP score, and 3246 had results of CAHPS survey questions. After merging, 3176 facilities had complete data for ICH-CAHPS, QIP, and star ratings. We considered 2 dependent variables (i.e., star rating and QIP) and 6 independent variables, along with facility characteristics as adjusters.

The scores for the ICH-CAHPS domains were transformed into a quantitative score from those domains with 3 categories using the formula of weighting in our primary analysis: (Topbox*1 +Middlebox*0.5 +Lowerbox*0)⁶. Therefore, the numeric score may be interpreted as a probability of having positive response; for example, 100% selecting Middlebox is numerically equivalent to 50% selecting Topbox and 50% selecting Lowerbox. For the domain with 2 categories (providing information to patients), we used the formula: Topbox*1 +Lowerbox*0. Both transformations result in the final scores between 0 and 1 for all domains. For the information domain, the numeric score can be interpreted as the percentage/probability of answers belonging to Topbox.

Due to the possible subjectivity in weighting, we employed two additional strategies as sensitivity analyses: 1) dichotomizing into Topbox vs. Middlebox/Lowerbox and 2) weights of 1–2/3–1/3. Since 5 domains have the same scale (with 3 categories vs. Information domain in 2 categories), more valid comparisons would be achieved when we compare 5 domains, instead of 6.

Statistical analysis

We summarized data using standard descriptive statistics, e.g., mean and standard deviation (SD) for continuous variables and frequency and percentage for categorical variables. With the 6 ICH-CAHPS domains as independent variables in separate regression models, we calculated odds ratios (ORs) from an ordered logit model for star ratings (as ordinal outcome), and beta (β) coefficients from a linear regression model for QIP (as continuous outcome), with the associated 95% confidence intervals (CIs) and p-values in unadjusted and adjusted regressions. For the latter, we adjusted for facility-level variables such as regions (Northeast, Midwest, South, West based on ESRD networks), facility size (measured by number of in-center hemodialysis stations), profit status (profit vs. non-profit), chain status (owned by chain vs not), and offering home dialysis status (Y/N) and peritoneal dialysis (Y/N)^7,8. These variables were pre-specified and scientifically supported covariates in the ESRD literature.

We included facilities with complete data in ICH-CAHPS survey, star rating and QIP score, i.e., imputation was not employed. We summarized measured variables for facilities with reported data vs. those with unreported data. We did not adjust analyses for multiple comparisons and present all results to inform readers objectively; for adjustment, readers may use 0.004(=0.05/(2*6)) in the interpretation of p-values. We conducted two sets of secondary or sensitivity analyses in order to check the robustness of the findings. First, we adopted the two different scoring schemes for survey data (as outlined above). Second, we repeated the analyses using the 2015 data.

All statistical analyses were performed with SAS 9.4 (SAS Institute, Cary, NC).

RESULTS

Characteristics of the 3176 dialysis facilities are presented in Table 1, including profit and chain status, region, offering peritoneal and home dialysis, and number of stations. Over 90% of facilities were chain-owned and reported for-profit status. About 40% were in the South and approximately 20% from each of the other 3 regions. Over 60% had answers in top box (with SD in 7.9 to 12.8 for 6 domains), 3 stars was most common (42%) and average score of QIP was 68.5 (SD=9.6; range=21–97).

Table 1.

Selected Characteristics of Dialysis Clinics (N=3176)

Variable	% or mean (Standard Deviation)
Facilities offer in-center peritoneal dialysis
No	47.4
Yes	52.6
Facilities offer home hemodialysis training
No	70.7
Yes	29.3
Chain owned
No	6.8
Yes	93.2
Profit or Non-Profit
For profit	91.0
Not-for-profit	9.0
Number of stations	22.1 (7.7)
Region
Northeast	17.0
South	42.2
Midwest	19.1
West	21.7
ICH-CAHPS Domains: weighted score*/top box only
Patients- nephrologists’ communication and caring	74.0 (8.0)/66.1 (9.1)
Patients-quality of dialysis center care and operations	71.3 (7.0)/61.1 (7.9)
Patients-rating of the nephrologist	72.8 (9.3)/60.5 (11.7)
Patients-rating of the dialysis facility	76.6 (9.7)/65.5 (12.8)
Patients-rating of the dialysis center staff	74.4 (9.5)/61.4 (12.5)
Providing information to patients	78.9 (5.8)
Quality Measures
Star rating
1 star	2.3
2 stars	13.1
3 stars	42.1
4 stars	26.6
5 stars	15.9
QIP score	68.5 (9.6)

Facilities with complete data in CAHPS, star rating and QIP were included.

^*Weights of 0–0.5–1 were used for Lower-Middle-Top boxes, respectively.

ICH-CAHPS: In-Center Hemodialysis Consumer Assessment of Healthcare Provider and Systems;

QIP: Quality Incentive Program

Facilities excluded from the analysis because of incomplete data reported fewer stations on average, and were more likely to be non-profit and from the Midwest region (Table S1). There did not appear to be meaningful differences in the QIP or star ratings despite slightly lower values among excluded facilities; average QIP score of 68.5 vs. 68.4 and 3.4 vs. 3.3 stars between included vs. excluded subgroups.

Table 2 shows the relationship of the ICH-CAHPS domains to the QIP score. Overall, there was a positive association between ICH-CAHPS domains and QIP. Yet domains in ICH-CAHPS pertaining to the dialysis facility and transmission of information had greater strength of association than domains related to nephrologist; the beta coefficients for dialysis facility related domains were 2.12, 1.90, and 1.83 versus 1.04 and 0.80 for the nephrologist related domains. The same pattern of associations was observed for star ratings in terms of the magnitude of association and ordering; facility-related and information domains showed stronger associations (OR = 1.38–1.54), compared to nephrologists-related domains (OR = 1.17–1.29); see Table 3. Particularly, patients’ satisfaction with the quality of dialysis center care and operations (i.e., “patients-quality of dialysis center care and operations”) and with information provided (“providing information to patients”) showed the strongest associations, while patients’ satisfaction with the nephrologist (“patients-rating of the nephrologist”) had the weakest relation. All associations were statistically significant, with p<0.0001.

Table 2.

Association between CAHPS domains and QIP (N=3176)

CAHPS Domains	β (95% CI)	Adjusted β* (95% CI)
Nephrologist-Related Domains
Patients-nephrologists’ communication and caring	1.04 (0.63–1.46)	0.90 (0.49–1.30)
Patients-rating of the nephrologist	0.80 (0.44–1.16)	0.70 (0.35–1.05)
Facility-Related Domains
Patients-quality of dialysis center care and operations	2.12 (1.64–2.59)	1.90 (1.41–2.38)
Patients-rating of the dialysis facility	1.90 (1.56–2.34)	1.69 (1.35–2.04)
Patients-rating of the dialysis center staff	1.83 (1.48–2.17)	1.62 (1.26–1.97)
Information Domain
Providing information to patients	2.48 (1.92, 3.05)	2.24 (1.67–2.81)

CI: confidence interval.

Beta (β) is the increase in QIP when 10% point increases in each domain. Of note, the first 5 domains were based on 3 categories so we used weighted score (0–0.5–1), while the last domain was based on 2 categories (0–1); thus comparisons among 5 domains would be more valid.

All betas are different from 0 with p<0.0001.

^*In the adjusted model, we adjusted peritoneal dialysis, home dialysis, profit status, chain status, number of stations, and region.

Table 3.

Association between CAHPS domains and Star Ratings (N=3176)

CAHPS Domains	OR (95% CI)	Adjusted OR* (95% CI)
Nephrologist-Related Domains
Patients-nephrologists’ communication and caring	1.29 (1.19–1.39)	1.24 (1.14–1.34)
Patients-rating of the nephrologist	1.19 (1.11–1.28)	1.17 (1.09–1.25)
Facility-Related Domains
Patients-quality of dialysis center care and operations	1.54 (1.41–1.69)	1.43 (1.29–1.57)
Patients-rating of the dialysis facility	1.47 (1.38–1.58)	1.40 (1.30–1.50)
Patients-rating of the dialysis center staff	1.46 (1.36–1.56)	1.38 (1.29–1.49)
Information Domain
Providing information to patients	1.53 (1.37–1.71)	1.50 (1.34–1.68)

CI: confidence interval; OR: odds ratio.

OR is the increase in odds of higher vs. lower rating when 10% point increases in each domain. Of note, the first 5 domains were based on 3 categories so we used weighted score (0–0.5–1), while the last domain was based on 2 categories (0–1); thus comparisons among 5 domains would be more valid.

All ORs are different from 1 with p<0.0001.

^*In the adjusted model, we adjusted peritoneal dialysis, home dialysis, profit status, chain status, number of stations, and region.

Sensitivity analyses (Table S2) for ICH-CAHPS survey domains with QIP and star ratings showed similar trends with different scoring schemes. Results pertaining to facility-related domains and information consistently had higher point and interval estimates (that is, beta coefficients for QIP and ORs for stars) than nephrologist-related domains in unadjusted as well as adjusted models. When we repeated the analyses using the 2015 dataset, we observed the same pattern of findings in terms of the relative magnitude and order of effect sizes or the strength of association for the different domains with QIP and star ratings; see Table S3.

DISCUSSION

In this analysis of US hemodialysis clinics, we examined the association of three publically reported clinical care measures — ICH-CAHPS, QIP, and star ratings. We hypothesized and found that high patient satisfaction, assessed by ICH-CAHPS, was associated with high facility quality, measured separately by QIP scores and star ratings. We also found that there were differences in the strength of association for the different ICH-CAHPS domains with QIP and star ratings. The domains related to the dialysis facility and its staff had a greater strength of association than the domains related to nephrologists’ care. Also, there was a similar trend in the strength of association of the domain related to the transmission of information compared to care by nephrologists. The observed trends were consistent for two outcomes measures (QIP and star ratings) and in two different time points (years 2016 and 2015).

The Agency of Healthcare Research and Quality (AHRQ) began developing CAHPS in 1995 to standardize surveys that could be used to assess patient experience in different types of health care settings⁹. The first survey focused on ambulatory care delivery^10,11, followed by those for hospitals¹², behavioral health¹³, and nursing homes¹⁴. The ICH-CAHPS survey began development in 2002 and was put into practice in 2015 with little empirical testing^15–17.

To our knowledge, this is the first study to evaluate patient satisfaction with quality measures in dialysis. Thus, it adds to the body of existing evidence on the relationship between patient satisfaction and clinical outcomes in other areas of medicine. Perhaps best summarized in a recent systematic review, patient experience is positively associated with safety outcomes such as decubitus ulcers and iatrogenic infections, as well as clinical effectiveness outcomes such as use of preventative services, adherence, and reduction in mortality¹⁸. The association of patient satisfaction with hospitalizations and emergency service utilization is mixed¹⁹, while one contemporary study demonstrated that patients with high satisfaction assessed by Hospital-CAHPS had a 40% lower risk of 30-day readmission (a key new CMS quality benchmark) compared to patients with low satisfaction.²⁰

How could patients’ perception of care quality influence QIP or star ratings? One possibility is that high patient satisfaction may reflect patient trust in staff and providers. Consistent with research on shared decision making^21–23, this trust could lead to enhanced communication and information flow between patients, providers or staff, and thereby lead to effective diagnosis and treatment planning. Additionally, high satisfaction could promote adherence to providers’ recommendations on diet, vascular access, and treatment duration that are ultimately captured in the QIP and star ratings quality metrics. In fields outside of dialysis, studies have demonstrated that satisfied patients are more adherent to physicians’ recommendations.^24,25

We were surprised to find systematic and robust differences in the strength of association between the multiple domains of ICH-CAHPS and QIP and star ratings. Potential explanations for these differences include the following. One possibility may be related to the difference in staff number and time relative to the nephrologist and/or their extenders. With a typical prescription of three times per week for hemodialysis, nurses and patient care technicians see patients thrice weekly, and are directly involved in vascular access cannulation and the administration of oral medications. In contrast, the current management paradigm in the US is for nephrologists or their extenders to have face-to-face encounters with dialysis patients only once weekly. Thus, while the care directed by nephrologists is essential, dialysis staff including nurses and technicians may have a direct and more frequent impact on the daily experience of patients during the three days per week schedule of hemodialysis. Unfortunately, there has been relatively limited discussion about the importance and contribution of care provided by dialysis staff.

A second possible explanation for the observed differences may relate to the reimbursement structure for dialysis services between staff and providers. With the advent of Prospective Payment System linked to a dynamic QIP, dialysis clinics receive payment for services at a rate determined by outcomes. The clinic performance for a given calendar year determines CMS reimbursement for the next calendar; reimbursement is reduced if outcomes are sub-standard. In contrast, there are no direct incentives for more than 4 encounters per month for nephrologists and their extenders²⁶. There are data suggesting that increased frequency of visits by nephrologists is associated with improvement in 30 day readmission, vascular access interventions, and others outcomes.^27–29

A third possible explanation, while speculative, may be sociodemographic similarities between patient and staff. Dialysis clinic staff – nurses, dialysis care technicians, and others—tend to come from the same communities as patients. This could facilitate communication and trust between them to a greater extent than between patients and nephrologist.

How could members of the dialysis care team—individual providers, clinic staff, large and independent dialysis organizations—use the findings of the study? Indeed, an overarching goal of public reporting has been to directly and indirectly encourage providers to develop care strategies and infrastructure that can improve patient outcomes and reduce costs. Programs like ICH-CAHPS are based on the assumptions that information on patient’s care experience is not only an important quality indicator but also can help providers and organizations improve services. Our findings suggest that large dialysis organizations could invest in infrastructure, training, and/or personnel to increase staff communication skills given the observed importance of transmission of information. Additionally, CMS may consider revising reimbursement methods to increase the frequency of patient encounters per month. Finally, for developers of survey instruments, the search for questions with high discrimination (for good vs. not good performers), high response rates and actual use by patients/consumers is always desired. Historically, surveys report that fewer than one in ten patients on dialysis actually have visited the DFC website, and thus the majority of patients are currently unlikely to use quality information in decision-making.³⁰

The limitations of this study should be noted. First, we conducted analyses on a relatively small number of variables without patient-level or area-level data in the cross-sectional manner. Yet our goal and approach were to evaluate a policy model that is already implemented via publically shared data by policy makers. Second, data were missing for many of the dialysis clinics-- a surprising finding since reporting is a requirement. This may be partially explained by the program being relatively new (since 2015) in the hemodialysis field, and the overall low response for surveys likes CAHPS observed elsewhere.³¹ When comparing the clinics that reported the information with clinics that did not, there was a higher frequency of small, independent and not-for-profit clinics among those with missing data. Importantly, the average QIP and star ratings did not differ between these two categories of clinics in a meaningful way. We speculated that the overrepresentation of small, independent and not-for-profit centers in our study sample may result in underestimation of the strength of association between the I-CAHPS domains with QIP and Star Ratings. Our speculation is based on the possibility that small, independent and not-for-profit centers are more likely to treat uninsured and socio-economically disadvantaged patients. Those patients may be more resource-constrained and less able to adhere to diet, binders, and seek the placement of vascular access. Third, our study addresses numerical association, not causation, which is a common limitation of observational studies.

In conclusion, the patient reported experience may be an informative and effective way to assess the quality of dialysis care. We found that patient satisfaction is directly associated with dialysis facility quality, whether assessed by QIP or star ratings. Yet there were consistent differences in this relationship according to patients’ satisfaction with nephrologists, or dialysis facility and information. Physicians, dialysis organizations, and payers should consider investment in refining patient reported measures that encourage use of information to improve care delivery and clinical outcomes for patients.

Figure 1:

CAHPS domains:

1: Patients-nephrologists’ communication and caring

2: Patients-rating of the nephrologist

3: Patients-quality of dialysis center care and operations

4: Patients-rating of the dialysis facility

5: Patients-rating of the dialysis center staff

6: Providing information to patients

Beta and odds ratio were adjusted peritoneal dialysis, home dialysis, profit status, chain status, number of stations, and region.

See footnotes in Tables 2 and 3.