Abstract
Background
The U.S. Centers for Medicare & Medicaid Services (CMS) assesses patient experiences of care as part of the End Stage Renal Disease (ESRD) Prospective Payment System and Quality Incentive Program. This article describes the development and evaluation of the Consumer Assessment of Healthcare Providers and Systems In-Center Hemodialysis survey (CAHPS® ICH survey).
Study Design
We conducted formative research to generate survey questions and conducted statistical analyses of survey responses to evaluate the survey’s measurement properties.
Setting and Participants
Formative research included 5 focus groups (2 with hemodialysis patients, 2 with caregivers, 1 with nephrologists) and 56 cognitive interviews with dialysis patients. We collected field test responses to the survey from 1454 dialysis patients receiving care at 32 facilities.
Measurements & Outcomes
We assessed the CAHPS ICH Survey.
Results
Response rate was 46%. Analyses support 3 multi-item scales: Nephrologists’ Communication and Caring (7 items, Cronbach’s alpha = 0.89); Quality of Dialysis Center Care and Operations (22 items, alpha = 0.93); and, Providing Information to Patients (11 items, alpha = 0.75). The communication scale was the most strongly correlated with the global rating of the ‘kidney doctor’ (r = 0.78). The Dialysis Center Care and Operations scale was most strongly correlated with the global ratings of staff (r = 0.75) and of the center (r = 0.69). Providing Information to Patients was most strongly correlated with the global rating of the staff (r=0.41).
Limitations
Males and younger patients were over-represented in the field test compared to the general U.S. population of dialysis patients. A relatively small number of patients completed the survey in Spanish.
Conclusions
This study provides support for the reliability and validity of the CAHPS ICH survey for assessing ESRD patient experiences of care at dialysis facilities. The survey can be used to compare care provided at different facilities.
Background
End-Stage Renal Disease (ESRD) is a complex condition, but with proper treatment patients can live long and productive lives. The preferred treatment for appropriate patients is a kidney transplant, but patients can be maintained for years with kidney dialysis.1-6 The 2013 USRDS Annual Data Report indicates that 430,273 ESRD patients in the U.S. were being treated with some form of kidney dialysis at the end of 2011. Multiple types of dialysis including peritoneal dialysis (PD) and hemodialysis (HD) are available, but more than 10 times as many ESRD patients receive in-center hemodialysis (ICH) treatment compared with PD and other forms of HD, such as home HD combined.7,8
The Consumer Assessment of Healthcare Providers and Systems (CAHPS®) surveys assess health care experiences.9 CAHPS surveys focus on aspects of quality of care that patients have identified as important and for which they are the best or only source of information. CAHPS surveys are available for ambulatory, inpatient and home-based care. The surveys include questions about specific experiences with care and ratings of the care received (more information is available at https://www.cahps.ahrq.gov/).
In 2002, the Centers for Medicare & Medicaid Services (CMS) in concert with the Agency for Healthcare Research and Quality (AHRQ) supported the development of a CAHPS survey to assess the experiences of care provided to patients at ICH facilities, known as the CAHPS ICH Survey. A recent study of 404 patients from 76 dialysis centers across the U.S. found support for the internal consistency reliability of 2 of the 3 multi-item scales, but the reliability for the composite on providing information to patients was suboptimal (alpha = 0.55).10 This article describes the development of the CAHPS ICH Survey and the results of a field test.
Methods
To develop and evaluate the survey, we conducted a literature review, focus groups, cognitive testing, a field test, and psychometric analyses of the field test data. At each step in the survey development process, we solicited the input of experts and representatives of groups likely to use the survey.
Identifying Domains of Interest
Literature review
Databases used for the literature search included MEDLINE®, CINAHL®, the Combined Health Information Database (CHID), Gale Group Health & WellnessSM, Current Index to Statistics, PsychFirst®, and WesInfo. In addition, we searched the Internet for organizations and services related to ESRD. Key words used for the search included: end-stage renal disease, hemodialysis, kidney failure, chronic kidney failure, outcome assessment, evaluation studies, questionnaires, health surveys, case management, patient satisfaction, quality of health care, patient participation, patient education, health care evaluation mechanisms, information dissemination, evaluation surveys, research design, relative value scales, and research instruments. Our search identified 10 important aspects of dialysis care (Table 1). Below we describe additional steps taken to help identify important topics or domains and questions.
Table 1. Important Aspects of Dialysis Care.
| # | Aspect of Dialysis Care |
|---|---|
| 1. | Communication with and education of patients |
| 2. | Concern and helpfulness of staff |
| 3. | Patient involvement in care |
| 4. | Care coordination |
| 5. | Patient perception of staff proficiency |
| 6. | Interpersonal relationships in the dialysis setting |
| 7. | Patient safety |
| 8. | Facility amenities and environment |
| 9. | Access and convenience of care |
| 10. | Handling of grievances and complaints |
Note: The table lists the 10 most important aspects of dialysis care identified through the literature review.
Call for Measures
We published a notice in the Federal Register about our intentions to develop the CAHPS ICH survey and inviting developers of extant measures to contribute their content for consideration. Eleven surveys were submitted in response to the notice and were added to two renal-specific surveys identified in the literature review. For each survey we compiled information about reliability and validity; breadth and magnitude of use (both in terms of the size and diversity of populations in which a survey had been implemented); number of years in use; and qualitative work done to develop the instrument. We prepared summaries for each instrument noting the number of items, response categories, question wording, topics covered, and evidence of reliability and validity.
We reviewed and sorted over 600 items from the 13 non-CAHPS surveys into the 10 domains identified in the literature review. We assessed each of these items for possible inclusion in the survey, determining which ones best captured critical issues unique to the dialysis population, as well as which items from existing CAHPS surveys might be applicable. The review process included two CMS renal clinicians who provided input into dimensions of dialysis care and experiences that dialysis patients can observe and reliably report upon --a key CAHPS survey design principle. Candidate items were reviewed in an iterative process and when necessary, item wording was modified according to principles of survey research and knowledge gained from prior CAHPS survey development and testing.
Focus groups
We conducted two focus groups with renal hemodialysis patients (7 male and 11 female), two focus groups with caregivers of hemodialysis patients (8 male and 13 female), and one focus group with 10 nephrologists. The patient and caregiver focus group participants were convenience samples drawn from the Palo Alto, CA and Raleigh-Durham, NC areas.
Focus groups to collect information on ESRD patient experiences with care were conducted using a scripted discussion guide and were recorded and transcribed. Topics covered included: patient/caregiver experiences completing surveys about dialysis; attribution of responsibility for care at the dialysis center; treatment options and quality of care; environment at the dialysis center; options and choice in the selection of a dialysis center; and, interest in patient reports of care at the dialysis center level.
Findings from the focus groups confirmed the most important domains to assess in the survey. Participants confirmed they would like access to patient experience information when selecting a dialysis facility for their own care. Some reported having experience completing surveys related to survey data being kept confidential. Nephrologists in other focus groups indicated that survey results should not be reported to the public because of concerns about data quality and patient confidentiality.
Draft Survey
We used information from the literature review, call for measures, focus groups and the Technical Expert Panel (TEP) to draft 75 questions for the field test survey.
Cognitive Interviews
Cognitive interviewing was used to evaluate the comprehensibility, respondent burden and content validity (e.g. relevance, understanding according to intended meaning) of the draft survey items.11,12 We conducted cognitive interviews with a total of 56 (40 English-language, 16 Spanish-language) dialysis patients (36% female, mean age = 52) recruited from ICH facilities in Raleigh-Durham, NC; and, Palo Alto and Los Angeles, CA.
The first round of cognitive testing indicated that patients often had more than one “kidney doctor” and found questions asking about a single one confusing; that most had difficulties estimating the amount of time they had been receiving hemodialysis; and, that trying to determine if patients had been informed of their treatment options (e.g., transplant) was difficult given the fact that most patients knew they were only eligible for dialysis.
Using the results of the first round of cognitive interviews and comments received in response to the Federal Register notice, we revised the survey and conducted a second round of cognitive interviews. We revised the survey further, and translated it to Spanish using two parallel forward translations reviewed by a bilingual committee, and prepared it for cognitive testing with Spanish-speaking dialysis patients. Findings were used to further refine the survey.
Stakeholder Input
We recruited a 19-member Technical Expert Panel (TEP) composed of nephrologists and other renal clinicians, ESRD Network executives, facility administrators and dialysis center staff, patients, patient advocates, public policy representatives, and researchers. We asked TEP members to review the draft survey, and presented information about the survey development and testing process at a special ESRD Stakeholders Meeting convened by CMS. Network executives, nephrologists, facility administrators and staff provided additional feedback on the draft survey during that meeting and in subsequent teleconferences.
Field Test Survey
We revised the survey to address problems identified through the cognitive interviews and feedback provided by TEP members. The resulting survey included 67 questions: 41 questions about care experiences (report questions); 10 questions to identify appropriate respondents for subsequent questions (screener questions): 3 global questions about kidney doctors, dialysis center staff, and the dialysis center, respectively (rating questions); and 13 questions to describe respondents. Questions about care experiences referred to patient experiences with their kidney doctor (7 items), experiences with dialysis center staff (24 items), the hemodialysis center (3 items), and treatment options (7 items).
Field Test
We conducted a field test to evaluate the reliability and validity of the CAHPS ICH survey measures, identify questions appropriate for public reporting and/or for internal quality improvement, evaluate survey data collection modes, and to identify questions on respondent characteristics that could be used to adjust for differences in the types or mix of patients receiving care at each center, when reporting survey results (case-mix adjusters).
We selected a sample of 30 geographically representative dialysis facilities from a list of those with at least 5 adult cases annually. Facility for testing were selected based on geographic location (Northeast, South, Midwest, West; and, rural versus urban), size (10-24 patients, 25-59 patients, 60-119 patients, and 120 patients or more), facility type (part of a large dialysis organization, hospital-based, non-hospital-based), and the racial/ethnic mix of patients to attempt to include a sufficient number of facilities with large numbers of Hispanic Spanish-speaking patients. Facilities were sampled from each region proportionate to the number of facilities: 11 I the Midwest, 9 in the South, 5 in the Northeast, and 5 in the West. Two facilities participating in a Quality Improvement project volunteered to participate in the field test and were later added to the original sample, bringing the total sample to 32 facilities.
Patients were eligible for the survey if they had received hemodialysis during 3 months or more at a selected facility. For facilities with up to 200 patients, a census of all patients were included in the sample; while for others, a systematic sample of 200 was drawn.
Two data collection modes (randomly assigned to patients) were used: telephone only and a mailed survey (two mailings plus a reminder) followed by a telephone call to non-respondents.13,14 Since vision problems, fatigue, literacy and cognitive difficulties are common in this patient population,15 we anticipated a response rate of about 40% based on the results of other surveys of patients on dialysis.
Data Analysis
We examined survey response rates16,17 and item distributions including ceiling and floor effects that can occur when people select response options at the extreme ends of the scales. We also assessed whether reliable and valid multi-item scales could be constructed.18 We evaluated the hypothesized multi-item scales using confirmatory factor analysis (CFA)19-23 to examine if responses to survey questions grouped together as predicted. To assess the hypothesized factor structure, we also examined goodness-of-fit results from a confirmatory factor analysis (CFA) using the SAS/STAT® CALIS procedure. We examined three indices of fit: the root mean square error of approximation (RMSEA),24 which describes how well the model fits the population covariance matrix; the normed-fit index (NFI), which compares the hypothesized model to a ‘worst case scenario’ model where all composite items are uncorrelated; and the comparative fit index,25 which is a variant of the NFI that takes into account sample size and performs well even with small samples. We followed common practice in evaluating model fit: CFI and NNFI greater than 0.95; and RMSEA to be less than 0.06; with a strict upper limit of 0.07.
We next conducted exploratory factor analysis (EFA) to identify alternative composites. The number of factors was determined by Guttmann’s weakest lower bound26 (number of factors with eigenvalues greater than one) in conjunction with a scree plot of the eigenvalues27 and examining the pattern of factor loadings upon rotation for simple structure10 (i.e., assessing the degree to which the number of factors extracted based on the first two criteria suggested a composite structure that was conceptually interpretable).
Multiple imputation (MI in SAS/STAT® version 9.2) was used to impute data missing due to structured item non-response from patients completing the survey (five imputations were generated). This procedure calculates the maximum-likelihood estimates of the covariance matrix under the missing-at-random (MAR) assumption.28,29
We also examined the correlation (corrected for item overlap) of each item with the composite or scale it was hypothesized to belong to and its correlations with the other scales. We estimated internal consistency reliability using Cronbach’s alpha.25 Since CAHPS ICH survey data are intended to be reported at the level of each dialysis center rather than the patient level, we also estimated the reliability of the items and composites by partitioning between center-level versus within-center variance in a one-way ANOVA.27 Finally, we computed the percentage of respondents at the highest- and lowest-possible composite scores.
Results
A total of 1,454 field test surveys were completed by patients receiving hemodialysis at centers (46% response rate). Among respondents, 56% completed the survey by telephone while 44% completed the survey by mail. Eighty-eight percent reported that English is the main language they speak at home and 93% completed the survey in English; 7% completed the survey in Spanish. Demographic characteristics of field test respondents are provided in Table 2. Demographic characteristics were not available for those who did not respond to the survey. The number of respondents per facility ranged from 3 to 85 patients, with an average of 45 and a median of 48. The facility with only 3 respondents was dropped from the estimate of facility level reliability.
Table 2. Demographic Characteristics of the ICH-CAHPS Field Test Participants (n = 1,454).
| Question | Response | % |
|---|---|---|
| Self-Rated Overall Health | Excellent | 18 |
| Very Good | 25 | |
| Good | 34 | |
| Fair | 18 | |
| Poor | 4 | |
| Missing/Don’t Know/Refused | 0.8 | |
| Highest Level of Completed Education | 8th grade or less | 14 |
| Some high school | 20 | |
| High school graduate or GED | 34 | |
| Some college or 2-year degree | 22 | |
| 4-year college graduate | 5 | |
| Missing/Don’t Know/Refused | 0.8 | |
| Race | American Indian/Alaska Native | 2 |
| Asian | 3 | |
| Black or African American | 36 | |
| Native Hawaiian/Other Pacific Islander |
0.6 | |
| White | 45 | |
| Two or more races | 7 | |
| Missing/Don’t Know/Refused | 7 | |
| Ethnicity | Non-Hispanic or Latino | 82 |
| Hispanic or Latino | 13 | |
| Missing/Don’t Know/Refused | 5 | |
| Age Category | 18 to 24 years | 0.6 |
| 25 to 34 years | 3 | |
| 35 to 44 years | 8 | |
| 45 to 54 years | 16 | |
| 55 to 64 years | 24 | |
| 65 to 74 | 25 | |
| 75 and older | 22 | |
| Missing/Don’t Know/Refused | 0.3 | |
| Gender | Male | 51 |
| Female | 49 |
Nine hypothesized multi-item scales were evaluated: Kidney Doctor Communication (7 items); Staff Communication (8 items); Privacy (2 items); Complaints (3 items); Staff Professionalism (6 items); Patient Involvement (5 items); Patient Education (3 items); Patient Safety (5 items); and Patient Rights (2 items). The CFA showed a poor fit between the data and this structure. Seven of the nine composites contained items that correlated more highly with another composite than with their own meaning that these composites did not measure distinct phenomenae according to patients’ experiences. The analysis also revealed that almost no one said that they had made a complaint (Q53), “In the last 12 months, did you make a complaint to any…agencies?” Thus, this item was excluded from subsequent analyses.
The EFA criteria suggested 3 or 4 underlying dimensions. Based on simple structure, we determined that three scales were preferred: Nephrologists’ Communication and Caring (6 items); Dialysis Center Care and Operations (17 items); and, Providing Information to Patients (9 items).
Item-total corrected correlations for Nephrologists’ Communication were all above 0.40 (see Table 3). Eleven of the 17 item-total correlations for Dialysis Facility Care and Operations were above 0.40; 5 of 9 item-total correlations for Providing Information to Patients were above 0.40. Internal consistency reliability estimates (alphas) for the three scales ranged from 0.75 to 0.93 (Table 3). Alphas by age (18-44 years vs. 45 and older), education (high school diploma or less vs. more than high school), and gender (male vs. female) subgroups were similar (the largest difference was 0.04 between two education subgroups for Providing Information to Patients composite). Alphas were estimated separately for survey responses in Spanish and were similar for the Nephrologist Communication (0.84) and Quality of Dialysis Center Care and Operations (0.89) composites; and lower for the Providing Information to Patients (0.58) composite.
Table 3. Item-Scale Correlations and Reliability Estimates for Three Multi-item Scales.
| Q # |
Nephrologists’ Communication and Caring | Alpha =0.891 |
Center-Level Reliability=0.772 |
|---|---|---|---|
| q3 | Doctor listens carefully | 0.78 | 0.73 |
| q4 | Doctor explains things | 0.74 | 0.64 |
| q5 | Doctor shows respect | 0.76 | 0.69 |
| q6 | Doctor spends enough time | 0.75 | 0.76 |
| q7 | Doctor cared about you | 0.78 | 0.74 |
| q9 | Doctor seemed informed | 0.56 | 0.89 |
| Q# | Quality of Dialysis Center Care and Operations | Alpha =0.93 | Center-Level Reliability=0.84 |
|---|---|---|---|
| q10 | Staff listen carefully | 0.77 | 0.67 |
| q11 | Staff explain in a way that is easy to understand | 0.74 | 0.64 |
| q12 | Staff show respect | 0.77 | 0.73 |
| q13 | Staff spend enough time | 0.76 | 0.72 |
| q14 | Staff cared about you | 0.79 | 0.95 |
| q15 | Staff makes you comfortable | 0.75 | 0.79 |
| q16 | Staff keep information private | 0.37 | 0.71 |
| q17 | Comfortable asking staff | 0.49 | 0.77 |
| q21 | Staff insert needle w/o pain | 0.44 | 0.55 |
| q22 | Staff check you closely | 0.70 | 0.72 |
| q24 | Staff manage problems | 0.70 | 0.23 |
| q25 | Staff professional | 0.72 | 0.75 |
| q26 | Staff discuss diet | 0.37 | 0.83 |
| q27 | Staff explain tests | 0.56 | 0.73 |
| q33 | On Machine w/in 15 minutes | 0.46 | 0.90 |
| q34 | Center Clean | 0.50 | 0.82 |
| q43 | Satisfied w/way Problems Handled | 0.71 | 0.32 |
| Q# | Providing Information to Patients | Alpha =0.75 | Center-Level Reliability=0.79 |
|---|---|---|---|
| q19 | Know how to care of access site | 0.33 | 0.42 |
| q28 | Staff give info on patient rights | 0.39 | 0.88 |
| q29 | Staff review patient rights | 0.46 | 0.70 |
| q30 | Staff told you what to do if health problem at home | 0.44 | 0.91 |
| q31 | Staff told you how to get off machine if emergency | 0.23 | 0.92 |
| q36 | Doctor/staff talk about which treatment is right for you |
0.56 | 0.60 |
| q38 | Doctor/staff explain why not eligible for transplant | 0.46 | 0.43 |
| q39 | Doctor/staff talk About Peritoneal | 0.33 | 0.75 |
| q40 | Involved in choosing treatment | 0.45 | 0.89 |
Coefficient alpha for the scales and item-total correlations (corrected for overlap) for the items presented in this column.
Center-level reliability of the composites was 0.77 for Kidney Doctor Communication (average n per center = 47), 0.84 for Dialysis Facility Care and Operations (average n per center = 47), and 0.79 for Patient Empowerment (average n per center = 47). Thirty percent of respondents’ doctors received the highest score on the Nephrologists’ Communication and Caring scale, but less than 10% were at the ceiling on the other two composites. The scales were very strongly intercorrelated (0.77 to 0.84).
Table 4 shows correlations between the global ratings and composites. The communication scale was the most strongly correlated with the global rating of the kidney doctor (r = 0.78). The dialysis facility care and operations scale was most strongly correlated with global rating of the staff (r = 0.75) and global rating of the center (r = 0.69). In a supplemental analysis of correlations of the items in the scales with the global rating items we found that patients’ responses to the question asking, “how often did you feel that the kidney doctors really cared for you as a person?” most strongly correlated with the global rating of nephrologists (r = 0.72), and that responses to the question asking “how often did you feel that dialysis center staff really cared for you as a person?” was most strongly correlated with the global ratings of dialysis center staff (r = 0.65), and of the dialysis center (r = 0.58).
Table 4. Product-Moment Correlations Between Scales and Global Rating Items.
| Scale | Dr Rating | Staff Rating |
Center Rating |
|---|---|---|---|
| Nephrologists’ Communication and Caring |
0.78 | 0.47 | 0.46 |
| Quality of Dialysis Center Care and Operations |
0.51 | 0.75 | 0.69 |
| Providing Information to Patients |
0.36 | 0.41 | 0.36 |
n = 1,451. All correlations significant at p < .001
Discussion
Results of the psychometric analyses strongly support the internal consistency reliability and validity of the CAHPS ICH Survey scales and show they can be used to discriminate variation in quality of care among dialysis facilities. Scales regarding care at dialysis facilities presented reliabilities exceeding 0.75. Scale scores were strongly related to patients’ global ratings of nephrologists, dialysis center staff and the center (correlations ranged from 0.36 to 0.78), providing evidence of construct validity and the pattern of correlations of global ratings with composite scores further support their validity.
The final version of the CAHPS In-Center Hemodialysis Survey is publicly available in both English and Spanish at https://ichcahps.org. The survey includes 58 core questions or items and 20 supplemental items that can be used to generate two types of results for reporting purposes: global ratings scaled from 0 to 10 to measure respondents’ assessment of their nephrologists, dialysis center staff, and the dialysis center where they receive care as well as three multi-item scales: (1) nephrologists’ communication and caring; (2) quality of dialysis center care and operations; and (3) providing information to patients (Table 5).26
Table 5. Reporting Measures for the CAHPS® In-Center Hemodialysis Survey.
| Nephrologists’ Communication and Caring (baseline 2005 question number if different) | Response Format | |
|---|---|---|
| Q3 | In the last 3 months, how often did your kidney doctors listen carefully to you? | Never Sometimes Usually Always |
| Q4 | In the last 3 months, how often did your kidney doctors explain things in a way that was easy to understand? | |
| Q5 | In the last 3 months, how often did your kidney doctors show respect for what you had to say? | |
| Q6 | In the last 3 months, how often did your kidney doctors spend enough time with you? | |
| Q7 | In the last 3 months, how often did you feel your kidney doctors really cared about you as a person? | |
| Q9 (Q8) | Do your kidney doctors seem informed and up-to-date about the health care you receive from other doctors? | Yes/No |
| Quality of Dialysis Center Care and Operations | Response Format | |
|---|---|---|
| Q10 (Q11) | In the last 3 months, how often did the dialysis center staff listen carefully to you? | Never Sometimes Usually Always |
| Q11 (Q12) | In the last 3 months, how often did the dialysis center staff explain things in a way that was easy to understand? | |
| Q12 (Q13) | In the last 3 months, how often did the dialysis center staff show respect for what you had to say? | |
| Q13 (Q14) | In the last 3 months, how often did the dialysis center staff spend enough time with you? | |
| Q14 (Q15) | In the last 3 months, how often did you feel the dialysis center staff really cared about you as a person? | |
| Q15 (Q16) | In the last 3 months, how often did the dialysis center staff make you as comfortable as possible during dialysis? | |
| Q16 (Q17) | In the last 3 months, did dialysis center staff keep information about you and your health as private as possible from other patients? |
|
| Q17 (Q20) | In the last 3 months, did you feel comfortable asking the dialysis center staff everything you wanted about dialysis care? |
|
| Q21 (Q24) | In the last 3 months, how often did dialysis center staff insert your needles with as little pain as possible? | |
| Q22 (Q25) | In the last 3 months, how often did dialysis center staff check you as closely as you wanted while you were on the dialysis machine? |
|
| Q24 (Q28) | In the last 3 months, how often was the dialysis center staff able to manage problems during your dialysis? | |
| Q25 (Q29) | In the last 3 months, how often did dialysis center staff behave in a professional manner? | |
| Q26 (Q32) | Please remember that for these questions, dialysis center staff does not include doctors. Dialysis center staff means nurses, technicians, dietitians and social workers at this dialysis center. In the last 3 months, did dialysis center staff talk to you about what you should eat and drink? |
Yes/No |
| Q27 (Q31) | In the last 3 months, how often did dialysis center staff explain blood test results in a way that was easy to understand? |
Never Sometimes Usually Always |
| Q33 (Q40) | In the last 3 months, when you arrived on time, how often did you get put on the dialysis machine within 15 minutes of your appointment or shift time? |
|
| Q34 (Q42) | In the last 3 months, how often was the dialysis center as clean as it could be? | |
| Q43 (Q51) | In the last 12 months, how often were you satisfied with the way they handled these problems? | |
| Providing Information to Patients | Response Format | |
|---|---|---|
| Q19 (Q22) | The dialysis center staff can connect you to the dialysis machine through a graft, fistula, or catheter. Do you know how to take care of your graft, fistula or catheter? |
Yes/No |
| Q28 (Q33) | As a patient you have certain rights. For example, you have the right to be treated with respect and the right to privacy. Did this dialysis center ever give you any written information about your rights as a patient? |
|
| Q29 (Q34) | Did dialysis center staff at this center ever review your rights as a patient with you? | |
| Q30 (Q35) | Has dialysis center staff ever told you what to do if you experience a health problem at home? | |
| Q31 (Q36) | Has any dialysis center staff ever told you how to get off the machine if there is an emergency at the center? | |
| Q36 (Q44) | You can treat kidney disease with dialysis, kidney transplant or with dialysis at home. In the last 12 months, did either your kidney doctors or dialysis center staff talk to you as much as you wanted about which treatment is right for you? |
|
| Q38 (Q46) | In the last 12 months, has either a doctor or dialysis center staff explained to you why you are not eligible for a kidney transplant? |
|
| Q39 (Q47) | In the last 12 months, did either your kidney doctors or dialysis center staff talk to you about peritoneal dialysis? | |
| Q40 (Q48) | In the last 12 months, were you as involved as much as you wanted in choosing the treatment for kidney disease that is right for you? |
|
| Global Ratings | Response Format | |
|---|---|---|
| Q8 (Q9) | Using any number from 0 to 10 where 0 is the worst kidney doctors possible and 10 is the best kidney doctors possible, what number would you use to rate the kidney doctors you have now? |
0-10 |
| Q32 (Q39) | Using any number from 0 to 10 where 0 is the worst dialysis center staff possible and 10 is the best dialysis center staff possible, what number would you use to rate your dialysis center staff? |
|
| Q35 (Q43) | Using any number from 0 to 10 where 0 is the worst dialysis center possible and 10 is the best dialysis center possible, what number would you use to rate your dialysis center? |
|
In 2011, CMS announced that use of this survey would be required as part of the value-based purchasing program for payments under the Medicare program. Since 2012, Medicare-certified ICH facilities have been required to administer the CAHPS ICH Survey to receive the annual payment update (APU) for 2014 and 2015. Facilities are required to attest they have conducted the survey in 2012 and 2013. In 2014, CMS is moving toward national implementation of the survey through approved survey vendors working under contract with Medicare-certified dialysis facilities, and will require survey data to be reported to CMS. Results from the national implementation of the ICH CAHPS Survey will be used by CMS to monitor performance of Medicare-certified dialysis facilities and to determine compliance with quality reporting requirements under the Quality Incentive Program (QIP). After national implementation of the survey, CMS will begin to publicly report survey results on the Dialysis Facility Compare link on the Medicare.gov Web site. Public reporting of survey results will be useful to patients receiving in-center hemodialysis, family members, and others interested in obtaining information to make more informed decisions when choosing a facility.
A limitation of the study is that males, younger patients and African Americans were over-represented among survey respondents compared to their numbers in the general US population of dialysis patients.9 Another limitation is that we had a relatively small number of patients who completed the survey in Spanish, a factor which may be related to the low internal consistency reliability for this subgroup on the composite measure regarding “providing information to patients”. Yet another limitation is that demographic characteristics were not available for those that did not respond to the survey; therefore non-response bias could not be estimated. Finally, patient’s perceived quality of care may not reflect delivery of CMS dialysis quality care indicators. Despite these limitations, our analyses indicate that the CAHPS ICH survey can be used to gather information from patients at the facility level that will allow CMS, dialysis networks, dialysis organizations, and individual facilities to compare patient experiences of care both within and across facilities. In addition, the information from the survey could be used to identify best practices that can be adopted or implemented in other hemodialysis facilities to improve quality of care for hemodialysis patients. However, further evaluation of the survey is needed among those not well represented in the study reported here to ensure that the survey works adequately with these patient populations.
Acknowledgments
Preparation of this manuscript was supported through cooperative agreements from the Agency for Healthcare Research and Quality (U18 HS09204 and U18 HS016980) and the Centers for Medicare and Medicaid Services. Ron D. Hays was supported in part by grants from the NIA (P30-AG021684) and the NIMHD (P20MD000182). The authors thank the members of the Technical Expert Panel, stakeholders, key informants, dialysis networks, and dialysis organizations and facilities that participated in the study, for their advice, collaboration, and support of this study. We also thank the respondents to the field test survey for sharing their experiences with in-center hemodialysis. Finally, we thank Keith Norris, M.D., and the anonymous reviewers of the paper for their suggestions.
Contributor Information
Beverly A. Weidmer, RAND Corporation
Paul D. Cleary, Yale School of Public Health
San Keller, American Institutes for Research
Christian Evensen, American Institutes for Research
Margarita P. Hurtado, American Institutes for Research
Beth Kosiak, Agency for Healthcare Research and Quality
Patricia M. Gallagher, University of Massachusetts
Roger Levine, American Institutes for Research
Ron D. Hays, UCLA Department of Medicine
References
- 1.Evans RW, Manninen DL, Garrison LP, et al. The Quality of Life of Patients with End-Stage Renal-Disease. New Engl J Med. 1985;312(9):553–559. doi: 10.1056/NEJM198502283120905. [DOI] [PubMed] [Google Scholar]
- 2.Himmelfarb J, Ikizler TA. Hemodialysis. N Engl J Med. 2010 Nov 4;363(19):1833–1845. doi: 10.1056/NEJMra0902710. [DOI] [PubMed] [Google Scholar]
- 3.Johnson D. CKD Screening and Management. In: Daugirdas J, editor. Handbook of Chronic Kidney Disease Management. Lippincott Williams and Wilkins; Philadelphia, PA: 2011. pp. 32–43. [Google Scholar]
- 4.Levin A, Hemmelgarn B, Culleton B, et al. Guidelines for the management of chronic kidney disease. CMAJ. 2008 Nov 18;179(11):1154–1162. doi: 10.1503/cmaj.080351. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Sarnak MJ, Levey AS, Schoolwerth AC, et al. Kidney disease as a risk factor for development of cardiovascular disease - A statement from the American Heart Association councils on kidney in cardiovascular disease, high blood pressure research, clinical cardiology, and epidemiology and prevention. Circulation. 2003 Oct 28;108(17):2154–2169. doi: 10.1161/01.CIR.0000095676.90936.80. [DOI] [PubMed] [Google Scholar]
- 6.Tonelli M, Wiebe N, Culleton B, et al. Chronic kidney disease and mortality risk: a systematic review. J Am Soc Nephrol. 2006 Jul;17(7):2034–2047. doi: 10.1681/ASN.2005101085. [DOI] [PubMed] [Google Scholar]
- 7.National Kidney and Urologic Diseases Information Clearinghouse [Accessed August 24, 2012];A service of the National Institute of Diabetes and Digestive and Kidney Diseases. 2012 www.kidney.niddk.nih.gov.
- 8.U.S. Renal Data System . USRDS 2013 Annual Data Report: Atlas of Chronic Kidney Disease and End-Stage Renal Disease in the United States. National Institute of Health, National Institute of Diabetes and Digestive and Kidney Disease; Bethesda, MD: 2013. [Google Scholar]
- 9.Darby C, Crofton C, Clancy CM. Consumer Assessment of Health Providers and Systems CAHPS): Evolving to Meet Stakeholder Needs. Am J MedQual. 2006 Mar-Apr;21(2):144–147. doi: 10.1177/1062860606286263. [DOI] [PubMed] [Google Scholar]
- 10.Wood R, Paoli CJ, Hays RD, Taylor-Stokes G, Piercy J, Gitlin M. Evaluation of the Consumer Assessment of Healthcare Providers and Syustems In-Center Hemodialysis Survey (CAHPS-ICH) Clinical Journalof the Ameridcan Society of Nephrology. doi: 10.2215/CJN.10121013. In Press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Harris-Kojetin LD, Fowler FJ, Brown JA, Schnaier JA, Sweeny SF. The use of cognitive testing to develop and evaluate CAHPS (TM) 1.0 core survey items. Med Care. 1999 Mar;37(3):Ms10–Ms21. doi: 10.1097/00005650-199903001-00002. [DOI] [PubMed] [Google Scholar]
- 12.Levine RE, Fowler FJ, Brown JA. Role of cognitive testing in the development of the CAHPS (R) Hospital Survey. Health Serv Res. 2005 Dec;40(6):2037–2056. doi: 10.1111/j.1475-6773.2005.00472.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Dillman DA, Smyth JD, Christian LM. Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method. John Wiley & Sons, Inc.; Hoboken, NJ: 2009. [Google Scholar]
- 14.Fredrickson DD, Jones TL, Molgaard CA, et al. Optimal design features for surveying low-income populations. J Health Care Poor U. 2005 Nov;16(4):677–690. doi: 10.1353/hpu.2005.0096. [DOI] [PubMed] [Google Scholar]
- 15.Weisbord SD, Carmody SS, Bruns FJ, et al. Symptom burden, quality of life, advance care planning and the potential value of palliative care in severely ill haemodialysis patients. Nephrol Dial Transpl. 2003 Jul;18(7):1345–1352. doi: 10.1093/ndt/gfg105. [DOI] [PubMed] [Google Scholar]
- 16.Asch DA, Jedrziewski MK, Christakis NA. Response rates to mail surveys published in medical journals. J Clin Epidemiol. 1997 Oct;50(10):1129–1136. doi: 10.1016/s0895-4356(97)00126-1. [DOI] [PubMed] [Google Scholar]
- 17.Sitzia J, Wood N. Response rate in patient satisfaction research: an analysis of 210 published studies. Int J Qual Health C. 1998 Aug;10(4):311–317. doi: 10.1093/intqhc/10.4.311. [DOI] [PubMed] [Google Scholar]
- 18.McGee J, Kanouse DE, Sofaer S, Hargraves JL, Hoy E, Kleimann S. Making survey results easy to report to consumers - How reporting needs guided survey design in CAHPS (TM) Med Care. 1999 Mar;37(3):Ms32–Ms40. doi: 10.1097/00005650-199903001-00004. [DOI] [PubMed] [Google Scholar]
- 19.Hu L, Bentler PM. Cutoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria Versus New Alternatives. Structural Equation Modeling. 1999;6(1):1–55. [Google Scholar]
- 20.Keller SD, O’Malley AJ, Hays RD, Zaslavsky AM, Hepner KA, Clearly PD. Methods Used to Streamline the CAHPS Hospital Survey. Journal of Health Services Research. 2005;40(6):2057–2077. doi: 10.1111/j.1475-6773.2005.00478.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Kenny DA. [Accessed August 27, 2013];Measuring Model Fit. 2003 http://davidakenny.net/cm/fit.htm.
- 22.Steiger JH. Understanding the Limitations of Global Fit Assessment in STructural Equation Modeling. Personality and Individual Differences. 2007;42(5):893–898. [Google Scholar]
- 23.Suhr DD. Exploratory or Confirmatory Factor Analysis?: a Paper. Located at: SUGI Proceedings. 2006:200–231. [Google Scholar]
- 24.Weech-Maldonado R, Weidmer B, Morales L, Schoeff D, Hays RD. In: Cynamon ML, Kulka RA, editors. Cross-cultural adaptation of survey instruments: the CAHPS experience; Seventh Conference on Health Survey Research Methods; Santa Monica, CA: RAND Corporation. 2001.pp. 75–82. [Google Scholar]
- 25.Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297–334. [Google Scholar]
- 26.Agency for Healthcare Research and Quality Reporting Measures for the CAHPS® In-Center Hemodialysis Survey. 2007 https://cahps.ahrq.gov/surveys-guidance/docs/509_ich_reporting_measures.pdf.
- 27.Hays RD, Revicki D. Reliability and validity (including responsiveness) In: Fayers P, Hays RD, editors. Assessing Quality of Life in Clinical Trials: Methods and Practice. Second ed. Oxford University Press; Oxford: 2005. pp. 25–39. [Google Scholar]
- 28.Hays RD, Chong K, Brown J, Spritzer KL, Horne K. Patient reports and ratings of individual physicians: An evaluation of the doctor guide and consumer assessment of health plans study provider-level surveys. Am J Med Qual. 2003 Sep-Oct;18(5):190–196. doi: 10.1177/106286060301800503. [DOI] [PubMed] [Google Scholar]
- 29.Solomon LS, Hays RD, Zaslavsky AM, Ding L, Cleary PD. Psychometric properties of a group-level Consumer Assessment of Health Plans Study (CAHPS) instrument. Med Care. 2005 Jan;43(1):53–60. [PubMed] [Google Scholar]