Abstract
Background Patient‐given global ratings are frequently interpreted as summary measures of the patient perspective, with limited understanding of what these ratings summarize. Global ratings may be determined by patient experiences on priority aspects of care.
Objectives (i) identify patient priorities regarding elements of care for breast cancer, hip‐ or knee surgery, cataract surgery, rheumatoid arthritis and diabetes, (ii) establish whether experiences regarding priorities are associated with patient‐given global ratings, and (iii) determine whether patient experiences regarding priorities are better predictors of global ratings than experiences concerning less important aspects of care.
Setting and participants Data collected for the development of five consumer quality index surveys – disease‐specific questionnaires that capture patient experiences and priorities – were used.
Results Priorities varied: breast cancer patients for example, prioritized rapid access to care and diagnostics, while diabetics favoured dignity and appropriate frequency of tests. Experiences regarding priorities were inconsistently related to global ratings of care. Regression analyses indicated that demographics explain 2.4–8.4% of the variance in global rating. Introducing patient experiences regarding priorities increased the variance explained to 21.1–35.1%; models with less important aspects of care explained 11.8–23.2%.
Conclusions Some experiences regarding priorities are strongly related to the global rating while others are poorly related. Global ratings are marginally dependent on demographics, and experiences regarding priorities are somewhat better predictors of global rating than experiences regarding less important elements. As it remains to be fully determined what global ratings summarize, caution is warranted when using these ratings as summary measures.
Keywords: CQ‐index, global rating, patient experiences, patient priorities
Introduction
Measurement of the patient perspective is now a common strategy to monitor quality of care in a number of countries. 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 Data on the patient perspective often include patient‐given global ratings of received health care. 9 , 10 These global ratings have been interpreted as summary information, as such an approach reduces the quantity and complexity of data produced by measurement of the patient perspective on various healthcare aspects. However, whether and to what extent global ratings are adequate summary measures of the patient perspective depends on what these ratings summarize. It has been proposed that an overall global rating of quality of care from the patient perspective may be a result of the experienced quality on healthcare aspects, weighted by the importance attributed to those aspects. 11 In other words, patient experiences regarding elements of a high priority would have more impact on a global rating compared to experiences on elements of a lower priority. A recently developed family of surveys that is known as the Consumer Quality Index (CQ‐index) provides data that allow this hypothesis to be tested.
The CQ‐index is a Dutch instrument inspired by two other types of surveys: the American CAHPS (Consumer Assessment of Health care Providers and Systems) 12 , 13 and the Dutch QUOTE (QUality Of care Through the patients’ Eyes). 11 , 14 , 15 , 16 The CQ‐index is characterized by its disease‐ and provider‐specific focus as well as the assessment of patient priorities, which both derive from the QUOTE. From the CAHPS, the CQ‐index adopted layout, response scales and standardized sampling, data collection, analysis and presentation. Similar to both the CAHPS and QUOTE, the CQ‐index focuses on patient experiences, rather than patient satisfaction. The underlying assumption is that measures of experienced quality of care will be less subjective than measures of satisfaction. Importantly, focus groups consisting of patients are employed during the development of a CQ‐index to ensure that the patient perspective is best represented in the instrument. 17 The content of a CQ‐index includes questions on experiences with health care, questions regarding patient priorities and assessment of a global rating of received health care from the patient perspective.
As indicated, it is important to assess how experiences on various healthcare aspects may be reflected by a global rating of care, as these ratings may be used as summary information. Several reports investigating relationships between global ratings and patient experiences are available. Using a translated version of the American hospital CAHPS, 6 Arah and colleagues studied a sample of discharged hospital patients and reported global ratings of hospital care to be highly associated with doctor’s care and communication, nurses care and communication, pain control and nursing services (r’s = 0.53–0.62; P’s < 0.05). In addition, global ratings of hospital care were also substantially related to discharge information, medication and physical environment (r’s = 0.32–0.46; P’s < 0.05). 9 Based on CAHPS‐data on experiences with healthcare providers from 114 063 adults, Otani reported that patients’ experiences with how well their doctor communicates was of substantially larger impact on global ratings than waiting times, conduct of office staff and experiences in getting the care needed. 10 Further, Zaslavsky et al. (2002) found that experiences with customer service of health plans, a dimension that also incorporates communication and information, was the best predictor of the global rating of the health plan. 18 Thus, from the literature available, communication appears the most consistent provider related concept determining global ratings. It is worth noting though, that some authors partition communication into an information component and a respect or dignity component. 19 Interestingly, while it is possible that provider related determinants of global ratings may depend on patient priorities, 11 which in turn may vary between patient groups, 15 , 16 , 20 the studies described above failed to differentiate between patient groups.
The present paper will further explore the concept of Sixma et al., 11 from which we derive that patient experiences regarding priorities should be represented in their global rating of care and more so compared to experiences regarding less important elements. In this context, we will present the top‐10 priorities for the following patient groups, measured by CQ‐index surveys: breast cancer patients, patients that underwent hip‐ or knee surgery, patients that underwent cataract surgery, rheumatoid arthritis patients and diabetics. These patient groups were selected because we had their data available and because they represent a wide variety of different healthcare processes. Priorities will be summarized by a classification of umbrella concepts. Despite sufficient overlap between the available classifications in the literature and attempts to synthesize these classifications, 19 , 21 , 22 there appears to be no overall consensus. Therefore, the present paper adopted the following concepts that are broadly consistent with the literature and appropriate to the surveys at issue: respect/dignity, 19 information/patient education, 19 accessibility of care 22 and professional conduct. The latter covers issues such as technical skills, up to date procedures and protocols, confidentiality etc. 21 , 23 The following research questions will be addressed:
-
1
Which elements of care have the highest priority for the patient groups at issue?
-
2
Are experiences regarding priorities related to the global rating?
-
3
Are experiences regarding priorities better predictors of the global rating compared to experiences regarding less important elements of care?
Methods
Participants
All data were collected in the Netherlands. Patients were identified through insurance companies and/or hospitals and approached by mail using a procedure known as the Dillman method, 24 which includes up to four mailings if necessary. The datasets of the breast cancer and rheumatoid arthritis patients are somewhat smaller as they were collected to determine the psychometric properties of the survey. The other datasets were collected to assess the discriminative power of the instruments in which case power calculations dictated larger samples than those required to establish psychometric properties. The dataset for breast cancer consisted of 356 patients (response = 56.6%); two were male and excluded from the analysis. * This was a subset of a larger database that covered both malignant and benign tumours; however, we only selected data from patients suffering from a malignant tumour as we felt that experiences of patients with malignant tumours and benign tumours would be to distinct to combine. Further, the dataset for patients that underwent hip or knee surgery consisted of 1686 patients (response = 75.0%), the dataset for patients that underwent cataract surgery consisted of 4640 patients (response = 71.7%), the dataset for rheumatoid arthritis consisted of 407 respondents (response = 71.3%) and the dataset for diabetes consisted 5438 participants (response = 62.5%). Data on the demographic characteristics age, self‐observed health, education and sex are presented in Table 1.
Table 1.
Respondent characteristics
| Education* | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | Missing | |
| Breast cancer | 10 (2.8%) | 41 (11.6%) | 78 (22.0%) | 91 (25.7%) | 40 (11.3%) | 25 (7.1%) | 45 (12.7%) | 11 (3.1%) | 13 (3.7%) |
| Hip/knee surgery | 20 (1.2%) | 436 (25.9%) | 538 (31.9%) | 285 (16.9%) | 126 (7.5%) | 68 (4.0%) | 69 (4.1%) | 23 (1.4%) | 121 (7.2%) |
| Cataract surgery | 90 (1.9%) | 1390 (30.0%) | 1259 (27.2%) | 808 (17.4%) | 269 (5.8%) | 163 (3.5%) | 262 (5.7%) | 82 (1.8%) | 312 (6.7%) |
| Rheumatoid arthritis | 17 (4.2%) | 55 (13.5%) | 116 (28.5%) | 95 (23.3%) | 37 (9.1%) | 27 (6.6%) | 31 (7.6%) | 1 (0.2%) | 28 (6.9%) |
| Diabetes | 608 (11.2%) | 1177 (21.6%) | 1401 (25.8%) | 932 (17.1%) | 432 (7.9%) | 162 (3.0%) | 218 (4.0%) | 44 (0.8%) | 464 (8.5%) |
| Age | |||||||||
| 18–24 | 25–34 | 35–44 | 45–54 | 55–64 | 65–74 | >75 | Missing | ||
| Breast cancer | 1 (0.3%) | 5 (1.4%) | 32 (9.0%) | 68 (19.2%) | 122 (34.5%) | 99 (28.0%) | 27 (7.6%) | 0 (0.0%) | |
| Hip/knee surgery | 1 (0.1%) | 8 (0.5%) | 22 (1.3%) | 85 (5.0%) | 358 (21.2%) | 653 (38.7%) | 544 (32.3%) | 15 (0.9%) | |
| Cataract surgery | 2 (0.0%) | 15 (0.3%) | 23 (0.5%) | 148 (3.2%) | 530 (11.4%) | 1482 (32.0%) | 2338 (50.4%) | 97 (2.1%) | |
| Rheumatoid arthritis | 0 (0.0%) | 9 (2.2%) | 25 (6.1%) | 58 (14.3%) | 127 (31.2%) | 128 (31.4%) | 58 (14.3%) | 2 (0.5%) | |
| Diabetes | 1 (0.0%) | 29 (0.5%) | 177 (3.3%) | 580 (10.7%) | 1409 (25.9%) | 1709 (31.43%) | 1447 (26.6%) | 86 (1.6%) | |
| Self‐observed health† | Sex | ||||||||
| 1 | 2 | 3 | 4 | 5 | Missing | Male | Female | Missing | |
| Breast cancer | 8 (2.3%) | 77 (21.8%) | 197 (55.6%) | 48 (13.6%) | 19 (5.4%) | 5 (1.4%) | ‐ | 354 (100%) | 0 (0%) |
| Hip/knee surgery | 14 (0.8%) | 214 (12.7%) | 847 (50.2%) | 280 (16.6%) | 285 (16.9%) | 46 (2.7%) | 471 (27.9%) | 1200 (71.8%) | 15 (0.9%) |
| Cataract surgery | 310 (6.7%) | 510 (11.0%) | 2286 (49.3%) | 1245 (26.9%) | 110 (2.4%) | 174 (3.8%) | 1685 (36.4%) | 2870 (61.9%) | 80 (1.7%) |
| Rheumatoid arthritis | 28 (6.9%) | 193 (47.4%) | 165 (40.5%) | 11 (2.7%) | 2 (0.5%) | 8 (2.0%) | 109 (26.8%) | 296 (72.7%) | 2 (0.5%) |
| Diabetes | 220 (4.0%) | 1933 (35.5%) | 2845 (52.3%) | 207 (3.8%) | 99 (1.8%) | 134 (2.5%) | 2316 (42.6%) | 3028 (55.7%) | 94 (1.7%) |
*1 no education, 2 primary school, 3 lower level secondary school, 4 intermediate secondary school, 5 technical/higher education, 6 advanced secondary school, 7 bachelors degree, 8 masters degree.
†1 poor, 2 fair, 3 good, 4 very good, 5 excellent.
Measurements of experiences, patient priorities and global ratings
The content of a CQ‐index questionnaire typically consists of questions regarding the frequency with which quality criteria were met on a scale from one to four (i.e. never, sometimes, usually, always) and the extent to which performance on quality criteria has raised problems on a scale from one to three (i.e. big problem, small problem, no problem). 17 Other answering categories (such as yes or no) are employed where categories regarding magnitude of problems or frequency of meeting quality criteria are not appropriate. In addition, a number of standard patient characteristics are assessed in all CQ‐index surveys, such as age, sex and educational level as well as questions regarding disease‐specific patient characteristics. Importantly, to capture the complexity of the healthcare process, large parts of each survey are designed to assess specific procedures applicable only to a subset of patients; patients to whom those specific procedures do not apply are requested to ignore the corresponding questions. In addition, patient priorities are measured using questions regarding importance attributed to certain aspects of health care are posed (i.e. not important, of some importance, important and extremely important). 17 Since importance scores are generally constant within specific patient groups, 25 they are not routinely examined in CQ‐index surveys, but assessed in a separate survey during its development and replicated when deemed appropriate. Because in this study, importance scores were only used to assess whether a healthcare aspect belonged to the top‐10 priorities or to the top‐10 least important aspects, importance scores will not be presented. Finally, respondents are requested to provide global ratings on (elements of) the care received using a 10‐point Likert scale, where one represents the worst possible care and ten represents the best possible care.
For each survey, we selected the top‐10 priorities. Since priorities were measured separately, we had to match priorities to experience items. On four occasions one priority corresponded to two experience items and on two occasions no appropriate match was found; the handling of these anomalies is commented on in the results section. In all other cases, each of the top‐10 priorities matched one experience item. We also selected the 10 least important aspects. Note that the latter aspects were still sufficiently important to be included in the survey. All least important aspects each corresponded to one experience item with one exception. For diabetes, the data available to us arose from a revised version of the survey in which the majority of the least important items were excluded; therefore, analyses including the 10 least important items will not be presented for diabetes. Experiences were coded such that higher values reflect better performance of the healthcare provider.
Data analyses
Since global ratings were skewed, these ratings were recoded. † Values 0–5 contained less than 5% of the data and were recoded to one; values 6–10 were recoded into 2–6 respectively. Following recoding, histograms of the global ratings resembled the normal distribution, but were still mildly skewed (skewness −0.06 to −0.88). Partial Pearson product–moment correlations were computed to determine associations between experienced quality of care on the top‐10 priorities and the global rating of care. On three occasions, there were two items which were both equally relevant to one of the important aspects. In these cases we correlated both items to the global rating and calculates the average of both correlation coefficients using the Fisher transformation. 26 In addition, the proportion of variance attributable to experiences on the top‐10 most and the top‐10 least important items was assessed using linear regression analyses. Within the correlational analyses we controlled for age, education, self‐observed health and sex for two reasons: (i) these demographic characteristics are commonly identified to affect global ratings, 9 , 27 and (ii) we were interested in the relationship between experiences and global ratings beyond that attributable to these characteristics. For similar reasons, the independent variables for the linear regression analyses with global rating of care as the dependent variable were entered in two steps: (i) demographic characteristics, (ii) the experiences regarding the top‐10 most or the top‐10 least important aspects. These demographic characteristics were measured as categorical variables and included in the analyses as if they were continuous variables, a strategy that is supported and validated by previous research on patient experiences. 27 , 28 Nevertheless, we also confirmed in our data that linear and categorical specifications of demographic covariates yielded similar results regarding the coefficients of interest, i.e. correlation coefficients and explained variance. In the regression analyses, missing data were substituted for the average value of the covariate or the reported patient experience to circumvent detrimental effects of missing data. For hip or knee surgery, cataract surgery and diabetes, we could check whether removing cases with missing data would change our findings and found that results were virtually identical. All analyses were performed using spss 14.0 (SPSS Inc., Chicago, IL, USA).
Results
The top‐10 priorities per CQ‐index, as rated by patients, are presented in Table 2 and summarized by attribution to the concepts ‘respect/dignity’, ‘information/patient education’, ‘accessibility’ and ‘professional conduct’. Information was a frequently occurring theme in the top‐10 importance scores of the CQ‐index surveys: five of the top‐10 most important items were attributable to information/patient education in hip or knee surgery, cataract surgery and rheumatoid arthritis. In contrast, the top‐10 most important items for breast cancer was characterized by accessibility (5 items) while the top‐10 most important items of the CQ‐index diabetes may be described by respect/dignity (3 items), information/patient education (3 items) and professional conduct (4 items). Finally, accessibility also frequently occurred in the top‐10 of the CQ‐index rheumatoid arthritis (3 items).
Table 2.
Top 10 most important items from the patient perspective for hip or knee surgery, breast cancer, cataract surgery, rheumatoid arthritis and diabetes. The column ‘experience’ contains means (SD) of the experience ratings, correlations between experiences regarding the top‐10 most important items and the number of cases included in the correlational analyses. Experiences were coded such that the higher the value, the better the performance of the healthcare provider
| Importance | Experience | ||||
|---|---|---|---|---|---|
| Item | Theme | M (SD) | Correlation global rating† | N | |
| Breast cancer | |||||
| 1 | Rapid appointment in the hospital following referral | A | 1.55 (0.79) | −0.05 | 324 |
| 2 | Rapid referral from general practitioner to hospital | A | 1.96 (0.20) | 0.14* | 293 |
| 3 | Rapid availability of research results | A | 2.51 (1.19) | −0.10‡ | 211 |
| 4 | Rapid surgery following diagnosis | A | 3.38 (1.45) | −0.22* | 278 |
| 5 | Result of X‐ray from population screening is rapidly available | A | 2.08 (0.76) | 0.10 | 115 |
| 6 | Information on sentinel node biopsy | I | 1.85 (0.36) | 0.00 | 292 |
| 7 | No second surgery required | P | 1.79 (0.41) | 0.08 | 227 |
| 8 | Information about side effects of further treatment | I | 1.98 (0.10) | 0.13 | 212 |
| 9 | To be taken seriously by health care providers | D | 3.69 (0.56) | 0.36*‡ | 125 |
| 10 | No complications following surgery | P | 1.36 (0.48) | −0.02 | 277 |
| HIP/knee surgery | |||||
| 1 | Information on what (not) to do following surgery | I | 3.80 (0.76) | 0.21* | 1523 |
| 2 | Frequency of cleaning room and bathroom | P | 3.50 (0.66) | 0.26* | 1508 |
| 3 | Anaesthetics as discussed prior to procedure | P | 3.82 (0.71) | 0.08* | 1505 |
| 4 | Information on what (not) to do following discharge | I | 2.77 (1.48) | 0.27* | 1505 |
| 5 | Physiotherapy adjusted to personal situation | A | 3.88 (0.60) | 0.09* | 1394 |
| 6 | Nurses explain things in an understandable manner | I | 3.47 (0.73) | 0.34* | 1523 |
| 7 | Doctors and nurse practitioners explain things in an understandable manner | I | 3.44 (0.78) | 0.30* | 1470 |
| 8 | Information on treatment following surgery | I | 3.74 (0.85) | 0.20* | 1523 |
| 9 | To be taken seriously by nurses | D | 3.68 (0.56) | 0.40* | 1523 |
| 10 | To be taken seriously by doctors and/or nurse practitioners | D | 3.67 (0.59) | 0.34* | 1475 |
| Cataract surgery | |||||
| 1 | Ophthalmologist provides information on risk of treatment | I | 1.68 (0.47) | 0.22* | 4153 |
| 2 | Information on what to do in case of an emergency following cataract surgery | I | 1.77 (0.42) | 0.19* | 4137 |
| 3 | Health plan pays full costs of cataract surgery | A | 1.99 (0.12) | 0.01 | 4186 |
| 4 | Ophthalmologist explains things in an understandable manner | I | 3.54 (0.75) | 0.28* | 4186 |
| 5 | Ophthalmologists, nurses and other hospital staff enquire whether you are allergic to certain medication | P | 2.64 (1.37) | 0.15*‡ | 4109 |
| 6 | The ophthalmologist listens to you attentively | D | 3.72 (0.56) | 0.37* | 4186 |
| 7 | Information on what (not) to do following surgery | I | 1.89 (0.31) | 0.12* | 4186 |
| 8 | Ophthalmologist, nurses and hospital staff explain things in an understandable manner | I | 2.68 (1.32) | 0.29* | 4034 |
| 9 | Ophthalmologist prescribes drugs that are covered by your health plan | A | 3.40 (1.12) | 0.05* | 4081 |
| 10 | No pain during surgery | P | 1.93 (0.25) | 0.11* | 4186 |
| Rheumatoid arthritis | |||||
| 1 | The health care provider considers your other medication when prescribing drugs | P | 3.74 (0.58) | 0.44* | 287 |
| 2 | The doctor explains the risks of an operation | I | 1.93 (0.25) | −0.24 | 34 |
| 3 | Rapid access to a rheumatologist following referral | A | 2.83 (1.06) | 0.04 | 85 |
| 4 | General practitioner quickly provides referral to rheumatologist | A | 2.93 (1.11) | 0.00 | 87 |
| 5 | You are being informed about long‐term consequences of an operation | I | 1.72 (0.45) | −0.07 | 35 |
| 6 | If discomfort from rheumatoid arthritis increases, the health care provider is available within a reasonable time | A | 3.49 (0.76) | 0.37* | 269 |
| 7 | The doctor explains the surgical procedure | I | 1.96 (0.20) | −0.32 | 35 |
| 8 | Health care provider takes you seriously | D | 3.85 (0.43) | 0.36* | 332 |
| 9 | The doctor provides rules for what (not) to do after discharge following surgery | I | 1.83 (0.38) | 0.12 | 35 |
| 10 | Discuss arrangements regarding what to do when rheumatoid arthritis deteriorates, as in an acute attack | I | 1.57 (0.50) | 0.15* | 337 |
| Diabetes | |||||
| 1 | Health care providers provide adequate assistance with the dosage of insulin | P | 3.44 (0.89) | 0.39* | 1114 |
| 2 | Eyes are checked yearly | P | 3.57 (0.76) | 0.11* | 4743 |
| 3 | Health care providers take me seriously | D | 3.75 (0.55) | 0.39*‡ | 3334 |
| 4 | Health providers provide decent patient education | I | 3.35 (0.90) | 0.49* | 4743 |
| 5 | Health care providers do not give conflicting information | I | 3.81 (0.60) | 0.09* | 2033 |
| 6 | Health care providers listen carefully and attentively | D | 3.69 (0.60) | 0.40*‡ | 3381 |
| 7 | Health care providers are skilled and knowledgeable on the subject of diabetes | P | – | – | |
| 8 | During scheduled control visits, the appropriateness of the treatment is discussed | P | 3.23 (0.99) | 0.45* | 4715 |
| 9 | Information about drugs is provided in an understandable manner | I | 3.55 (0.75) | 0.39* | 3346 |
| 10 | Health care providers take patient measured blood values seriously | D | – | – | |
N, number of patients on which the correlation is based; A, accessibility; D, respect/dignity; I, information and patient education; P, professional conduct; –, No corresponding experience item available for this importance item.
*P < 0.05.
†While controlling for age, sex, education and self‐observed health.
‡These importance items corresponded to more than one experience item of the survey. Both items were correlated to the global rating and the average of those correlations and the minimal N are reported.
The correlational analyses showed that, on the whole, relationships between experiences on important aspects and the global rating varied substantially in magnitude (see Table 2). When comparing between CQ‐index surveys, it can be observed that the top‐10 of some surveys, such as hip/knee surgery and diabetes, include many items that are substantially related to the global rating (r’s > 0.20) whereas the top‐10 of other surveys, such as breast cancer and cataract surgery, mainly contained items that are moderately or not related to the global rating (see Table 2). Note also that some experiences were negatively related to the global rating, that is, higher scores on some accessibility items in breast cancer and some information items in rheumatoid arthritis were accompanied by lower global ratings. More specifically, the global rating of breast cancer was related to some items concerning accessibility (r’s = −0.22 and 0.14, P’s < 0.05) and strongly related to one item of respect/dignity (r = 0.36, P < 0.05), but not related to items regarding professional conduct and information/patient education. The global rating for hip or knee surgery on the other hand, was primarily related to items concerning respect/dignity (r’s = 0.34 and 0.40, P’s < 0.05) and information/patient education (r’s = 0.20–0.40, P’s < 0.05). In addition, the cataract surgery global rating was most strongly associated with one aspect of respect/dignity (r = 0.37, P < 0.05) and related to items concerning information/patient education (r’s = 0.12–0.29, P’s < 0.05) and professional conduct (r’s = 0.11 and 0.15, P’s < 0.05). Associations to items of accessibility were either negligible or non‐existent. The correlates of the global rating of rheumatoid arthritis care were mixed with items regarding information/patient education (r = −0.32, P = 0.06; r = 0.15, P <0.05), accessibility (r = 0.37, P < 0.05), professional conduct (r = 0.44, P < 0.05) and respect/dignity (r = 0.36, P < 0.05). Finally, the correlates of the diabetes global rating were also mixed, including items of information/patient education (r’s = 0.09–0.49, P’s < 0.05), professional conduct (r’s = 0.11–0.45, P’s < 0.05) and respect/dignity (r’s = 0.39–0.40, P’s < 0.05). In sum, experiences regarding priorities were inconsistently related to the global rating except for experiences regarding dignity that were always strongly and positively related. Since correlation analyses are affected by distributional properties of the variables involved, i.e. variance in a dependent variable cannot be explained by an independent variable that does not vary, the means and standard deviations of the independent variables are also presented in Table 2. Although the independent variables differ in their means and standard deviations, there appears to be no clear relationship between these differences and the magnitude of the correlation coefficients.
Linear regression analyses were performed to determine how much of the variance in global rating may be attributed to the combined experiences on the top‐10 most important items. This is not simply the sum of the variances accounted for by individual experience items as these variances may overlap. The global rating was the dependent variable and the first set of independent variables were the patient characteristics age, sex, self‐observed health status and educational level, as we wanted to control for these variables. Subsequently, the experiences on the top‐10 priorities were entered.
The results of the linear regression analyses are presented in Table 3. For all surveys, the model including the demographic characteristics, as well as the model including both demographic characteristics and experiences significantly predicted the global rating. The proportion of variance accounted for in the models containing demographic characteristics only, ranged from 2.4% to 8.4%. Of the demographic characteristics at issue, self‐observed health most consistently predicted global rating, followed by age and education, while sex was not related to the global rating. Introducing the experiences on the top‐10 priorities improved the explained variance for all surveys; the proportion of variance accounted for ranged from 21.1% (cataract surgery) to 35.1% (diabetes) (see Table 3).
Table 3.
Linear regression analyses with global rating of care as the dependent variable, demographic characteristics as independent variables entered in step1 and experiences on the top‐10 most important items as independent variables entered in step 2
| CQI | Model | Age | Sex | Self‐observed health | Education | Degrees of freedom¶ | R 2†† |
|---|---|---|---|---|---|---|---|
| Breast cancer | dc†,‡ | 0.08 | – | 0.29* | −0.04 | 3, 353 | 0.084* |
| dc, experiences§ | 0.02 | – | 0.19* | −0.02 | 16, 353 | 0.276* | |
| Hip/knee surgery | dc | 0.01 | −0.05 | 0.25* | −0.16* | 4, 1685 | 0.071* |
| dc, experiences | 0.06* | −0.02 | 0.13* | −0.11* | 14, 1685 | 0.333* | |
| Cataract surgery | dc | 0.07* | −0.03 | 0.15* | −0.07* | 4, 4634 | 0.029* |
| dc, experiences | 0.08* | −0.02 | 0.10* | −0.05* | 15, 4634 | 0.211* | |
| Rheumatoid arthritis | dc | 0.13* | −0.03 | 0.11* | −0.05 | 4, 406 | 0.024* |
| dc, experiences | 0.13* | 0.00 | 0.09 | −0.03 | 14, 406 | 0.222* | |
| Diabetes | dc | 0.10* | 0.00 | 0.21* | −0.02 | 4, 5437 | 0.053* |
| dc, experiences | 0.09* | 0.01 | 0.12* | −0.03* | 15, 5437 | 0.351* |
*P < 0.05.
†Demographic characteristics.
‡Age: from 1 (18–24 years) to 7 (older than 75); Sex: 0 = male, 1 = female; self‐observed health: from 1 (poor) to 5 (excellent); education: from 1 (no education) to 8 (university degree or higher).
§Experiences on top‐10 priorities.
¶The number of independent variables may differ from the number of demographic variables plus ten as in some cases, several experience items were available for one importance item.
††Adjusted R 2, represents proportion of variance accounted for.
The combined experiences for priorities may explain more variance in the global rating compared to experiences on aspects of a lesser importance. The final analyses addressed this issue and were similar to the linear regression analyses described above, except that instead of the top‐10 priorities, the 10 least important items were entered. It is worth noting though, that the least important items were still sufficiently important to be included in the survey. The results of these analyse are presented in Table 4 and show that the models with experiences regarding unimportant items also account for a substantial proportion of the variance in global rating (11.8–23.3%; see Table 4), but, with the exception of cataract surgery, not as much compared to models with experiences regarding important items (21.1–35.1%; see Table 3).
Table 4.
Linear regression analyses with global rating of care as the dependent variable, demographic characteristics as independent variables entered in step1 and experiences on the 10 least important items as independent variables entered in step 2
| CQI | Model | Age | Sex | Self‐observed health | Education | Degrees of freedom | R 2¶ |
|---|---|---|---|---|---|---|---|
| Breast cancer | dc†‡, experiences§ | 0.11* | – | 0.26* | −0.01 | 13, 353 | 0.118* |
| Hip/knee surgery | dc, experiences | 0.03 | −0.03 | 0.17* | −0.11* | 14, 1685 | 0.232* |
| Cataract surgery | dc, experiences | 0.06* | −0.02 | 0.10* | −0.02* | 14, 4634 | 0.229* |
| Rheumatoid arthritis | dc, experiences | 0.13* | −0.01 | 0.12* | −0.04 | 14, 406 | 0.068* |
*P < 0.05.
†Demographic characteristics.
‡Age: from 1 (18–24 years) to 7 (older than 75); Sex: 0 = male, 1 = female; self‐observed health: from 1 (poor) to 5 (excellent); education: from 1 (no education) to 8 (university degree or higher).
§Experiences on 10 least important items.
¶Adjusted R 2, represents proportion of variance accounted for.
Discussion
The present paper showed that patient priorities varied between patient groups and that experiences regarding priorities are inconsistently related to the global rating. Overall, however, a global rating seems to better represent experiences regarding priorities than experiences regarding less important elements of care.
Consistent with previous research, the top‐10 priorities of breast cancer patients primarily consisted of items regarding acessibility, 20 while the top‐10 priorities for cataract surgery were characterized by information/patient education. 15 The top‐10 priorities for hip‐ or knee surgery were also dominated by items concerning information/patient education. It would appear that surgery is a powerful trigger of a need for information. Indeed, information‐related items in rheumatoid arthritis were also predominantly focussed on a possible surgery. Rheumatoid arthritis patients also prioritized items concerning accessibility while the top‐10 priorities for diabetics on the other hand were evenly distributed among professional conduct, respect/dignity and information/patient education.
Although differences in priorities between patient groups have been observed previously, the determinants of those differences are, as far as the authors are aware, largely ignored. One possible determinant that the authors would consider worth investigating, though, is the concept of a worst case scenario. The worst case scenario for breast cancer for example would be death. Perhaps, fear of death and need of reassurance may account for importance attributed to fast access and rapid availability of test results in breast cancer. Diabetes patients on the other hand may be most concerned by regulation of their blood sugar levels, in which case appropriate frequency of tests and competence of the healthcare provider would be helpful, as would information and patient education for self management. For surgery however, the worst case scenario maybe discomfort and/or complications following surgery; patients may aim to control and avoid this scenario by a focus on information such as ‘information on what (not) to do following surgery and discharge’ and ‘information on risk of treatment’. Thus, relevant worst case scenarios may be considered as a possible determinant of patient priorities. However, as the design of the present study was not set up to identify determinants of priorities, conclusions can only be drawn in future research addressing this issue.
The correlational analyses afforded an opportunity to assess the extent to which experiences regarding the most prominent themes in each of the top‐10 patient priorities are related to the global rating of care. Overall, associations between experiences and global ratings differed within and between surveys, except for items concerning respect/dignity which were consistently, substantially and positively related to the global rating across all surveys.
As indicated, respect/dignity may be viewed as an element of communication and, as such, the present associations between respect/dignity and the global rating resonate with previous findings. 9 , 10 , 29 As far as we are aware, an explanation accounting for the consistent and substantial correlations between respect/dignity and the global rating is lacking. One possibility refuted by the current data considers that patients would rate dignity as more important than other quality aspects; respect/dignity was never the most important nor the most prevalent theme in the present paper, which shows that importance is not the feature distinguishing respect/dignity from the other quality aspects. Other hypotheses that future research should address would include the following: (i) compared to other quality aspects, experiences regarding dignity are either more prominently experienced, and/or more easily judged and/or more easily remembered, (ii) dignity is a more subjective or global experience than most other quality aspects and therefore more susceptible to the influences that also affect a global rating. In the latter explanation, the relationship between experiences regarding respect/dignity and the global rating would be, at least in part, a result of a component common to both.
To our surprise there were a few items that were negatively related to the global rating such that the more a priority was met, the lower the global rating. In breast cancer for example, negative associations between items concerning accessibility reached significance for ‘rapid surgery following diagnosis’ and almost reached significance for ‘rapid availability of diagnostic results’. Although explanations remain speculative, the authors would like to entertain disease severity as a possible explanation linking fast access to lower global ratings in breast cancer. It is conceivable that more severe cases of breast cancer receive a faster access to surgery and diagnostics while the overall experience would also be more traumatic in these cases, which may result in lower global ratings. Other negative associations occurred for information‐related items concerning an operation in rheumatoid arthritis. The latter correlations were based on a small subset of the rheumatoid arthritis sample, as these items were not applicable to the majority of patients; the conventional criterion for statistical significance was therefore not reached. Even so, the observation that all information‐related items concerning an operation were negatively related to the global rating in rheumatoid arthritis remains intriguing, although the authors are at a loss to explain this. Future research is needed to establish whether the negative associations reported here are robust, before further speculation or investigation on possible explanations.
In accordance with previous research, analyses showed that a patient’s global rating of care is, to a limited extent, influenced by demographic characteristics. 3 , 27 The impact of demographic characteristics was more pronounced for breast cancer, hip‐ or knee surgery and diabetes compared to rheumatoid arthritis and cataract surgery. The extent to which demographic characteristics affect the global rating seems to be driven by ‘self‐observed health’; on the whole, self‐observed health was the most consistent demographic characteristic in the analyses and, the surveys displaying a larger impact of demographic characteristics on the global rating also show larger coefficients for self‐observed health. Previous research also identified self‐observed health as a potent predictor of global ratings. 27 Age and educational level affected the global rating for some, but not all surveys, while sex did not significantly affect the global rating for any survey. Introducing the experiences regarding the top‐10 most important items consistently and substantially improved the variance accounted for: 21.1–35.1% of the total variance was explained.
Although the correlational analyses showed that experiences regarding important aspects of health care are inconsistently related to the global rating, it is still possible that on the whole, experiences regarding important healthcare aspects explain more variance in the global rating compared to experiences regarding less important aspects. A final regression analyses was performed in which demographic characteristics and the ten least important items were entered to the model. Note that these items were still sufficiently important to be included in the questionnaire. Comparing the variance in global rating accounted for between the models with experiences regarding priorities and models with experiences regarding less important items revealed that on the whole, experiences regarding priorities are somewhat better reflected in a global rating. The limited power of importance as a determinant of associations between experiences and global ratings has been observed previously. Gustafson et al. 30 explored whether correlating experiences with the global rating was an appropriate strategy to identify what patients find important and showed that such a strategy results in a different set of priorities compared to asking patients to rate the importance of a number of aspects directly.
When interpreting the present data, several possible limitations should be considered. It is worth noting, for example, that sample sizes differed between and within surveys. Differences in sample size between surveys is explained by differences in the number of individuals that were approached and that responded, whereas differences within surveys are explained by missing data or items not being applicable to subsets of patients. These differences may explain why some correlations reached significance, while other correlations of a similar magnitude did not. In addition, the classification used to attribute importance items to umbrella concepts may be questioned as, in the absence of a generally accepted classification, we decided to select the most suitable concepts from several classifications to describe our data. However, given the differences between patient groups, patient priorities and patient experiences, adapting a classification to the patient groups at issue may be the appropriate way. Further, demographic characteristics were measured as categorical variables, but included in our study as if they were continuous variables. However, this strategy would not appear to be a problem as it is supported by previous research on patient experiences 27 , 28 and yielded results which were also consistent with other reports, including those handling demographic characteristics as categorical variables. 7 , 8 , 9 , 27 Further, we also confirmed in our data that categorical specifications yield similar results. Finally, we used responses to individual items, rather than the more stable composite scores, because we could not develop composites regarding the most important items such that they would be comparable between surveys. However, the majority of the findings reported here apply to a number of items within or between surveys and, for those findings, this limitation may be considered minor as replication is available within the present paper. Nevertheless, caution is warranted and replication is required, particularly where the correlational analyses on a single item are not replicated using similar items within the same survey or in other surveys or studies.
In conclusion, the present data indicate that patient priorities vary between patient groups. These differences in patient priorities stress the need for disease‐specific surveys such as the CQ‐index. Experiences regarding patient priorities were inconsistently related to patients’ global rating. Thus, a global rating summarizes some, but not all important elements of care. Nevertheless, on the whole, experiences regarding priorities appear to be somewhat more reflected by a global rating than experiences regarding less important elements of care. The findings of the present paper question the validity of global ratings as summary measures of patient‐experienced quality of care, since it remains difficult to establish what these retains summarize and, because some important aspects of care are not reflected in these ratings.
Footnotes
The two male breast cancer patients were excluded from the analyses to circumvent controlling for sex in the correlational and regression analyses on the basis of only two males.
Initially we conducted analyses without recoding the global rating, which yielded results that were remarkably similar to the results using a recoded global rating reported here.
References
- 1. Fritt Sykehusvalg . Available at: http://www.frittsykehusvalg.no/start/, accessed 11 December 2008.
- 2. Your Hospitals . Available at: http://www.health.vic.gov.au/yourhospitals/, accessed 10 December 2008.
- 3. Østerbye T, Sevaldsen J, Skjødt HK, Freil M. Patient’s Experiences in Danish Hospitals. Copenhagen: The Unit of Patient Evaluation, 2007. [Google Scholar]
- 4. CAHPS Home Page . Available at: https://www.cahps.ahrq.gov/default.asp, accessed 14 December 2008.
- 5. Redding D, Boyd J. Data briefing. Involvement leads to satisfaction. Health Service Journal, 2008; 17. [PubMed] [Google Scholar]
- 6. Delnoij DMJ, Asbroek GT, Arah O et al. Made in the USA: the import of American Consumer Assessment of Health Plan Surveys (CAPHS) into the Dutch social insurance system. European Journal of Public Health, 2006; 16: 652–659. [DOI] [PubMed] [Google Scholar]
- 7. Stubbe JH, Gelsema T, Delnoij DM. The Consumer Quality Index Hip Knee Questionnaire measuring patients’ experiences with quality of care after a total hip or knee arthroplasty. BMC Health Services Research, 2007; 7: 60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Stubbe JH, Brouwer W, Delnoij DM. Patients’ experiences with quality of hospital care: the Consumer Quality Index Cataract Questionnaire. BMC Ophthalmology, 2007; 7: 14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Arah OA, Ten Asbroek AH, Delnoij DM et al. Psychometric properties of the Dutch version of the Hospital‐level Consumer Assessment of Health Plans Survey instrument. Health Services Research, 2006; 41: 284–301. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Otani K. Enrollees’ global rating process of health care with the national CAHPS Benchmarking Database. Health Care Management Review, 2006; 31: 205–212. [DOI] [PubMed] [Google Scholar]
- 11. Sixma HJ, Kerssens JJ, Campen CV, Peters L. Quality of care from the patients’ perspective: from theoretical concept to a new measuring instrument. Health Expectations, 1998; 1: 82–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Hargraves JL, Hays RD, Cleary PD. Psychometric properties of the Consumer Assessment of Health Plans Study (CAHPS) 2.0 adult core survey. Health Services Research, 2003; 38: 1509–1527. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Carman KL, Short PF, Farley DO, Schnaier JA, Elliott DB, Gallagher PM. Epilogue: early lessons from CAHPS Demonstrations and Evaluations. Consumer Assessment of Health Plans Study. Medical Care, 1999; 37: MS97–MS105. [PubMed] [Google Scholar]
- 14. Hekkink CF, Sixma HJ, Wigersma L et al. QUOTE‐HIV: an instrument for assessing quality of HIV care from the patients’ perspective. Quality and Safety in Health Care, 2003; 12: 188–193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Nijkamp MD, Sixma HJ, Afman H et al. Quality of care from the perspective of the cataract patient. QUOTE cataract questionnaire. Journal of Cataract and Refractive Surgery, 2002; 28: 924–931. [DOI] [PubMed] [Google Scholar]
- 16. Van Campen C, Sixma HJ, Kerssens JJ, Peters L, Rasker JJ. Assessing patients’ priorities and perceptions of the quality of health care: the development of the QUOTE‐Rheumatic‐Patients instrument. British Journal of Rheumatology, 1998; 37: 362–368. [DOI] [PubMed] [Google Scholar]
- 17. Sixma H, Delnoij DM, Stubbe JH et al. Handboek CQI meetinstrumenten. Utrecht: Centrum Klantervaring Zorg, 2007.
- 18. Zaslavsky AM, Cleary PD. Dimensions of plan performance for sick and healthy members on the Consumer Assessments of Health Plans Study 2.0 survey. Medical Care, 2002; 40: 951–964. [DOI] [PubMed] [Google Scholar]
- 19. Shaller D. Patient‐centered care: what does it take? 2007. The Commonwealth Fund.
- 20. De Kok M, Scholte RW, Sixma HJ et al. The patient’s perspective of the quality of breast cancer care. The development of an instrument to measure quality of care through focus groups and concept mapping with breast cancer patients. European Journal of Cancer, 2007; 43: 1257–1264. [DOI] [PubMed] [Google Scholar]
- 21. Thiedke CC. What do we really know about patient satisfaction? Family Practice Management, 2007; 14: 33–36. [PubMed] [Google Scholar]
- 22. De Silva A. A Framework for Measuring Responsiveness. Geneva, 1999, Unpublished paper. GPE Discussion Paper No. 32. [Google Scholar]
- 23. Elwyn G, Buetow S, Hibbard J, Wensing M. Measuring quality through performance. Respecting the subjective: quality measurement from the patient’s perspective. British Medical Journal, 2007; 335: 1021–1022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Dillman DA. Mail and Internet Surveys: The Tailored Design Method. New York: John Wiley & Sons Inc, 2000. [Google Scholar]
- 25. De Boer D, Hendriks M, Damman OC et al. Ervaringen met de zorg en de zorgverzekeraars. CQ‐meting Zorg en Zorgverzekering. Utrecht: NIVEL, 2007. [Google Scholar]
- 26. Neter J, Kutner M, Nachtsheim C, Wasserman W. Applied Linear Statistical Models. Chicago, IL: Irwin Publishing, 1996. [Google Scholar]
- 27. O’Malley AJ, Zaslavsky AM, Elliott MN, Zaborski L, Cleary PD. Case‐mix adjustment of the CAHPS Hospital Survey. Health Services Research, 2005; 40: 2162–2181. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Zaslavsky AM, Zaborski LB, Ding L, Shaul JA, Cioffi MJ, Cleary PD. Adjusting performance measures to ensure equitable plan comparisons. Health Care Financing Review, 2008; 22: 109–126. [PMC free article] [PubMed] [Google Scholar]
- 29. Chang JT, Hays RD, Shekelle PG et al. Patients’ global ratings of their health care are not associated with the technical quality of their care. Annals of Internal Medicine, 2006; 144: 665–672. [DOI] [PubMed] [Google Scholar]
- 30. Gustafson DH, Arora NK, Nelson EC, Boberg EW. Increasing understanding of patient needs during and after hospitalization. Joint Commission Journal on Quality and Patient Safety, 2001; 27: 81–92. [DOI] [PubMed] [Google Scholar]