Relationship Between Clinical Quality and Patient Experience: Analysis of Data From the English Quality and Outcomes Framework and the National GP Patient Survey

Abstract

PURPOSE

Clinical quality and patient experience are both widely used to evaluate the quality of health care, but the relationship between these 2 domains remains uncertain. The aim of this study was to examine this relationship using data from 2 established measures of quality in primary care in England.

METHODS

Practice-level analyses (N = 7,759 practices in England) were conducted on measures of patient experience from the national General Practice Patient Survey (GPPS), and measures of clinical quality from the national pay-for-performance scheme (Quality and Outcomes Framework). Spearman’s rank correlation and multiple linear regression were used on practice-level estimates.

RESULTS

Although all the correlations between clinical quality summary scores and patient survey scores are positive, and most are statistically significant, the strength of the associations was weak, with the highest correlation coefficient reaching 0.18, and more than one-half were 0.11 or less. Correlations with clinical quality were highest for patient-reported access scores (telephone access 0.16, availability of urgent appointments 0.15, ability to book ahead 0.18, ability to see preferred doctor 0.17) and overall satisfaction (0.15).

CONCLUSION

Although there are associations between clinical quality and measures of patient experience, the 2 domains of care quality remain predominantly distinct. The strongest correlations are observed between practice clinical quality and practice access, with very low correlations between clinical quality and interpersonal aspects of care. The quality of clinical care and the quality of interpersonal care should be considered separately to give an overall assessment of medical care.

INTRODUCTION

Providing high-quality clinical care and providing a good patient experience are priorities for most health care systems.¹ Patient experience is a valuable outcome in its own right, beyond any relationship with clinical outcomes. Nevertheless, there are a number of possible causal pathways that might link patient experience and clinical or organizational quality which are relevant to the assessment and management of care. These pathways include patients being more satisfied with care of better technical quality, poor access in which patients fail to attend scheduled appointments and therefore receive a poorer standard of care, poor communication leading to reduced compliance with treatment, and poor continuity of care reducing compliance with appointments and treatment. In addition, it is possible that increased attention paid to clinical aspects of care, for example as a result of pay-for-performance schemes such as the Quality and Outcomes Framework (QOF),² could inadvertently lead to a deterioration in patient-centered care and hence to patient-reported experience.

Although there is a considerable literature dedicated to the measurement of clinical quality and patient experience, respectively, there has been little research exploring the relationship between these 2 key domains of quality. Studies in primary care settings have reported findings ranging from no association^3–5 to modest associations^6,7 between measures of patient experience and clinical quality, whereas studies conducted in secondary care generally report positive associations with correlations, some modest (eg, 0.19),⁸ and some substantial (eg, 0.63).⁹ Methodological issues may contribute to the wide range of results reported in these studies, including sample size, choice of indicators used to define clinical quality and patient experience, and whether the analyses were conducted at individual, practice, or health plan level. In particular, analyses conducted with small samples may underestimate correlations in the absence of shrinkage estimation, aggregation, or compositing (combining a number of indicators).

In this study, we set out to investigate the relationship between 2 established measures of quality in UK primary care: the QOF, a major pay-for-performance scheme that focuses largely on clinical aspects of care, and the General Practice Patient Survey (GPPS), which measures patient experience of family practices through a large cross-sectional population survey.^10,11 For the purpose of this report, we use the term clinical quality to denote technical aspects of medical care.

METHODS

We explored associations of clinical and patient experience dimensions of quality at the level of the primary care practice.

Data

Data on patient experience were taken from the GPPS (http://www.gp-patient.co.uk), which is sent to 5.5 million patients in England each year. It is sent as a mail questionnaire to patients who have been continuously registered with a family practice for at least 6 months. It can also be answered online or on the telephone in 15 languages. In England, almost all the population is registered with a family practice. Family practices have an average of 3.5 primary care physicians and are responsible for registered lists averaging of 6,745 patients. Stratified random samples are drawn from family practice lists, oversampling from small practices and from those with low response rates with previous surveys, resulting in an average of 260 patients respondents per practice. Further details of the survey and its development can be found elsewhere.^10,12 The analysis reported here uses data from year 4 of the survey (April 2009 to March 2010) to which 2,169,718 patients responded (response rate of 39%).

Data on clinical quality were taken from 2009/2010 clinical quality indicators from the UK pay-for-performance QOF,² calculated for each indicator as the percentage of eligible patients for whom the indicator was met, correcting for exception reporting to give the population level of achievement¹³ (see the Supplemental Appendix, available at http://annfammed.org/content/11/5/467/suppl/DC1).

Practice data were provided by the National Health Service Information Centre for 2009, including practice patient numbers, broken down by sex and age, and information on each individual physician, including sex, country and year of medical qualification, and the amount of time worked (measured in full-time-equivalents).

Analysis

For GPPS questions, we first linearly rescaled items on a scale of 0 to 100. We then calculated shrunken estimates of practice scores from mixed-effects models, case-mix adjusted for age, sex, ethnicity, deprivation, and self-rated health. For QOF practice scores, we calculated an overall clinical summary score for each practice using a shrunken estimate of the proportion of patients for whom each measure was met, weighted by the point score for that indicator in the Quality and Outcomes Framework.

Shrunken estimates¹⁴ (also known as empirical Bayes estimates, best linear unbiased predictions, or BLUPs) were used to reduce the effects of measurement error by adjusting practice scores toward the overall mean according to the amount of measurement error, thereby obtaining more accurate estimates of the underlying association of clinical quality and patient experience at the practice level.¹⁵ Similarly, use of patient-mix adjustment removes effects of patient characteristics on measured quality and thus improves the accuracy of the estimates of the underlying 2 dimensions of quality and their correlation.^16,17 To further limit measurement error, only those practices with at least 100 GPPS respondents, at least 1 clinical domain with at least 100 patients, and no missing data for practice population were included.

We summarize the shrunken estimates of practice performance on the GPPS questions and the QOF clinical quality summary score using medians and interquartile ranges. We initially looked at simple associations between clinical quality and GPPS scores by calculating the Spearman’s rank correlation coefficient between each practice’s summary QOF score and practice GPPS score, using Bonferroni corrections to account for multiple testing. We then performed a series of linear regressions with the QOF clinical summary score as the outcome variable. In each model 1 GPPS score was an explanatory variable along with either (1) no other fixed effects or (2) practice population demographics. We also performed a number of linear regressions to calculate the proportion of variance in the QOF clinical summary score associated with different groups of GPPS questions assessing several areas of patient experience (access, continuity of care, communication, overall satisfaction, confidence and trust in doctor, care planning). We report the results of the regression as standardized regression coefficients (equivalent to correlation coefficients in the bivariate case).

Shrunken case-mix adjusted estimates of the GPPS scores were obtained using SAS 9.2 (SAS Institute Inc). All other analysis was performed using Stata 11.2 (StataCorp LP). Full details of the statistical methods used are given in the Supplemental Appendix.

We also broke down the clinical quality data by type of indicator, categorized as recording/review, tests/specialist referral, treatment, intermediate outcome. We calculated correlations between GPPS items and each category of clinical quality indicator (Supplemental Appendix).

RESULTS

A total of 7,759 practices in England (93%) had data that could be included in the analyses. For individual clinical domains in the QOF, the number of included practices ranged from 1,312 (16%) for the cancer domain to 7,758 (93%) for the smoking cessation domain. Table 1 shows the medians and interquartile ranges for the shrunken estimates of practice performance on the GPPS questions and QOF clinical quality summary score. Median practice performance on these measures varies from 13.2% to 94.6%, with most showing an appreciable range of practice performance.

Table 1.

Summary Statistics for GPPS Item Scores and QOF Clinical Summary Score (N = 7,759 Practices)

Item	Median	Interquartile Range
GPPS
Q4 Helpful receptionists	83.3	79.5 – 86.9
Q5a Get through on the phone	69.1	59.4 – 78.0
Q5b Speak to a doctor on the phone	53.8	44.1 – 62.1
Q7 See a doctor in the next 2 days	84.1	76.8 – 89.9
Q10 Get an appointment in advance	76.8	65.7 – 86.1
Q16 See preferred doctor	76.0	69.0 – 82.4
Q17 Satisfied opening hours	80.6	77.5 – 83.4
Q20 Doctor patient communication	84.2	81.1 – 86.6
Q21 Confidence and trust in doctor	84.7	81.0 – 87.5
Q24 Nurse patient communication	85.5	83.3 – 87.4
Q25 Overall satisfaction	86.2	82.9 – 88.9
Care planning
Q28a Doctor/nurse took notice of patient views	94.6	93.5 – 95.5
Q28b Given information	90.9	89.5 – 92.2
Q28c Doctor/nurse patient agreement	89.6	87.7 – 91.0
Q28d Given written document	22.9	20.5 – 25.4
Q28e Given a “care plan”	13.2	12.0 – 14.5
Q29 Discussion improved management of health	66.4	63.9 – 68.6
QOF clinical summary score	80.6	78.5 – 82.4

GPPS = General Practice Patient Survey; Q =question; QOF = Quality Outcome Framework.

GPPS items reflect scores on a scale of 0 to 100, and QOF clinical summary scores are a weighted average of percentage achievement.

Table 2 shows the results of the rank correlation analysis (column 2). Although most of the correlations are statistically significant, the strength of the associations is generally weak, with the highest being 0.18 and more than one-half at 0.11 or less. Correlations with clinical quality were highest for patient assessments of access (telephone access 0.16, availability of urgent appointments 0.15, ability to book ahead 0.18, ability to see preferred doctor 0.17) and overall satisfaction (0.15). Correlations for individual clinical conditions are shown in Supplemental Table 1, available at http://annfammed.org/content/11/5/467/suppl/DC1. In the sensitivity analysis (not shown) using unshrunken clinical quality scores, most correlation coefficients in the Supplemental Table changed by less than 0.002, with only 8 changing by more than 0.02.

Table 2.

Coefficients Between GPPS Item Scores and QOF Clinical Summary Score (N = 7,759 Practices)

		Linear Regression Analysis
GPPS Item	Spearman Rank Correlation Coefficientsa	Crudeb	Adjusted for Practice Population Characteristicsc
Q4 Helpful receptionists	0.15	0.15	0.15
Q5a Get through on the phone	0.16	0.15	0.16
Q5b Speak to a doctor on the phone	0.11	0.12	0.11
Q7 See a doctor in the next 2 days	0.15	0.15	0.14
Q10 Get an appointment in advance	0.18	0.16	0.16
Q16 See preferred doctor	0.17	0.16	0.16
Q17 Satisfied opening hours	0.13	0.13	0.13
Q20 Doctor patient communication	0.09	0.10	0.08
Q21 Confidence and trust in doctor	0.10	0.12	0.10
Q24 Nurse patient communication	0.11	0.11	0.10
Q25 Overall satisfaction	0.15	0.17	0.16
Care planning
Q28a Doctor/nurse took notice of patient views	0.04	0.05	0.03
Q28b Given information	0.08	0.09	0.08
Q28c Doctor/nurse patient agreement	0.08	0.09	0.08
Q28d Given written document	0.08	0.09	0.09
Q28e Given a “care plan”	0.11	0.10	0.10
Q29 Discussion improved management of health	0.11	0.12	0.11

GPPS = General Practice Patient Survey; Q = question; QOF = Quality Outcome Framework.

^aAll significant at P <.001, except Q28a, where P = .44 after applying Bonferroni corrections for 374 tests.

^bAll significant at P <.001.

^cAll significant at P <.001, except Q28a, where P =.003.

The results of the regression analysis modeling the association between the QOF clinical summary score and the various GPPS items are also shown in Table 2 (columns 3 and 4). All associations were statistically significant, but the size of the associations is small, with standardized regression coefficients ranging from 0.05 to 0.17. In the sensitivity analysis (not shown) using unshrunken QOF proportions, standardized regression coefficients are smaller in all cases (as expected), by up to 0.03. Adjustment for practice-level case mix leads to a small decrease in the standardized regression coefficients for most GPPS items.

Table 3 shows the proportion of variance in the QOF clinical summary score associated with different groups of GPPS questions. Access has the strongest association with QOF scores, followed by overall satisfaction and continuity of care, care planning, communication, and confidence and trust in the doctor. We can also consider the proportion of variance uniquely associated with each GPPS domain, ie, the variance explained above and beyond other factors. When considered, access and care planning are each uniquely associated with more than 3 times as much variance in QOF scores as any other GPPS domain (not shown in table).

Table 3.

Regression Analysis Showing Proportion of QOF Variance Associated With Different Groups of GPPS Items

Explanatory Variables	Proportion of QOF Variance Associated With GPPS Dimension (R2)
Access (Q4, Q5a, Q5b, Q7, Q10, and Q17)	0.0366
Continuity (Q16)	0.0243
Communication (Q20 and Q24)	0.0139
Overall satisfaction (Q25)	0.0254
Confidence and trust in doctor (Q21)	ç0.0105
Care planning (Q28a, Q28b, Q28c, Q28d, Q28e, Q29)	0.0192

GPPS = General Practice Patient Survey; Q = question; QOF = Quality Outcome Framework.

Note: see Table 1 for description of questions.

Supplemental Table 1 shows a breakdown of correlations by type of quality indicator (recording/review, tests/specialist referral, treatment, intermediate outcome). In general, the correlations were highest for associations between GPPS scores and intermediate clinical outcomes. Intermediate outcome indicators denote percentages of patients achieving, for example, a recommended cholesterol, hemoglobin A_1C, or blood pressure measurement. The associations with GPPS scores were again small, with no correlation coefficients above 0.18.

DISCUSSION

Our results show that across general practices in England there are statistically significant but weak associations between patient-reported experience of care as measured by the GPPS and the technical quality of care as measured by the QOF. These positive associations persist after adjustment for population-level factors (Table 2).

Among the different areas covered by the GPPS, questions about access (ability to get through on the telephone and to make appointments) have the strongest correlations with the clinical scores, followed by questions addressing continuity of care, overall satisfaction, and then care planning. Dimensions of patient experience are themselves interrelated, however, and it is notable that considerably more variation in QOF scores are associated with access and care planning than with any of the other quality dimensions. These results could suggest a more direct causal link between improved access and improved care planning on clinical quality than for other patient experience dimensions. For example, by enhancing access, practices may increase patient encounters and create added opportunities to address aspects of care identified in the QOF. In concordance with these findings, previous research has reported positive correlations between patient-reported access and process measures of clinical quality in family practice.⁶ Furthermore, a previous analysis in England reported positive associations between patient ratings of access drawn from a precursor of the GPPS and clinical quality scores from QOF scores; it also reported that improved access scores were associated with reduced emergency admissions.¹⁸

Associations of overall patient satisfaction with clinical quality were positive in our study, unlike findings from smaller US primary care studies,^3,4 which found no relation between these 2 domains. The lack of a strong association between patients’ overall satisfaction and any technical quality measures may not be surprising, as we know from qualitative research that patients’ assessment of technical quality is heavily colored by communication with the physician.¹⁹ This association is consistent with evidence that patient reports of overall satisfaction correlate closely to communication ratings.²⁰ As such, our results offer little evidence that improved technical care would lead to improved patient satisfaction. Correlations between patient satisfaction and clinical summary scores are consistently positive, however, and this finding challenges the suggestion that, by encouraging doctors to concentrate on technical aspects of care, incentive schemes such as the QOF will lead to deterioration in the doctor-patient relationship.

Our findings are consistent with a situation in which physicians who are committed to high quality in one dimension are somewhat more likely to be committed to high quality in the other, but with the dimensions being sufficiently distinct that improving one has little, if any, direct effect on improving the other. Alternatively, it may be that the modest positive association between satisfaction and clinical quality reflects 2 separate causal links: (1) a strong contribution of access, care planning, and other specific dimensions of patient experience to overall patient satisfaction, and (2) a modest contribution of access and care planning to clinical quality.

Our findings are relevant more widely to health systems investing in health care monitoring and improvement initiatives. The lack of negative correlations in our study tells us that focus on one domain of health care quality need not necessarily be at the expense of another. At the same time, the small magnitude of the associations cautions against relying on a single measure and emphasizes the need to maintain a multidimensional approach by constructing quality indicators across all domains. Such an approach promotes a quality measurement model where such patient priorities as access and continuity are integrated alongside clinical and organizational effectiveness.

Most family practice research on the relationship between quality of care and patient experience has relied on process measures of technical quality. The QOF contains a combination of process and outcome measures. When we split our clinical quality data according to the type of indicators, correlations between outcome measures and patient survey data were higher, compared with correlations between process measures and patient survey data. Overall, satisfaction and continuity (seeing preferred doctor) were patient survey domains that showed the highest correlations with outcome measures of clinical quality. Sequist et al analyzed the relationship at the patient and practice level on US data drawn from the Health Plan Employer Data and Information Set (HEDIS), which considers process and outcome measures of clinical quality separately. In contrast with our results, the study found no positive correlations between patient experience and clinical outcome measures.⁷

The QOF and GPPS are established measures of family practice quality in England. Both are the product of rigorous research and subject to regular revision. Limitations intrinsic to both sources of data may affect our findings, however. Although concerns relating to both a low response rate (38%) and reliability have been raised with respect to the GPPS, there is little evidence that low response rates have introduced bias,¹¹ and research shows that most survey questions used in this study meet stringent guidelines for reliability.²¹ The QOF has been criticized for failing to capture important elements of family practice.²² In fact, many important aspects of clinical care are not measured, either because they are not amenable to reliable and widespread measurement, or because they are yet to be incorporated into the QOF. Even though the QOF remains an imperfect measure of technical quality of care, it is the most reliable and comprehensive measure available.

Our data only allowed for practice-level analysis. We examined to what extent practice scores for patients’ experience correlated with practice scores for clinical quality. Even though this ecological association is of inherent interest, it does not allow us to draw conclusions about patient-level associations. Studies using the individual as the unit of analysis would enable us to further elucidate the relationship between patient experience and quality of technical care.

Our findings support the hypothesis that although there are positive associations between clinical quality of care and measures of patient experience, these 2 domains of care quality remain predominantly distinct, with statistically significant but very low correlations. The strongest correlations are between clinical quality and access. Clinical and patient experience domains of quality need to be considered separately when assessing the overall performance of a family practice.