Do the Quality and Outcomes Framework patient experience indicators reward practices that offer improved access?

Abstract

Background

The Quality and Outcomes Framework (QOF) includes indicators for patient experience, but there has been little research on whether the indicators identify practices that deliver good patient access.

Aim

To determine whether practices that achieved high QOF patient experience points in 2005/2006 or 2006/2007 also delivered good patient access.

Design of study

Use of publicly available data to investigate two hypotheses: practices with more positive access survey findings in 2006/2007 will be more likely to have achieved maximum QOF patient experience points in the same year; and practices with maximum QOF patient experience points in 2005/2006 will have higher access survey findings in 2006/2007.

Setting

Two-hundred and twenty-four East Midlands general practices.

Method

For hypothesis one, binary logistic regression was used, with achievement of maximum QOF points as the dependent variable, and access survey findings, responder variables, and practice variables as independent variables. For hypothesis two, general linear models were used, with access survey findings as the independent variables, and achievement of maximum QOF points and the responder and practice variables as dependent variables.

Results

The findings did not support the first hypothesis. For the second hypothesis, achievement of maximum QOF points was only significantly associated with patient satisfaction with opening hours (positive correlation). QOF points were not associated with any other aspect of access.

Conclusion

The QOF patient experience indicators do not reward practices that offer good patient access. A standard patient survey with financial incentive may be more effective in identifying and rewarding practices that offer better access, including opportunity to book appointments with a particular doctor.

INTRODUCTION

The Quality and Outcomes Framework (QOF) is designed to reward practices that deliver high-quality care. Quality is measured by 146 indicators in four key domains: clinical, organisational, patient experience, and additional services. Practices are allocated points for targets achieved, and the number of points gained is translated into financial reward.¹ The majority of studies investigating the impact of the QOF on quality of care have concentrated on the clinical indicators that account for most of the points that determine the financial reward. Although improvements among the clinical indicators have been reported,^2–5 few studies have investigated the impact of the patient experience indicators.

In 2005–2006 and 2006–2007, four indicators of patient experience were included in the QOF, although two of the 2005–2006 indicators were modified for 2006–2007 (Table 1).⁶ A total of 100 points was allocated to patient experience in 2005/2006, and 108 points in 2006/2007. The indicator PE5 is being dropped for 2008/2009, and PE2 and PE6 are being dropped for 2009/2010.⁷ Although the indicators do not deal specifically with access or continuity, in both years they required practices to undertake a patient survey using one of two approved instruments, these being the General Practice Assessment Questionnaire (GPAQ) and the Improving Practice Questionnaire (IPQ). The GPAQ includes questions on practice opening hours, the number of days' wait for an appointment with a particular doctor or with any doctor, aspects of the consultation, and being able to get through to the practice by telephone.^8,9 The IPQ includes questions on access, continuity, availability of information, and the patient's experience of the doctor's interpersonal skills.¹⁰ Thus, practices using these instruments that achieve maximum scores in the patient experience domain will have undertaken a survey, considered access and continuity, reflected on the results, and enacted changes or a detailed action plan.

Table 1.

The QOF indicators for the years 2005/2006 and 2006/2007.

	Number of points awarded
Indicator code	2005/2006	2006/2007	Indicator
PE1	30	33	The length of routine booked appointments in the practice is not less than 10 minutes. (If the practice routinely sees extras during booked surgeries, then the average booked consultation length should allow for the average number of extras seen in a surgery session. If the extras are seen at the end, then it is not necessary to make this adjustment.) For practices with only an open surgery system, the average face-to-face time spent by the GP with the patient is at least 8 minutes. Practices that routinely operate a mixed economy of booked and open surgeries should report on both criteria.
PE2	40	25	The practice will have undertaken an approved patient survey each year.
PE3	15	–	The practice will have undertaken a patient survey each year, will have reflected on the results, and have proposed changes if appropriate.
PE4	15	–	The practice will have undertaken a patient survey each year and discussed the results as a team and with either a patient group or non-executive director of the primary care organisation (PCO). Appropriate changes will have been proposed with some evidence that the changes have been enacted.
PE5	–	20	The practice will have undertaken a patient survey each year and, having reflected on the results, will produce an action plan that: summarises the findings of the survey; summarises the findings of the previous year's survey; and reports on the activities undertaken in the past year to address patient experience issues.
PE6	–	30	The practice will have undertaken a patient survey and, having reflected on the results, will produce an action plan that: sets priorities for the next 2 years; describes how the practice will report the findings to patients (for example, posters in the practice, a meeting with a patient practice group or a PCO-approved patient representative); describes the plans for achieving the priorities, including indicating the lead person in the practice; and considers the case for collecting additional information on patient experience, for example through surveys of patients with specific illnesses, or consultation with a patient group.

In 2006/2007, a new financial incentive (called a directed enhanced service or DES) was introduced in England to improve patient access to primary care.¹¹ This provided a reward to practices that offered patients the opportunity to consult a GP within 2 working days, the opportunity to book appointments in advance, ease of telephone access, and opportunity for the patient to consult their practitioner of preference.¹² One-third of the reward was offered to practices that aspired to these goals, and two-thirds awarded according to the findings of a new survey of patient access.¹³

How this fits in

The Quality and Outcomes Framework is a major initiative intended to transform the delivery of primary care services. Some clinical indicators have improved following introduction of the Framework, but concerns remain about the value for money of the scheme. This study suggests that current patient experience indicators do not identify practices that provide good patient experience of access.

This paper reports a study of the patient experience QOF points and patient access survey findings. It was hypothesised that:

• practices with more positive patient access survey findings in 2006/2007 will be more likely to have achieved maximum QOF patient experience points in the same year (that is, QOF patient experience points and access survey results are associated); and

• that practices with maximum QOF patient experience points in 2005/2006 will have a higher access survey score in 2006/2007 (that is, practices that undertake their own patient survey and enact appropriate changes if indicated will provide improved access in the following year).

METHOD

All practices in three local primary care trusts (PCTs: Leicester City, Leicestershire County and Rutland, and Northamptonshire) that had completed the access survey in 2007 were included. QOF data for the years 2005/2006 and 2006/2007 were obtained from the NHS Information Centre.¹⁴

The patient access survey was administered by MORI to around 5.2 million people registered with 8241 practices between January and March 2007.¹⁵ Approximately 2.3 million people responded (44% response rate). The survey questionnaire was sent to systematic samples of patients aged ≥18 years who were registered with each practice, the sample size being adjusted according to practice size. The samples were drawn from either general practice computer systems or the registration details on a central NHS database (National Health Application and Infrastructure Services). The average sample size was 432 patients per practice (699 in those practices in which computer records could not be used to eliminate patients who had not had an appointment in the past 6 months).¹⁵ The questionnaire asked five questions about aspects of access: whether they had been able to get an appointment within 2 days, able to book in advance (more than 2 days in advance), and able to get an appointment with a particular GP, and satisfaction with (a) ease of getting through to the practice on the telephone, and (b) opening hours. The questionnaire also sought information on responders' sex, age, and ethnic group, and numbers of consultations in the past year. The data are publicly available.¹⁶ Data were also obtained on numbers of patients registered with each practice and a measure of the level of socioeconomic deprivation of the practice population (the Index of Multiple Deprivation (IMD) 2007).¹⁷

Statistical analysis

Following descriptive reporting of the data, regression models were undertaken for each hypothesis. For the first hypothesis, binary logistic regression was undertaken, with QOF points in 2006/2007 as the dependent variable, grouped into two categories: maximum points for patient experience and less than maximum points. The total patient experience points was the sum of PE2, PE5, and PE6; PE1 being omitted since it related to length of consultations and not access. The independent variables were the five access questions, access survey responder variables: age group (percentage of responders aged 18–44, 45–64 and ≥65 years); percentage of responders who were male; percentage giving their ethnicity as white (white British, white Irish, or other white category); percentage response rate for the practice; the percentage of responders reporting different numbers of appointments with their practice in the past year (categorised into 0, 1–4, and ≥5), and practice variables: number of patients in the practice; the practice IMD score; and the PCT of the practice. For the second set of hypotheses, the five access questions were the dependent variables (each dependent variable was arcsine transformed) in five general linear models using backward stepwise regression, the independent variables being achievement of maximum QOF score in 2005/2006, plus the responder and practice variables. Analyses were undertaken with SAS (version 9.1).

RESULTS

Of the 230 practices in the three PCTs, 224 registered for the enhanced access scheme patient surveys. Maximum QOF patient experience points were achieved by 183 practices across both years, with 201 just in 2005/2006 and 188 just in 2006/2007. Practice characteristics are shown in Table 2, and the percentages of responders answering positively to the five access survey questions are shown in Table 3, according to whether practices did or did not achieve maximum QOF patient experience points in 2005/2006.

Table 2.

Practice characteristics (n = 224).

	Less than maximum QOF points, 2005/2006	Maximum QOF points in 2005/2006
Number of practices	17	201
Number of patients/practice, mean (SD)	4229 (2742)	7563 (4667)
IMD score, mean (SD)	25.42 (10.81)	19.13 (11.09)
Percentage of ethnic minority patients among responders, median (IQR)	94 (80–95)	96 (85–99)
Practice response rate to 2006/2007 access survey, median (IQR)	46 (38–50)	51 (43–57)
Percentage of male responders, median (IQR)	44 (40–45)	42 (39–44)
Percentage of responders aged:
≤44 years, median (IQR)	32 (24–36)	30 (25–36)
45–64 years, median (IQR)	36 (32–39)	35 (29–39)
≥65 years, median (IQR)	34 (24–39	31 (26–36)

IMD = Index of Multiple Deprivation. IQR = interquartile range. SD = standard deviation.

Table 3.

Access survey results, 224 practices, according to achievement of maximum QOF patient experience points 2005/2006.

	Less than maximum QOF points, mean (SD)	Maximum QOF points, mean (SD)	P-valuea
Able to make appointment with particular doctor	88.2 (8.5)	85.0 (10.3)	0.874
Satisfied with opening hours	80.7 (7.8)	83.7 (6.8)	0.927
Satisfied with telephone access	84.1 (12.7)	83.7 (14.1)	0.935
Able to book appointments ≥2 days in advance	68.7 (19.3)	68.4 (21.7)	0.078
Able to access a GP within 48 hours	87.1 (7.6)	86.9 (9.4)	0.055

^aIndependent t-test. SD = standard deviation.

The regression analysis did not support the first hypothesis; there was no association between QOF points and the access survey findings. The only variable associated with achievement of maximum QOF patient experience was the number of patients registered with the practice (odds ratio 1.0002, 95% confidence interval = 1.0001 to 1.0003, P = 0.001, Hosmer and Lemeshow test of goodness of fit, non-significant [P = 0.23]). Figure 1 shows the fitted probability of achieving maximum points with increasing numbers of patients.

Figure 1.

Plot of total patients against probability of practices achieving maximum QOF points, 2006/2007.

Table 4 presents the results of the regression models for each of the five questions of the access survey, showing which independent variables were associated with increased satisfaction in response to the five access questions. The analysis failed to support the second hypothesis that conducting a patient survey and enacting changes in response to the findings as required by the 2005/2006 QOF indicators was associated with improved access survey findings in the following year. Only in the case of satisfaction with opening hours was there a significant association between QOF patient experience points and experience of access. With respect to satisfaction with opening hours, Table 4 shows that in practices that achieved maximum QOF patient experience points, more patients tended to report satisfaction; satisfaction tended to be lower in practices with larger patient list sizes, and lower if a greater proportion of patients reported having one to four appointments in the past year rather than five or more appointments, or no appointments; satisfaction was higher in practices with a higher IMD score, higher when the proportion of white responders was higher, and higher as the response rate increased.

Table 4.

Prediction of access survey results in 2006/2007 by QOF patient experience points in 2005/2006 (general linear models, backward stepwise regression used to eliminate non-significant predictors).

	Regression coefficient (β)	P-value
Access survey question: satisfied with opening hours
Practice achieved maximum patient experience QOF score	0.08640	<0.001
Total patient list size	−0.00001	<0.001
Responders having 1–4 appointments in last 12 months, %	−0.00560	<0.001
IMD 2007	0.00246	0.008
Responders of white ethnicity, %	0.00293	<0.001
Response rate, %	0.00317	<0.001
r2 = 0.39a
Access survey question: % of responders able to see a particular doctor
Total patient list size	−0.00001	<0.001
Practice IMD 2007 score	−0.00321	0.038
Responders of white ethnicity, %	0.00314	<0.001
Response rate, %	0.00463	0.031
Responders aged 18–44 years, %	0.00327	0.046
r2 = 0.33a
Access survey question: % of responders able to book an appointment in advance
Total patient list size	−0.00002	<0.001
Responders of white ethnicity, %	0.00479	<0.001
Response rate, %	0.01120	<0.001
Responders aged 18–44 years, %	0.00780	<0.001
Male responders, %	0.00760	0.026
r2 = 0.11a
Access survey question: % of responders satisfied with phone access.
Total patient list size	−0.00002	<0.001
Responders of white ethnicity, %	0.00479	<0.001
Response rate, %	0.01120	<0.001
Responders aged 18–44 years, %	0.00780	<0.001
Male responders, %	0.00760	0.026
r2 = 0.31a
Access survey question: % or responders able to see a GP in 48 hours.
Total patient list size	−0.00001	<0.001
Responders of white ethnicity, %	0.00240	<0.001
Response rate, %	0.01100	<0.001
Responders aged 18–44 years, %	0.00700	<0.001
Responders having 1–4 appointments in last 12 months, %	−0.00650	0.001
r2 = 0.28a

^ar² = total variance explained by the model. IMD = Index of Multiple Deprivation.

Increasing size of practice was associated with declines in patient experience of all aspects of access (opening hours, being able to see a particular doctor, telephone access, being able to see a GP in 48 hours, and being able to book an appointment in advance). People who were white, younger, and male tended to experience better access. Improved access was also associated with a higher practice response rate to the survey. This suggests that the data were not missing at random, and that there may be a bias operating.

DISCUSSION

Summary of main findings

No association was found between achievement of maximum QOF patient experience points and patient experience of access in 2006/2007 (hypothesis one). Furthermore, there was an association between QOF patient experience points in 2005/2006 and patient-reported access in 2006/2007, but only for satisfaction with opening hours (hypothesis two); for all four other aspects of access there was no significant association. In contrast, larger practices were more likely to achieve maximum QOF patient experience points,¹⁸ but smaller practices were more likely to have better patient-reported access.

Strengths and limitations of the study

This study is the first to the authors' knowledge to bring together QOF indicators and findings of the national patient access survey. It was undertaken in an area with an ethnically and socioeconomically diverse patient population, and since almost all the practices took part in the access survey, the majority of practices were included. There are some limitations to the access survey data. The published data give information on percentages of responders in different age, sex, and ethnic groups, but not the numbers. Information on practice non-responders is not available. While it was possible to include a wide variety of potentially explanatory variables relating to the responders and practices, it is likely that other variables explaining either QOF points or access survey results were omitted. It should also be pointed out that the response of patients to surveys may not change quickly following changes in service provision, a factor that might explain some of the discrepancy between the QOF patient experience indicators and access survey findings. Nevertheless, the study raises questions about the ability of the QOF patient experience indicators to identify practices that offer better patient experience of access.

Comparison with existing literature

In addition to questions on access, the national patient survey has questions relevant to aspects of continuity: being able to book an appointment in advance and being able to consult a preferred practitioner.¹⁹ Concern has been expressed about the impact of the QOF on continuity,²⁰ and although the present study is unable to determine whether the QOF has impaired continuity, it does indicate that the QOF patient experience indicators are failing to encourage practices to improve these two aspects of continuity. This could have to do with the design of the QOF, which may encourage a more clinical or disease-orientated type of care that can sometimes conflict with the patient's own agenda.²¹ It has been established previously that patients tend to report that larger practices offer worse access in comparison with smaller practices.^22,23 The present findings indicate that larger practices still have difficulty ensuring high levels of access and continuity. The large new primary care providers being established in some cities will need to pay particular attention to access and continuity.

Implications for clinical practice

Improvements in some clinical indicators following the introduction of the QOF have been established,^2–5 but questions continue to be raised about the value for money of the scheme. A recent parliamentary report concluded that the QOF had not led to improved access to general practice,²⁴ and two patient experience indicators are being removed from the QOF.⁷ The present study supports the view that the patient experience indicators have not been successful in promoting improved access or continuity. The study does not show why the QOF has not been successful. It is possible, for example, that practices have been attempting for many years to improve access and have already reached the best attainable level of performance. Other explanations include competing priorities for practices within the QOF, and the failure of the QOF indicators to target activities that have an impact on patient experience. Since most practices achieved maximum points for the patient experience indicators, it may be argued that the indicators are not effective in discriminating between different levels of performance. An independent patient survey that directly investigates patients' experiences of access, coupled to a financial incentive, may be a more effective means of promoting improved access and continuity.

Funding body

No external funding was used in undertaking this study

Ethics committee

Ethics approval for this study was not required since it was based on publicly available data

Competing interests

Professor Baker was chair of the expert advisory group on the QOF patient experience indicators, 2006–2008. There are no other potential conflicts of interest to report.

Discuss this article

Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp-discuss