Pay-for-Performance in the United Kingdom: Impact of the Quality and Outcomes Framework—A Systematic Review

Abstract

PURPOSE

Primary care practices in the United Kingdom have received substantial financial rewards for achieving standards set out in the Quality and Outcomes Framework since April 2004. This article reviews the growing evidence for the impact of the framework on the quality of primary medical care.

METHODS

Five hundred seventy-five articles were identified by searching the MEDLINE, EMBASE, and PsycINFO databases, and from the reference lists of published reviews and articles. One hundred twenty-four relevant articles were assessed using a modified Downs and Black rating scale for 110 observational studies and a Critical Appraisal Skills Programme rating scale for 14 qualitative studies. Ninety-four studies were included in the review.

RESULTS

Quality of care for incentivized conditions during the first year of the framework improved at a faster rate than the preintervention trend and subsequently returned to prior rates of improvement. There were modest cost-effective reductions in mortality and hospital admissions in some domains. Differences in performance narrowed in deprived areas compared with nondeprived areas. Achievement for conditions outside the framework was lower initially and has worsened in relative terms since inception. Some doctors reported improved data recording and teamwork, and nurses enhanced specialist skills. Both groups believed that the person-centeredness of consultations and continuity were negatively affected. Patients’ satisfaction with continuity declined, with little change in other domains of patient experience.

CONCLUSIONS

Observed improvements in quality of care for chronic diseases in the framework were modest, and the impact on costs, professional behavior, and patient experience remains uncertain. Further research is needed into how to improve quality across different domains, while minimizing costs and any unintended adverse effects of payment for performance schemes. Health care organizations should remain cautious about the benefits of similar schemes.

INTRODUCTION

Strong primary care is widely accepted to be a prerequisite for effective, efficient, equitable health systems and to lead to better population health.¹ Introduced in 2004, the UK Quality and Outcomes Framework (QOF) is arguably the most comprehensive national primary care pay-for-performance (P4P) scheme in the world.² The QOF is more than a payment scheme; it is a complex intervention comprising a number of elements (Table 1), including financial incentives and information technology (computerized prompts and decision support), designed to promote structured and team-based care with the aim of achieving evidence-based quality targets.³

Table 1.

Domains of the Quality and Outcomes Framework

Clinical

Secondary prevention of coronary heart disease

Cardiovascular disease: primary prevention

Heart failure

Stroke and transient ischemic attack

Hypertension

Diabetes mellitus

Chronic obstructive pulmonary disease

Epilepsy

Hypothyroidism

Cancer

Palliative care

Mental health

Asthma

Dementia

Depression

Chronic kidney disease

Atrial fibrillation

Obesity

Learning disabilities

Smoking

Organizational

Records and information

Information for patients

Education and training

Practice management

Medicines management

Patient Experience

Length of consultations

Patient survey (access)

Additional Services

Cervical screening

Child health surveillance

Maternity services

Contraception

It was one component in the reorganization of primary care resulting from a new General Medical Services contract for general practitioners that led to a practice-based, rather than practitioner-based, contract and investment to reward quality of care through both fixed and performance-related funding streams. The financial incentives are substantial, with a maximum of 1,000 points available to practices, and an average payment per practice in 2011–2012 of £130 (US $204) for each point achieved.⁴ More than one-half of these points are allocated to clinical indicators, which currently cover 20 chronic conditions.⁵

In 2009–2010, practices in England achieved an average of 937 points, with a range in each of the 152 primary care trusts from 878 to 972 points.⁶

Since its introduction, the effects of the QOF on quality of care have been the subject of considerable debate, which is now being informed by an accumulating body of research. As national governments seek to improve the quality of health systems in the face of financial stringency, searching analysis of this evidence is timely. The successes and failures of the QOF as a national centralized system may be predictive of the effects of P4P in the United States.⁷

P4P schemes have been extensively reviewed.^7–14 A recent Cochrane review found that, whereas they improved patients’ well-being, the effects of financial incentive schemes on the quality of primary health care were “modest and variable.”¹⁵ Previous reviews have focused on a single dimension of care or have had strict inclusion criteria with few articles retrieved or have not been conducted systematically. The huge variation in P4P schemes in different countries has made it difficult for reviewers to draw generalizable conclusions, whereas the uniform design of the QOF lends itself to close scrutiny. The research evidence about the QOF has grown rapidly and merits systematic review. We sought to examine the impact of the QOF on the quality of UK primary medical care, using broad inclusion criteria.

METHODS

We searched MEDLINE, EMBASE, and PsycINFO databases to identify all publications that sought to evaluate the QOF. The following search terms were used as free text in the title, abstract, or key words: (quality outcomes framework) OR (QOF) OR (pay for performance) OR (P4P) OR (pay-for-performance) AND (England) OR (Scotland) OR (Wales) OR (UK) OR (United Kingdom) OR (Great Britain). We limited the search to publications from January 1, 2004, to July 31, 2011, and limited the search to publications in the English language. We retrieved 575 references from MEDLINE (n = 348), EMBASE (n = 294), and PsycINFO (n = 55). We further searched by hand the reference lists of these articles for additional relevant studies. After eliminating duplicates and screening abstracts for relevance, 305 articles were excluded. The remaining 124 were read and rated by 2 authors (S.G. and N.S.) using a modified Downs and Black rating scale for observational studies (n = 110) and a Critical Appraisal Skills Programme rating scale for qualitative studies (n = 14). Thirty articles were excluded because they did not meet quality scores (less than 5 of 7 for observational or less than 7 of 10 for qualitative studies), did not evaluate the effect of the QOF, were a repeat publication, or were a review of previous research. Discrepancies in coding and areas of disagreement were resolved through discussion and adjudication by the third author (A.N.S.). The flow of information through the review is presented as a PRISMA flowchart (Figure 1).¹⁶

Figure 1.

Flow of information through the review.

Several typologies have been used to define quality in primary care,^17–20 and the Institute of Medicine’s 6 dimensions of quality have been widely used: safety, effectiveness, patient-centeredness, timeliness, efficiency, and equity.²⁰ We used these dimensions to categorize the included studies.

RESULTS

Our search and review retained 94 articles, which we have categorized into 4 areas: effectiveness, efficiency, equity, and patient experience. These correspond to dimensions of the Institute of Medicine’s definition,²⁰ with the exception of safety and timeliness, for which we did not find any relevant studies. Several studies have been considered in more than one of these dimensions. Most of the qualitative studies examined professionals’ experience and team working and were considered separately as a fifth area.

Effectiveness

Supplemental Table 1 (47 studies) displays descriptions of the studies on the impact of the QOF on effective care (available at http://annfammed.org/content/10/5/461/suppl/DC1). The QOF has helped consolidate evidence-based methods for improving care by increasing the use of computers, decision support, clinician prompts, patient reminders, and recalls.²¹ It has resulted in better recorded care, enhanced processes, and improved intermediate outcomes for most conditions, notably diabetes. These improvements decreased after the first year of the QOF, however, and subsequent increases have followed secular trends. For example, Doran et al showed that achievement rates improved for most indicators in the preincentive period. There were significant increases in the rate of improvement in the first year of the incentive scheme (2004–2005) for 22 of the 23 incentivized indicators. These reached a plateau after 2004–2005, but quality of care in 2006–2007 remained higher than predicted by preincentive trends for 14 incentivized indicators. There was no overall effect on the rate of improvement for nonincentivized indicators in the first year of the scheme, but by 2006–2007 achievement rates were significantly below those predicted by preincentive trends.²² These improvements have been accompanied by rising prescription rates in associated drug categories.²³ Performance improvements for those conditions that were not included in the QOF were significantly lower than for incentivized indicators, and these differences increased over time.^22,24

Overall, modest population mortality reductions have been estimated.²⁵ Fleetcroft et al modeled a potential saving of 11 lives per 100,000 people per year aggregated across all clinical indicators and domains in the first year of the contract, with no further gain in the second year as performance for a typical practice already exceeded the target payment levels.²⁶

Efficiency

Supplemental Table 2 (5 studies) displays descriptions of the studies on the impact of the QOF on efficiency and costs (available at http://annfammed.org/content/10/5/461/suppl/DC1). There is limited evidence that increasing the quality of ambulatory care may reduce admission rates and hence costs for some conditions.²⁵ For example, epilepsy care as incentivized by the QOF was associated with fewer epilepsy-related emergency admissions.²⁷

There have been few attempts to model the cost effectiveness of QOF attainments.^28,29 Walker et al could assess only a minority of indicators, and concluded that QOF incentive payments were cost-effective, even with only modest improvements in care, although they took no account of the costs of administering the QOF scheme. They found average indicator payments ranged from £0.63 to £40.61 per patient, and the percentage of eligible patients treated ranged from 63% to 90%. The improvements in performance required for QOF payments to be cost-effective varied by indicator from less than 1% to 20%. There was no relationship between the size of payments in a clinical domain and the likely resulting health gain.^30,31

Equity

Supplemental Table 3 (25 studies) displays descriptions of the studies on the impact of the QOF on equity (available at http://annfammed.org/content/10/5/461/suppl/DC1). The QOF was not specifically designed to reduce health inequalities resulting from socioeconomic disadvantage. Nevertheless, inequalities in processes of care comparing the most and least deprived areas have narrowed. For example, Doran et found that the gap in median achievement comparing practices from the most deprived and least deprived quintiles narrowed from 4% to 0.8% between 2004 and 2007.³² In contrast, there have been variable effects on inequalities in care for long-term conditions based on age, sex, and ethnicity. Differences in care according to age for cardiovascular disease and diabetes narrowed after the introduction of the QOF as a result of greater improvement in those patients (usually older groups) receiving worse care. Disparities between men (who more often received better care) and women for cardiovascular disease and diabetes persisted or increased. Similarly, ethnic variations in quality of care have been reduced. For example, Millet et al found that improvements in blood pressure control were greater in the black group than the white, with disparities evident at baseline being attenuated (black 54.8% vs white 58.3% reaching target in 2005). Lower recording of blood pressure in the south Asian group, evident in 2003, was attenuated in 2005.³³ The QOF has encouraged greater consistency of care irrespective of deprivation, but the practitioners’ option to exclude (exception report) hard-to-reach patients from the population used to determine payment may limit its impact on health inequalities.³⁴

Patient Experience

Supplemental Table 4 (7 studies) displays descriptions of the studies on the impact of the QOF on patient experience (available at http://annfammed.org/content/10/5/461/suppl/DC1). There were no significant changes in quality of care reported by patients between 2003 and 2007 for communication, nursing care, coordination, or overall satisfaction.³⁵

Continuity of care worsened for patients with chronic disease, and only access to urgent appointments improved significantly but modestly for these patients but not for adult patients more generally; overall, patients reported seeing their usual physician less often and gave lower satisfaction ratings for continuity of care.³⁶ The few detailed ethnographic studies suggest that some practice teams have changed their consultations and clinical care in ways that may result in patients receiving a more biomedical type of care.³⁷ There are also health professionals who acknowledge that an emphasis on protocol-driven care (“box-ticking”) may have distracted them from patient-led consultations and listening to patients’ concerns.³⁸

Professionals and Team Working

Supplemental Table 5 (6 studies) displays descriptions of the studies on the impact of the QOF on professionals and team working (available at http://annfammed.org/content/10/5/461/suppl/DC1). Interviews with doctors and nurses suggest that the QOF has had positive effects on practice organization. For example, on team working and the diversification of nursing roles, both groups acknowledge that an enhanced role for nurses in managing long-term conditions could result in potential deskilling of doctors. They regret a decline in personal continuity of care between doctors and patients.^39,40 The QOF appears to have introduced new hierarchies within practice teams and led to greater stratification of medical roles.⁴¹ Some team members resent not benefiting financially from payments, and there are concerns about an ongoing culture of performance monitoring in the United Kingdom.⁴² Quality of care may have become too narrowly focused on QOF domains and targets, with less regard to other areas for practice development, innovation, and quality improvement.⁴³

DISCUSSION

There are inevitably conflicting findings from this large and diverse body of research, but some consistent themes have emerged. The implementation of the QOF has helped consolidate evidence-based methods.⁴⁴ It has been associated with an increased rate of improvement of quality of care during the first year of implementation, returning to preintervention rates of improvement in subsequent years. There have been modest reductions in mortality and hospital admissions in some areas, and where they have been assessed, these modest improvements appear cost-effective. The QOF has led to narrowing of differences in performance in deprived areas compared with areas not deprived.⁴⁵ It has strengthened team working.

The effect of the QOF in unincentivized areas has been disappointing. Prescription rates for antidepressants, statins, and other drugs have increased, but this increase is not clearly attributable to the QOF.^46–48 The costs of administering the scheme are substantial, and some staff are concerned that primary care has become more biomedical in focus and less patient centered.

The QOF has strengthened team working and promoted a diversity of new roles, especially for nurses. Indeed, the QOF may have diminished the workload of general practitioners, enabled them to concentrate on more complex care, and led to teams in which work and knowledge is more distributed among its members.

The QOF has been described as scientific bureaucratic medicine, where indicators and guidelines are perceived as threatening professionalism in various ways.⁴⁹ For better or worse, the QOF can be seen to have reduced clinical autonomy and provided performance data that can be used to compare clinicians nationally.

Remarkably little is known of what patients make of these changes, although anecdotal reports point to unintended consequences detracting from patient-centered care.⁵⁰ The fear expressed by some that adherence to single disease-based guidelines might override respect for patient autonomy, lead clinicians to ignore comorbidities, promote a mechanistic approach to chronic disease management, or reduce clinical practice to a series of dichotomized decisions at the expense of personal aspects of care,⁵¹ has not been borne out by the research to date.

Strengths and Limitations

The great majority of studies used statistical analyses of trends or before and after comparisons, there being no possibility of controlled trials. The influence of many other regulatory, workforce-related, and educational changes on the quality of general practice is hard to disentangle. The development of evidence-based medicine, guidelines, and the introduction of other contractual incentives predated the QOF.

Some of the most intriguing findings, particularly concerning the culture of care and professional and patient experience, derived from the small number of qualitative studies. The incorporation of qualitative research into the conventional processes of systematic review presents epistemological and methodological challenges that are unresolved.⁵² Strengths of this review are that it has been conducted systematically, it includes more studies on the QOF than any previous review, and it considers a broad range of outcomes.

To what extent the apparent improvements in quality of care are the result of improved data entry remains unclear, but some of the documented improvement is likely to be due to recording of care previously delivered.⁵³ Several factors impair the QOF’s impact at the population level. Setting targets below 100% and the process of exception reporting reduces the public health effectiveness of population targets by shifting the focus of the practice away from patients who are harder to reach.⁵³ More fundamentally, payment for adhering to guidelines cannot be assumed to improve health status, regardless of whether it improves performance: improved processes (eg, treating hypertension) may not always translate to improved outcomes (eg, stroke prevention); and other powerful confounding influences affect outcomes, such as differential access to care, nonmodifiable risk factors (genetics, familial), or patterns of comorbidity. Process measures are often preferred for incentive schemes, as they are under the control of the health system and can be more efficient.^7,54 The QOF’s evidence base will only ever be partial because its indicators by their very nature will focus on measuring the measurable.

Implications for Policy

The lessons that policy makers draw must, of course, take account of the different historical and organizational contexts in which their health system operates. A sensible verdict regarding the QOF’s effectiveness must balance a nuanced assessment of health and other gains against its costs, many of which are hard to describe, let alone quantify.

The system of P4P and how it is designed will affect how it professionals feel and behave. Family doctors in the United Kingdom, despite feeling that the QOF has changed the nature of the consultation, appear less negative toward P4P than doctors working in California, where lack of new funding, rewards directed at organizations rather than individual physicians, lack of identification with or ownership of measures, and thresholds rather than incremental levels of improvement have led to resentment and greater evidence of dysfunctional or coercive behavior toward patients regarded as noncompliant.⁵⁵ It may be that the nature of medical practice is too complex to be improved by simple financial incentives.⁵⁶

The limited evidence for cost-effectiveness and opportunity cost of the scheme is a central critique for the QOF’s detractors. If £1 billion a year of additional funding to general practice has yielded only modest improvements in measured quality of care, might greater benefits have been achieved if this investment had funded an alternative approach to quality improvement? The opportunity costs of the QOF are to a great extent unknown and unknowable, but the imperfect evidence available suggests that the same benefits could be maintained at reduced cost, particularly if systems are designed to involve clinicians and align with their values.^57,58 More sophisticated modeling is required.

Developing the QOF

Although some have argued for discarding the QOF, it seems wiser to concentrate on addressing weaknesses rather than throwing away the gains. There is no reason why both technical aspects of quality and personal care cannot improve together.⁵⁹ The involvement of the National Institute for Health and Clinical Excellence has greatly strengthened the QOF’s scientific underpinnings.⁶⁰ There will always be a fine judgment about timing, level of evidence required, and whether to accept a consensus rather than evidence-based indicator. An argument for greater consistency of care should not prevail where evidence is lacking. The evidence base for existing indicators needs to be under constant review. Some indicators for which performance has reached a ceiling may need to be retired,⁶¹ although performance may not be maintained,⁶² and new indicators should be introduced after piloting.⁶³

Gaming is known to occur in many systems that are driven by P4P; however, there has been little evidence of gaming in the QOF despite, or perhaps because of, a rigorous system of checks at various levels.⁶⁴ On the contrary, practices are exceeding the upper payment thresholds and levels of exception reporting continue to fall year on year.⁶⁵ Nevertheless, vigilant monitoring systems are needed. The balance of fixed vs performance-related funding should be reviewed. There is merit in linking the size of financial rewards to the public health impact of attaining individual indicators.⁶⁶

In view of our findings that observed improvements in care from a very large payment-for-performance scheme in the United Kingdom were modest, with uncertainty about possible adverse effects, we recommend that policy makers continue to exercise caution about implementing similar schemes. Consideration should be given to improving different dimensions of quality, including user experience and equity. Costs should be monitored and balanced against benefits. Wherever possible, schemes should be designed in collaboration with health service researchers to evaluate the benefits of minor differences in system design. Payment for performance is still an imperfect approach to improving primary care, and should be considered as only one option alongside alternative quality improvement methods.

Future Research

The conclusions of this review are based on the available observational evidence, with all its limitations. They raise many questions about the design and implementation of payment for performance in health care. There is a clear need for more experimental research in many areas, and we suggest 5 high-priority areas. First, does the size of incentive payment affect achievement? Psychological research has surprisingly found that large incentives for tasks requiring greater levels of cognition may lead to lower levels of achievement, yet no research has been done on this finding in health care.⁵⁶

Second, how can the patient-user experience be better assessed and more directly linked to the payment of financial incentives? Third, do incentives lead to a trade-off between technical and patient-centered dimensions of quality, or can they produce improvements across different dimensions of quality? Fourth, what effect do incentives have on such harder-to-measure outcomes as the interpersonal aspects of care and care for underserved populations? Fifth, what is the optimum time for a quality indicator to be included in a payment scheme before being reviewed or replaced by a different incentive?