Abstract
Background
Improving care for people with long-term conditions is central to NHS policy. It has been suggested that the Quality and Outcomes Framework (QOF), a primary care pay-for-performance scheme that rewards practices for delivering effective interventions in long-term conditions, does not encourage high-quality care for this group of patients.
Aim
To examine the evidence that the QOF has improved quality of care for patients with long-term conditions.
Design and setting
This was a systematic review of research on the effectiveness of the QOF in the UK.
Method
The authors searched electronic databases for peer-reviewed empirical quantitative research studying the effect of the QOF on a broad range of processes and outcomes of care, including coordination and integration of care, holistic and personalised care, self-care, patient experience, physiological and biochemical outcomes, health service utilisation, and mortality. Because the studies were heterogeneous, a narrative synthesis was carried out.
Results
The authors identified three systematic reviews and five primary research studies that met the inclusion criteria. The QOF was associated with a modest slowing of both the increase in emergency admissions and the increase in consultations in severe mental illness (SMI), and modest improvements in diabetes care. The nature of the evidence means that the authors cannot be sure that any of these associations is causal. No clear effect on mortality was found. The authors found no evidence that the QOF influences integration or coordination of care, holistic care, self-care, or patient experience.
Conclusion
The NHS should consider more broadly what constitutes high-quality primary care for people with long-term conditions, and consider other ways of motivating primary care to deliver it.
Keywords: incentive reimbursement; primary health care; quality indicators, health care; quality of health care
INTRODUCTION
The UK’s Quality and Outcomes Framework (QOF) is the world’s largest pay-for-performance scheme in primary care. It rewards general practices financially for delivering interventions and achieving patient outcomes using evidence-based indicators developed by the National Institute for Health and Care Excellence (NICE).1 Although the QOF is voluntary, nearly 99% of practices in England participate, on average deriving 10–15% of total practice income from the scheme.2
The introduction of the QOF in 2004 was a part of a new national contract for GPs, driven by the need to respond to years of underinvestment in general practice compared with other parts of the health service, low morale among GPs, and variations in the quality of primary medical care.3,4 The QOF was intended to provide a mechanism to motivate GPs and to increase funding for their practices, and the vast majority of practices took up the opportunity for additional income. Evidence from the early years of the scheme suggested it reduced variations between practices in the delivery of incentivised interventions,5 and contributed to progress towards better use of electronic records and nurse-led multidisciplinary care of long-term conditions.3 After the first year of the QOF, most practices achieved near-maximum remuneration from the scheme.2
Arguably, then, the QOF achieved what it set out to do. But this may have come at a cost. It has been suggested that practices prioritise QOF-related activities at the expense of other aspects of care, because of their reliance on QOF income.6,7
A decade after the introduction of the QOF, NHS strategy, set out in the 2014 Five Year Forward View,8 is now focused on other challenges. These include finding new ways to manage people with long-term conditions, whose care is estimated to consume 70% of health service resources.8 Most clinical QOF indicators relate to the care of long-term conditions and are based on good evidence,4 but tend to be limited in scope, focusing on single, biomedical dimensions of care. Appendix 1 provides a brief description of the 68 QOF indicators relating to care of long-term conditions in 2016–2017; the total number of indicators for that year was 77.
In 2015, the Royal College of General Practitioners called for the replacement of the QOF to allow GPs ‘to focus on providing the best possible holistic care’.9 NHS England, in April 2016, undertook to review the QOF, acknowledging that it may have ‘served its purpose’ and may be ‘a barrier to holistic management’.10 In early 2017, the British Medical Association called for the QOF to be suspended to reduce bureaucratic pressures and free up clinical time.11 Scotland abolished the QOF in 2016.3
The Policy Research Unit in Commissioning and the Health Care System was commissioned to undertake a review, led by the Centre for Health Services Studies at the University of Kent, to report in September 2016. The authors aimed to examine the evidence that the QOF has improved care and outcomes for patients with long-term conditions, including elements of care highlighted as priorities in the Five Year Forward View,8 such as coordinated and integrated care, holistic and personalised care, and self-care.
How this fits in
The usefulness of the Quality and Outcomes Framework (QOF) as a tool for promoting progress towards the vision of the Five Year Forward View for care of long-term conditions has been questioned. This systematic review found no convincing evidence that the QOF can promote better integrated care, personalised, holistic care, or self-care — or, indeed, improve any other outcomes in people with long-term conditions. The NHS should consider other ways of supporting general practice to deliver the vision of the Five Year Forward View.
METHOD
The authors searched for reports of empirical quantitative research examining the effectiveness of the QOF in the management of long-term conditions, published in peer-reviewed journals in English. They included studies of populations registered with GPs in the UK, and excluded studies of locally designed and implemented pay-for-performance schemes, and studies of limited geographical scope (which were defined as examining data from fewer than four primary care trusts in England, or fewer than 100 practices in Scotland) because of likely low generalisability. The authors included studies where the comparator was any other method of funding general practice, concurrent or historical, and, if there was no concurrent comparator, where the analysis controlled for underlying trends. They set no limits on outcomes except that they were measured quantitatively and related to patients with long-term conditions, including:
measures of health or morbidity: biochemical and physiological measures, mortality, hospital admissions;
biomedical aspects of delivery of care: diagnostics, plans, referrals, and ongoing monitoring, clinical interventions (for example, prescriptions, immunisations), consultation rates;
broader aspects of care: coordination, continuity or integration of care, holistic care (that is, that considers multiple morbidity and social context, personalised for the patient), self-care; and
patient perspectives: patient experience, quality of life, or satisfaction.
The authors included randomised controlled trials, longitudinal studies where the analysis attempted to control for underlying trends, controlled before- and-after studies, and systematic reviews of these. They excluded cross-sectional studies examining how outcomes varied according to QOF achievement, because of the lack of suitable controls, the near-universal high level of achievement, and the high likelihood of confounding of associations between QOF achievement and outcome by other factors. The authors also excluded studies in which the researchers estimated or modelled outcomes rather than reporting empirical data (more details of the inclusion and exclusion criteria are available from the authors). The authors searched electronic databases (Cochrane Database, Medline, Embase, and Health Management Information Consortium) for studies published between 2004 (the year the QOF was introduced) and May 2016 (see Box 1 for search terms). They examined references of identified papers to search for further reports and asked experts for references to other relevant research.
Box 1. Search terms.
Search 1
Quality Outcomes Framework (keyword) OR
Quality and Outcomes Framework (keyword OR
QOF (keyword)
Search 2
Pay-for-performance (keyword) or Reimbursement (Medical Subject Heading (MeSH) term) AND
Primary health care (keyword) or Primary Health Care (MeSH term) OR
Primary medical care (keyword) or Family practice (MeSH term) OR
General practice (keyword) or General Practice (MESH term)
Two of the authors assessed suitability for inclusion for each abstract identified, and, where there was no consensus, asked a third author to adjudicate. Data were extracted independently from all papers by two authors.
The authors assessed quality of randomised controlled trials using adaptations of the Cochrane Collaboration’s tool,12 and longitudinal studies and systematic reviews using tools adapted from those developed by the National Institutes of Health.13,14
RESULTS
Identification and description of studies
Figure 1 shows the process of identification of studies. The three most recent systematic reviews asking the same questions as this review had search dates in 201215,16 and 2015.17 These included 20 studies of the QOF in total.18–37 The systematic review with the 2015 search date17 identified two studies of the QOF,22,32 both of which had been published in 2011 and had been included in one or other of the two reviews with search dates in 2012.
Figure 1.
Search process and results.
In all three reviews, while the authors set few limits on outcomes, study outcomes were solely derived from either QOF indicators themselves or prescribing data, with two exceptions: one study examined adherence to British Thoracic Society spirometry standards36 and the other cardiovascular events in patients with hypertension.32
The authors found five primary research studies meeting the search criteria published since 2012.38–42 The first examined trends in mortality rates for conditions covered by the QOF, comparing the UK with other high-income countries with no pay-for-performance schemes in primary care.38 The others examined the effect of introducing the QOF on patient management and outcomes, either nationally,39 or in samples of practices participating in the UK General Practice Research Database (GPRD) or Clinical Practice Research Datalink (CPRD) (627 UK practices,40 516 UK practices,41 148 English practices42).
The studies’ outcomes were: mortality from long-term conditions;38 hospital admissions for a range of QOF and non-QOF conditions;39 consultation rates in severe mental illness (SMI);40 prescribing in type 2 diabetes;41 and a composite indicator derived from QOF data on processes and outcomes of care in type 2 diabetes.42 Because of a high degree of study heterogeneity, the authors carried out a narrative synthesis. Table 1 summarises the design and results of primary research studies.
Table 1.
Primary research studies examining the effect of QOF on patient management and outcomes
| Authors | Year of publication | Geographical area and period under study | Main outcome | Key results |
|---|---|---|---|---|
| Ryan A, et al 38 | 2016 | Populations of UK and 27 other high income countries,1994–2010. | Mortality levels from chronic diseases targeted by QOF. | Mortality fell in all countries over the period. QOF not associated with any step change in mortality in the UK. Difference between mortality/100 000 between UK observed and expected −3.7 (95% CI = −8.2 to 0.8). |
| Harrison MJ, et al 39 | 2014 | All English practices, 1998–2010. | Rates of emergency admissions:
|
Emergency admission rates increased by 34%, but rate of increase lower for conditions for which care incentivised by QOF than other types of emergency admission.
The lower increase in emergency admission rates among those for whom care incentivised by QOF was mainly driven by admissions for coronary heart disease. |
| Kontopantelis E, et al 40 | 2015 | A total of 627 practices in the CPRD across UK, 2000–2011. | Primary care consultation rates in people with SMI and people without SMI. | Mean consultation rates between 2000 and 2011:
For both, trend of increasing rates before 2004. Significant step change increase in 2004 for both groups, much bigger step change for people with SMI. After this, rate of increase declined in both groups. Face-to-face consultation rate:
|
| Gallagher N, et al 41 | 2015 | A total of 516 practices in the GPRD across UK, 1999–2008. | The % of newly diagnosed patients with type 2 diabetes prescribed medication within 24 months after diagnosis. | Pre-intervention 1999–2003: rate decreasing by 1.4% per year (95% CI = 0.8 to 2.1). Post-intervention 2004–2008: rate increased by 1.6% per year (95% CI = 0.8 to 2.3). |
| Kontopantelis E, et al 42 | 2013 | A total of 148 practices in the GPRD across England, 2001–2006. | Delivery of care of type 2 diabetes — composite of achievement of the 17 diabetes QOF indicators, including processes and outcomes. | Pre-intervention 2001–2003: delivery of care improving. Post-intervention 2004–2006: delivery of care improved over and above the previous trend. In the first year, 14% improvement in score over and above expected (95% CI = 13.7 to 14.6). By third year, 7% improvement in score over and above expected (95% CI = 6.7 to 8.0). |
CI = confidence interval. CPRD = Clinical Practice Research Datalink. GPRD = General Practice Research Database. QOF = Quality and Outcomes Framework. SMI = severe mental illness.
None of the relevant studies identified by the systematic reviews, or the primary studies published since 2012, examined the effect of the QOF on broader aspects of care or patient perspectives.
Quality
The systematic reviews were of good quality. Due to the nature of the intervention, the primary research studies were all before- and-after studies using interrupted time series or difference-in-differences methods and, as such, were of good quality for observational studies. However, because of the study designs, the authors cannot be sure that the QOF was responsible for any change in outcomes.
Findings
Systematic reviews
The first systematic review concluded that the QOF had had a limited impact on health outcomes.15 The second systematic review concluded that the effect of pay-for-performance remained uncertain.16 The third systematic review concluded that there was limited evidence of the effects of financial incentives.17
Primary research
The study examining trends in mortality in the UK compared with other countries found no effect of the QOF, although the synthetic control approach43 adopted in the study required the use of conservative tests for statistical inference.38 The study examining emergency admissions before and after the introduction of the QOF found that the trend of increasing emergency hospital admission rates (which increased overall by 34% between 2004 and 2010) was modestly lower for conditions incentivised in the QOF compared with conditions that were not incentivised in the QOF, by 3% in the first year rising to 8% in 2010.39 The difference was mainly driven by relative reductions in emergency admission rates for coronary heart disease.
The study examining consultation rates found a trend of increasing rates overall during the period, with a small step change in 2004; the rate of increase was greater in people with SMI than overall.40 The face-to-face consultation rate in SMI increased from about nine to 11 per patient per year from 2000 to 2011, and in other people it stayed stable at about five per patient per year over the same period.
The study of prescribing in type 2 diabetes found a modest increase in prescribing of antidiabetic medication (changing the direction of the trend of decreasing initiation rates to increasing initiation rates) after the introduction of the QOF.41 The increase was sustained at a similar rate until 2008.
The study examining effects on a composite indicator of process and outcomes in type 2 diabetes found a modest improvement of 14% over and above the underlying trend in the first year after the introduction of the QOF, declining to 8% in the third year.42 Table 1 summarises the results of the studies.
DISCUSSION
Summary
The authors found evidence that the QOF may be associated with a modest reduction in emergency admission rates in long-term conditions, a modest increase in consultation rates in SMI, and modest improvements in certain limited aspects of the care of diabetes. They found no clear evidence that these changes have led to any effect on mortality. Because of the design of the studies, it is not possible to be sure that any of the positive effects seen are causally related to the QOF.
The authors found no evidence to suggest that the QOF influences, positively or negatively, other aspects of care, such as integration or coordination of care, holistic or personalised care, or self-care, nor any evidence of its effects on patients’ quality of life, experience, or satisfaction.
The QOF is unlikely to advance progress towards the vision of the Five Year Forward View for the care of long-term conditions. To deliver the aims of the Five Year Forward View, the NHS should consider more broadly — beyond what is measured by the QOF — what constitutes high-quality primary care for people with long-term conditions, and consider managing performance on this basis. In the context of a demoralised primary care workforce, it is important also to consider ways other than financial incentives to motivate primary care teams to deliver high-quality care.
Strengths and limitations
To the authors’ knowledge, this review is the first to have specifically addressed the effect of the QOF on those aspects of care for long-term conditions that are prioritised by national policy. As with any systematic review, the authors’ conclusions are constrained by the limited quantity and quality of the primary research published to date. Although the search for quantitative research was comprehensive, the authors did not include qualitative research, which may provide other insights.
Research to date has not attempted to identify the effects of the QOF on any of the broader aspects of care for long-term conditions, having examined effects only on more easily measurable outcomes, for example, those collected as part of the QOF, or routinely available data on mortality, emergency admissions, consultation rates, and prescribing. The authors found no evidence of attempts to evaluate the QOF using validated measures of quality of care in general practice. Perhaps this is because defining and measuring quality of general practice is complex.44
The lack of effect of the QOF on mortality is surprising, given that the indicators are based on high-quality evidence of effectiveness of interventions. Why this is the case is not clear. The wider determinants of population health (including low income, experience of inequality or discrimination, or quality of air, education, housing, or work conditions45) may be much more important than the quality of care in determining mortality. Also, it is recognised that the effectiveness of interventions demonstrated in randomised controlled trials, which include highly selected study participants, is often diluted in routine clinical practice.46 Perhaps non-incentivised activities are more important in determining mortality in the patient population. It is also possible that practices misreport performance so as to exaggerate the quality of care, although there is little evidence that this is a significant problem.47
The authors found evidence that the QOF was associated with a modest slowing of the increase in emergency admissions for conditions for which care is incentivised by the QOF, and an increase in primary care attendances for people with serious mental illness. Whether the QOF is responsible for these is unclear; many other factors are likely to have influenced admission and attendance rates over the period, including changes in medical technology or access to other services, or national standards for management of long-term conditions. In any case, among interventions to prevent emergency admissions, pay-for-performance is unlikely to be one of the most effective.48,49
It could be argued that some QOF indicators — in palliative care, cancer, SMI, dementia, and rheumatoid arthritis — incentivise multidisciplinary meetings, reviews, and care plans, considered necessary elements of holistic and integrated care (as set out in recent guidance from NICE).50 However, to achieve the indicators, practices are not required to demonstrate that their activities ensure that holistic or integrated care has been delivered.
Implications for research and practice
The authors found no convincing evidence that the QOF promotes better care and outcomes for people with long-term conditions. QOF may also have negative effects. If practices have achieved maximum or near maximum points under the scheme (which is true for most practices), they have little motivation to improve achievement further. It is likely that the QOF diverts practices and professionals from ways of providing high-quality primary care that is not QOF-related. Moreover, the QOF does not incentivise practices to improve care for patients with the most complex needs in primary care, because these are more likely to be excepted from the scheme.51 Raising thresholds for achievement may be counterproductive — there is evidence that it leads to increased exception reporting, raising apparent achievement with no real increase in the desired activity.52
The Chief Executive of the NHS announced in October 2016 that the QOF would be phased out.53 What would happen to the quality of primary care if the QOF is completely abolished is not clear, although it seems unlikely that standards would drop significantly, because the activities rewarded in the QOF are now firmly embedded in practice. There is some limited evidence to suggest that performance did not fall following the withdrawal of certain individual indicators from the scheme.54 Abolishing the QOF may also allow practices to prioritise other activities, which could lead to better care.
The QOF provides a major component of practice income; if it were abolished, practices would need to be assured of a stable income. Losing this is likely to have detrimental effects on patient care and further worsen recruitment and retention in primary care, which is once again in a precarious position.55
Alternative methods of rewarding good practice are being considered for new models of primary care.56 Any replacement for the QOF needs to consider the evidence of effectiveness of pay-for-performance in primary care, and the evidence of what motivates primary care professionals to provide high-quality care.57
Acknowledgments
The authors thank Ms Anna Peckham, consultant librarian, for her assistance in conducting the searches of electronic databases, Ms Linda Jenkins for her help with downloading and interpreting QOF reporting data, and Dr Mark Ashworth for his advice on interpreting the results.
Appendix 1. QOF indicators 2016/2017 relating to care of long-term conditions
| Long-term condition | QOF code | Brief description of indicator |
|---|---|---|
| Asthma | AST001 | Register of patients with asthma |
| AST002 | Percentage of patients with asthma and measures of variability or reversibility recorded | |
| AST003 | Percentage of patients with asthma who have had control assessed | |
| AST004 | Percentage of patients with asthma with record of smoking status | |
|
| ||
| Atrial fibrillation | AF001 | Register of patients with atrial fibrillation |
| AF006 | Percentage of patients with atrial fibrillation in whom stroke risk has been assessed | |
| AF007 | Anticoagulant therapy in those with atrial fibrillation and high risk of stroke | |
|
| ||
| Cancer | CAN001 | Register of patients with cancer |
| CAN003 | Percentage of patients with cancer who have been reviewed | |
|
| ||
| Chronic kidney disease (CKD) | CKD001 | Register of patients with chronic kidney disease |
|
| ||
| Chronic obstructive pulmonary disease (COPD) | COPD001 | Register of patients with COPD |
| COPD002 | Percentage of patients with COPD with diagnosis confirmed by post-bronchodilator spirometry | |
| COPD003 | Percentage of patients with COPD who have had a review with assessment of breathlessness | |
| COPD004 | Percentage of patients with COPD with a record of forced expiratory volume in 1 second (FEV1) | |
| COPD005 | Percentage of patients with severe COPD with record of oxygen saturation | |
| COPD007 | Percentage of patients with COPD who have had influenza immunisation | |
|
| ||
| Coronary heart disease (CHD) | CHD001 | Register of patients with CHD |
| CHD002 | Percentage of patients with CHD with blood pressure 150/90 mmHg or less | |
| CHD005 | Percentage of patients with CHD taking aspirin, an alternative antiplatelet therapy, or an anticoagulant | |
| CHD007 | Percentage of patients with CHD who have had influenza immunisation | |
|
| ||
| Dementia | DEM001 | Register of patients with dementia |
| DEM004 | Percentage of patients with dementia whose care plan has been reviewed face-to-face | |
| DEM005 | Percentage of patients with a new diagnosis of dementia with record of tests to exclude reversible cause | |
|
| ||
| Depression | DEP003 | Percentage of patients with new diagnosis of depression with review soon after diagnosis |
|
| ||
| Diabetes mellitus | DM002 | Percentage of patients with diabetes with blood pressure 150/90 mmHg or less |
| DM003 | Percentage of patients with diabetes with blood pressure 140/80 mmHg or less | |
| DM004 | Percentage of patients with diabetes with total cholesterol 5 mmol/l or less | |
| DM006 | Percentage of patients with diabetes and nephropathy taking angiotensin converting enzyme inhibitors or angiotensin receptor blockers (ACEIs or ARBs) | |
| DM007 | Percentage of patients with diabetes with glycosylated haemoglobin 59 mmol/mol or less | |
| DM008 | Percentage of patients with diabetes with glycosylated haemoglobin 64 mmol/mol or less | |
| DM009 | Percentage of patients with diabetes with glycosylated haemoglobin 75 mmol/mol or less | |
| DM012 | Percentage of QOF patients with diabetes with a record of a foot examination and foot risk classification | |
| DM014 | Patients newly diagnosed with diabetes referred to a structured education programme | |
| DM017 | Register of patients with diabetes | |
| DM018 | Percentage of patients with diabetes who have had influenza immunisation | |
|
| ||
| Epilepsy | EP001 | Register of patients with epilepsy |
|
| ||
| Heart failure | HF001 | Register of patients with heart failure |
| HF002 | Percentage of patients with heart failure confirmed by an echocardiogram or by specialist assessment | |
| HF003 | Percentage of patients with heart failure taking ACEIs or ARBs | |
| HF004 | Percentage of patients with heart failure taking ACEIs or ARBs plus beta-blocker | |
|
| ||
| Hypertension | HYP001 | Register of patients with hypertension |
| HYP006 | Percentage of patients with hypertension with blood pressure of 150/90 mmHg or less | |
| CVD-PP001 | Percentage of patients with hypertension and high cardiovascular risk treated with statins | |
|
| ||
| Learning disability | LD003 | Register of patients with learning disability |
|
| ||
| Mental health | MH001 | Register of patients with serious mental health problems |
| MH002 | Percentage of patients with serious mental health problems with comprehensive care plan | |
| MH003 | Percentage of patients with serious mental health problems with record of blood pressure | |
| MH007 | Percentage of patients with serious mental health problems with record of alcohol consumption | |
| MH008 | Percentage of women with serious mental health problems with cervical screening test performed | |
| MH009 | Percentage of patients on lithium therapy having renal and thyroid function monitored | |
| MH010 | Percentage of patients on lithium therapy with lithium levels in therapeutic range | |
|
| ||
| Osteoporosis | OST002 | Percentage of patients 50–74 with confirmed osteoporosis taking bone-sparing agent |
| OST004 | Register of patients with osteoporosis | |
| OST005 | Percentage of patients aged >75 with osteoporosis taking bone-sparing agent | |
|
| ||
| People with palliative care needs | PC001 | Register of patients in need of palliative care/support |
| PC002 | Regular multidisciplinary case review meetings for people receiving palliative care | |
|
| ||
| Peripheral arterial disease | PAD001 | Register of patients with peripheral arterial disease |
| PAD002 | Percentage of patients with peripheral arterial disease with blood pressure 150/90 mmHg or less | |
| PAD004 | Percentage of patients with peripheral arterial disease taking aspirin or an alternative antiplatelet | |
|
| ||
| Rheumatoid arthritis | RA001 | Register of patients with rheumatoid arthritis |
| RA002 | Percentage of patients with rheumatoid arthritis who have had a face-to-face review | |
|
| ||
| Stroke or transient ischaemic attack (STIA) | STIA001 | Register of patients with STIA |
| STIA003 | Percentage of patients with STIA with blood pressure 150/90 mmHg or less | |
| STIA007 | Percentage of patients with non-haemorrhagic stroke or TIA taking antiplatelet agent, or anticoagulant | |
| STIA008 | Percentage of patients with STIA referred for further investigation | |
| STIA009 | Percentage of patients with STIA who have had influenza immunisation | |
| Several long-term conditions | SMOK002 | Percentage of patients with long-term conditions with record of smoking status |
| SMOK005 | Percentage of smokers with long-term conditions offered smoking cessation support | |
QOF = Quality and Outcomes Framework.
Funding
This study was funded by the Policy Research Unit in Commissioning and the Healthcare System, and by the Department of Health. The views expressed are those of the researchers and not necessarily those of the Department of Health.
Ethical approval
Not applicable.
Provenance
Freely submitted; externally peer reviewed.
Competing interests
The authors have declared no competing interests.
Discuss this article
Contribute and read comments about this article: bjgp.org/letters
REFERENCES
- 1.Sutcliffe D, Lester H, Hutton J, Stokes T. NICE and the quality and outcomes framework (QOF) 2009–2011. Qual Prim Care. 2012;20(1):47–55. [PubMed] [Google Scholar]
- 2.Health and Social Care Information Centre . NHS payments to general practice, England, 2014–15. HSCIC; 2015. http://content.digital.nhs.uk/catalogue/PUB18468 (accessed 11 Sep 2017) [Google Scholar]
- 3.Roland M, Guthrie B. Quality and outcomes framework: what have we learnt? BMJ. 2016;354:i4060. doi: 10.1136/bmj.i4060. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Lester H, Campbell S. Developing quality and outcomes framework (QOF) indicators and the concept of ‘QOFability’. Qual Prim Care. 2010;18(2):103–109. [PubMed] [Google Scholar]
- 5.Doran T, Fullwood C, Kontopantelis E, Reeves D. Effect of financial incentives on inequalities in the delivery of primary clinical care in England: analysis of clinical activity indicators for the quality and outcomes framework. Lancet. 2008;372(9640):728–736. doi: 10.1016/S0140-6736(08)61123-X. [DOI] [PubMed] [Google Scholar]
- 6.Roland M, Campbell S. Successes and failures of pay for performance in the United Kingdom. N Engl J Med. 2014;370(20):1944–1949. doi: 10.1056/NEJMhpr1316051. [DOI] [PubMed] [Google Scholar]
- 7.Kramer G. Payment for performance and the QOF: are we doing the right thing. Br J Gen Pract. 2012. DOI: https://doi.org/10.3399/bjgp12X630151. [DOI] [PMC free article] [PubMed]
- 8.NHS England, Care Quality Commission, Public Health England, Health Education England, Monitor, Trust Development Authority . NHS five year forward view. London: NHS England; 2014. [Google Scholar]
- 9.Royal College of General Practitioners . A blueprint for building the new deal for general practice in England. London: RCGP; 2015. http://www.rcgp.org.uk/policy/rcgp-policy-areas/blueprint-for-general-practice.aspx (accessed 11 Sep 2017) [Google Scholar]
- 10.NHS England General practice forward view. 2016. https://www.england.nhs.uk/gp/gpfv/(accessed 11 Sep 2017)
- 11.Bostock N. GPC frustrated by NHS England failure to agree national QOF suspension. GP Online. 2017. Feb 3, http://www.gponline.com/gpc-frustrated-nhs-england-failure-agree-national-qof-suspension/article/1423088 (accessed 11 Sep 2017)
- 12.Higgins JPT, Altman DG, Gøtzsche PC, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928. doi: 10.1136/bmj.d5928. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.National Heart, Lung, and Blood Institute Quality assessment tool for before-after (pre-post) studies with no control group. https://www.nhlbi.nih.gov/health-pro/guidelines/in-develop/cardiovascular-risk-reduction/tools/before-after (accessed 21 Aug 2017)
- 14.National Heart, Lung, and Blood Institute Quality assessment of systematic reviews and meta-analyses. https://www.nhlbi.nih.gov/health-pro/guidelines/in-develop/cardiovascular-risk-reduction/tools/sr_ma (accessed 11 Sep 2017)
- 15.Langdown C, Peckham S. The use of financial incentives to help improve health outcomes: is the quality and outcomes framework fit for purpose? A systematic review. J Public Health. 2014;36(2):251–258. doi: 10.1093/pubmed/fdt077. [DOI] [PubMed] [Google Scholar]
- 16.Houle SKD, McAlister FA, Jackevicius CA, et al. Does performance-based remuneration for individual health care practitioners affect patient care? A systematic review. Ann Intern Med. 2012;157(12):889–899. doi: 10.7326/0003-4819-157-12-201212180-00009. [DOI] [PubMed] [Google Scholar]
- 17.Rashidian A, Omidvari AH, Vali Y, et al. Pharmaceutical policies: effects of financial incentives for prescribers. Cochrane Database Syst Rev. 2015;8:CD006731. doi: 10.1002/14651858.CD006731.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Alshamsan R, Lee JT, Majeed A, et al. Effect of a UK pay-for-performance program on ethnic disparities in diabetes outcomes: interrupted time series analysis. Ann Fam Med. 2012;10(3):228–234. doi: 10.1370/afm.1335. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Calvert M, Shankar A, McManus RJ, et al. Effect of the quality and outcomes framework on diabetes care in the United Kingdom: retrospective cohort study. BMJ. 2009;338:b1870. doi: 10.1136/bmj.b1870. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Campbell SM, Reeves D, Kontopantelis E, et al. Effects of pay for performance on the quality of primary care in England. N Engl J Med. 2009;361(4):368–378. doi: 10.1056/NEJMsa0807651. [DOI] [PubMed] [Google Scholar]
- 21.Coleman T, Lewis S, Hubbard R, Smith C. Impact of contractual financial incentives on the ascertainment and management of smoking in primary care. Addiction. 2007;102(5):803–808. doi: 10.1111/j.1360-0443.2007.01766.x. [DOI] [PubMed] [Google Scholar]
- 22.Doran T, Kontopantelis E, Valderas JM, et al. Effect of financial incentives on incentivised and non-incentivised clinical activities: longitudinal analysis of data from the UK Quality and Outcomes Framework. BMJ. 2011;342:d3590. doi: 10.1136/bmj.d3590. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Gulliford MC, Ashworth M, Robotham D, Mohiddin A. Achievement of metabolic targets for diabetes by English primary care practices under a new system of incentives. Diabet Med. 2007;24(5):505–511. doi: 10.1111/j.1464-5491.2007.02090.x. [DOI] [PubMed] [Google Scholar]
- 24.Hippisley-Cox J, Vinogradova Y, Coupland C. Time series analysis for selected clinical indicators from the quality and outcomes framework — 2001–2006. NHS digital; 2006. https://digital.nhs.uk/catalogue/PUB01968 (accessed 13 Sep 2017) [Google Scholar]
- 25.MacBride-Stewart SP, Elton R, Walley T. Do quality incentives change prescribing patterns in primary care? An observational study in Scotland. Fam Pract. 2008;25(1):27–32. doi: 10.1093/fampra/cmm074. [DOI] [PubMed] [Google Scholar]
- 26.McGovern MP, Boroujerdi MA, Taylor MW, et al. The effect of the UK incentive-based contract on the management of patients with coronary heart disease in primary care. Fam Pract. 2008;25(1):33–39. doi: 10.1093/fampra/cmm073. [DOI] [PubMed] [Google Scholar]
- 27.McGovern MP, Williams DJ, Hannaford PC, et al. Introduction of a new incentive and target-based contract for family physicians in the UK: good for older patients with diabetes but less good for women? Diabet Med. 2008;25(9):1083–1089. doi: 10.1111/j.1464-5491.2008.02544.x. [DOI] [PubMed] [Google Scholar]
- 28.Millett C, Bottle A, Ng A, Majeed A. Pay for perfomance and the quality of diabetes management in individuals with and without co-morbid medical conditions. J R Soc Med. 2009;102(9):369–377. doi: 10.1258/jrsm.2009.090171. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Millett C, Gray J, Saxena S, et al. Ethnic disparities in diabetes management and pay-for-performance in the UK: the Wandsworth Prospective Diabetes Study. PLoS Med. 2007;4(6):e191. doi: 10.1371/journal.pmed.0040191. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Millett C, Gray J, Saxena S, et al. Impact of a pay-for-performance incentive on support for smoking cessation and on smoking prevalence among people with diabetes. CMAJ. 2007;176(12):1705–1710. doi: 10.1503/cmaj.061556. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Millett C, Gray J, Wall M, Majeed A. Ethnic disparities in coronary heart disease management and pay for performance in the UK. J Gen Intern Med. 2009;24(1):8–13. doi: 10.1007/s11606-008-0832-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Serumaga B, Ross-Degnan D, Avery AJ, et al. Effect of pay for performance on the management and outcomes of hypertension in the United Kingdom: interrupted time series study. BMJ. 2011;342:d108. doi: 10.1136/bmj.d108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Simpson CR, Hannaford PC, Lefevre K, Williams D. Effect of the UK incentive-based contract on the management of patients with stroke in primary care. Stroke. 2006;37(9):2354–2360. doi: 10.1161/01.STR.0000236067.37267.88. [DOI] [PubMed] [Google Scholar]
- 34.Simpson CR, Hannaford PC, Ritchie LD, et al. Impact of the pay-for-performance contract and the management of hypertension in Scottish primary care: a 6-year population-based repeated cross-sectional study. Br J Gen Pract. 2011. DOI: https://doi.org/10.3399/bjgp11X583407. [DOI] [PMC free article] [PubMed]
- 35.Steel N, Maisey S, Clark A, et al. Quality of clinical primary care and targeted incentive payments: an observational study. Br J Gen Pract. 2007;57(539):449–454. [PMC free article] [PubMed] [Google Scholar]
- 36.Strong M, South G, Carlisle R. The UK Quality and Outcomes Framework pay-for-performance scheme and spirometry: rewarding quality or just quantity? A cross-sectional study in Rotherham, UK. BMC Health Serv Res. 2009;9:108. doi: 10.1186/1472-6963-9-108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Vamos EP, Pape UJ, Bottle A, et al. Association of practice size and pay-for-performance incentives with the quality of diabetes management in primary care. CMAJ. 2011;183(12):E809–E816. doi: 10.1503/cmaj.101187. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Ryan A, Krinsky S, Kontopantelis E, Doran T. Long-term evidence for the effect of pay-for-performance in primary care on mortality in the United Kingdom: a population study. Lancet. 2016;388(10041):268–274. doi: 10.1016/S0140-6736(16)00276-2. [DOI] [PubMed] [Google Scholar]
- 39.Harrison MJ, Dusheiko M, Sutton M, et al. Effect of a national primary care pay-for-performance scheme on emergency hospital admissions for ambulatory care sensitive conditions: controlled longitudinal study. BMJ. 2014;349:g6423. doi: 10.1136/bmj.g6423. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Kontopantelis E, Olier I, Planner C, et al. Primary care consultation rates among people with and without severe mental illness: a UK cohort study using the Clinical Practice Research Datalink. BMJ Open. 2015;12(5):e008650. doi: 10.1136/bmjopen-2015-008650. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Gallagher N, Cardwell C, Hughes C, O’Reilly D. Increase in the pharmacological management of type 2 diabetes with pay-for-performance in primary care in the UK. Diab Med. 2015;32(1):62–68. doi: 10.1111/dme.12575. [DOI] [PubMed] [Google Scholar]
- 42.Kontopantelis E, Reeves D, Valderas JM, et al. Recorded quality of primary care for patients with diabetes in England before and after the introduction of a financial incentive scheme: a longitudinal observational study. BMJ Qual Saf. 2013;22(1):53–64. doi: 10.1136/bmjqs-2012-001033. [DOI] [PubMed] [Google Scholar]
- 43.Dimick JB, Ryan AM. Methods for evaluating changes in health care policy: the difference-in-differences approach. JAMA. 2014;312(22):2401–2402. doi: 10.1001/jama.2014.16153. [DOI] [PubMed] [Google Scholar]
- 44.Young RA, Roberts RG, Holden RJ. The challenges of measuring, improving, and reporting quality in primary care. Ann Fam Med. 2017;15(2):175–182. doi: 10.1370/afm.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Marmot M, Bell R. Fair society, healthy lives. Public Health. 2012;126(Suppl 1):S4–S10. doi: 10.1016/j.puhe.2012.05.014. [DOI] [PubMed] [Google Scholar]
- 46.Ford I, Norrie J. Pragmatic trials. N Engl J Med. 2016;375(5):454–463. doi: 10.1056/NEJMra1510059. [DOI] [PubMed] [Google Scholar]
- 47.Ashworth M, Kordowicz M. Quality and outcomes framework: smoke and mirrors? Qual Prim Care. 2010;18(2):127–131. [PubMed] [Google Scholar]
- 48.Purdey S, Huntley A. Predicting and preventing avoidable hospital admissions: a review. J R Coll Physicians Edinb. 2013;43(4):340–344. doi: 10.4997/jrcpe.2013.415. [DOI] [PubMed] [Google Scholar]
- 49.Wallace E, Smith SM, Fahey T, Roland M. Reducing emergency admissions through community based interventions. BMJ. 2016;352:h6817. doi: 10.1136/bmj.h6817. [DOI] [PubMed] [Google Scholar]
- 50.National Institute for Health and Care Excellence . Multimorbidity: clinical assessment and management. NG56. London: NICE; 2016. https://www.nice.org.uk/guidance/ng56 (accessed 11 Sep 2017) [Google Scholar]
- 51.Martin JL, Lowrie R, McConnachie A, et al. Physical health indicators in major mental illness: analysis of QOF data across UK general practice. Br J Gen Pract. 2014. DOI: https://doi.org/10.3399/bjgp14X681829. [DOI] [PMC free article] [PubMed]
- 52.Kontopantelis E, Doran T, Gravelle H, et al. Family doctor responses to changes in incentives for influenza immunization under the UK Quality and Outcomes Framework pay-for-performance scheme. Health Serv Res. 2012;47(3 Pt 1):1117–1136. doi: 10.1111/j.1475-6773.2011.01362.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Matthews King A. QOF has reached the end of its useful life, says NHS chief. Pulse. 2016. Oct 19, http://www.pulsetoday.co.uk/your-practice/qof/qof-has-reached-the-end-of-its-useful-life-says-nhs-chief/20033054.article (accessed 13 Sep 2017)
- 54.Kontopantelis E, Springate D, Reeves D, et al. Withdrawing performance indicators: retrospective analysis of general practice performance under UK quality and outcomes framework. BMJ. 2014;348:g330. doi: 10.1136/bmj.g330. [Erratum in BMJ 2014; 349: g7730.] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Roland M, Everington S. Tackling the crisis in general practice. BMJ. 2016;352:i942. doi: 10.1136/bmj.i942. [DOI] [PubMed] [Google Scholar]
- 56.NHS England New care models The multispecialty community provider (MCP) emerging care model and contract framework. 2016. https://www.england.nhs.uk/wp-content/uploads/2016/07/mcp-care-model-frmwrk.pdf (accessed 11 Sep 2017)
- 57.Peckham S, Marchand C, Peckham A. General practitioner recruitment and retention: an evidence synthesis. Project report. Policy Research Unit in Commissioning and the Healthcare System, University of Kent; 2016. https://kar.kent.ac.uk/58788/ (accessed 11 Sep 2017) [Google Scholar]