Impacts of pay for performance on the quality of primary care

Abstract

Increasingly, financial incentives are being used in health care as a result of increasing demand for health care coupled with fiscal pressures. Financial incentive schemes are one approach by which the system may incentivize providers of health care to improve productivity and/or adapt to better quality provision. Pay for performance (P4P) is an example of a financial incentive which seeks to link providers’ payments to some measure of performance. This paper provides a discussion of the theoretical underpinnings of P4P, gives an overview of the health P4P evidence base, and provide a detailed case study of a particularly large scheme from the English National Health Service. Lessons are then drawn from the evidence base. Overall, we find that the evidence for the effectiveness of P4P for improving quality of care in primary care is mixed. This is to some extent due to the fact that the P4P schemes used in primary care are also mixed. There are many different schemes that incentivize different aspects of care in different ways and in different settings, making evaluation problematic. The Quality and Outcomes Framework in the United Kingdom is the largest example of P4P in primary care. Evidence suggests incentivized quality initially improved following the introduction of the Quality and Outcomes Framework, but this was short-lived. If P4P in primary care is to have a long-term future, the question about scheme effectiveness (perhaps incorporating the identification and assessment of potential risk factors) needs to be answered robustly. This would require that new schemes be designed from the onset to support their evaluation: control and treatment groups, coupled with before and after data.

Introduction to pay for performance in health care

Health care expenditure comprises a substantial proportion of public expenditure and gross domestic product in developed nations.¹ The demand for health care continues to increase due to the simultaneous effects of aging populations and rising expectations.² On the supply side, increasing fiscal pressures have resulted in health care budgets being squeezed. These forces have resulted in an increasing challenge for health systems to maintain or improve the quality and quantity of care provided.

Improving the productivity and/or quality of the health system within available resources is one way in which to expand service delivery. Financial incentive schemes are one approach by which the system may incentivize providers of health care to improve productivity and/or adapt to better quality provision. Incentivizing providers of health care reduces the financial risk to purchasers of care by directing the activities of providers towards some measurable predefined target; in addition, one can view incentive schemes as reducing the risk of poor/inadequate health care outcomes. One type of payment mechanism, pay for performance (P4P), seeks to link providers’ payments to some measure of performance. We define P4P as a payment system whereby payments are clearly linked to a quality target. Therefore, P4P does not include schemes whereby payments are fixed, such as fee-for-service schemes or block contacts, even if better performance is implicitly incentivized. The adoption of P4P schemes is becoming more widespread internationally in health care despite an ambiguous evidence base. For example, ambiguity exists concerning the clinical effectiveness of P4P because there is uncertainty about the main purpose of P4P schemes – whether they should be used to set minimum standards or to drive quality upwards.³ Ambiguity is also created due to the varied designs of P4P schemes and the context in which they are introduced. What works in one country’s health care system may not work in another’s. Further complications can be created as P4P can often have unintentional effects on the behavior of doctors. Targets linked to income may induce doctors to divert effort away from other aspects of care which are not financially incentivized, such as continuity of care or non-incentivized conditions.

This paper provides a general overview of P4P theory and applications, with particular emphasis on how P4P transfers risk between purchasers and providers of health care and the impacts this transfer of risk has been found to have on the quality of primary care. Although several systematic reviews exist assessing the evidence of financial incentives on health-care, these relate to a multitude of incentive schemes and health care sectors. The single review assessing primary care had strict exclusion criteria, resulting in the removal of studies of schemes such as the Quality and Outcomes Framework (QOF) that explicitly sought to incentivize quality in primary care. Our approach provides an overview of the findings across the reviews to identify the evidence on P4P schemes (where incentives are related to predefined targets) on the quality of primary care. In addition, we provide a case study of the evidence from the QOF. The paper concludes with a discussion of the key policy implications revealed from the literature on the design of P4P schemes in primary care.

The theory of P4P

P4P is a potential solution to the principal–agent problem. The principal–agent problem captures the situation in which one party (the agent) is required to act on the behalf of another (the principal), with differing (utility-maximizing) objectives for each.⁴ The main objective for the commissioner may be some measure of outcomes (such as mortality rates), whereas for the provider the objective may be to retain their salaries regardless of these outcomes and might also be assumed to include a preference for maintaining a smaller workload. However, outcomes are only a proxy for provider effort and behavior, as they are also the product of unobservable phenomena and may lie outside of the control of providers (for example, patient health behaviors). The inability to perfectly identify provider behavior and the potential conflicts between commissioner and provider objectives results in the misalignment of provider behavior and effort – the principal–agent problem.

In a simple treatment of the model, the solution is to provide the agent with an incentive to carry out the delegated work. Given that the agent seeks to maximize utility, the incentive must meet two conditions: first, the increase in the utility of the agent created by the incentive must be greater than the decrease in the agent’s utility resulting from carrying out the delegated work. This is known as the incentive compatibility constraint. Second, the increase in the agent’s utility created by the incentive must be greater than the utility gained from not undertaking the delegated work. This is known as the participation constraint. The solution is effective, as the interests of the principal and the agent are aligned as a result of the incentive.

Mainstream theories of P4P often need some adaptation before they can be applied to health care. There are four main reasons for this: dual agency, measurement difficulties, team production, and intrinsic motivation.⁵ Dual agency arises as a doctor (the agent) must act on behalf of the patient and the health care commissioner (two principals).⁶ The two principals have different objectives, and the agent must try to please both. Measuring the amount of effort an employee applies can be difficult in health care. The output might be observable (patient health, for example), but this is a product of the employee’s effort as well as patient characteristics such as compliance. Health care is provided by teams such as hospitals or practices and not solely provided by a single individual. As a result, performance pay is often targeted at these teams, meaning that the incentives must be designed not to induce free-riding, where individual effort is reduced because performance is measured at a team level.⁷ Intrinsic motivators such as altruism or professionalism become a factor, as a doctor can receive utility from their own income but also the quality of care they provide and their reputation.⁸

P4P allocates risks across both parties and deals with the problem of observability. However, when agents are required to perform multiple tasks, P4P not only allocates risks, it also allocates the focus of the agent between their multiple duties.⁹ Providers may thus have an incentive to focus narrowly on (divert effort to) one element of performance (for example, waiting times) at the expense of other important dimensions that are not necessarily measureable (for example, patient experience). This unintended consequence is the first implication of the multidimensional model. The second implication is that job design plays an important role for incentives. Incentives will be less likely to lead to diversion of effort if different tasks can be easily delegated to separate agents.

Diversion of effort is only one unintended consequence which might result from an incentive scheme; there are a host of additional unintended consequences which can be stimulated, including:¹⁰ 1) tunnel vision (diversion of effort away from nonpayment domains); 2) suboptimization (discordance of agents’ objective from overall organizational objectives); 3) myopia (short-termism); 4) measure fixation (focus on measures of success and not underlying objectives); 5) misrepresentation (manipulation of measures such that reported performance exceeds actual performance); 6) misinterpretation; 7) gaming (changing actual behavior to make the scheme fit the agent: an example involves deliberate underperformance for public sector organizations to keep future service targets relatively achievable); and 8) ossification (organizational paralysis and innovation stifling caused by narrow targets).

Discussion: evidence demonstrating the impacts of P4P on the quality of primary care

There have been several systematic reviews of the P4P evidence base. Many focus primarily on hospitals, and previous reviews in primary care have tended to exclude studies of important schemes such as the QOF due to the lack of pre/post data and control/intervention groups. We provide an overview of the evidence in the following section.

Financial incentive schemes have tended to vary considerably in their design.¹¹ Typically, P4P schemes make payments conditional on providing a prespecified level or providing a change in activity or quality of care. Many studies have looked at these types of financial incentive schemes. One study considered the impact of such a scheme on hemoglobin testing in California,¹² while others have looked at pathway compliance under financial incentives in a Medicaid health maintenance organization.^13,14 Further studies analyzed the effects of P4P for diabetes care in a managed care organization in upstate New York;¹⁵ and evaluated the impact of using financial incentives for providing smoking cessation advice in a large medical group practice in the United States (US).¹⁶ These “conditional” financial incentive schemes have been found to have some limited effects, but there is no evidence of an effect on health outcomes.¹¹

A recent systematic review of the evidence base for the effects of financial incentives on the quality of health care provided by primary care physicians found seven relevant studies.¹⁷ Three of these studies evaluated single-threshold target payments in the US^12,16 and Germany,¹⁸ one considered a mixed fee per patient and threshold target system in the US,¹⁹ one examined a fixed fee per patient achieving a specified outcome in the US,²⁰ another considered a US tournament-based scheme,²¹ and the final study included evaluating the English experience of changing blended payments to a salaried system.²² Six of these seven studies showed modest and inconsistent positive effects on quality of care for some primary outcome measures, and one found no effect whatsoever. The systematic review noted that study design led to substantial risk of bias for the majority of these studies (particularly self-selection into schemes by physicians).¹⁷Another study, published in 2013, is a systematic review of the existing systematic reviews.²³ This review is broad in its scope, considering literature assessing a range of P4P-related subjects, including effectiveness of schemes, cost-effectiveness of schemes, unintended consequences, and design of schemes. It considers reviews which include studies evaluating each of these aspects and summarizes the findings across the included evidence. On effectiveness, this review suggests that the evidence from randomized control trials indicates a consistent concluding message across studies: results are mixed, and there is insufficient evidence to support the use of P4P in improving quality of preventative and chronic care. Reviews of nonrandomized studies show consistent positive but modest effects overall. Reviews also indicate that studies have generally found that convincing evidence on cost-effectiveness is notable by its absence. In terms of unintended consequences, three reviews have found some weak evidence of risk selection by providers under P4P and public reporting. The evidence regarding spillover effects is mixed, with evidence of both positive and negative spillovers in nonpayment aspects of care. The evidence in reviews has found that studies of gaming have been mixed – some have found evidence of gaming of performance system by physicians, but others have found that no such evidence exists. The evidence base is also ambiguous as to the effects on intrinsic motivation of providers, with some reporting no effects on motivation but with other (United Kingdom-based) studies finding that it may have reduced perceived physician autonomy and undermined providers’ sense of professionalism. Regarding the design features of P4P schemes, results from this review of reviews suggest that P4P may be more effective when directed at individuals compared to large groups, possibly because the risks are not shared under an individual scheme. More positive effects were also found for schemes that adopted absolute and not relative targets, potentially suggesting that “room for improvement” and benchmarking should be an important consideration for the design of P4P schemes. Finally, a higher degree of provider engagement and the collaborative design of schemes was found to correlate with better results.

Case study: an example showing the impact of P4P on the quality of primary care

The QOF was introduced in 2004. The scheme made general practitioner (GP) income dependent on the performance on several quality and outcome indicators, and affected the majority of GP practices in the United Kingdom.²⁴ The QOF accounts for around 20% of the budget for general practice. On the introduction of the QOF GPs saw their personal income increase by around £25,000, a 30% increase on the previous year.²⁵

Practice performance was measured on four domains of quality: clinical, organizational, additional services, and patient experience. Points could be earned in each domain, which would then be converted into practice income at a rate of £75 per point for the average practice in 2004/2005.²⁴ A total of 1,050 points were available in 2004/2005; the breakdown of points and examples of quality indicators for each domain is given in Table 1.²⁶ Other factors determined how much income would result from the number of points earned, such as the size of the practice population and the prevalence of disease in the area. Average performance was high since the first year, when practices earned 95.5% of total points and 96.7% of clinical indicator points.²⁷

Table 1.

Example indicators from 2004/2005

Clinical indicators (total 550 points)

• BP 2. The percentage of patients with hypertension whose notes record smoking status at least once (10 points)

• DM 6. The percentage of patients with diabetes in whom the last HbA1c is 7.4 or less in last 15 months (16 points)

Organizational indicators (184 points)

• The records, hospital letters, and investigation reports are filed in date order or available electronically in date order (3 points)

• The practice has arrangements for patients to speak to GPs and nurses on the telephone during the working day (1 point)

Additional services (36 points)

• Antenatal care and screening are offered according to current local guidelines (6 points)

• The practice has a system for informing all women of the results of cervical smears (2 points)

Patient experience (100 points)

• The practice will have undertaken an approved patient survey each year (40 points)

Abbreviations: BP, blood pressure; DM, diabetes mellitus; GP, general practitioner; HbA_1c, hemoglobin A_1c.

The future of the QOF remains unclear. Changes planned for 2014/2015 will see 25% of the total points available removed from the QOF.²⁸ The income that would have been linked to these points will be transferred to core practice funding, which is determined by capitation. This move suggests a desire to streamline the QOF while avoiding unpopular cuts to the health care budget and GP incomes.

Impact on the quality of primary care

Owing to its large scope and scale, the QOF has been the subject of a substantial body of research into the effects of P4P on quality of care. The financial incentives were significant, and the clinical indicators were chosen based on evidence of the effectiveness of the indicators for improving patient care. Practice performance on QOF indicators was published from 2005 onwards; as these data were not collected prior to the introduction of the scheme, it is not possible to make direct assessments of the impact of the QOF on the specific factors incentivized. Studies that have evaluated the impact of the QOF on the quality of primary care have hence made use of alternative data collected before and after the introduction of the QOF.

Intended effects on quality

Previous results have depended on the choice of disease, with only certain diseases seeing significant improvements due to the QOF. Between 2000 and 2007, just over 470,000 patients with hypertension had their quality of care recorded on various measures ranging from monitoring to control.²⁹ Across all of these measures, the quality of care was high; this did not change significantly when the QOF was introduced. The high pre-QOF performance on hypertension indicators was mentioned as the likely reason for the lack of response to the incentives.²⁹

Data extracted from patient records has been used to track changes in quality of care for asthma, coronary heart disease, and diabetes between 1998 and 2005.³⁰ This time period included two observations from before the QOF (1998 and 2003) and one observation from after the QOF (2005). Improvements above trend in quality were found for asthma and diabetes, but not for coronary heart disease. Further analysis with the addition of 2007 data found that the increases seen for diabetes and asthma were not maintained; instead, performance was stable between 2005 and 2007.³¹

Before and after data on 42 measurement and prescribing indicators from English practices were used to investigate the impact of the QOF.³² Performance increased in incentivized areas of quality following the introduction of the QOF, but this appears to have been largely due to a step-increase in quality immediately after the QOF introduction, as performance increases were not found in later years of the QOF.

A systematic review on the impact of the QOF included 94 studies in total.³³ The review reinforces the findings discussed above: that performance increases were accelerated in the first year of the scheme for incentivized quality, but then returned to pre-QOF rates.

Research has shown that when indicators are removed from the QOF, performance remains at the incentivized level even in the absence of the financial incentives.³⁴ This result suggests that the incentives have a persistent effect after they have been removed. This could be due to investments made while the incentives were active, meaning that maintaining the same level of quality would come at a small cost to the practice. Or, as the authors state, these indicators that had their incentives removed covered aspects of care that were incentivized elsewhere in the scheme. If the latter point is true, it would suggest incentives are spilling over from other indicators and the true impact of removing incentives is still unknown.

By encouraging a standard of performance, P4P may help reduce any existing inequalities in health and/or health care utilization between groups of the population. Data from the first year of the QOF showed that practices in less deprived areas delivered a higher quality of care.³⁵ Practices with lower QOF performance have a greater chance to improve on quality than practices with already high performance. By the third year of the QOF, the performance gap between the most and least deprived had disappeared.^36,37 Despite the lack of socioeconomic inequalities, there were still disparities in quality between age, sex, and ethnicity.³⁸

Unintended effects on quality

A comparison of patient perceptions collected in surveys from 1998, 2005, and 2007 provided a test for potential spill-over effects and unintended consequences of the QOF.³¹ These revealed no effect on patients getting appointments and on communication, and decreases in patient-rated continuity of care.

Further work into possible spill-over effects focused on the recording of incentivized and non-incentivized risk factors for patients with and without diseases targeted by the QOF.³⁹ Analyzing a period of time from 2000 to 2006, rates of recording were found to have increased for all the various groups of patients used, suggesting positive spillover effects on quality. The effect sizes did differ, however, and was largest for incentivized indicators for patients with targeted diseases. Increases in recording rates for risk factors that were not incentivized but where the patient had a disease that was targeted elsewhere in the QOF also suggested a positive spillover effect.

Using 42 measurement and prescribing indicators from English practices, Doran et al found performance was below trend for non-incentivized areas of quality by the third year of the QOF, which suggests effort was diverted away from these areas.³²

Further evidence of unintended consequences on non-incentivized quality has been reported.³³ With respect to non-incentivized clinical areas, performance was relatively low prior to the QOF and has dropped further since 2004. Doctors, nurses, and patients felt that the QOF has had a negative effect on continuity of care.

Strong financial incentives can lead to undesirable behavior, such as cheating.⁴⁰ With respect to the QOF, this cheating could come in the form of manipulating the number of eligible patients to artificially improve performance. Research into this form of cheating found some evidence that it was occurring.⁴¹ The evidence for cheating within the QOF is limited, and further research is needed to fully understand the prevalence of this behavior and its wider effects.

Limitations of the design of the QOF for measuring the quality of primary care

There are several design features of the QOF which have made analysis more difficult; the most significant of these being the simultaneous implementation without a trial period, control/treatment groups, or data collected before the financial incentives started. This design severely hindered the ability to detect a causal effect of the policy using accurate incentivized indicators of quality.

Within the QOF, points were transferred between indicators, and new indicators were introduced when old ones were retired.⁴² This feature allowed the QOF to adapt to meet the changing health needs of the target population. What could not be changed, however, was linking the financial rewards to an absolute measure of quality of care. This meant that after the first year of the QOF, when performance was high but not increasing further, large payments were still being earned based on existing performance. The financial incentive to improve quality disappeared once quality reached a certain level. Had the QOF rewarded improvements in quality, we may have seen a more continued upwards trend in quality.

Income per QOF point was decided based on some prediction of how practices would perform. It was predicted that around 75% of potential points would be achieved; when the figure came out at 90%, the scheme proved more expensive than initially thought.⁴³ More accurate predictions could have made the incentives more effective by setting targets at higher levels. When the QOF was revised for 2006/2007, more difficult targets were set; from 2015/2016 targets will be revised yearly in line with practice performance.²⁸

The future importance of P4P in ensuring the quality of primary care

The existing evidence suggests P4P has the potential to have an effect on the quality of primary care. In many P4P schemes, risks have not been managed sufficiently. Typically schemes are not piloted, and data on targets (and nontargets) before the scheme are rarely observed; these create barriers for identifying and anticipating the effects and potential risks of P4P.

P4P is a method by which risk is transferred from purchasers to providers of health care. Financial incentives put an element of providers’ incomes at risk by making payments conditional on achievement of certain targets/adherence to certain behaviors. Financial incentives can also direct provider behavior towards achieving a standard of health-care and (potentially) health outcomes, hence reducing the risk of health care delivery that may not be seen as welfare maximizing to the purchaser. The design of P4P, however, can create several types of risk for purchasers.

The focus on particular elements of quality can lead to unintended consequences on other aspects of quality in health-care. Reducing the risk of poor health care outcomes for a specific unit of measurement may increase the risk of adverse outcomes across other non-incentivized aspects of quality. In addition, achievement of quality goals by providers is determined in part by patient lifestyle behaviors and compliance, which are beyond the control of providers. Thus, providers are exposed to the risk that their measured performance may be contaminated by factors beyond their control.

In addition to the unit of measurement, the method by which performance is measured has also been found to influence provider behavior. The size, frequency, and recipient (practice or practitioner) of P4P income are linked to the risk of providers gaming of the scheme.

As most P4P schemes are introduced with little or no prior trial, determining cost-effectiveness is problematic. The analysis of schemes is largely based on a comparison of before and after the introduction of the scheme, with little evaluation into the performance of the scheme under experimental conditions. Such approaches would better lend themselves to more rigorous evaluation of factors such as cost-effectiveness, and of how modifications to a scheme may affect the potential negative impacts of risk transfer.

P4P has the potential to influence, and be influenced by, other sectors of health care. Policies which change behavior within primary care can have wider effects on the larger health care system. For example, the QOF incentivized better management of chronic conditions, and better management has been shown to reduce emergency hospital admissions for diabetes⁴⁴ and stroke.⁴⁵ The recent generation of Accountable Care Organizations (ACOs) to align primary care doctors and hospitals in the US⁴⁶ give a good opportunity to explore the impacts of quality incentive schemes within and between health care sectors. The ACOs would be responsible for coordinating the care for their patient population, measuring performance, and linking payments to improvements in quality. The types of performance incentivized can be set locally and include a wide range of activities happening in a primary care or hospital setting; for example, cancer screening, hospital readmissions or quality of life outcomes.⁴⁷ ACOs represent an ambitious change in policy for the US and the appropriate performance measures as well as the means to collect performance data need to be used.⁴⁸ Several publicly funded (Medicare and Medicaid) programs have been started, and these have been the focus of the current research.⁴⁹ Research shows the new programs have reduced costs in some participating institutions and, more generally, for patients eligible for Medicare and Medicaid,⁵⁰ and has improved quality of care chronic conditions and pediatric care.⁵¹

Conclusion

Overall, the evidence for the effectiveness of P4P for improving quality of care in primary care is mixed. This is to some extent due to the fact that the P4P schemes used in primary care vary considerably. There are many different schemes that incentivize different aspects of care, in different ways and in different settings. This makes evaluation problematic. The QOF is the largest example of P4P in primary care and is in its tenth year. Evidence suggests incentivized quality initially improved following the introduction of the P4P scheme, but this was short-lived. There are examples where the transfer of risk induced by P4P schemes result in positive spillover effects on non-incentivized quality, and improvements in equity in quality. However, there is also evidence where the transfer of risk has resulted in negative spillovers for non-incentivized quality. In addition, the identification of the effects of P4P is problematic, as targets from other incentive schemes outside of primary care may impact on the quality of primary care and vice versa. If P4P in primary care is to have a long-term future, the question about scheme effectiveness (perhaps incorporating the identification and assessment of potential risk factors) needs to be answered robustly. This would require that new schemes be designed from the onset to support their evaluation: control and treatment groups coupled with before and after data.