Pay for Performance in Primary Care in England and California: Comparison of Unintended Consequences

Abstract

PURPOSE We undertook an in-depth exploration of the unintended consequences of pay-for-performance programs In England and California.

METHODS We interviewed primary care physicians in California (20) and England (20) and compared unintended consequences in each setting. Interview recordings were transcribed verbatim and subjected to thematic analysis.

RESULTS Unintended consequences reported by physicians varied according to the incentive program. English physicians were much more likely to report that the program changed the nature of the office visit. This change was linked to a larger number of performance measures and heavy reliance on electronic medical records, with computer prompts to facilitate the delivery of performance measures. Californian physicians were more likely to express resentment about pay for performance and appeared less motivated to act on financial incentives, even in the program with the highest rewards. The inability of Californian physicians to exclude individual patients from performance calculations caused frustration, and some physicians reported such undesirable behaviors as forced disenrollment of noncompliant patients. English physicians are assessed using data extracted from their own medical records, whereas in California assessment mostly relies on data collected by multiple third parties that may have different quality targets. Assessing performance based on these data contributes to feelings of resentment, lack of understanding, and lack of ownership reported by Californian physicians.

CONCLUSIONS Our study findings suggest that unintended consequences of incentive programs relate to the way in which these programs are designed and implemented. Although unintended, these consequences are not necessarily unpredictable. When designing incentive schemes, more attention needs to be paid to factors likely to produce unintended consequences.

INTRODUCTION

To improve health care quality, payers are increasingly using financial incentives to reward physicians and medical groups that meet specific performance targets. There has been a rapid recent growth in the number of these pay-for-performance programs, which provide financial incentives for quality improvement, in primary health care.¹ In the United Kingdom an ambitious and costly pay-for-performance program—Quality and Outcomes Framework (QOF)—was introduced for all primary care physicians in 2004.² In the United States, almost all pay-for-performance programs include incentives for primary care physicians,³ and the largest physician incentive program, a statewide initiative in California,⁴ involves even more physicians than in the UK program. In California many primary care doctors participate in larger physician organizations (such as prepaid multispecialty medical groups and independent practice associations, or IPAs,^5,6 which contract with health plans on their behalf), and incentive payments are made to these larger organizations as opposed to individual physicians or practices. In England, payments are made directly to practices, which are mostly groups of between 1 and 10 primary care physicians. These payments contribute as much as 30% of practice income. In most (but not all) of the California settings, the amount received by physicians is much lower.

Although pay-for-performance programs may deliver on their stated goals,^7,8 they could have unintended effects on other aspects of care or on physician motivation. These effects include encouraging physicians to avoid sicker patients,⁹ exacerbating disparities,^10,11 and neglecting types of care for which quality is not measured.¹² It is therefore important when designing incentive programs to consider unintended effects in addition to desired benefits. A combination of theory and empirical data can provide important lessons to inform the design of pay-for-performance programs.¹³ Because externally imposed incentive programs can undermine internal motivation,^14,15 careful attention should be paid to the manner in which incentive programs are implemented. Although economic and psychological theories alert us to potential unintended effects of external incentives,¹⁶ there has been no assessment to date of the extent to which such effects are manifest in practice in the California initiative and only limited assessment within English primary care.^17,18 We pay particular attention to unintended consequences in each setting.

METHODS

In this study, we examined the effects of financial incentives of pay for performance in England and California to understand how differences in the design and implementation of these programs influence their impact in primary care settings. Both programs involve paying physicians based on performance against targets, but the number of targets is much greater in the English initiative. The English program allows physicians to exclude patients (or report exceptions) if they refuse treatment, whereas excluding noncompliant patients is not permitted in the California program. UK physicians face a single payer and 1 pay-for-performance program. We compared physician attitudes to this scheme with physician attitudes to the statewide initiative in California. Physicians in California, however, face other targets and pay-for-performance initiatives in a context of multiple payers and payment rules. In addition, in England, the pay-for-performance initiative was part of a broader program of reform that greatly increased investment in primary care. In contrast, in the US context, there has been little new investment in primary medical care.

To understand each system and the unexpected consequences that might arise from pay for performance, we conducted face-to-face interviews with 40 primary care physicians. In the English sample (20) physicians were drawn from 2 regions (northwest and southwest). In the California sample (20) physicians were drawn from 4 organizations that ranged in size from 600 to 3,000 physicians and health care clinicians. Three of these organizations (A, B, and C) were IPAs, and 1 organization (D) was a medical group with salaried physicians.⁶ In the largest of these organizations (A), a decision had been taken to link a large percentage (up to 30%) of physician remuneration to the achievement of quality targets. In the other physician organizations, the percentage of remuneration linked to targets was substantially less (less than 5%). All physicians in the English sample used electronic medical records compared with only 7 physicians in the US sample.

The study was approved by the relevant institutional human subject review committees. The sample was identified using snowballing (a small number of informants put the researcher in touch with others, who then nominated colleagues and other contacts, and so on), a sampling technique used in qualitative research.¹⁹ In the US context, this technique resulted in a sample in which physicians participating in group- or IPA-level activities in addition to providing patient care (eg, membership of the IPA or group quality improvement committee, board-level participation) were overrepresented. The physicians interviewed worked predominantly in urban settings, though the populations served ranged from affluent to disadvantaged. To capture a broad spectrum of experiences and views, we sampled both salaried and self-employed physicians. The interviews were conducted by 1 researcher (R.M.) using the same topic guide, transcribed verbatim, and analyzed thematically using Atlas Ti software (ATLAS.ti GmbH, Berlin, Germany).

Both authors (a social scientist and a primary care physician-academic, both experienced qualitative researchers) read all transcripts independently, using a constant comparative method to interpret the data.²⁰ Key concepts were identified using an open-coding method. Once coding was complete, the codes that had common elements were merged to form categories. Disagreements were discussed until a consensus was achieved. The interview schedule was open-ended and addressed reasons for entering primary care, likes and dislikes about work, and attitudes toward and impact of incentives on practice. There were systematic differences in the themes that recurred in the 2 countries, and we report these differences.

RESULTS

Three major themes emerged from the analysis: changes in the nature of the office visit, threats to the physician-patient relationship, and threats to professional autonomy.

Changes in the Nature of the Office Visit

English physicians were much more likely than their Californian counterparts to report that the pay-for-performance program had changed the nature of the office visit. The requirement to enter data into the electronic medical record to respond to the large number of targets was described as reducing eye contact, increasing time spent on data collection in the office visit, and potentially crowding out the patient’s agenda.

Compared with California physicians, English physicians faced a much larger number of targets (80 vs 12 clinical targets in the statewide program at the time the interviews were conducted), and the program in England relied exclusively on data captured from electronic medical records. Pop-up boxes on the computer screen highlighted any areas of activity required to meet targets, prompting clinicians to take action or enter data during the office visit.

You look at the screen and the screen’s completely obscured by the list of yellow boxes, and it’s always trying to balance up the mood the patient’s in and getting the boxes ticked, especially with people that don’t come in that often. You know, they come in and tell you, you know, that “Oh, my son’s died last week,” and you go, “Yeah, yeah, whatever. Do you smoke?” or “Yeah, watch, watch your weight” and stuff (physician identifier [ID] 16, practice 2).

One of the things that happens is the patient comes in, the boxes pop up, and you get straight into doing all that stuff… and they’re out of the room.… And I just think there is just more chance to, you know, miss [something significant], and that’s such an important bit, isn’t it (ID11, practice 3)?

The pay-for-performance initiative appeared to have little impact on the nature of the office visit in Californian settings. Few of the California physicians used electronic medical records, which meant that computerized prompts and electronic templates were largely absent in these settings. Californian respondents were also much less aware of targets than their English counterparts.

Doctors have no idea [what the targets are], but they would guess some of the things probably right, like the mammogram…, the cervical smear.... They would guess that and probably diabetes…. If you are a sophisticated person, you might be able to find it on the Internet (ID8, organization C).

Californian physicians affiliated with the physician organization (organization A, comprising 3,000 physicians and health care clinicians) that in our sample offered rewards comparable to those of the English program, however, were much more likely to be aware of targets.

What I think, probably, having the measures has done for us is made our outreach a little bit more aggressive, so that we track and are able to look and see, “OK, you haven’t gotten your mammogram. We’re going to send you a letter, rather than just remind you next time you happen to wander into the office” (ID13, organization A).

All the Californian physicians reported receiving feedback on performance from their physician organization (IPA or medical group), and some received requests to take action (eg, being provided with names of patients overdue for a Papanicolaou smear). Where physicians chose to act on this information, their response was normally to get office staff to contact and remind these patients, so the flow of office visits was for the most part unchanged by pay-for-performance targets.

Threats to the Ongoing Physician-Patient Relationship

Although the absence of electronic records and computerized prompts meant that targets were seen as less disruptive of the flow of office visits among US physicians, adverse effects on physician-patient relationships were nevertheless identified, especially among physicians affiliated with organization A, the organization with the largest financial rewards. Physicians affiliated with this group expressed resentment about patients who refused to comply with their advice. In extreme cases patient noncompliance led to physicians telling patients they would be disenrolled unless they changed their behavior.

I tell them to leave. I told someone, you’re killing my pay for performance. You are the one that keeps being my outlier. Go join another medical group (ID14, organization A).

The inability to exclude patients who refuse treatment or testing (unlike the UK system) appeared to increase pressure to cajole and persuade patients to secure their compliance. Other strategies reported by physicians included accusing patients of damaging their physician’s rating or lying to patients about the financial consequences of their refusing to comply.

Some physicians also reported bypassing informed consent procedures to meet screening targets for Chlamydia trachomatis. In addition to considerations of ethics, choosing not to request informed consent raises questions about the potential damage to doctor-patient relationships when patients who are tested without their knowledge are subsequently found to have a positive test for C trachomatis.

Well, everybody who didn’t have one, we sent out a form with a letter for Chlamydia screening. And we got 5 people who actually came back and did it, out of I don’t know how many hundred. So now, anybody who comes in and is in that age, I just tell them to get a urine. And I just send it in. This is life: I just send it in. If we’re going to be rated on it by somebody, that’s fine. We do it (ID5, organization B).

English physicians also described targets as giving them a greater incentive to encourage patients to comply. The English program, however, allows physicians to exclude or file an “exception report” for patients if they refuse treatment.²¹ No English physicians reported resorting to the methods described by Californian respondents or reported similar feelings of resentment toward patients.

I think you’d always have a discussion with the patients about pros and cons of treatment, and it would always try to be a 2-way process.… I don’t think QOF makes me treat people differently (ID3, practice 4).

You know, if I get somebody in who says, “Look, I don’t care about my blood pressure, I’m not taking your tablets.” Fine it’s your life,” okay (ID12, practice 6).

Perceived Impact on Autonomy

In both contexts, the development of clinical indicators involved a high degree of participation by primary care professionals and their representatives. Whereas in England all primary care physicians were allowed to vote before the introduction of the new incentives arrangements, in California proposed indicators were published on the Internet and public comment was invited. Following this process, indicators were pilot tested before being incorporated into the incentive program. Despite efforts in both contexts to consult primary care physicians and to encourage commitment to the process, and despite the larger number of targets in England, English physicians were generally more supportive of and accepted the targets that formed part of the pay-for-performance program. Differences in attitudes appeared to be related to perceptions about the implications of the respective target regimes for clinical autonomy.

Most of the English physicians suggested that the targets were helpful and did not appear to view them as undermining their ability to act as autonomous professionals.

Although some Californian physicians were supportive of pay for performance, most expressed much less satisfaction with it than their English counterparts. The incentive program was perceived as something externally imposed and managed, which made physicians feel that their autonomy was being challenged or that they were not trusted to perform in the absence of incentive payments.

At the same time US physicians believed they were being held accountable for things beyond their control. Data on their performance were generally collected by third parties, and the situation was compounded by the physicians’ inability to exclude patients from performance data if patients refused treatment or the targets were inappropriate for other reasons.

I think things like the QOF are very difficult to argue with. I think we’ll all have views as to whether there is missed opportunities in that and whether the disease areas are absolutely the ones that should be in there. Or whether all the indicators are quite as sensitive as they should be, but I think, you know, generally if those indicators are met, we should see caring improve...it’s been a generally good thing (ID18, practice 5).

We see the QOF as the bare minimum, and I think we should be doing over and above that.... I think it’s fine (ID2, practice 4).

Physicians are monitored more than anybody. Are attorneys monitored? No. Are dentists monitored? No, not as far as I know. Are chiropractors monitored? No. So, it seems to be that physicians have either rolled over and given over their rights, and maybe they’ll be pushed to a certain point where they will rise up and say, “No more.” I don’t know (ID20, organization D).

The system was viewed by many as unfair and opaque because it failed to take account of variations in practice populations, comprised indicators that were not amenable to control by physicians, withheld money that was due to physicians, and added to workload.

You might get a little bonus of money that some doctors would’ve considered part of their rightful payment to begin with. So the whole notion of withholds leaves a very bad taste. If you say, “What do you think of withholds?” that’s like saying, “What do you think of hemlock?” Not something I’d want to take. The problem with paying for performance…is it smacks of a withhold…. There must be adequate accounting methodologies to account for the recalcitrant patient or the patient who refuses for whatever reason treatments and therapies, and that’s not in the current model. It’s a source of disgruntlement (ID11, organization A).

DISCUSSION

This study suggests that the unintended consequences of pay-for-performance programs are likely to vary according to the design and implementation of these programs. The strength of the study lies in its in-depth qualitative approach, which allows us to examine some unintended effects that may result from financial incentive programs. Our results support the importance of a number of key design features of pay-for-performance programs described in recent analyses of financial incentives in American medicine.^13,22

This study has a number of limitations, however. Our sampling method was not designed to provide representative views of those working in primary medical care; rather, we wanted to contribute to existing conceptual models of pay for performance and to identify areas of importance that may need more detailed investigation. In the US sample, respondents were more likely to be participating in activities at the medical group level in addition to providing patient care (eg, membership in the group quality improvement committee, group board participation), making their assessment more positive than the general population of primary care physicians who are subject to financial incentives for quality. The pay-for-performance initiative in England is part of a strategy of increasing investment in primary health care, and the generous financial incentive program has been broadly well received by primary care physicians in England. The introduction of pay for performance (alongside other contractual changes) has been associated with improvements in recruitment and retention of physicians in England,²³ whereas in California there have been no similar increases in primary care resources, and the gap between primary care and specialist remuneration remains large.²⁴

A key issue that came out of our interviews was resentment about loss of autonomy by Californian physicians. Paradoxically, this issue was less prominent in UK interviews, despite the much larger number of performance measures on which UK physicians are judged. There are a number of possible reasons for this difference. First is that UK physicians may be more familiar with an environment in which their performance is subject to external influence, for example, nationally agreed-upon guidelines for the management of common diseases have been widely used in the United Kingdom in the last few years. Even so, the large number of performance measures may also have been responsible for the main unintended consequence of the UK incentive program, namely, the crowding out of the patient’s agenda due to the large number of targets and the computerized support required to deliver the targets. The response of many UK physicians to this burden has been to hire more nurses to deal with more routine protocol-driven aspects of care to minimize the impact on their own personal practice. Some still find, however, that the pay-for-performance program makes a significant and negative intrusion into office visits.

A second common reason for resentment among Californian physicians was the inability to exclude individual patients from quality calculations. Our interviews contained reports of seriously dysfunctional or coercive behavior by physicians in relation to patients who were regarded as noncompliant, including forced disenrollment. Rosenthal and Adams Dudley¹³ identify the ability to prioritize quality indicators for underserved populations as a key feature of pay-for-performance programs, and some characteristics of the Californian programs we observed would actively disadvantage patients from such groups. In the UK program, physicians are permitted to exclude patients from quality calculations. One concern is that the privilege might be abused by physicians for their own financial gain rather than be used when clinically appropriate, although such abuse does not appear to have occurred on a widespread basis in the United Kingdom.²¹

Understanding the Incentives

In California, third party payers have expressed disappointment that “breakthrough improvements in health care services,” the declared aim of incentive programs, have not been achieved.²⁵ Our study found, however, that many physicians were unaware of the target contents or had a poor understanding of the relation between their performance and incentives payments received. This lack of awareness is partly because individual US physicians may contract with several different payers who have different quality targets. In contrast, the United Kingdom has one nationally agreed-upon set of incentives. In addition, US health plans have moved away from contracting with individual physicians and small practices, with 61% now targeting larger physician organizations instead.²⁶ Incentives in the United Kingdom are applied at the level of the individual family practice (1 to 10 family physicians), and physicians are generally very familiar with the incentives, partly because of the widespread use of computerized prompts in electronic medical records. As a result, despite the UK program being nationally negotiated, we found a more direct sense of involvement by UK than US physicians in our sample, which may have contributed to the feelings of frustration and resentment expressed by the US participants.

Rewarding Improvement

Seventy percent of US programs pay for the achievement of quality thresholds¹³ in contrast to the United Kingdom, where rewards are dependent on incremental levels of improvement. In addition, UK payments are based on data entered by physicians in the medical records and are therefore under the direct control of the physician, whereas data are more commonly collected by third parties in the United States. Both of these factors may contribute to the greater concern about loss of control expressed by US physicians, even though they are assessed on considerably fewer targets than their UK counterparts.

The unintended consequences of incentive programs reported by physicians in this study appeared to vary according to the design and implementation of the programs. Programs involving surveillance and external rewards have the potential to damage intrinsic motivation (the desire to undertake a task for its own sake)^27,28 and potentially do more harm than good. Reward systems that promote feelings of competence and autonomy, however, are likely to enhance intrinsic motivation,²⁹ and a context perceived as supportive rather than pressuring will further enhance motivation.³⁰ Incentive programs involving extrinsic rewards implicitly assume that rewards are necessary to induce desired behaviors. Physicians are not driven only by money, however. As Frey³¹ argues in his “not just for the money” theory of personal motivation, financial rewards are only one of a number of important motivators of professional behavior, and among the most negative comments in our interviews were those from physicians in the Californian program with the biggest financial rewards. It would therefore be a mistake to think that the problems of pay for performance could all be addressed by getting the correct level of financial reward.

The potential adverse effects of external incentives on motivation are likely to be diminished where individuals identify with the goals and values of incentive programs and feel that they have a degree of autonomy in their delivery. In other words, when designing incentive programs, it is important to consider the manner in which they are implemented and the extent to which the context is perceived as supportive. Ideally, incentive programs should aim to induce “identified regulation,” a state in which external incentives are aligned with internal drivers and, “where people have a full sense that the [incentivised] behavior is an integral part of who they are and…is self-determined.”¹⁵ A context in which physicians are held accountable for factors beyond their control, where physicians feel they are under surveillance by health plans and their physician organizations, and where performance regimes are viewed as opaque or unfair as reported by some of our Californian physicians, is likely to be perceived as challenging rather than supportive. In such circumstances, increasing the percentage of income available for meeting targets risks damaging motivation and patient care rather than encouraging the sort of quality improvements sought by health plans.

Our findings suggest that the unintended consequences of pay-for-performance programs may differ according to the design and implementation of the program. Our study raises concerns about the dysfunctional behaviors arising from the introduction of financial incentives and their likely consequences for patient care. Although these consequences are unintended, they are not necessarily unpredictable or unavoidable.