Unintended Consequences of Implementing a National Performance Measurement System into Local Practice

ABSTRACT

BACKGROUND

Although benefits of performance measurement (PM) systems have been well documented, there is little research on negative unintended consequences of performance measurement systems in primary care. To optimize PM systems, a better understanding is needed of the types of negative unintended consequences that occur and of their causal antecedents.

OBJECTIVES

(1) Identify unintended negative consequences of PM systems for patients. (2) Develop a conceptual framework of hypothesized relationships between PM systems, facility-level variables (local implementation strategies, primary care staff attitudes and behaviors), and unintended negative effects on patients.

DESIGN, PARTICIPANTS, APPROACH

Qualitative study design using dissimilar cases sampling. A series of 59 in-person individual semi-structured interviews at four Veterans Health Administration (VHA) facilities was conducted between February and July 2009. Participants included members of primary care staff and facility leaders. Sites were selected to assure variability in the number of veterans served and facility scores on national VHA performance measures. Interviews were recorded, transcribed and content coded to identify thematic categories and relationships.

RESULTS

Participants noted both positive effects and negative unintended consequences of PM. We report three negative unintended consequences for patients. Performance measurement can (1) lead to inappropriate clinical care, (2) decrease provider focus on patient concerns and patient service, and (3) compromise patient education and autonomy. We also illustrate examples of negative consequences on primary care team dynamics. In many instances these problems originate from local implementation strategies developed in response to national PM definitions and policies.

CONCLUSIONS

Facility-level strategies undertaken to implement national PM systems may result in inappropriate clinical care, can distract providers from patient concerns, and may have a negative effect on patient education and autonomy. Further research is needed to ascertain how features of centralized PM systems influence whether measures are translated locally by facilities into more or less patient-centered policies and processes.

Electronic supplementary material

The online version of this article (doi:10.1007/s11606-011-1906-3) contains supplementary material, which is available to authorized users.

“As we bustle from one well documented chart to the next, no one is counting whether we are still paying attention to the human beings” Dena Rifkin, New York Times 11/17/2009 Page D5.

INTRODUCTION

Performance measurement (PM) systems are central to many healthcare organizations’ quality assurance programs.¹ Orienting quality management around quantitative measures has led to documented improvement in a variety of domains.^2–4 In the Veterans Health Administration (VHA), implementation of a national clinical PM system was associated with dramatic improvements in clinical outcomes, effectively transforming VHA into a quality leader.^5–7 However, in recent years, scores on many VHA measures have stabilized at high levels, indicating that the benefit of these measures has mostly been realized. Within VHA and other organizations with mature PM systems, there may be more potential for improvement by concentrating on minimizing negative effects of PM than by focusing on further improving already high scores.

Concerns have been expressed about negative unintended effects ⁸ of PM on unmeasured aspects of care,^9–12 on patient-centered care,^6,13 and on provider morale and professionalism.^8,9,14–19 Chassin et al. recently argued that only measures with “little or no chance of inducing unintended adverse consequences” should be used for accountability purposes.²⁰ However, only a few empirical studies have delineated adverse effects such as decreased continuity of care,² provider avoidance of sicker patients,²¹ and increased disparities.²² None of these studies have focused on the pathways by which PM programs produce these effects—information that is critical to designing system improvements. We conducted a qualitative study to explore possible relationships between a centralized clinical PM system, facility-level practices to implement the PM system into daily care, and unintended negative consequences for patients.

METHODS

Design, Setting and Participants

We conducted semi-structured, in-person, individual interviews of staff at four VHA facilities between February and July 2009. To capture diverse perspectives we identified facilities using a dissimilar cases sampling strategy.²³ Facilities were chosen based on: 1) facility size (number of outpatients) and 2) performance on an index of primary care clinical PM scores created by averaging scores on cancer screening, cardiovascular health, endocrinology, infection control, and tobacco. After eliminating facilities in the middle two quartiles on either of the two measures, we selected one facility from each of four groups (large high-performers, small high-performers, large low-performers, small low-performers).

At each site we interviewed primary care staff including clinic leaders (primary care clinic physician and nurse directors), physicians, non-physician practitioners (e.g. nurse practitioners), and intake nurses. To better understand the processes used to implement PMs we also interviewed facility leaders and quality/PM officers. We conducted 60 interviews, one of which was excluded because it was inadvertently not recorded. Table 1 shows facility and participant characteristics.

Table 1.

Participating VHA Facilities

	Facility 1	Facility 2	Facility 3	Facility 4
Region	South central	Midwest	Midwest	Southeast
Number of veterans served	87,000	20,000	34,000	126,000
% of Clinical primary care PMs above VHA goal	69%	77%	100%	85%
Number of interviews					Total
Facility administrators	2	2	3	2	9
QI / PM specialists	2	2	2	1	7
Clinic physician leaders	2	0	1	0	3
Clinic nurse leaders	0	1	1	1	3
Physicians	3	5	3	3	14
Non-physician practitioners	3	1	3	3	10
Intake nurses	3	3	5	2	13
Total	15	14	18	12	59

Data Collection

Interviews lasted 60 to 90 minutes and were led by one of the researchers using a guide developed through a review of the PM literature. A series of formative interviews (led by the first author) were conducted to refine the order of topics and phrasing of questions in the interview guide. Interviews focused on the effects of PM on patients, providers, clinics, and facilities. (Provider and leadership interview guides are available in the online Appendix.) Additional modifications to the guide were made as the interviews progressed to ensure that emerging topics were thoroughly explored. The lead interviewer directed the discussion to each topic in the guide, investigating how the PM system was implemented in primary care and its effects on the patient, provider, encounter, clinic, and facility. Probes were used to uncover participant perceptions of the processes by which PMs result in unintended consequences. Participants were given a list of VHA primary care clinical PMs as a discussion aid (Table 2). Interviews were recorded and transcribed.

Table 2.

Interview Discussion Aide: 2009 VHA Primary Care Clinical Performance Measures

FY09 primary care clinical performance measures	FY 09 Benchmark %
Patients screened for obesity & offered weight management treatment if “at risk” in the past year	65
Substance use
Patients screened for alcohol misuse using audit-C in the past year	95
Brief alcohol counseling within 14 days for alcohol misuse screening result of 5 or greater	NA
Tobacco users who have been provided with brief counseling within the past year	84
Tobacco users who have been offered medications to assist with cessation within the past year	73
Tobacco users who have been offered a referral to smoking cessation specialty program in the past year	93
Mental Health Screening
Patients screened at required intervals for PTSD using PC-PTSD	90
Suicide risk evaluation following positive PC-PTSD screen within 24 hours	60
Timely triage of patients with positive PC-PTSD screening result within 24 hours	NA
Patients screened for depression using the PH-Q2 OR PH-Q9 annually	90
Suicide risk evaluation following a positive annual depression screen within 24 hours	60
Patients with new Dx of depression with adequate medication for at least 84 treatment days	61
Timely triage of patients with positive depression screen within 24 hours	NA
Screened initially for traumatic brain injury	90
Timely contact for follow-up following an initial positive TBI screen	NA
Screened initially for TBI and if positive screen follow up evaluation within 30 days of initial screen	NA
Hypertension
Hypertension patients on antihypertensive mono-therapy receiving thiazide	23
Hypertension patients on antihypertensive multi-therapy receiving thiazide	70
Cancer Screening
Breast cancer screening mammography among women age 50–69 within past 2 years	72
Cervical cancer screening pap smear among women age 21–64 within past 3 years	90
Colorectal cancer screening test among veterans age 50–80 (time interval dependant upon test)	79
Immunizations
Patients age 65 or older receiving influenza immunization between 9/1 and 3/31	83
Patients age 50 to 64 receiving influenza immunization between 9/1 and 3/31	66
Patients age 65 or older ever receiving pneumococcal vaccination	94
Vascular Disease
AMI, CABG, PTCA, or ischemic vascular disease patients with full lipid panel in past year	95
AMI, CABG, PTCA, or ischemic vascular disease patients with full lipid panel in past year & LDL-C <100	66
Patients with active diagnosis of hypertension whose most recent blood pressure recording was <140/90	75
Diabetes
Diabetes mellitus patients having HbA1c testing performed during the past year	95
Diabetes mellitus patients with HbA1c >9 or not done (poor control) in past year	15
Diabetes mellitus patients having full lipid panel in past year	95
Diabetes mellitus patients with full lipid panel in past year and most recent LDL-C is <100 mg/dL	67
Diabetes mellitus patients with most recent blood pressure within past year <140/90	79
Diabetes mellitus patients with nephropathy screening test or evidence of nephropathy	93
Diabetes mellitus patients with retinal exam biannually or annually if retinopathy in prior exam	88
Separate performance scores are calculated for patients who have been diagnosed with both spinal cord injury and diabetes for tobacco, influenza,  blood glucose, blood pressure and retinal exam measures

Measurement/Analysis

We first applied codes to quotes in interview transcripts using qualitative analysis software (NVivo 8.0). Initial “topcodes” were identified inductively by reviewing 15 interviews. As coding progressed, topcodes were refined to reflect emerging constructs. For each transcript, a second coder reviewed the initial topcoding. Discrepancies were resolved through discussion. (See Table 3 for a list of final topcodes.) Next, we applied more specific subcodes to quotes within each topcode. Subcodes were created by one reviewer and then applied by a second coder who also refined the list of subcodes. When subcodes were finalized, the first author created two new topcode categories — “positive consequences” and “negative consequences” — and assigned appropriate subcodes to one of these two topcodes. Subcodes assigned to these two topcodes were reviewed with the study team and refined. We conducted a thematic analysis²⁴ on the “negative consequences” codes by running a set of queries on the dataset to identify potential relationships between codes (e.g. “extra time/work” frequently appeared near “technical issues”) and by re-reading passages from the “negative consequences” subcodes to (1) confirm that each passage represented the “negative consequence” it was coded with, (2) confirm that relationships identified through queries were represented in the data, and (3) identify candidate quotes that concisely and accurately conveyed thematic categories.

Table 3.

List of Topcodes Applied to Transcripts

1.	Patient affect/reactions/satisfaction during encounter
2.	Effects on patient health or other benefits-harms to patient
3.	Patient compliance/effect on patients preventive health behaviors
4.	Motivating/educating patients
5.	Complex patients and special cases
6.	Procedures/process to address PMs during the encounter
7.	Role distinctions/who does what
8.	Affective staff reactions/ways PMs benefit or hinder staff
9.	Staff education/training
10.	Unintended staff behavior
11.	Feedback/evaluation/recognition/perceptions of own performance
12.	Technical issues
13.	Staff time issues or extra work
14.	Measurement Issues
15.	Evidence base
16.	Leadership emphasis/comments about leader attitudes or behavior
17.	Ideas for improvement/changes made to improve

We report the most commonly mentioned negative unintended consequences as well as several unintended consequences that were mentioned by a smaller number of participants but were considered by the study team to be relevant to practitioners and researchers interested in further exploring PM systems. We also provide a framework of relationships between PMs, facility/clinic implementation practices, staff attitudes and behavior, and unintended consequences that was developed based on the relationships identified between codes (Fig. 1).

Figure 1.

Pathways from national performance measurement systems to unintended effects on patient care.

We provide counts of the number of participants mentioning each unintended consequence in parentheses. Unless otherwise noted, each count includes at least one participant at all sites. (Readers should note that our methodology was designed to uncover a range of unintended consequences of PM that may exist in primary care, not to quantify the frequency with which each unintended consequence occurs or compare frequencies across facilities or respondent groups. The counts provided give some indication of the relative salience of each issue among participants; however, they should not be interpreted as a measure of the frequency of occurrence or magnitude.) Example quotes illustrating each type of unintended consequence were chosen by team consensus and are provided in italics. Because of the small number of facilities, and because facility characteristics were conflated with the order of site visits, we do not stratify our findings by facility subgroups.

RESULTS

Attitudes toward PM varied from very positive to very negative. Many participants noted benefits of PM that extend beyond increasing the proportion of patients receiving the measured aspects of care. For example, feedback from the PM system helps some clinic staff feel more confident that their care is thorough; performance scores can be a source of pride and positive competition; and incorporating preventive health PMs into the clinical encounter sends the message to patients that VHA cares about more than just their immediate needs. However, because the primary objective of our research was to identify opportunities for improvement, we focus our analysis on three unintended negative effects of PM on patients.

Within VHA, PMs are defined and monitored at the national level, but facilities are given considerable latitude over the methods they establish to achieve high scores. Providers in our study tended to focus more on these local policies, processes, and strategies than on the nationally generated PM definitions and scores. As depicted in Figure 1, our analyses suggest several pathways by which local processes developed in response to national PM policies have unintended effects on provider attitudes and behaviors and, in turn, on patient care.

Inappropriate Clinical Care

Treating the Measure not the Patient (Figure Pathway D1→C1→B1, B2→A1). Although all four facilities have systems in place to generate provider-level performance scores, providers at three sites were not required to regularly review their scores and many indicated that they were unaware of their scores. One site, however, distributed weekly reports to each provider containing lists of patients who were not in compliance with performance measures. Providers were asked to indicate in writing reasons for non-compliance or actions that would be taken to achieve compliance. Many participants at this site expressed concerns that too much emphasis was placed on PM scores (Site 3 – 10 participants mentioned).

I think it has become a contest, the number game is just a big thing lately. (Registered Nurse – Site 3)

Pressure to achieve high performance scores can incentivize providers to take actions to improve scores even if these actions are not in the patients’ best interests, including overuse of medications (32 mentioned concerns about inappropriateness of PMs for some patients, 17 mentioned specific concerns about medications or procedures).

There are a lot of providers that are really, really pushed by these performance measures and just add on drugs, add on drugs to treat the performance measure. (Physician – Site 2)

They’ll have a patient that will come into clinic and their blood pressure is up and they give them an oral medication in clinic and have the nurse re-check it . . . Clonidine actually brings blood pressure down quickly but then you get a rebound hypertension afterwards if you don’t follow-up . . . I’ve heard of that happening with one provider who no longer works here. (Facility Leader – Site 4)

Delegation of care to intake nurses (Figure pathway D2, D3→B2→A1). Computerized clinical reminders (CCRs) are commonly used throughout VHA to integrate PMs into clinical practice. At all four facilities intake nurses are responsible for the CCRs associated with the majority of PMs. Nurses feel that they are expected to complete all active CCRs at each visit, and do so with few exceptions. Several nurses (7) and physicians (2) expressed concerns that this can lead to inappropriate use.

I have some very healthy people in their 20s who all of a sudden come up as supposedly needing pneumococcal vaccine and I'm like ‘no, they don't.’ Unfortunately . . . the nurses get that one and so they’ll just go ahead and give a pneumococcal vaccine. (Physician – Site 1)

Decreased Provider Focus on Patient Concerns and Patient Service

PM Improvement Strategies that Inconvenience/Frustrate Patients (Figure Pathway D4→B1→A2). Sites implemented a variety of strategies to improve PM scores. Most of these seem beneficial to patients, including creating specialized clinics to address PM issues in certain populations (e.g. diabetes clinic), increasing time allotted per patient visit (all four sites now have 30 minute appointments), and calling patients to remind them to complete home colorectal cancer screening tests. Strategies employed at two sites (Sites 2 and 3) however appeared to unnecessarily inconvenience patients.

We’ve had days where it was towards the end of the month….and we have nurses calling and saying “Why don’t you come get your blood pressure here”. If we have somebody with 190, we’re not even going to call him, we’re going to call him to come in and take care of his blood pressure next month. But for purposes of (the performance measure) – we have patients that when they leave here, their blood pressure at that visit was 140 – if I bring him back in and we redo it, it may be lower or it may be higher – so we’ve done that. We’ve fixed a lot of them that way. (Facility Leader – Site 3)

As an emergency room nurse – they kept telling us . . . that we had to have all these (primary care) reminders done on every patient in the emergency room…we got told over and over … do the reminders, do the reminders … when people are really sick it’s just silly, it’s not the time to be doing all that teaching. (Intake Nurse – Site 2)

Repetition of Clinical Reminders (Figure Pathway D3,D5→B3→A2). Many CCRs at participating facilities are locally programmed to remain active even if a patient refuses a PM-related intervention. Although this is not mandated, CCR programmers are often instructed to match the CCR programming as closely as possible to national PM definitions, which do not exclude patient refusals from the score. As a consequence, many nurses (14) say they waste time asking patients the same questions at multiple visits.

There are some [patients] that are saying I’m just flat-out not doing it, don’t ask me anymore. And one of the nurses actually told me that somebody threw FOBT cards at her. (Intake Nurse – Site 1)

Although participants noted that many patients are used to the repetition of PM-related questions, a few primary care staff (3) at one site (Site 2) mentioned that some patients attempt to avoid it by refusing to respond or by adapting their responses.

A few of the guys…know exactly how many to answer (the alcohol screen) – it says like “less than 14 drinks a week” and they have that number memorized. How many did you have? ‘I had two a day, seven days a week, I had 14.’ (Intake Nurse – Site 2)

Patient Health Concerns Marginalized (Figure Pathway D3, D5, D6→C2, B3, B4→A2). Several inefficiencies in local PM documentation systems create time pressure during the clinical encounter for both intake nurses and providers. Although national VHA PMs are calculated through chart reviews, facilities in our study generated their own local performance scores through the CCR system. Therefore many providers (15) who generally enter the PM data in their progress notes believe they must also re-enter the data in the CCR system.

You have to go back into the reminder and click on these three boxes and then document that you talked with him about alcohol when I’ve already talked with him and I’ve documented it in my note . . . which is a bit frustrating. (Physician – Site 2)

A few providers (3) at two sites (Sites 2 and 3) indicated that even when they are not personally bogged down with PM-related CCRs, the time intake nurses spend with the patient on PM CCRs reduces the time available for provider interaction with the patient.

Sometimes the nurses will be spending up to 20–25 minutes with the patient to try to address all of the performance measures that she feels her job depends on. And then you get the patient 25 minutes late. (Physician – Site 2)

When time is limited, patient health concerns may be given lower priority than PM-related areas of care (23 mentioned).

Occasionally a patient will come in with a complaint and the providers will make sure all the alerts are answered rather than addressing the complaint per se. Because you have this and this to do and you don’t address the fact that they have low back pain because that’s not a performance measure or their ankle hurts or something. (Facility Leader – Site 4)

Our agenda is often guided in some way shape or form by these performance measures, which I think largely are pretty reasonable and pretty evidence-based practices but aren’t always recognized by patients as being the most important thing, and I think can potentially get us down this path of ‘OK, this is the work I need to get done now and I’ve got a lot of it and you need to sit and listen to me so I can turn off my reminders.’ (Physician – Site 2)

Compromises to Patient Education and Autonomy

Quality of Patient Education may Suffer (Figure Pathway D2, D3, D5→C2→B5→A3). There were mixed opinions on how PMs affect patient education. While some felt that PMs are beneficial because more topics are covered, several nurses (3) from one participating site (Site 3) indicated that the emphasis placed on completing all PM-related CCRs during each visit left little time for them to provide the quality of education necessary for patients to make informed decisions.

Every nurse in primary care complains that they don’t have enough time to educate the patient on anything…I think most people work in an assembly line; they just click the buttons to meet the criteria but really not doing justice to the patient….They go through the list of reminders and click, click, click and patient moves on to see the provider. Next patient, get the vitals, click, click, click, move on. (Intake Nurse – Site 3)

Patient’s Decision to Decline Care not Accepted (Figure Pathway D1, D3, D5→C1→B6→A3). Although both nurses and physicians felt patients have a right to refuse PM-related care, several previously described PM implementation strategies make it difficult for them to accept patient refusals, leading to pressure on patients to comply (33 mentioned).

The number one change I would make is when I‘m discussing something with a patient or I’m saying their A1c is too high or their cholesterol needs changing and they absolutely refuse to have any intervention - when I document that I have (had) this lengthy talk with them and told them what the possible consequences of not doing that would be - it should be reflected …as a positive. (Nurse Practitioner – Site 1)

The system does not allow you to refuse . . . It’s like you’re trying to break them down and eventually make them give in and say ‘oh okay, I’ll take the flu shot.’ (Intake Nurse – Site 3)

Adverse Consequences on Primary Care Team Dynamics

Participants described several additional unintended consequences of PMs on primary care team dynamics. Although participants did not directly link these to negative consequences for patients (and are therefore not included in Fig. 1), they may have important implications for the quality of care provided. At two of the four facilities (Sites 2 and 3), nurses were asked to check on the providers to be sure that they completed and documented PM-related interventions. Several providers (4) and nurses (6) expressed concerns with this arrangement.

At the end of the day I have to go through and police all their charts to see if they’ve done (their PM documentation), but if they haven’t finished the chart or dictated on it until the next day, then I’m expected to … continue to nag the physician and that really irritates them. (Intake Nurse – Site 2)

Additionally, several nurses (3) at two sites (Sites 1 and 3) resented the fact that physicians receive bonuses based in part on PM scores but that nurses don’t.

I don’t get any monetary returns from my doing my part, and I’ve heard . . . that providers do. To me it’s like I’m helping the provider get a bonus – but it’s part of their job anyways, that’s not fair to me. (Registered Nurse – Site 3)

COMMENTS

Primary care staff in our study described several ways in which PMs may lead to inappropriate care, may take the focus off of patient concerns and patient service, and may make it more difficult for patients to make informed value-consistent decisions (especially when the patient’s values conflict with interventions deemed appropriate by PMs). These problems have undoubtedly existed to some extent in all healthcare systems even before the advent of modern performance measurement systems. Indeed, PM systems have likely been responsible for improvements in some areas within these quality domains while simultaneously exacerbating problems in other areas (e.g. reducing inappropriate underuse of interventions while increasing inappropriate overuse). Although our research methodology does not allow us to identify which, if any, unintended consequences are inherent in all PM systems, it is noteworthy that few of the unintended negative consequences identified in our research appeared to be directly related to national PM policies. Instead, the influence of PM policy on patient care appeared to be mediated through facility-level efforts to implement these national policies. This finding suggests that greater collaboration may be needed between developers of PMs and those responsible for facility implementation so that PM developers understand the variation in how measures are adapted into practice and local implementers have a thorough understanding of the rationale and evidence underlying the measure.

As hypothesized elsewhere,^9–11,22 we found some evidence that PMs can drive inappropriate care, such as polypharmacy. Prescribing drugs involves greater risk but is a more efficient way to meet the PMs than behavioral interventions and providers may feel incentivized by the PMs to provide medications rather than behavioral health counseling. The administration of more drugs than is medically indicated may occur in over half of elderly patients²⁵ and is a risk factor for morbidity and mortality.^26–29 The examples of overtreatment provided by participants were sometimes reported second hand. It is unknown whether they represent perception or reality. Even if more perceived than real, perceptions may shape norms and have unintended effects on provider behavior and satisfaction; or to quote the sociologist W.I. Thomas: “If men define situations as real, they are real in their consequences.”³⁰

This work contributes to a growing body of evidence suggesting that PM systems can have negative effects on provider-patient communication. As others have observed,^14,31 we found that PMs increase providers’ workload during the clinical encounter which can crowd out education and the discussion of issues that are of higher priority to patients (including, on occasion, the chief complaint). It is therefore perhaps not surprising that the establishment of a national pay for performance system in the UK failed to improve patient ratings of patient-provider communication even though this measure was incentivized.² PMs that count patient refusals against PM scores may add to the communication problem by making it more difficult for providers to accept a patient’s choice to decline care.

Avenues for Improvement

This work does not provide sufficient evidence to conclude that the frequency and seriousness of unintended negative consequences of PM are sufficient to warrant system-wide change. Our data does indicate that PM systems do carry some risk of negatively impacting the quality of patient care. Organizations wishing to take the lead in improving their PM systems may therefore be interested in exploring ways to minimize the risk of these problems. The following approaches may address several of the negative unintended consequences identified in this research:

1. Integrate the development of PMs and local implementation strategies through collaboration between PM system administrators and facility staff.

2. Develop PMs monitoring overtreatment to balance the current focus on undertreatment.

3. Modify PMs to credit appropriate provider treatment behavior as well as achievement of patient health goals. (See Kerr and colleagues work on “tightly linked” PMs.)^32,33

4. Track the number of patients who refuse care and exclude them from measure denominators. Although some gaming may occur^14,34,35 research suggests that widespread abuse is unlikely.³⁵

VHA is currently making several changes to the national PM system to incorporate insights from this and other research. A newly convened national clinical reminder standardization workgroup will develop reminders that facilitate patient-centered care decisions. New PMs are being considered that will reward clinically appropriate action, even if the patient has not achieved specific targets. Finally, more flexible incentive plans were recently introduced that hold facilities accountable for fewer measures, chosen to address areas with the greatest opportunity for local improvement.

Limitations

Our methodology allowed us to explore a range of unintended consequences of PM and identify clinic processes and provider actions that appear to lead to their occurrence. However, this methodology is not well suited for assessing the frequency with which the identified effects occur or evaluating the balance of positive and negative effects of PM. Additional quantitative studies are needed to address these issues. Additionally, our findings may not generalize to other healthcare systems. However, most of the VHA’s primary care PMs are similar or identical to HEDIS (Healthcare Effectiveness Data and Information Set) measures which are used by over 90% of U.S. health plans.³⁶ Our data consisted of facility staff self-reports (and in some cases second-hand reports on the behavior of others). Participants may have incorrectly inferred negative effects on patients, may have been reluctant to share perceived deficiencies in personal or facility practices, or may have been more likely than non-participants to express the views summarized above.

CONCLUSIONS

Although quantitative performance measurement systems have important benefits for patients, they may also: (1) lead to inappropriate clinical care; (2) decrease providers’ focus on patient concerns and patient service, and (3) compromise patient education and autonomy. These negative unintended consequences are largely determined by facility-level strategies undertaken to improve PM scores. Additional study is needed to determine the prevalence of unintended effects and to better understand how to implement systems in ways that minimize them.