Exempting dissenting patients from pay for performance schemes: retrospective analysis of exception reporting in the UK Quality and Outcomes Framework

Abstract

Objective To examine the reasons why practices exempt patients from the UK Quality and Outcomes Framework pay for performance scheme (exception reporting) and to identify the characteristics of general practices associated with informed dissent.

Design Retrospective analysis.

Setting Data for 2008-9 extracted from the clinical computing systems of general practices in England.

Participants 8229 English family practices.

Main outcome measures Rates of exception reporting for 37 clinical quality indicators, associations of patient and general practice factors with exception rates, and financial gain for practices relating to their use of exception reporting.

Results The median rate of exception reporting was 2.7% (interquartile range 1.9-3.9%) overall and 0.44% (0.14-1.1%) for informed dissent, but variation in rates was wide between practices and across indicators. Common reasons for exception reporting were logistical (40.6% of exceptions), clinical contraindication (18.7%), and patient informed dissent (30.1%). Higher rates of informed dissent were associated with: higher numbers of registered patients, higher levels of local area deprivation, and failure of the practice to secure maximum remuneration in the previous year. Exception reporting increased the cost of the scheme by £30 844 500 (€36 877 700; $49 053 200) (£0.58 per patient), with two indicators accounting for a quarter of this additional cost.

Conclusions The provision to exception report enables practices to exempt dissenting patients without being financially penalised. Relatively few patients were excluded for informed dissent, however, suggesting that the incentivised activities were broadly acceptable to patients.

Introduction

Since early 2000 payers across healthcare systems worldwide have experimented with pay for performance schemes that explicitly link doctors’ remuneration to quality of care, with mixed success.^{1 2 3} In the United Kingdom, a national scheme for primary care—the Quality and Outcomes Framework—was introduced in 2004, providing financial incentives to family practices for meeting targets on a range of clinical, organisational, and patient experience indicators.⁴ Most practices have performed well under the scheme,^{5 6 7 8} but improved patient outcomes have not been consistently evident.^{9 10 11} Even if they successfully stimulate improved performance, pay for performance schemes have several potential unintended consequences. In particular, given a method of remuneration that financially rewards doctors for performing procedures, prescribing drugs, and controlling biological variables, patients may be coerced or refused care if they are non-compliant.^{12 13} Understanding these potential risks, the designers of the Quality Outcomes Framework included two mechanisms intended to protect patients from coercive care. Firstly, upper payment thresholds are set below 100%, so practices do not have to achieve the targets for all patients to receive the maximum payment. Secondly, doctors are permitted to use their clinical judgment to remove inappropriate patients from achievement calculations for clinical indicators, a process known as exception reporting. The box gives permitted reasons for exception reporting, including logistical considerations (for example, recent registration of the patient with the practice), clinical reasons (for example, a contraindication to treatment), and patient informed dissent (that is, not agreeing to the investigation or treatment). Patients recently registered with the practice or with a recent diagnosis are automatically excepted by clinical computing systems, whereas practices must actively identify patients who meet other exception reporting criteria. The provision to except dissenting patients is intended to counter any financial conflict of interest for doctors in respecting a patient’s choice to refuse an intervention incentivised under the scheme.

Reasons practice staff may use to exempt patients from quality assessment*

Logistical

• The patient has recently received a diagnosis or recently registered with the practice†

• A specified investigative service is unavailable to the practice

Clinical—contraindication or intolerance

• The patient has had an allergic or other adverse reaction to a specified drug or has another contraindication to the drug

• The patient has not tolerated the drug

• The patient is taking the maximal tolerated dose of a drug, but the levels remain suboptimal

Clinical—patient unsuitable

• The indicator is judged inappropriate for the patient because of particular circumstances, such as terminal illness or extreme frailty‡

• The patient has a supervening condition that makes the specified treatment clinically inappropriate

• The patient has received at least three invitations for a review during the preceding 12 months but has not attended

Informed dissent

• The patient refuses to be reviewed‡

• The patient does not agree to a specific investigation or treatment§

The principal drawback of exception reporting is that it allows practices to receive the maximum remuneration without necessarily providing the required care for all eligible patients.¹⁴ If exception rules are applied too readily or inappropriately, high achievement scores will mask suboptimal care.^{4 15} To date, overall exception reporting rates have generally been low (less than 6%),¹⁶ with little evidence of widespread fraud.¹⁷ However, a few practices have achieved high scores by excluding unusually large numbers of patients.^{18 19} Concerns over inappropriate use of exception reporting have led to calls for the provision to be amended.²⁰ Without the ability to exception report, practices could be penalised for respecting the wishes of dissenting patients, particularly in localities where rates of dissent are high. Recent qualitative work suggests that doctors perceive exception reporting to be an important, defensible safeguard against inappropriate or over treatment of patients.²¹ However, to date no research has been carried out into the reasons why practices exception report patients and in particular how high rates of patient dissent are for different population groups. We examined the reasons given by practices for exception reporting patients and identified patient and practice characteristics associated with informed dissent.

In 2008-9 the Quality and Outcomes Framework consisted of 127 quality indicators, of which 65 covered the maintenance of disease registers, organisational activities, and patient experience of care. The remaining 62 indicators covered clinical activities across 15 clinical areas, for which practices are permitted to exception report patients for specified reasons (box). Practices are expected to satisfy certain criteria before exception reporting patients. For exceptions due to informed dissent, practices are required to make personal contact with the patient and to record the patient’s reasons for rejecting the intervention in the notes. Practices are awarded points based on the proportion of appropriate patients (that is, those not exception reported) for whom targets are achieved, between a lower achievement threshold of 40% and an upper threshold that varies by indicator (table 1). The maximum number of points awarded also varies by indicator. In 2008-9 each point earned practices £126, adjusted for the relative prevalence of the disease and the size of the practice population. Practice performance is monitored by primary care trusts, which have the power to audit patient records and to investigate the use and misuse of exception reporting.

Table 1.

 Quality and Outcomes Framework clinical indicators included in main analysis (n=37)

Indicator (No of eligible patients)	Description	Indicator type	Upper payment threshold	Maximum points available	No of exceptions		% excepted		Median exception rate (interquartile range) (%)
					Informed dissent	Total	Informed dissent	Total	Informed dissent	Total
Atrial fibrillation:
AF 4 (89 230)	Newly diagnosed with record of ECG or specialist confirmed diagnosis	Measure	90	10	103	12 200	0.1	13.7	0.0 (0.0-0.0)	10.0 (0.0-20.0)
Asthma:
AST 3 (266 871)	Record of smoking status (ages 14-19)	Measure	80	6	5489	6731	2.1	2.5	0.0 (0.0-0.0)	0.0 (0.0-1.6)
AST 6 (3 191 842)	Asthma review	Measure	70	20	106 734	161 486	3.3	5.1	0.2 (0.0-2.1)	2.0 (0.9-5.0)
AST 8 (292 139)	Measures of variability or reversibility (new diagnoses, age ≥8)	Measure	80	15	3308	24 299	1.1	8.3	0.0 (0.0-0.0)	6.7 (2.3-12.3)
Hypertension:
BP 4 (7 120 728)	Record of blood pressure	Measure	90	20	44 326	72 803	0.6	1.0	0.1 (0.0-0.6)	0.5 (0.2-1.2)
BP 5 (7 119 579)	Blood pressure ≤150/90 mm Hg	Outcome	70	57	62 427	288 092	0.9	4.0	0.3 (0.0-0.9)	3.4 (2.3-5.1)
Cancer:
CAN 3 (137 921)	Patient review within six months of diagnosis	Measure	90	6	302	1963	0.2	1.4	0.0 (0.0-0.0)	0.0 (0.0-0.0)
Coronary heart disease:
CHD 5 (1 883 107)	Record of blood pressure	Measure	90	7	9433	19 546	0.5	1.0	0.0 (0.0-0.6)	0.6 (0.0-1.5)
CHD 6 (1 882 959)	Blood pressure ≤150/90 mm Hg	Outcome	70	19	15 056	55 394	0.8	2.9	0.0 (0.0-1.0)	2.4 (1.2-4.0)
CHD 7 (1 882 998)	Record of total cholesterol	Measure	90	7	16 746	49 734	0.9	2.6	0.0 (0.0-1.0)	1.9 (0.8-3.5)
Heart failure:
HF 2 (117 872)	Diagnosis confirmed by electrocardiogram or by specialist	Measure	90	6	1164	9678	1.0	8.2	0.0 (0.0-0.0)	4.5 (0.0-12.5)
Chronic kidney disease:
CKD 2 (1 736 280)	Record of blood pressure	Measure	90	6	1720	7855	0.1	0.5	0.0 (0.0-0.0)	0.0 (0.0-0.5)
CKD 3 (1 736 709)	Blood pressure ≤140/85 mm Hg	Outcome	70	11	6137	140 254	0.4	8.1	0.0 (0.0-0.3)	6.2 (2.9-11.3)
Dementia:
DEM 2 (232 047)	Care has been reviewed	Measure	60	15	750	17 717	0.3	7.6	0.0 (0.0-0.0)	5.3 (0.0-11.1)
Depression:
DEP 1 (3 656 353)	Case finding for patients with diabetes or coronary heart disease	Measure	90	8	22 044	97 001	0.6	2.7	0.0 (0.0-0.1)	1.8 (1.1-3.0)
DEP 2 (452 152)	Assessment of severity at outset of treatment	Measure	90	25	717	57 789	0.2	12.8	0.0 (0.0-0.0)	10.5 (3.4-20.0)
Diabetes:
DM 2 (2 208 812)	Record of body mass index	Measure	90	3	32 539	68 271	1.5	3.1	0.6 (0.0-1.9)	2.2 (1.0-4.2)
DM 5 (2 208 753)	Record of HbA1c or equivalent	Measure	90	3	24 405	60 682	1.1	2.7	0.5 (0.0-1.5)	2.2 (1.1-3.7)
DM 7 (2 208 395)	HbA1C ≤10%	Outcome	90	11	34 506	112 299	1.6	5.1	0.8 (0.0-2.1)	4.4 (2.8-6.8)
DM 11 (2 208 729)	Record of blood pressure	Measure	90	3	19 129	32 422	0.9	1.5	0.3 (0.0-1.2)	1.0 (0.3-2.0)
DM 12 (2 208 341)	Blood pressure ≤145/85 mm Hg	Outcome	60	18	38 944	132 815	1.8	6.0	0.9 (0.0-2.3)	5.3 (3.3-7.8)
DM 13 (2 071 049)	Record of testing for microalbuminuria	Measure	90	3	46 817	116 231	2.3	5.6	1.0 (0.0-2.9)	4.5 (2.5-7.5)
DM 16 (2 208 701)	Record of total cholesterol	Measure	90	3	26 889	54 211	1.2	2.5	0.5 (0.0-1.6)	1.8 (0.8-3.4)
DM 20 (2 208 047)	HbA1C ≤7.5%	Outcome	50	17	54 237	203 978	2.5	9.2	1.2 (0.0-3.2)	8.0 (5.3-12.1)
DM 22 (2 208 667)	Record of estimated glomerular filtration rate or serum creatinine testing	Measure	90	3	23 976	44 472	1.1	2.0	0.5 (0.0-1.5)	1.5 (0.6-2.8)
Epilepsy:
EPI 6 (326 216)	Record of seizure frequency	Measure	90	4	4400	12 461	1.3	3.8	0.0 (0.0-0.0)	1.3 (0.0-5.0)
EPI 7 (326 205)	Drug review involving patient or carer, or both	Measure	90	4	3984	11 728	1.2	3.6	0.0 (0.0-0.0)	1.2 (0.0-4.8)
EPI 8 (326 149)	Seizure free for past 12 months	Outcome	70	6	6531	52 604	2.0	16.1	0.0 (0.0-0.0)	12.5 (4.4-23.9)
Psychotic illness:
MH 4 (55 039)	Record of serum creatinine and thyroid stimulating hormone	Measure	90	1	447	1646	0.8	3.0	0.0 (0.0-0.0)	0.0 (0.0-0.0)
MH 7 (23 198)	Patients not attending for annual review followed-up within 14 days	Measure	90	3	1196	2501	5.2	10.8	0.0 (0.0-0.0)	0.0 (0.0-0.0)
MH 9 (386 713)	Patient review and health promotion advice	Measure	90	23	13 668	54 356	3.5	14.1	0.0 (0.0-3.6)	10.0 (4.2-18.8)
Smoking:
SMO 3 (11 474 188)	Record of smoking status (patients with coronary heart disease, stroke, hypertension, diabetes, chronic obstructive pulmonary disease, chronic kidney disease, asthma, schizophrenia, psychosis)	Measure	90	33	6699	77 197	0.1	0.7	0.0 (0.0-0.0)	0.6 (0.4-0.9)
Stroke:
STR 5 (899 490)	Record of blood pressure	Measure	90	2	4493	15 440	0.5	1.7	0.0 (0.0-0.4)	1.0 (0.0-2.5)
STR 6 (899 399)	Blood pressure ≤150/90 mm Hg	Outcome	70	5	7021	40 640	0.8	4.5	0.0 (0.0-0.9)	3.7 (1.8-6.1)
STR 7 (899 422)	Record of total cholesterol	Measure	90	2	8844	43 458	1.0	4.8	0.0 (0.0-1.1)	3.3 (1.2-6.3)
STR 13 (90 057)	Referred for further investigation	Measure	80	2	926	17 809	1.0	19.8	0.0 (0.0-0.0)	16.0 (0.0-28.6)
Hypothyroidism:
THY 2 (1 536 123)	Record of thyroid function tests	Measure	90	6	1927	7038	0.1	0.5	0.0 (0.0-0.0)	0.0 (0.0-0.7)

Methods

We carried out a retrospective study of exception reporting by English general practices in 2008-9, identifying practice and population predictors of exception reporting rates through multilevel multivariate linear regression.

Data sources

We used data from the Quality Management and Analysis System (QMAS), the national information system supporting the Quality and Outcomes Framework, published by the National Health Service Information Centre. The Quality Management and Analysis System holds publicly available data on the achievement of indicators for 8229 English practices (over 99.9% of all practices). Detailed information on exception reporting (including reason for exception) is not publicly reported and we specifically acquired the relevant dataset for this study. Data on patient and practice characteristics were obtained from the Office for National Statistics website²² and the General Medical Services database. Practices were grouped into fourths on the basis of the local level of area deprivation, using boundary datasets from the UKBORDERS website²³ to assign index of deprivation scores to practices at the postcode level.

Data analyses

For each practice and each clinical indicator we calculated the rate of exception reporting as the number of excepted patients divided by the number of eligible patients:

ER_i=E_i/(E_i+D_i)

Where E_i is the number of patients exception reported for that indicator and D_i is the number of patients meeting the criteria for the indicator and not excepted by the practice. As the distributions of exception reporting rates are skewed, we report medians. We calculated the overall rates for a practice (that is, exception rates across multiple indicators) by summing exceptions for all indicators and dividing by the sum of eligible patients. Patients eligible for multiple indicators are double counted in the overall rates, and these rates therefore represent the proportion of “opportunities” to perform the incentivised activity that resulted in an exception report, rather than the proportion of patients excepted.

We calculated exception reporting rates separately for each of the main reasons: logistical, clinical (contraindication or intolerance), clinical (patient unsuitable), and informed dissent. Under the Quality Management and Analysis System, if the indicator is met the patient is counted in D_i and not in E_i, even if the patient was previously exception reported—that is, the patient is not counted as having been exception reported. In addition, the Quality Management and Analysis System only records one exception reporting reason for each patient. For patients who satisfy two or more criteria, the first criterion encountered in the business rules is recorded (see appendix on bmj.com).

We also estimated the average financial gain (G_i) from exception reporting, following a previously published method.¹⁶ For each indicator in each practice this is calculated as the difference between the estimated remuneration received and the remuneration that would have been received had the practice not excepted any patients (see appendix on bmj.com).

Because patterns of exception reporting differ according to the type of activity,¹⁶ we classified indicators into three categories: measurement (for example, monitoring blood pressure levels), treatment (for example, prescribing β blockers), and outcomes (for example, controlling blood pressure levels). For 25 indicators (see supplementary table on bmj.com), including all treatment indicators, the Quality Management and Analysis System does not differentiate between patient refusal of treatment and clinical contraindications, rendering some exceptions non-ascribable. These indicators were excluded from all analyses relating to reasons for exception reporting. Exception reporting rates were analysed for the remaining 37 indicators (table 1), by category and by reason for exclusion. Sensitivity analyses based on the excluded indicators are given in the appendix on bmj.com.

We used two multilevel mixed effects multiple linear regression models to identify indicator, practice, and population characteristics as predictors of overall and of patient informed dissent exception reporting rates. The structure of the data was three level, with indicators crossed with practices and nested within primary care trusts. To account for this structure and to model variability at each level, we used mixed effects models with the xtmixed command in Stata. Owing to computational limitations, models of indicator variability (both fixed and random effects) failed to converge, hence we used a three level model with indicator observations nested within practices and practices nested within primary care trusts. We assessed collinearity by estimating variance-inflation factors for the independent variables. All factors were in the range 1.0 to 2.1, below the conservative threshold for collinearity. All statistical comparisons were made at an α level of 5%. Stata v11.1 software was used for all analyses.

Results

Rates of exception reporting

In 2008-9 the median exception reporting rate across all 62 clinical indicators was 4.5% (interquartile range 3.4-5.8%). Median rates for individual indicators ranged from 0.0% (for seven indicators) to 24.4% (CHD 10: β blocker therapy for patients with coronary heart disease) (table 1). Rates were generally lower for measurement indicators (median 2.4%) than for treatment and intermediate outcomes indicators (median 10.0% and 5.7%, respectively).

The median exception rate across the 37 indicators for which reasons for exception reporting were ascribable was 2.7% (interquartile range 1.9-3.9%). Figure 1 gives the proportions of patients excepted for each indicator. Logistical exceptions were used most frequently (40.6% of exceptions) and clinical contraindications least frequently (7.6% of exceptions, table 2). Logistical exceptions were particularly common for indicators involving referrals to other agencies or complex assessments for patients with a new diagnosis—for example, diagnosis of atrial fibrillation confirmed by specialist assessment (AF 4), assessment of the severity of depression (DEP 2), and confirmation of stroke by further investigation (STR 13).

Fig 1 Proportion of patients exception reported by indicator and reason, 2008-9. For 37 indicators for which reasons for exception reporting were ascribable (see table 1). Indicators ordered by type of activity (measurement or outcome) and by rate of exception reporting attributable to informed dissent

Table 2.

 Proportion of exception reports attributable to each exception reporting category, by type of indicator

Reason for exception report	Type of indicator (%)
	Measurement	Intermediate outcome	All
Unknown*	3.5	2.2	2.9
Logistical	35.9	45.9	40.6
Clinical—contraindication	0.0	16.2	7.6
Clinical—patient unsuitable	23.1	13.8	18.7
Informed dissent	37.4	21.9	30.1
Total No of exceptions	1 158 735	1 026 076	2 184 811

Based on 37 indicators for which reasons for exception reporting were ascribable (see table 1).

*In these cases a “general” exception was applied to the patient and the exact reason for the exception report is not recorded.

Informed dissent accounted for 37.4% of measurement exceptions, 21.9% of intermediate outcomes exceptions, and 30.1% of exceptions overall. The median exception rate for informed dissent across all 37 indicators was 0.44% (interquartile range 0.14-1.1%), but a minority of practices had considerably higher rates: 10% of practices excepted over 2.2% of patients for informed dissent and 1% of practices excepted over 5.7%. Median rates for individual indicators ranged from 0.0% (25 of the 37 indicators) to 1.2% (DM 20: patients with diabetes with HbA_1C levels ≤7.5%, table 1).

The estimated median rate of informed dissent across the 25 indicators excluded from the main analysis was between 0.73% (assuming no non-ascribable exceptions were due to informed dissent) and 4.8% (assuming all non-ascribable exceptions were due to informed dissent, see appendix on bmj.com). Based on these assumptions, the overall median rate of informed dissent for all 62 clinical indicators was between 0.53% (interquartile range 0.18-1.3%) and 1.7% (1.2-2.5%).

Factors associated with exception reporting

Table 3 gives the results of the regression analyses for the 37 indicators for which reasons for exception reporting were ascribable. Higher overall exception reporting rates were associated with lower payment thresholds, higher points values (maximum remuneration), and lower numbers of eligible patients. After controlling for other factors, exception reporting rates did not differ significantly between intermediate outcomes and measurement indicators. The generally low rates of exception reporting previously noted for measurement indicators may therefore be attributable to characteristics such as their generally high upper payment thresholds and low points values.

Table 3.

 Results of regression analysis—factors associated with exception reporting rates

Variable	All exceptions			Informed dissent
	Coefficient	P value	95% CI	Coefficient	P value	95% CI
Indicator characteristics
Upper payment threshold (per 1% increase)	−0.09	<0.001	−0.10 to −0.09	−0.02	<0.001	−0.02 to −0.02
Indicator type (intermediate outcome)	−0.05	0.321	−0.15 to 0.05	−0.10	<0.001	−0.14 to −0.07
Maximum points/remuneration available	0.11	<0.001	0.11 to 0.11	0.00	0.045	0.00 to 0.00
No of eligible patients (per 100 increase in disease register size)	−0.63	<0.001	−0.64 to −0.62	−0.04	<0.001	−0.04 to −0.04
Practice characteristics
Maximum points scored in previous year (2007/8)*	−2.33	<0.001	−2.44 to −2.22	−0.33	<0.001	-0.37 to −0.29
% of doctors aged ≥55†	−0.00	<0.001	−0.01 to −0.00	−0.00	0.449	−0.00 to 0.00
% of women doctors†	0.00	0.244	−0.00 to 0.00	−0.00	0.054	−0.00 to 0.00
Personal Medical Services contract	0.17	0.004	0.06 to 0.29	0.01	0.785	−0.05 to 0.07
No of patients (per 1000 increase in list size)†	0.28	<0.001	0.26 to 0.29	0.07	<0.001	0.06 to 0.08
Patient and area characteristics
% of patients aged ≥65†	0.03	<0.001	0.02 to 0.04	0.00	0.327	−0.00 to 0.01
% of female patients†	−0.00	0.987	−0.02 to 0.02	−0.00	0.685	−0.02 to 0.01
% of patients from ethnic minority groups†	−0.00	0.197	−0.01 to 0.00	−0.01	<0.001	−0.01 to −0.00
Population density in locality†	0.00	0.178	−0.00 to 0.00	0.00	<0.001	0.00 to 0.00
Material deprivation in locality‡:
1st fourth (most affluent)	—	—	—	—	—	—
2nd fourth	0.13	0.082	−0.02 to 0.28	0.10	0.021	0.02 to 0.18
3rd fourth	0.44	<0.001	0.27 to 0.60	0.18	<0.001	0.09 to 0.27
4th fourth (most deprived)	0.69	<0.001	0.51 to 0.87	0.33	<0.001	0.23 to 0.42

Based on 37 indicators for which reasons for exception reporting were ascribable (table 1).

*For each specific indicator.

†Data for 2006/7.

‡Measured by index of deprivation 2007.

Rates of exception reporting varied with practice characteristics: higher rates were associated with younger age profiles of doctors, failure to secure maximum remuneration on that indicator in the previous year, and a higher number of registered patients. Higher rates were also associated with patient and local area characteristics—for example, higher levels of local area deprivation. Most of these effects were small. The most influential factor was previous performance on the pay for performance scheme: practices that failed to acquire all available points for a specific indicator in the previous year (2007-8) had, on average, exception reporting rates 2.3% higher than those that succeeded. With respect to area deprivation, median exception reporting rates across the 37 indicators varied from 2.6% (interquartile range 1.6-3.7%) for practices in the most affluent fourth to 2.9% (2.0-4.2%) for practices in the most deprived fourth.

Factors associated with higher levels of informed dissent exceptions were broadly comparable with those for overall exceptions, but effect sizes tended to be smaller and were non-significant in some cases. With respect to area deprivation, median exception reporting rates for informed dissent varied from 0.42% (0.15-1.0%) for practices in the most affluent fourth to 0.45% (0.14-1.3%) for practices in the most deprived fourth.

Financial gain associated with exception reporting

By removing patients from the denominator for achievement calculations, exception reporting increased a practice’s reported achievement rate. If a practice’s achievement rate before exception reporting was below the upper payment threshold, then exception reporting would also increase remuneration. Figure 2 shows the remuneration practices received for each of the 62 clinical indicators attributable to achieving the clinical targets and to exception reporting. For most indicators the cost of exception reporting was relatively low, as most practices exceeded the upper payment thresholds even before exception reporting was taken into account.

Fig 2 Total remuneration for all practices attributable to achievement of targets and exception reporting, by clinical indicator (all 62). Total remuneration is based on population achievement rates. Remuneration attributable to achieving targets is based on reported achievement rates. Remuneration attributable to exception reporting is the difference between total remuneration and remuneration attributable to achieving targets

Indicators ordered by remuneration attributable to exception reporting

Overall, 5.4% of clinical points scored by practices—and therefore remuneration received—was attributable to exception reporting. This equates to about £30 844 500 (€36 877 700; $49 053 200) for all English practices, £3834 for the average practice, and £0.58 per patient. Mean gain ranged from £3586 per practice in the most affluent fourth to £4093 in the most deprived fourth. The cost of exception reporting varied widely by indicator, from £1630 for DM 11 (recording the blood pressure of patients with diabetes) to £4.5m for DEP 2 (assessing the severity of depression). Just two of the 62 indicators, DEP 2 and MH 9 (reviewing physical and social care for people with psychotic illness), accounted for £8.4m—over a quarter of the total cost associated with exception reporting.

For the 37 indicators for which reasons for exception reporting were ascribable, 4.9% of remuneration received was attributable to exception reporting (£19 188 917 for all English practices, £2386 for the average practice, and £0.36 per patient). Out of this total, the gain attributable to informed dissent exceptions was £2 406 500 nationally, £300 per practice and £0.05 per patient. Mean gain ranged from £244 per practice in the most affluent fourth to £351 in the most deprived fourth. The cost of informed dissent exceptions was relatively low because most applied to measurement indicators, which attract less remuneration.

Discussion

Respecting a patient’s decision to refuse an investigation or treatment, even if the decision is considered wrong or irrational by the attending clinician, is central to medical professionalism²⁴ and a legal requirement in most circumstances. By making incomes partly dependent on patient compliance, pay for performance schemes bring a clinician’s financial self interest into conflict with their duty not to pressure patients to accept medical advice. This can have unintended consequences—for example, under California’s primary care pay for performance scheme, some practices forced the disenrollment of non-compliant patients,¹³ and in a national survey, US primary care physicians identified performance measures as a driver of more aggressive practice.²⁵ Under the UK Quality and Outcomes Framework, dissenting patients are removed from payment calculations, making the quality targets fairer for practitioners and providing some protection from coercion for patients.²¹ Our paper suggests that rates of informed dissent in the UK scheme are low, with little variation in rates across the spectrum of deprivation.

Strengths and limitations of the study

This is the first study to examine the reasons used by practices for exception reporting patients under the Quality and Outcomes Framework, and in particular the use of the informed dissent criterion. The study is subject to several limitations. Firstly, practices might not record all expressions of informed dissent for several reasons—for example, if the practice has exceeded the upper payment threshold for the year or if another exception reporting criterion applies for a given patient. In the absence of other permissible reasons for excluding patients, rates of informed dissent would possibly be higher than we report. Secondly, the central Quality Management and Analysis System database only records the first exception reporting reason encountered in the business rule algorithm for each indicator (see appendix on bmj.com). Our figures will therefore underestimate the true extent of informed dissent for indicators where dissent codes appear later in the algorithm. Finally, we were unable to analyse in detail the reasons for exclusion for 25 indicators. Rates of informed dissent may be higher for these indicators (see appendix on bmj.com), but precise figures can only be determined by auditing individual practices.

Patterns of exception reporting

As with previous studies,^{16 26} we found that overall rates of exception reporting were generally low but varied widely by indicator. Examining the reasons used to exception report patients provides additional insights. For example, the unusually high number of exceptions for certain measurement indicators (such as confirmation of stroke diagnosis) were mainly logistical, pointing to potential problems with access to specialist services and the timescales allocated to these quality indicators. Even greater variation existed between indicators in the costs associated with exception reporting: over £8m (27% of the total cost) was attached to just two indicators (reviewing patients with psychotic illness and assessing the severity of depression). It is notable that the latter indicator proved unworkable for many doctors and was recommended for removal from the scheme by the Quality and Outcomes Framework advisory committee of the National Institute for Health and Clinical Excellence in 2011.²⁷

Lower exception reporting rates were associated with indicators for which upper payment thresholds were high. These thresholds were intended to represent what is maximally attainable by the average practice, and they therefore reflect the anticipated level of difficulty of indicators. Fewer patients were therefore excepted for indicators perceived to be less challenging to achieve—for example, most measurement indicators. Some practice characteristics, such as list size, were also associated with exception reporting rates, despite having no direct bearing on patient eligibility for exception reporting. Fraudulent or inappropriate activity is one possible explanation for these relations, particularly given that practices that fail to achieve maximum remuneration in one year except more patients the following year, a finding noted in previous years of the scheme.¹⁹ Although monitoring of exception reporting by primary care trusts may have discouraged gross misuse and driven exception reporting rates down, the frequency and thoroughness of monitoring varies across trusts,²⁸ and low level misuse may have gone undetected. However, it is also possible that propensity to legitimately except patients or thoroughness in documenting exceptions is related to certain practice characteristics. For example, larger practices tend to be better organised²⁹ and may therefore be better at identifying patients who should be excepted. Conversely, practices with larger disease registers for a given list size may have detected more patients with less severe disease, who may be less likely to meet exception reporting criteria. Nevertheless, our findings raise the possibility of gaming, and as financial incentive schemes rely on accurate and honest reporting of performance this issue warrants further and more direct investigation than we have provided in this paper.

Characteristics of patients attending practices were also associated with exception reporting rates. Practices located in more deprived areas tended to except more patients, both overall and for reasons of informed dissent. Given that patients excepted for informed dissent are effectively drawn from a subset of patients not excepted for other reasons, it is possible that the relation between deprivation and informed dissent would be stronger in the absence of other reasons for exception reporting. There has been concern that high levels of exception reporting in practices serving deprived populations may be disguising unmet need.²⁶ Our findings suggest that the differences in exception reporting rates attributable to deprivation, although significant, are small. This could therefore be seen in a more positive light as doctors in more deprived areas, where health literacy is likely to be more limited, seem to be listening to and responding appropriately to patients’ concerns about incentivised procedures. This potential explanation resonates with the views of family doctors,²¹ but further qualitative work is needed to explore these hypotheses in more detail. In contrast, financial “gains” from exception reporting varied substantially with deprivation. Practices in more deprived areas tended to have lower achievement rates for the clinical indicators and so were more likely to achieve below the upper payment thresholds and therefore to benefit financially from excepting patients.

Conclusions and policy implications

Under the UK Quality and Outcomes Framework variations in exception reporting rates and associated costs are wide, both between practices and across indicators, which require further investigation and careful monitoring. The high exception rates for certain quality indicators also raise questions for policy makers about the appropriateness of these indicators and whether the current exception reporting system is being used appropriately to deal with the potential unintended consequences of financial incentives.

Exception reporting does, however, provide some protection from inappropriate and coercive treatment of patients whose providers are subject to financial incentives. Relatively few patients are excepted for reasons of informed dissent in the United Kingdom, which suggests that activities incentivised in the scheme are broadly acceptable to patients (although this does not imply approval of the incentive scheme itself). It could be argued that dissent is so uncommon that a specific exception reporting provision is not required, as long as upper thresholds are set below 100%. However, removing the provision might subject a substantial minority of patients attending practices performing below the upper payment thresholds to coercion. Thousands of patients expressed their wish not to receive interventions under the framework (for example, over 100 000 patients with asthma actively declined a review). At relatively low cost, the provision to exception report enables patients’ voices to be heard and counters some of the critiques of the scheme as endangering the doctor-patient relationship.³⁰

What is already known on this topic

• The provision to exempt dissenting patients from pay for performance schemes removes the financial conflict of interest for doctors in respecting a patient’s choice to refuse an incentivised intervention

• Exception reporting rates have generally been low in the United Kingdom, with little evidence of widespread, large scale fraud

• Concerns over inappropriate use of exception reporting, however, persist in the absence of data on the exact reasons why practices except patients

What this study adds

• In the UK pay for performance scheme the most common reasons for excepting patients were logistical (40.7% of exceptions), contraindication to treatment (18.7%), and patient informed dissent (30.1%)

• Higher rates of informed dissent within practices were associated with: higher numbers of registered patients, higher levels of local area deprivation, and failure of the practice to secure maximum remuneration in the previous year

• For most of the quality indicators, less than 2% of patients were excepted for reasons of informed dissent, suggesting that the incentivised activities were broadly acceptable to patients

Cite this as: BMJ 2012;344:e2405

Web Extra. Extra material supplied by the author

Business rules for clinical indicators

Example of business rules for clinical indicators (fig A1) and exception reporting rates for indicators excluded from main analysis (fig A2)

Table showing clinical indicators excluded from main analysis