In England primary care groups will have a key role in promoting the health and improving the health care of their local population.1 By April 1999 these groups, involving all primary care professionals, will provide and commission health care for roughly 100 000 people in each locality. Primary care groups will be accountable to health authorities and “will agree targets for improving health, health services and value for money.”1 There will be several primary care groups in each district health authority. This new approach offers primary care the opportunity to further integrate health promotion and health care at the individual and population levels.
The present UK government intends to manage the performance of the “new NHS”; the word performance appeared 88 times in its recent white paper.1 It has published a national framework for assessing performance as a consultation document,2 and primary care groups within health authorities will be judged to have “performed” well on the basis of the indicators listed in table 1. Most are attributable in part to primary care, but only some are linked to interventions that will necessarily lead to improved health outcomes. The government has also proposed four targets for England in its green paper Our Healthier Nation.3 Approaches taken by health authorities, and presumably by primary care groups, will be “fully monitored by the Regional Offices of the NHS Executive.”3 These targets for reduced death rates from heart disease and stroke, cancer, suicide, and accidents are all outcome indicators but, again, are only partly attributable to primary care.
Summary points
The NHS Executive and Department of Health have proposed a wide range of performance indicators many of which are applicable to future primary care groups
Some of these indicators reflect access and efficiency, but few of the effectiveness indicators are based on primary care interventions for which there is evidence that increased uptake results in improved health outcomes
We present a method to identify important primary care interventions of proved efficacy and suggest performance indicators that could monitor their use
Our evidence based approach may be a complementary way of identifying areas for performance indicators to those proposed by the NHS Executive and Department of Health
Our suggested indicators are more likely to help turn evidence into everyday practice and to have an impact on the population’s health
Table 1.
Indicators proposed in national framework for assessing performance2 that are most relevant to primary care
| Performance indicators | Available data source | Attributable to health care | Evidence that improved indicator value leads to improved health outcomes |
|---|---|---|---|
| Fair access | |||
| To elective surgery: rates of CABG and PTCA, of hip and knee replacement, and of cataract replacement | Yes | Yes | Unclear what ideal rates should be. Only effective if performed on appropriate patients |
| To family planning services: conception rates for those aged <16 | Yes | Limited extent | Yes |
| To cancer screening services: % of target population screened for breast and cervical cancer | Yes | Yes, but people can refuse | Yes, though evidence for cervical screening based on observational data |
| To district nurse contacts: district nurse and assisted district nurse contacts for those aged ⩾75, and district nurse contacts lasting >30 minutes for same age group | Yes | Yes | No |
| Effective delivery of appropriate health care | |||
| % of target population vaccinated and % of all orchidopexies for those aged <5 | Yes | Yes, but people can refuse vaccination | Vaccination, yes; orchidopexy, limited extent |
| % of target population screened for breast and cervical cancer (as above) | Yes | Yes, but people can refuse | Yes, though evidence for cervical screening based on observational data |
| Rates of CABG and PTCA, of hip and knee replacement, and of cataract replacement | Yes | Yes | Only effective if performed on appropriate patients |
| Age and sex standardised admission rates for severe ENT infection, kidney or urinary tract infection, heart failure (“avoidable admissions”) | Yes | Yes, but patients can self refer to accident and emergency units | Unclear how |
| Age and sex standardised admission rates for asthma, diabetes, and epilepsy (“largely managed in a primary care setting”) | Yes | Yes, but patients can self refer to accident and emergency units | Unclear how |
| Volume of prescribing of benzodiazepines, and ratio of antidepressants to benzodiazepines | Yes | Yes | To some extent |
| Composite measure of prescribing of combination and modified release products plus “drugs of limited clinical value” and inhaled corticosteroids | Yes | Yes | To some extent |
| Efficiency | |||
| % generic prescribing | Yes | Yes | No, but savings made can be invested in effective interventions |
| Health outcomes of NHS care | |||
| Conception rates for those aged <16 | Yes | Limited extent | Yes as this is a health outcome indicator |
| Notification rates for pertussis and measles | Yes | Yes | Yes as this is a health outcome indicator |
| Emergency hospital admissions for people aged >75 | Yes | To some extent, but also reflects social provision | Possibly, but definition of emergency may vary |
| Rates of emergency psychiatric readmission | Yes | To some extent, but also reflects social provision | Possibly, but definition of emergency may vary |
CABG=coronary artery bypass graft; PTCA=percutaneous transluminal coronary angioplasty; ENT=ear, nose, and throat.
The indicators in the other categories—“Health improvement” (deaths from all causes and cancer registrations) and “Patient/carer experience of the NHS”—are less relevant to primary care.
Performance indicators for practices
—Previous governments have attempted to use performance indicators for group practices of general practitioners, such as those linked to payments for uptake of immunisations and cervical smears. Health authorities have also tried to use practice based performance indicators,4 with varying degrees of success.5,6 The recent availability of data on prescribing analysis and cost has allowed health authorities to look at practice prescribing in more detail and to develop indicators reflecting “good and bad” prescribing.7 Campbell et al have identified a number of valid practice indicators from over 240 under consideration for use by health authorities in England and Wales.8
Performance indicators for primary care groups
—To maximise their usefulness, performance indicators for primary care groups should meet certain minimal criteria before any consideration of their introduction into routine use. They should be attributable to health care,9 sensitive to change,10 based on reliable and valid information, precisely defined, reflect important clinical areas, and include a variety of dimensions of care. The US National Library of Healthcare Indicators describes several “definable, measurable and improvable domains of performance” for its indicators.11 These are attributes of organisational performance related to “doing the right things” (such as appropriateness, availability, and efficacy) and “doing things right” (such as effectiveness, efficiency, respect and caring, safety, and timeliness).11 For those indicators that reflect appropriateness, availability, efficacy, and effectiveness there should be robust evidence that the interventions on which they are based lead to improved health outcomes. Use of such indicators to monitor performance may be one way to promote the wider use of evidence based interventions—for example, in the secondary prevention of coronary heart disease.12,13
However, there is more to primary care than the use of evidence based interventions. Other important dimensions to primary care include consultation skills, the advocacy role of members of the primary care team for individual patients, communication within the practice team, access to primary care, managing a business within a regulatory framework, and coordination with community, secondary care, and local authority services. The use of evidence based interventions and related performance indicators as presented in this paper can therefore only represent some aspects of primary care. Further research is needed to address the feasibility of developing meaningful performance indicators reflecting these other dimensions.
The aim of our study was to develop a method to identify important, evidence based interventions in primary care suitable for linking to performance indicators for primary care groups. Our objectives were to (a) identify interventions of proved efficacy for which primary care teams have a key responsibility; (b) estimate the number of preventable deaths or events in a primary care group locality of 100 000 people if all those eligible were receiving the intervention; and (c) compare the potential indicators we derived with the indicators currently proposed by the government.
Methods
There is no simple definition of primary care.14 Aspects of primary care include general practice, community nursing, midwifery, health visiting, pharmacy, dentistry, optometry, and other professions. For the purpose of this study, we identified primary care interventions of proved efficacy from systematic reviews and for which we judged primary care teams to have the major responsibility. We searched the Cochrane Database of Systematic Reviews and the Database of Abstracts of Reviews of Effectiveness15 and Effective Health Care bulletins and obtained the primary sources referred to in the abstracts.
Mant and Hicks proposed a method to compare the relative sensitivity of indicators to monitor differences in care for the hospital treatment of myocardial infarction.10 We developed their approach and for each primary care intervention estimated:
(a) Reduced risk of death or events for those receiving the intervention compared with those not receiving it over a certain period—the relative risk reduction (%)
(b) Mortality or event rate of those not receiving the intervention (the controls) over a certain period
(c) The difference in risk of death or events between those receiving the intervention and those not receiving it—the absolute risk reduction (a×b)
(d) The number of patients needed to receive the intervention in order to prevent one of them dying or developing an adverse event—the number needed to treat (1/c, the reciprocal of the absolute risk reduction16)
(e) The proportion (and number) of patients likely to be eligible to receive the intervention in a locality of 100 000
(f) The adjusted relative risk reduction to take into account those eligible for the intervention over a certain period (a×e)
(g) The adjusted absolute risk reduction (c×e%) and number of preventable deaths or events in the locality over a certain period if all those eligible received the intervention
(h) Current rate of uptake of the intervention in those eligible in the primary care group (estimated from published studies, local data, or local opinion)
(i) Additional number of preventable deaths or events if all those eligible in the locality received the intervention (g×(1−h)).
We also made brief comments on the interventions such as potential side effects and whether the intervention was likely to be cost effective.
Estimates of the potential impact of interventions
Table 2 lists the primary care interventions we examined and whether we were able to obtain key information as described in the previous section. For the purposes of this brief discussion, we focus our illustrative method on the first eight interventions listed for which we were able to easily translate risk reduction into improvement in health outcome.
Table 2.
Availability of evidence or information on primary care interventions
| Intervention | Availability of information
|
Comments | |||||
|---|---|---|---|---|---|---|---|
| Relative risk reduction | Absolute risk reduction | % of patients eligible | Current uptake rate in those eligible | Cost effectiveness | Risk reduction translated to improved health outcome at population level | ||
| Aspirin for patients at high risk of coronary or ischaemic cerebrovascular events | fYes | Yes | Yes | Yes | Yes | Yes | Further details in tables 3 and 4 |
| Control of hypertension | Yes | Yes | Yes | Yes | Yes | Yes | Further details in tables 3 and 4 |
| Advice on stopping smoking or nicotine replacement therapy | Yes | Yes | Yes | Yes | Yes | Yes | Further details in tables 3 and 4 |
| Angiotensin converting enzyme inhibitors for patients with heart failure | Yes | Yes | Yes | Yes | Yes | Yes | Further details in tables 3 and 4 |
| Statins for patients at high risk of coronary heart disease (secondary prevention) | Yes | Yes | Yes | Yes | Yes | Yes | Further details in tables 3 and 4 |
| Statins for patients at low risk of coronary heart disease (primary prevention) | Yes | Yes | Yes | No | Yes | Yes | Further details in tables 3 and 4 |
| Warfarin for stroke prophylaxis in non-valvular atrial fibrillation | Yes | Yes | Yes | Yes | Yes | Yes | Further details in tables 3 and 4 |
| Influenza vaccination for those aged >65 | Yes | Yes | Yes | Yes | Yes | Yes | Further details in tables 3 and 4 |
| Diabetes care | Yes | Yes | Yes | Yes | Yes | No | Diabetes care in general practice can achieve standards of care equivalent to or better than hospital outpatient care but does not lead to reduced mortality or hospital admissions.17 Difficult to translate other reported end points such as glycaemic control and losses to follow up into outcomes such as non-fatal events |
| Cervical screening | No | No | Yes | Yes | Yes | No | Observational data suggest that cervical screening programmes are effective. Difficult to translate such observational data into relative and absolute risk reductions |
| Brief interventions to reduce alcohol consumption | Yes | Yes | Yes | Yes | Yes | No | Unable to translate evidence on efficacy in reducing alcohol consumption18 into improved health outcomes (fatal and non-fatal events) at population level |
| Contraceptive prescribing with family planning advice | No | No | Yes | Yes | Yes | No | Experimental and observational evidence of relative effectiveness of different contraceptives. Difficult to translate such data into relative and absolute risk reductions between users and non-users |
| Immunisations (except against influenza in elderly) | No | No | Yes | Yes | Yes | No | Observational data strongly suggest that immunisation programmes are effective.19 Difficult to translate such observational data into relative and absolute risk reductions |
| Treatment of obesity in adults | Yes | Yes | Yes | Yes | Yes | No | Unable to translate evidence of efficacy in reducing weight20 into improved health outcomes (fatal and non-fatal events) at population level |
Yes=evidence or information readily available. No=evidence or information not readily available.
Table 3 shows the relative risk reductions for these eight interventions together with the number of patients likely to be eligible in a locality and the number of preventable deaths or events in the locality if all those eligible received the intervention. (Full details of how we estimated stages (a) to (i) for each intervention and the assumptions we made are listed in the appendices on the BMJ website). Some interventions, despite having high numbers needed to treat, could have a considerable impact on the health of a population. For example, 108 people aged over 65 need to receive influenza vaccination each year to prevent one death, but in a population of 100 000 this intervention could prevent 146 deaths each year.
Table 3.
Primary care interventions: relative risk reductions, eligible patients, numbers needed to treat, and total number of preventable deaths or events
| Events measured | Relative risk reduction (95% CI)* | No of eligible patients in a population of 100 000 | No of patients needed to treat to prevent one event† | No of preventable events if all eligible patients receive intervention | Comments‡ |
|---|---|---|---|---|---|
| Aspirin for patients at high risk of coronary or ischaemic cerebrovascular events | |||||
| Total deaths over 1 year | 17% (11% to 23%)21 | 3 00022 | 62 | 48 | Trials reviewed varied in length of follow up. Largest benefits were seen in first year of follow up. Side effects can be a problem and include intracerebral and gastrointestinal haemorrhage |
| Vascular deaths over 1 year | 18% (12% to 24%)21 | 67 | 45 MI, stroke, or vascular deaths | ||
| Non-fatal MI over 1 year | 35% (27% to 43%)21 | 68 | 44 | ||
| Non-fatal stroke over 1 year | 31% (21% to 41%)21 | 111 | 27 | ||
| Control of hypertension | |||||
| Total mortality over 4 years | 16% (4% to 27%)23 | 17 88824 | 63 | 286 | There is inconsistency in what constitutes controlled hypertension. Side effects can be a problem |
| CHD deaths over 4 years | 25% (13% to 36%)23 | 100 | 179 | ||
| Cerebrovascular deaths over 4 years | 43% (21% to 58%)23 | 116 | 154 | ||
| Advice on stopping smoking or nicotine replacement therapy | |||||
| Total deaths over 1 year§ | 13% | 30 00024 | 256 | 120 | Relative risk reduction extrapolated from a UK cohort and may be an overestimate because of confounding |
| Angiotensin converting enzyme inhibitors for patients with heart failure | |||||
| Total deaths over 90 days§ | 23% (12% to 33%)25 | 1 50026 | 20 | 76 | Patients in many of the trials were younger than those in the general population |
| Statins for patients at high risk of coronary heart disease (secondary prevention) | |||||
| Total deaths over 5 years | 30% (15% to 42%)27 | 1 96828 | 29 | 69 | Cholesterol lowering is only one of many possibilities to reduce CHD risk. Stroke risk may also be reduced. Essentially, all the deaths prevented were coronary deaths |
| All coronary deaths over 5 years | 42% (17% to 54%)27 | 28 | 70 | ||
| Major coronary events over 5 years | 34% (25% to 41%)27 | 13 | 154 | ||
| Statins for patients at low risk of coronary heart disease (primary prevention) | |||||
| Total deaths over 5 years | 22% (0 to 40%)29 | 1 39428 | 111 | 14 | CHD rates have been falling in UK for two decades. This must be taken into account when making projections of population benefits from interventions such as lipid lowering drugs |
| CHD deaths over 5 years | 28% (−10% to 52%)29 | 155 | 9 | ||
| CHD events over 5 years | 31% (17% to 43%)29 | 41 | 34 | ||
| Warfarin for stroke prophylaxis in non-valvular atrial fibrillation | |||||
| Deaths over 1 year | 33% (9% to 51%)30 | 1 90031 | 56 | 33 | Side effects can be a problem and include intracerebral haemorrhage. Pooled data were from hospital based trials |
| Stroke events over 1 year | 68% (50% to 79%) | 33 | 58 | ||
| Influenza vaccination for those aged >65 | |||||
| Deaths each year | 68% (56% to 76%)32 | 15 70034 | 108 | 146 | Effectiveness of vaccine depends on vaccine strain being sufficiently similar to epidemic strain |
| Influenza episodes each year | 58% (26% to 77%)33 | 57 | 273 | ||
MI=myocardial infarction; CHD=coronary heart disease.
Reduced risk of death or events for those receiving the intervention compared with those not receiving it.
Calculated from the attributable risk reduction presented in the appendices available on the BMJ website.
References for these statements appear in the appendices available on the BMJ website.
We were unable to calculate other events from the studies that we used.
Table 4 shows the estimated current uptake for each intervention, the additional number of preventable events if all eligible patients receive the intervention, and, briefly, the likely cost effectiveness of the intervention. The additional number of preventable deaths or events with full uptake is highly dependent on the estimated current uptake rate. The few studies that have examined these rates suggest that uptake is low. Considerable improvements in health outcomes would result if primary care groups with low uptake of these interventions—apart from use of statins for patients at low risk of coronary heart disease—improved their uptake rates. For example, a locality would prevent 24 deaths each year if all high risk patients took aspirin rather than the 50% who currently do so.
Table 4.
Primary care interventions: estimated current uptake, additional number of preventable deaths or events with full uptake in a population of 100 000, and likely cost effectiveness
| Intervention | Estimated current | Additional No of preventable deaths or events with full uptake | Likely cost effectiveness† |
|---|---|---|---|
| uptake among eligible patients (%)* | |||
| Aspirin for patients at high risk of coronary or ischaemic cerebrovascular events | 50 | 24 deaths over 1 year; 22 vascular deaths over 1 year; 22 non-fatal MIs over 1 year; 59 non-fatal strokes over 1 year | Likely to be cost effective |
| Control of hypertension | 40 | 171 deaths over 4 years; 107 CHD deaths over 4 years; 92 cerebrovascular deaths over 4 years | Very cost effective for first line drugs, but less so for more expensive drugs and for older people |
| Advice on stopping smoking or nicotine replacement therapy | 34 | 79 deaths over 1 year | Likely to be cost effective |
| Angiotensin converting enzyme inhibitors for patients with heart failure | 20 | 60 deaths over 90 days | Likely to be cost effective, especially if treatment started in primary care. The SOVLD trial showed a reduction in hospital admissions of 30% in those receiving the drug |
| Statins for patients at high risk of coronary heart disease (secondary prevention) | 25 | 52 deaths over 5 years; 53 coronary deaths over 5 years; 116 major coronary events over 5 years | Greatest in patients with highest risk for CHD. Should be used in patients with additional risk factors |
| Statins for patients at low risk of coronary heart disease (primary prevention) | 25 | 11 deaths over 5 years; 7 CHD deaths over 5 years; 25 CHD events over 5 years | Should be used in patients with additional risk factors. Cost effectiveness low in patients with no previous MI or angina |
| Warfarin for stroke prophylaxis in non-valvular atrial fibrillation | 30 | 23 deaths each year; 40 stroke events each year | Cost effective, especially if at least one additional risk factor for stroke |
| Influenza vaccination for those aged >65 | 30 | 102 deaths each year; 191 influenza episodes each year | Likely to be cost effective. UK government has recently recommended that all those aged ⩾75 should receive vaccine |
MI=myocardial infarction; CHD=coronary heart disease.
Details on how we calculated the estimated current uptake rates are in the appendices available on the BMJ website.
References for these statements appear in the appendices available on the BMJ website.
For these eight interventions that improve health outcomes, table 5 lists the possible performance indicators that could measure their use in primary care groups. There are indicators of the proportion of the population with diagnoses of hypertension, coronary heart disease, atrial fibrillation, and heart failure. Comparing observed with expected proportions could highlight inadequate detection of these diseases or incomplete Read coding within primary care. Data sources for all these indicators are available in practices with well computerised records.
Table 5.
Primary care interventions that improve health outcomes and possible performance indicators that reflect their use
| Intervention and possible indicators | Evidence that improved indicator value reflects improved health outcomes |
|---|---|
| Aspirin for patients at high risk of coronary or ischaemic cerebrovascular events | |
| % of population with diagnosis of IHD | Yes, if appropriate treatment follows diagnosis. Need to compare observed and expected prevalence to estimate undetected IHD |
| % of population with diagnosis of IHD who take aspirin* | Yes, but need to ensure that observed prevalence of IHD is similar to expected. Need to record advice to buy aspirin |
| % of population with a diagnosis of ischaemic stroke or TIAs† | Yes, if appropriate treatment follows diagnosis. Need to compare observed and expected prevalence to estimate undetected stroke or TIAs |
| % of population with diagnosis of ischaemic stroke or TIAs who take aspirin* | Yes, but need to ensure that observed prevalence of stroke and TIAs is similar to expected. Need to record advice to buy aspirin |
| Control of hypertension | |
| % of population whose BP recorded in past 5 years*† | Yes, if appropriate treatment follows diagnosis |
| % of population with diagnosis of hypertension | Yes, if appropriate treatment follows diagnosis. Need to compare observed and expected prevalence to estimate undetected hypertension |
| % of population identified as hypertensive whose most recent systolic BP <160 mm Hg* | Yes, but need to ensure that observed prevalence of hypertension is similar to expected |
| % of population identified as hypertensive whose most recent diastolic BP <90 mm Hg | Yes, but need to ensure that observed prevalence of hypertension is similar to expected |
| % of population identified as hypertensive whose BP recorded in past year† | Yes, if appropriate treatment follows a set of abnormal readings |
| % of those with diagnosis of IHD whose BP recorded in past year† | Yes, if appropriate treatment follows a set of abnormal readings |
| % of those with diagnosis of ischaemic stroke or TIA whose BP recorded in past year | Yes, if appropriate treatment follows a set of abnormal readings |
| Advice on stopping smoking or nicotine replacement therapy | |
| % of population whose smoking status recorded | Yes, if appropriate treatment follows recording status as current smoker |
| % of population who are current smokers and have received advice on stopping smoking or nicotine replacement therapy | Yes |
| % of those with diagnosis of IHD whose smoking status recorded | Yes, if appropriate treatment follows recording status as current smoker |
| % of those with diagnosis of IHD who are current smokers who have received advice on stopping smoking or nicotine replacement therapy | Yes |
| % of those with a diagnosis of ischaemic stroke or TIA who have their smoking status recorded | Yes, if appropriate treatment follows recording status as current smoker |
| % of those with a diagnosis of ischaemic stroke or TIA who are current smokers who have received smoking cessation advice or nicotine replacement therapy | Yes |
| Use of angiotensin converting enzyme inhibitors in those with heart failure | |
| % of population with a diagnosis of heart failure | Yes, if appropriate treatment follows diagnosis. Need to compare observed and expected prevalence to estimate undetected heart failure |
| % of population with heart failure who have a prescription for ACE inhibitors | Yes, if diagnosis is confirmed by echocardiography |
| Lipid lowering drugs for patients with established cardiovascular disease | |
| % of those with a diagnosis of IHD who have had a cholesterol measurement | Yes, if appropriate treatment follows diagnosis |
| % of those with a diagnosis of IHD with a raised cholesterol who are prescribed lipid lowering drugs | Yes |
| Warfarin for stroke prophylaxis in NVAF | |
| % of population with diagnosis of NVAF | Yes, if appropriate treatment follows diagnosis. Need to compare observed and expected prevalence to estimate undetected NVAF |
| % of general practice patients with diagnosis of NVAF who have a prescription for anticoagulants | Yes, need to ensure that observed prevalence of NVAF is similar to expected |
| Influenza vaccination in those aged over 65 years | |
| % of population aged >65 who receive annual influenza vaccination | Yes |
IHD=ischaemic heart disease; TIA=transient ischaemic attack; BP=blood pressure; ACE=angiotensin converting enzyme; NVAF=non-valvular atrial fibrillation
All these indicators are attributable to health care.
Similar indicators may be suggested by Department of Health commissioned expert groups examining a wide range of indicators for both stroke and myocardial infarction (M Goldacre, personal communication).
Similar indicators presented in US National Library of Healthcare Indicators.11
Methodological issues
Our sources of evidence were not comprehensive. We selected the eight interventions because of the ease of obtaining information, including the ability to translate evidence on efficacy into improvement in health outcomes at a population level. It would be possible to use this method for other primary care interventions and to overcome some of the difficulties listed in table 2. We used our sources of evidence in an illustrative way to demonstrate the potential for developing performance indicators based on interventions of proved efficacy.
We used end points from randomised controlled trials based in primary care, which are usually mortality and major non-fatal events. These end points for events such as further strokes or myocardial infarction are defined variously as, for example, “major coronary events,” “vascular deaths,” and “coronary heart disease events.” The terms presented in the tables are those used in the relevant trials. These events are rare within an individual general practice and underestimate the burden of morbidity.
Our method at present takes no account of years of life lost or the difference between prevalent and incident cases. We used prevalent cases, and we recognise that absolute gains would fall over time. The effects of the interventions included are over different time scales, and there are wide confidence intervals for the size of these effects and estimates of prevalence. The effect is also dependent on patient compliance; patient preferences and contraindications would further reduce the number eligible for these interventions. We interpreted odds ratios reported in systematic reviews as relative risks and may have therefore overstated any effect size.35 Interventions interact in complex ways, but our method presents them in isolation. We only briefly mention potential side effects and likely cost effectiveness of these interventions. Ideally, we would want to compare the overall cost per life year gained for each intervention.
Despite these methodological difficulties, we believe that this approach is a useful complement to that used by the NHS Executive and Department of Health in developing performance indicators relevant to primary care. Our method identifies those interventions that are attributable to primary care and estimates the relative importance of these in terms of reduced mortality or non-fatal events. It helps to emphasise the importance of examining healthcare needs both at the individual and population levels by taking into account the prevalence of conditions and the current uptake of interventions. This method could be used to develop performance indicators for areas other than primary care.
Requirements for developing evidence based indicators
Before using the indicators proposed in table 5 it is essential to develop clear definitions of the numerators and denominators for each indicator. Sufficient numbers and standard definitions are required to enable comparisons between practices in a primary care group. Indicators require evaluation both before and after their introduction into routine use to fulfil practical and scientific criteria.36 We are currently attempting to derive and evaluate the indicators in table 5 for all patients aged 45-69 in 19 local practices of a future primary care group. These indicators require collection of extra data, and some might argue that primary care teams cannot cope with yet more tasks. However, in the United Kingdom well over a million hours every month are already spent collecting data in primary care,37 and yet there is little consensus on which data should be collected. Focusing data collection on meaningful indicators and abandoning it in less relevant areas could result in an overall reduction in workload. If the government is to use performance indicators as a method of improving health and health care it is important to encourage health professionals to focus on data collection linked to interventions over which they have substantial control and which improve health outcomes.
Comparison with the performance indicators currently proposed
There are considerable differences between the evidence based performance indicators that we generated (table 5) and those in the national framework for assessing performance relevant to primary care (table 1). Some of the latter are important, relating to efficiency and access, but many of the others could create perverse incentives to change practice.38,39 For example, the indicators relating to district nurses may encourage district nurses to reduce the number of appropriate visits to patients aged under 75 years. Similarly, in order to seem to “perform” well, general practitioners may reduce the number of appropriate hospital admissions for anyone aged over 75 or those with pyelonephritis, heart failure, or asthma. They may even stop notifying pertussis or measles. Our evidence based indicators may be less likely to encourage perverse incentives as they are based on robust evidence. However, health authorities and primary care groups would have to use such indicators appropriately and ensure that the risks and benefits of interventions were considered, especially in elderly patients.
Conclusions
Applying evidence from clinical trials and systematic reviews to individual patients in primary care is complex and challenging.40 Overcoming operational issues and changing clinical behaviour require a multifaceted approach.41,42 The use of performance indicators by themselves as a method to improve the effectiveness of health care in primary care groups is unlikely to succeed. However, the use of evidence based indicators linked to interventions that improve health outcomes, such as those suggested in table 5, could be an important adjunct if used in interactive practice or primary care group educational meetings.43 Primary care group indicators should be based on robust evidence. If not, their use is unlikely to lead to improved health outcomes. Our method may be a complementary way of identifying areas for performance indicators to those proposed by the NHS Executive and Department of Health. Our suggested indicators are more likely to help turn evidence into everyday practice and to have an impact on the population’s health.
Footnotes
Funding: The study was funded by the Department of Health. The views expressed here are those of the authors and not necessarily those of the Department of Health.
Conflict of interest: None.
References
- 1.Secretary of State for Health. The new NHS. London: Stationery Office; 1997. [Google Scholar]
- 2.NHS Executive. The new NHS, modern and dependable: a national framework for assessing performance. London: Department of Health; 1998. [Google Scholar]
- 3.Secretary of State for Health. Our healthier nation. London: Stationery Office; 1998. [Google Scholar]
- 4.Majeed FA, Voss S. Performance indicators for general practice. BMJ. 1995;311:209–210. doi: 10.1136/bmj.311.6999.209. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Shanks J. Primary care outcome measures in Lambeth, Southwark and Lewisham. In: McColl AJ, Roderick P, Gabbay J, editors. Improving health outcomes—Case studies on how English health authorities use population based health outcome assessments. Vol. 2. Southampton: Wessex Institute for Health Research and Development; 1997. pp. 155–166. ) [Google Scholar]
- 6.Stein K. Supplementary and primary care indicators in Southampton and South West Hampshire HC. In: McColl AJ, Roderick P, Gabbay J, editors. Improving health outcomes—Case studies on how English health authorities use population based health outcome assessments. Vol. 2. Southampton: Wessex Institute for Health Research and Development; 1997. pp. 167–188. ) [Google Scholar]
- 7.Majeed A, Evans N, Head P. What can PACT data tell us about prescribing in general practice? BMJ. 1997;315:1515–1519. doi: 10.1136/bmj.315.7121.1515. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Campbell SM, Roland MO, Qualyle JA, Shekelle PG, Buetow S. Quality indicators for general practice; which ones can general practitioners and health authority managers agree are important and how useful are they? J Public Health Med (in press). [DOI] [PubMed]
- 9.Leyland AH, Boddy FA. League tables and acute myocardial infarction. Lancet. 1998;351:555–558. doi: 10.1016/S0140-6736(97)09362-8. [DOI] [PubMed] [Google Scholar]
- 10.Mant J, Hicks N. Detecting differences in quality of care: the sensitivity of measures of process and outcome in treating acute myocardial infarction. BMJ. 1995;311:793–796. doi: 10.1136/bmj.311.7008.793. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Joint Commission on Accreditation of Health Care Organisations. National library of healthcare indicators. Oakbrook Terrace, IL: Joint Commission of Healthcare Organisations; 1997. [PubMed] [Google Scholar]
- 12.Van der Weijden T, Grol R. Preventing recurrent coronary heart disease. BMJ. 1998;316:1400–1401. doi: 10.1136/bmj.316.7142.1400. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Campbell NC, Thain J, Deans HG, Ritchie LD, Rawles JM. Secondary prevention in coronary heart disease: baseline survey of provision in general practice. BMJ. 1998;316:1430–1434. doi: 10.1136/bmj.316.7142.1430. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.National Working Group on R&D in Primary Care. R&D in Primary Care. Bristol: NHS Executive South and West; 1997. [Google Scholar]
- 15.The Cochrane Library [database on disk and CDROM]. Cochrane Collaboration. Oxford: Update software; 1997. Updated quarterly. [Google Scholar]
- 16.Sackett DL, Richardson WS, Rosenberg W, Haynes RB. Evidence-based medicine—How to practice and teach EBM. London: Churchill Livingstone; 1997. [Google Scholar]
- 17.Griffin S, Kinmonth AL. Diabetes care: the effectiveness of systems for routine surveillance for people with diabetes. In: Cochrane Database of Systematic Reviews. The Cochrane Library. Cochrane Collaboration. Oxford: Update software, 1997. Updated quarterly. [DOI] [PubMed]
- 18.Freemantle N, Song F, Sheldon T, Long A. Brief interventions and alcohol use. York: NHS Centre for Reviews and Dissemination; 1993. [Google Scholar]
- 19.Department of Health. Immunisation against infectious disease. London: HMSO; 1996. [Google Scholar]
- 20.University of York. The prevention and treatment of obesity. Effective Health Care. 1997;3(2):1–12. [Google Scholar]
- 21.Altman R, Carreras L, Diaz R, Figueroa E, Paolasso E, Parodi JC, et al. Collaborative overview of randomised trials of antiplatelet therapy—I: Prevention of death, myocardial infarction, and stroke by prolonged antiplatelet therapy in various categories of patients. BMJ. 1994;308:81–106. [PMC free article] [PubMed] [Google Scholar]
- 22.Moher M, Schofield T, Weston S, Fullard E. Managing established coronary heart disease. BMJ. 1997;315:69–70. doi: 10.1136/bmj.315.7100.69. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Mulrow C, Lau J, Cornell J, Brand M. Antihypertensive drug therapy in 60 to 80 year olds using diuretic and beta-blocker therapies. In: Cochrane Database of Systematic Reviews. The Cochrane Library. Cochrane Collaboration. Oxford: Update software, 1997. Updated quarterly.
- 24.Joint Health Surveys Unit. Health survey for England 1995. vol 1. Findings. London: Stationery Office; 1997. [Google Scholar]
- 25.Garg R, Yusuf S. Overview of randomized trials of angiotensin-converting enzyme inhibitors on mortality and morbidity in patients with heart failure. JAMA. 1995;273:1450–1456. [PubMed] [Google Scholar]
- 26.McDonagh TA, Morrison CE, Lawrence A, Ford I, Tunstall-Pedoe H, McMurray JJ, et al. Symptomatic and asymptomatic left-ventricular systolic dysfunction in an urban population. Lancet. 1997;350:829–833. doi: 10.1016/S0140-6736(97)03033-X. [DOI] [PubMed] [Google Scholar]
- 27.Randomised trial of cholesterol lowering in 4444 patients with coronary heart disease: the Scandinavian simvastatin survival study (4S) Lancet. 1994;344:1383–1389. [PubMed] [Google Scholar]
- 28.NHS Executive. Standing Medical Advisory Committee statement on use of statins. Wetherby: Department of Health; 1997. (EL (97)41.) [Google Scholar]
- 29.Shepherd J, Cobbe SM, Ford I, Isles CG, Lorimer AR, MacFarlane PW, et al. Prevention of coronary heart disease with pravastatin in men with hypercholesterolemia. West of Scotland Coronary Prevention Study Group. N Engl J Med. 1995;333:1301–1307. doi: 10.1056/NEJM199511163332001. [DOI] [PubMed] [Google Scholar]
- 30.Risk factors for stroke and efficacy of antithrombotic therapy in atrial fibrillation. Analysis of pooled data from five randomized controlled trials. Arch Intern Med. 1994;154:1449–1457. [PubMed] [Google Scholar]
- 31.Langenberg M, Hellemons BSP, Van Ree JW, Vermeer F, Lodder J, Schouten HJA, et al. Atrial fibrillation in elderly patients: Prevalence and comorbidity in general practice. BMJ. 1996;313:1534. doi: 10.1136/bmj.313.7071.1534. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Gross PA, Hermogenes AW, Sacks HS, Lau J, Levandowski RA. The efficacy of influenza vaccine in elderly persons. A meta- analysis and review of the literature. Ann Intern Med. 1995;123:518–527. doi: 10.7326/0003-4819-123-7-199510010-00008. [DOI] [PubMed] [Google Scholar]
- 33.Govaert TM, Thijs CT, Masurel N, Sprenger MJ, Dinant GJ, Knottnerus JA. The efficacy of influenza vaccination in elderly individuals. A randomized double-blind placebo-controlled trial. JAMA. 1994;272:1661–1665. [PubMed] [Google Scholar]
- 34.Office for National Statistics. National population projections—Series PP2 No 20. London: Stationary Office; 1997. [Google Scholar]
- 35.Davies HT, Crombie IK, Tavakoli M. When can odds ratios mislead? BMJ. 1998;316:989–991. doi: 10.1136/bmj.316.7136.989. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.McKee M, Sheldon T. Measuring performance in the NHS. BMJ. 1998;316:322. doi: 10.1136/bmj.316.7128.322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Newrick DC, Spencer JA, Jones KP. Collecting data in general practice: need for standardisation. BMJ. 1996;312:33–34. doi: 10.1136/bmj.312.7022.33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Ghali WA, Ash AS, Hall RE, Moskowitz MA. Statewide quality improvement initiatives and mortality after cardiac surgery. JAMA. 1997;277:379–382. [PubMed] [Google Scholar]
- 39.Schneider EC, Epstein AM. Influence of cardiac-surgery performance reports on referral practices and access to care. A survey of cardiovascular specialists. N Engl J Med. 1996;335:251–256. doi: 10.1056/NEJM199607253350406. [DOI] [PubMed] [Google Scholar]
- 40.Fahey TP. Applying the results of clinical trials to patients in general practice: perceived problems, strengths, and challenges for the future. Br J Gen Pract. 1998;48:1173–1178. [PMC free article] [PubMed] [Google Scholar]
- 41.Grol R. Beliefs and evidence in changing clinical practice. BMJ. 1997;315:418–421. doi: 10.1136/bmj.315.7105.418. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Haines A, Donald A. Making better use of research findings. BMJ. 1998;317:72–75. doi: 10.1136/bmj.317.7150.72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA. Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. BMJ. 1998;317:465–468. doi: 10.1136/bmj.317.7156.465. [DOI] [PMC free article] [PubMed] [Google Scholar]