Measuring NHS performance 1990–2009 using amenable mortality: interpret with care

Abstract

Objectives

The new performance framework for the NHS in England will assess how well health services are preventing people from dying prematurely, based on the concept of mortality amenable to healthcare. We ask how the different parts of the UK would be assessed had this measure been in use over the past two decades, a period that began with somewhat lower levels of health expenditure in England and Wales than in Scotland and Northern Ireland but which, after 1999, saw the gap closing.

Design

We assessed the change in age-standardized death rates in England and Wales, Northern Ireland and Scotland in two time periods: 1990–1999 and 1999–2009. Mortality data by five-year age group, sex and cause of death for the years 1990 to 2009 were analysed using age-standardized death rates from causes considered amenable to healthcare. The absolute change was assessed by fitting linear regression and the relative change was estimated as the average annual percent decline for the two periods.

Setting

United Kingdom.

Participants

Not applicable.

Main outcome measures

Mortality from causes amenable to healthcare.

Results

Between 1990 and 1999 deaths amenable to medical care had been falling more slowly in England and Wales than in Scotland and Northern Ireland. However the rate of decline in England and Wales increased after 1999 when funding of the NHS there increased. Examination of individual causes of death reveals a complex picture, with some improvements, such as in breast cancer deaths, occurring simultaneously across the UK, reflecting changes in diagnosis and treatment that took place in each nation at the same time, while others varied.

Conclusions

Amenable mortality is a useful indicator of health system performance but there are many methodological issues that must be taken into account when interpreting it once it is adopted for routine use in England.

Introduction

There has been a longstanding interest among policymakers and researchers to better understand whether and to what extent healthcare contributes to improving population health. One approach to measure this is mortality from causes considered amenable to healthcare (‘amenable mortality’) (Box 1).¹

Box 1. The development of the concept of mortality amenable to healthcare.

The concept of ‘avoidable mortality’ as it has been used over the last 25 years, stems from the Working Group on Preventable and Manageable Diseases led by David Rutstein of Harvard Medical School in the USA in the 1970s, identifying over 90 conditions as ‘sentinel health events’ with cases of disease, disability or death from these conditions to be considered to be preventable and/or treatable by appropriate and timely medical care.¹ Subsequent writers have more specifically related this concept to deaths, introducing the terms ‘avoidable’ and ‘amenable to medical care’ as a means to identify potential problems in healthcare justifying further inquiry. The concept was adopted in Europe, using work by Holland and colleagues in particular, who published a series of European Community atlases of avoidable deaths.^20,21 Mackenbach evaluated the concept further, showing that the introduction of specific interventions coincided with accelerating falls in mortality from the conditions they were intended to treat.²² They thus concluded that the healthcare interventions they examined added substantially to gains in overall life expectancy. In 2004 Nolte and McKee undertook a systematic review of the assumptions underlying the concept.¹² Their updated list forms the basis for the measure now being evaluated by the English Department of Health⁷ and the Organisation for Economic Cooperation and Development.¹⁵ Examples of its practical application include studies showing how the Soviet health system failed to modernize in the 1960s,²³ the impact of German unification on healthcare outcomes,¹¹ and how the American system lagged behind health systems in other industrialized countries.⁹

However, one challenge of using amenable mortality as a means to assess health system performance internationally is the difficulty to disentangle factors other than healthcare that are important determinants of population health against the diversity of healthcare systems. The NHS in the UK provides an important natural experiment with which to study the relationship between health system factors and health outcomes as some of the contextual factors can be considered as controlled for because of the common institutional, organizational and financial origins of the NHS in the four countries. At the same time, although the basic principles of the NHS have historically been similar throughout the UK, there have been important differences. Thus, the NHS in Scotland and Northern Ireland has been funded more generously than in England and Wales, with allocations governed by the 1978 Barnett Formula. However, recognizing the low spending on the NHS in international terms, in 1997 the incoming Labour government committed to increasing NHS spending across the UK to the European average and, although this has not yet been achieved, it did result in marked increase in per capita spending in England and Wales relative to the other countries from 1999 onwards² and a closing of the gap with Scotland (and narrowing that with Northern Ireland) (Figure 1).³

Figure 1.

Changes in relative expenditure per capita on the NHS in the constituent parts of the UK: ratios compared to Scotland

Source: House of Commons library²

Although both this spending increase and the performance of the NHS in the by now devolved Scottish and Northern Irish administrations, have been studied, research⁴ has paid limited attention to health outcomes.^5,6 In this paper, we ask whether and to what extent the changes of the NHS in each part of the UK, including the increase in spending in England, are reflected in health outcomes, using the concept of amenable mortality. This is an important question given that the concept was proposed for routine use within the new NHS Outcomes Framework in England as a means of capturing, ‘at a high level, how successful the NHS is’ in meeting the objective of preventing people from dying prematurely.⁷ Under the proposed reforms, it is important that healthcare providers and commissioners, including the proposed Commissioning Consortia, understand fully what amenable mortality is able to tell us.

Methods

We obtained data on deaths by five-year age group, sex and cause of death for the years 1990–2009 from the World Health Organization's (WHO) mortality database.⁸ Deaths were classified according to the ninth and 10th revisions of the International Classification of Diseases (ICD). Population data were obtained from the same source. Data were extracted for England and Wales, Scotland and Northern Ireland. Separate data for Wales are not available from this source.

We calculated age-standardized death rates from causes considered amenable to healthcare by sex, using the list of conditions applied in previous analyses (Appendix 1 – available online at http://jrsm.rsmjournals.com/cgi/content/full/104/9/370/DC1).⁹ In keeping with that work, we considered half of deaths from ischaemic heart disease to be amenable to healthcare. Age-standardized death rates by sex were calculated for single causes and cause groups and for all ages (0–74 years) by direct standardization to the European standard population.¹⁰

We included deaths under 75 years only, primarily because of the difficulty of reliably assigning a single cause of death to the often multiple conditions present among those dying at older ages.¹¹ This includes almost 50% of mortality in the populations in question. In line with earlier work, we applied lower age limits for some causes,¹² such as diabetes (<50 years).

We adjusted for known discontinuities related to the introduction of automated cause of death coding and the application of selection rule 3 to assigning underlying cause of death in England and Wales in 1993, which resulted, among other things, in an over-estimation of deaths assigned to pneumonia and an under-estimation of those assigned to cerebrovascular disease between 1993 and 1999; we used comparability ratios proposed by the Office for National Statistics.^13,14

We assessed the change in age-standardized death rates in England and Wales, Northern Ireland and Scotland in two time periods: 1990–1999 and 1999–2009. The second period coincided with the beginning of increased spending in England and the creation of the Scottish parliament. As the latter came into being on 1 July 1999, we consider 1999 as the year of transition, and include it in both periods. We show both relative and absolute changes as each conveys complementary information. We estimated the absolute annual change (‘slope’) between 1990 and 1999 and between 1999 and 2009 by fitting a linear regression function to each of the two time periods. The relative change was estimated as the average annual percent decline throughout 1990–2009. Calculations were carried out using Microsoft Excel® 2007.

Results

In 1990, amenable mortality, for each sex, was highest in Scotland (male deaths: 238.6 per 100,000 population; female deaths: 172.3/100,000), followed by Northern Ireland (male deaths: 223.0/100,000; female deaths: 165.6/100,000) and England and Wales (male deaths: 187.0/100,000; female deaths: 140.0/100,000) (Table 1). By 2009, amenable mortality had fallen substantially in all countries, to 105.4 per 100,000 population among men/boys in Scotland (women/girls: 82.3), 87.0 per 100,000 population in Northern Ireland (women/girls: 74.0) and 83.5 in England and Wales (women/girls: 67.9).

Table 1.

Age-standardized death rates (per 100,000) among men and women from selected causes in England and Wales, Northern Ireland and Scotland, 1990, 1999 and 2009

Nation	Time period	Age-standardized death rate (per 100,000 population)
		Conditions amenable to healthcare								IHD (50%)	Other
		Amenable (including 50% IHD)	Other circulatory diseases	Treatable cancers	Respiratory diseases	Perinatal, maternal and congenital conditions	Infectious diseases	Diabetes	Remainder amenable diseases
Men
England and Wales	1990	187.04	40.16	23.24	10.64	9.49	1.81	0.75	9.17	91.78	399.09
	1999	137.00	30.52	18.18	12.64	7.34	2.31	0.77	7.97	57.26	330.97
	2009	83.50	16.87	14.46	8.64	6.04	1.74	0.74	5.84	29.18	264.38
Northern Ireland	1990	223.01	39.61	25.86	28.45	7.58	1.14	0.60	8.22	111.53	426.97
	1999	153.44	34.85	19.56	13.05	8.29	0.70	0.52	6.86	69.62	358.13
	2009	87.03	15.85	14.97	9.64	5.97	1.73	1.60	5.30	31.98	292.22
Scotland	1990	238.62	54.83	23.62	22.28	8.25	2.75	0.75	11.64	114.49	472.42
	1999	171.94	40.30	23.17	11.13	6.19	3.43	0.98	11.00	75.73	431.77
	2009	105.44	22.17	17.54	9.59	5.16	3.71	1.38	7.88	38.01	348.10
Women
England and Wales	1990	139.98	31.52	54.02	6.62	7.79	1.18	0.59	5.56	32.70	199.53
	1999	104.54	23.41	39.76	8.56	5.98	1.72	0.46	4.81	19.84	182.04
	2009	67.85	12.23	30.37	5.92	5.19	1.35	0.44	3.82	8.54	151.75
Northern Ireland	1990	165.57	39.66	50.91	20.41	5.76	0.91	0.93	6.06	40.93	208.76
	1999	109.44	25.85	37.73	9.27	5.80	0.70	0.52	5.16	24.42	190.62
	2009	73.99	13.33	32.51	5.90	4.71	2.00	0.95	3.10	11.51	174.86
Scotland	1990	172.33	42.38	52.10	13.28	7.18	2.06	0.22	7.62	47.49	246.70
	1999	122.31	30.57	41.52	8.17	4.86	1.95	0.83	6.20	28.21	233.33
	2009	82.28	17.33	32.84	6.61	4.29	2.36	1.02	4.27	13.56	200.85

For completeness, the tables show both the annual absolute change in mortality and the annual percentage change. While both are important, they provide different but complementary information. The same absolute reduction in two time periods of overall decline will represent a greater percentage reduction in the second period as the starting point is lower. However, the scope for further declines may be diminished in the second period as the easiest gains have already been made. Consequently, to allow the reader to interpret the data fully, we show both but in our discussion we focus on the latter.

The pace of change differed between countries and over time. In England and Wales the annual percentage rate of decline in amenable mortality was initially less than in Scotland and Northern Ireland (male deaths 2.68% versus 2.79% and 3.12%, respectively; female deaths 2.53% versus 2.9% and 3.39%, respectively). However, its rate of decline increased during the second period (1999–2009), with the percentage annual decline among men/boys overtaking that in Scotland while that for women/girls overtook the two other nations (Table 2; Figure 2). The rapid rate of decline seen for women/girls in Northern Ireland in the first period slowed subsequently.

Table 2.

Absolute and relative changes in age-standardized death rates per 100,000 population per year among men and women in England and Wales, Scotland and Northern Ireland between 1990–1999 and 1999–2009

		Change in age-standardized death rate (per 100,000 population)
Nation/sex	Time period	Annual change	Conditions amenable to healthcare								IHD (50%)	Other
			Amenable (including 50% IHD)	Other circulatory diseases	Treatable cancers	Respiratory diseases	Perinatal, maternal and congenital conditions	Infectious diseases	Diabetes	Remainder amenable diseases
Men
England and Wales	1990–1999	Annual absolute decline (slope)*	5.58	1.06	0.62	–0.26	0.22	–0.07	–0.01	0.08	3.93	7.41
	1999–2009	Annual absolute decline (slope)	5.23	1.36	0.37	0.33	0.12	0.04	0.01	0.18	2.81	6.33
	1990–1999	Annual relative decline (%)†	2.68	2.40	2.17	–1.87	2.26	–2.79	–0.31	1.31	3.76	1.71
	1999–2009	Annual relative decline (%)	3.55	4.07	1.86	2.87	1.61	2.26	0.44	2.43	4.46	1.83
Northern Ireland	1990–1999	Annual absolute decline (slope)	8.17	0.95	0.67	1.57	0.21	0.03	0.02	0.02	4.70	7.61
	1999–2009	Annual absolute decline (slope)	5.90	1.48	0.27	0.46	0.22	–0.09	–0.03	0.24	3.35	4.86
	1990–1999	Annual relative decline (%)	3.12	1.20	2.44	5.41	–0.93	3.91	1.35	1.66	3.76	1.61
	1999–2009	Annual relative decline (%)	3.93	4.96	2.14	2.38	2.54	–13.48	–18.98	2.07	4.92	1.67
Scotland	1990–1999	Annual absolute decline (slope)	7.57	1.56	0.11	1.12	0.25	–0.10	–0.08	–0.01	4.71	4.22
	1999–2009	Annual absolute decline (slope)	6.18	1.68	0.51	0.21	0.17	–0.07	–0.08	0.36	3.40	8.26
	1990–1999	Annual relative decline (%)	2.79	2.65	0.19	5.01	2.51	–2.45	–3.09	0.55	3.39	0.86
	1999–2009	Annual relative decline (%)	3.52	4.09	2.21	1.26	1.51	–0.75	–3.69	2.58	4.53	1.76
Women
England and Wales	1990–1999	Annual absolute decline (slope)	3.95	0.89	1.65	−0.22	0.19	–0.08	0.01	0.05	1.46	1.70
	1999–2009	Annual absolute decline (slope)	3.49	1.08	0.86	0.22	0.08	0.03	0.00	0.11	1.11	2.81
	1990–1999	Annual relative decline (%)	2.53	2.57	2.64	–2.92	2.32	–4.65	2.25	1.36	3.93	0.88
	1999-2009	Annual relative decline (%)	3.19	4.34	2.15	2.80	1.21	1.96	0.41	1.87	5.18	1.51
Northern Ireland	1990–1999	Annual absolute decline (slope)	6.17	1.41	1.71	1.10	–0.05	0.00	0.03	0.08	1.90	1.84
	1999–2009	Annual absolute decline (slope)	3.56	1.03	0.70	0.42	0.09	–0.05	–0.01	0.15	1.24	1.86
	1990–1999	Annual relative decline (%)	3.39	3.48	2.59	5.46	–0.08	2.34	4.38	1.48	4.03	0.87
	1999–2009	Annual relative decline (%)	2.95	4.40	1.26	3.30	1.71	–16.89	–7.41	3.64	4.81	0.75
Scotland	1990–1999	Annual absolute decline (slope)	6.04	1.59	1.48	0.59	0.16	–0.01	–0.03	0.07	2.19	0.97
	1999–2009	Annual absolute decline (slope)	3.79	1.40	0.79	0.05	0.15	–0.11	–0.05	0.15	1.41	2.85
	1990–1999	Annual relative decline (%)	2.90	2.79	2.03	3.85	3.23	0.53	–28.26	1.86	4.06	0.54
	1999–2009	Annual relative decline (%)	2.98	3.94	1.90	1.73	1.07	–1.91	–2.19	2.84	4.72	1.27

*A positive absolute decline indicates a reduction in age-standardized death rates. A negative absolute decline suggests an increase in age-standardized death rates

^†A positive relative decline suggests that there is a decline in the average annual percentage change of age-standardized death rates. A negative relative decline suggests an increase in the percentage change of age-standardized death rates

Figure 2.

Annual rate of decline in amenable mortality in the years 1990–1999 and 1999–2009

The annual percentage decline in death rates from causes other than those amenable to healthcare (last column of Tables 1 and 2) was somewhat slower (about two-thirds that of amenable mortality for men and about half for women) but it did increase in the second period everywhere except among women in Northern Ireland.

Aggregate data are not easy to interpret so it is necessary to examine specific causes (Table 2). Deaths from treatable cancers declined in male and female patients in each nation during 1990–1999. Among women they continued to do so, but more slowly in each nation in 1999–2009. Scottish men experienced an accelerating decline in these deaths in the second period. The annual percentage decline from ischaemic heart disease accelerated in each nation, for both sexes, after 1999, as did deaths from other circulatory diseases, which include hypertensive disease, cerebrovascular disease and chronic rheumatic heart disease. Progress in reducing deaths from respiratory diseases has been mixed, with an initial worsening trend among English men being reversed but a slowing of progress in Scotland and Northern Ireland. Death rates from diabetes worsened in Scotland in both periods, as in Northern Ireland after 1999.

Discussion

The principal findings of this study provide new insights into the performance of the NHS in each part of the UK over the past two decades. We already know that the rate of decline in amenable mortality in the United Kingdom as a whole, from 1998 onwards, has been substantially greater than in almost all other industrialized countries, which is consistent with the substantial increase in overall NHS funding from historically low levels.^9,15 The faster rate of decline observed in Scotland and Northern Ireland in the period before 1999 is consistent with the hypothesis that the higher levels of spending may have allowed it to benefit disproportionately from the expansion in effective treatments in the late 1980s and early 1990s. The acceleration in the rate of improvement in England and Wales from 1999 onwards is consistent with a process of catching up, suggesting that the NHS in England did indeed convert some of the additional resources into better care.

However, the picture is complicated when individual causes of death are considered. Trends in treatable cancer improved more among women than men, with the greatest gains in the first period in all countries. Cancer deaths among women considered treatable are dominated by breast cancer and the early 1990s saw both the introduction of a screening programme across the UK coupled with significant advances in treatment that were introduced everywhere.¹⁶ By 1999, these were widely implemented and, it would appear, a large proportion of those who might benefit were doing so, limiting the scope for additional gains. Thus, it would not be expected that the increased spending in England and Wales would differentially affect this cause of death. As the example of breast cancer shows, it is always difficult to disentangle the precise contributions of different factors to changes in specific causes of death. There are many things going on at once, so the accelerated reduction in deaths from cardiovascular disease occurred at a time when not only health spending was increasing but also smoking was being banned in public places and new systems of paying general practitioners, with an emphasis on meeting unmet needs, were being introduced. Perhaps the most that can be said is that, collectively, these measures seem to have worked. More worrying, however, is the increasing death rate (or among English and Welsh women, slowing in the rate of decline) from diabetes. This illustrates the main value of amenable mortality, as a high level indicator of progress that points to areas requiring more detailed study.

This analysis has strengths and weaknesses. Its main strength is that it captures the experience of the entire population in the relevant ages in each of the four countries. In addition, there is now a broad international consensus on the conditions included within the definition of amenable mortality, facilitating comparisons with other similar studies.

Its weaknesses relate to certain inevitable caveats that must be observed when seeking to interpret the data further. First, it is necessary to take account of variable lag periods between clinical interventions and outcomes. The impact of differences in the initial treatment of cancer on subsequent mortality may only become fully apparent after five years. In contrast, the results of surgery for major trauma will be apparent at once.

Second, it is important not to over-interpret short-term fluctuations in deaths from specific causes in territories with small populations. Thus, the small increase in death rates from treatable cancer in Northern Ireland, with a population of only 1.7 million, must be treated with considerable care.

Third, while our results have been adjusted for known problems with coding of causes of death, it will be important to be aware of any discontinuities in other countries where comparisons are to be undertaken. The effects of introducing new versions of the ICD are well-known; less so are changes to the rules for assigning ICD codes to causes of death, such as the introduction of automated coding in the UK in the late 1980s that we adjusted for in our analysis.

Fourth, it is necessary to take account of changes in incidence of underlying disease. This can never be done precisely, given the very limited data on disease incidence, but knowledge of general trends should be used to aid interpretation.¹² Thus, it might be expected that the rising levels of obesity in the UK¹⁷ would increase deaths from diabetes, all else being equal. This is addressed, in part, by restricting deaths from diabetes to those under the age of 50 years, at which ages most deaths will be associated with type 1 diabetes. However, the situation is more complex with ischaemic heart disease as there are countervailing forces at work. The increasing prevalence of obesity will increase the risk in the population while reduced smoking will decrease it, illustrated by the marked fall in hospital admissions with myocardial infarction following the introduction of smoking bans,¹⁸ as mentioned above. Disentangling this complex picture is a priority given evidence of a flattening out in the rate of mortality decline among young people.¹⁹

Fifth, the representation of the UK in international health statistics remains problematic, and we were unable to extract data separately for Wales for the entire period under analysis.

Finally, the attribution of an outcome to a particular aspect of healthcare is intrinsically difficult because of the multifactorial nature of most outcomes. As a consequence, when interpreting findings, a degree of judgement is needed, based on an understanding of the natural history and scope for prevention and treatment of the condition in question. This is explicitly recognized in the decision to include only 50% of deaths from IHD as evidence suggests that IHD is only partly amenable to healthcare⁹ although it is recognized in the literature on amenable mortality that is a somewhat arbitrary choice.

The study has important implications for the ongoing debate on NHS reform and, in particular, the consequences of the increase in funding for the English NHS after 1999. While the interpretation of our findings must be laced with some caveats, it does seem that the rate of decline in deaths amenable to healthcare in England and Wales had been slower than in Scotland or Northern Ireland before 1999, at a time when it was relatively less generously funded. However, the subsequent injection of additional funds did allow it to accelerate. This is consistent with the evidence from international comparisons of the remarkable progress achieved by the United Kingdom as a whole in recent years.

The study also raises some questions that it has not been possible to address here. In the space available it has not been possible to examine in detail the reasons for the changes in individual causes in each country. This is a fertile area for future research.

DECLARATIONS

Competing interests

None declared

Funding

EN, MM, MK and BK have worked on projects on amenable mortality for the EU (AMIEHS Project) and the European Observatory on Health Systems and Policies that have fed into this paper. EN was also supported by an NIHR Career Scientist Award

Ethical approval

Not applicable

Guarantor

MD

Contributorship

EN, MD and MM conceived the idea for this paper and had equal contribution to data analysis and writing the manuscript; MM and EN have published extensively in the area of amenable mortality and were invited to advise the English Department of Health on its use in the Outcomes Framework; MK and BK contributed to the revisions and MK contributed to the analyses. The principal data source was the World Health Organization (WHO) database

Acknowledgements

None

Reviewer

Nigel Edwards