Measuring the health of nations: analysis of mortality amenable to health care

Abstract

Objective To assess whether and how the rankings of the world's health systems based on disability adjusted life expectancy as done in the 2000 World Health Report change when using the narrower concept of mortality amenable to health care, an outcome more closely linked to health system performance.

Design Analysis of mortality amenable to health care (including and excluding ischaemic heart disease).

Main outcome measure Age standardised mortality from causes amenable to health care

Setting 19 countries belonging to the Organisation for Economic Cooperation and Development.

Results Rankings based on mortality amenable to health care (excluding ischaemic heart disease) differed substantially from rankings of health attainment given in the 2000 World Health Report. No country retained the same position. Rankings for southern European countries and Japan, which had performed well in the report, fell sharply, whereas those of the Nordic countries improved. Some middle ranking countries (United Kingdom, Netherlands) also fell considerably; New Zealand improved its position. Rankings changed when ischaemic heart disease was included as amenable to health care.

Conclusion The 2000 World Health Report has been cited widely to support claims for the merits of otherwise different health systems. High levels of health attainment in well performing countries may be a consequence of good fortune in geography, and thus dietary habits, and success in the health effects of policies in other sectors. When assessed in terms of achievements that are more explicitly linked to health care, their performance may not be as good.

Introduction

In its 2000 World Health Report, the World Health Organization published a set of rankings of the world's health systems.¹ In brief, the overall performance of the health systems was assessed as a composite measure including level and distribution of health attainment, responsiveness of the health system, and degree of fairness of financing. Aggregate performance was compared with what might be expected given the country's level of economic and educational development.

This approach has attracted intensive debate, ranging from the implied values underlying the approach to technical considerations.²,³ Despite the volume of debate, a recent review characterised this process as a dialogue of the deaf.⁴

One issue still unresolved is the attribution of health attainment to health systems.⁵ The World Health Report refers to growing evidence of health gains to be achieved from health care and from health related policies in other sectors, such as vehicle safety. However, many determinants of health still lie outside health care. One possible solution uses mortality data at a population level that are available in many countries and is based on the concept that deaths from certain causes should not occur in the presence of timely and effective health care. This has given rise to the development of a variety of terms including “avoidable mortality” and “mortality amenable to medical/health care.”⁶ Originally developed in the mid-1970s to measure the quality of health care, the concept was subsequently adopted widely, especially in Europe.^7-10

The 2000 World Health Report approvingly cited research on amenable mortality but then used a much broader measure of health, disability adjusted expectancy, without attempting to disaggregate mortality that is and is not amenable to health care. The broad definition of a health system that this implies is, however, inconsistent with the scope of the other two measures used in the report, responsiveness and fairness of financing, which relate directly to the healthcare system. An important question emerging from this debate is whether and how rankings of health systems change when using a more clearly defined measure of health system outcome, mortality amenable to health care. We compared the results achieved by the two approaches in industrialised countries.

Methods

Mortality and population data were extracted from WHO mortality files for 1998 (1997 for Canada).¹¹ Data include deaths, coded according to the ninth and 10th revisions of the international classification of diseases, by sex and five year age bands (with infant deaths listed separately).

Data were examined for the countries of western Europe, the United States, Canada, Australia, New Zealand, and Japan, countries with high quality mortality data by cause of death. We excluded countries if data were unavailable for years more recent than 1996 (Belgium, Switzerland) and those with small populations (Luxemburg, Iceland).

The selection of causes of death considered amenable to health care was derived from a recent review of “avoidable” mortality.⁶ The detailed justification for selection of conditions is set out in that review but, in brief, the final list was a modification of work by Tobias and Jackson, who updated earlier work by Mackenbach and Charlton and coworkers (table).⁸,¹²,¹³

Table 1.

Causes of death considered amenable to health care

		International classification of diseases
Cause of death	Age	9th revision	10th revision
Intestinal infections	0-14	001-9	A00-9
Tuberculosis	0-74	010-8, 137	A15-9, B90
Other infections (diphtheria, tetanus, poliomyelitis)	0-74	032, 037, 045	A36, A35, A80
Whooping cough	0-14	033	A37
Septicaemia	0-74	038	A40-1
Measles	1-14	055	B05
Malignant neoplasm of colon and rectum	0-74	153-4	C18-21
Malignant neoplasm of skin	0-74	173	C44
Malignant neoplasm of breast	0-74	174	C50
Malignant neoplasm of cervix uteri	0-74	180	C53
Malignant neoplasm of cervix uteri and body of uterus	0-44	179, 182	C54, C55
Malignant neoplasm of testis	0-74	186	C62
Hodgkin's disease	0-74	201	C81
Leukaemia	0-44	204-8	C91-5
Diseases of the thyroid	0-74	240-6	E00-7
Diabetes mellitus	0-49	250	E10-4
Epilepsy	0-74	345	G40-1
Chronic rheumatic heart disease	0-74	393-8	I05-9
Hypertensive disease	0-74	401-5	I10-3, I15
Cerebrovascular disease	0-74	430-8	I60-9
All respiratory diseases (excluding pneumonia and influenza)	1-14	460-79, 488-519	J00-9, J20-99
Influenza	0-74	487	J10-1
Pneumonia	0-74	480-6	J12-8
Peptic ulcer	0-74	531-3	K25-7
Appendicitis	0-74	540-3	K35-8
Abdominal hernia	0-74	550-3	K40-6
Cholelithiasis and cholecystitis	0-74	574-5.1	K80-1
Nephritis and nephrosis	0-74	580-9	N00-7, N17-9, N25-7
Benign prostatic hyperplasia	0-74	600	N40
Maternal death	All	630-76	O00-99
Congenital cardiovascular anomalies	0-74	745-7	Q20-8
Perinatal deaths, all causes, excluding stillbirths	All	760-79	P00-96, A33
Misadventures to patients during surgical and medical care	All	E870-6, E878-9	Y60-9, Y83-4
Ischaemic heart disease*	0-74	410-4	I20-5

^*See text.

As in our earlier work, an age limit was set at 75 years, as avoidability of death and reliability of death certification become increasingly questionable at older ages. We recognise that any upper age limit is essentially arbitrary, but this value is consistent with life expectancy at birth in many industrialised countries. However the logic of this would suggest setting different limits for men and women because of the sex gap in life expectancy. We recognise this as an important issue for debate, but we do not believe that it has yet been resolved.

Different age limits were set for diabetes mellitus (under 50) because the preventability of deaths at older ages from diabetes remains controversial. For some other causes, a limit of 15 years was set (see table) as related deaths other than in childhood are likely to reflect some other disease process. The age limit for leukaemia was extended to 44 years because of recent evidence showing substantial improvements in mortality from leukaemia in the European Union up to age 44 since 1960.¹⁴

To calculate mortality from conditions amenable to health care, we combined single causes and groups of causes. We computed age standardised death rates per 100 000 population, for both sexes combined, by direct standardisation to the European standard population.¹⁵ Ischaemic heart disease was treated separately, as the precise contribution of health care to reductions in deaths from this condition remains unresolved.¹⁶ However, accumulating evidence suggests its impact to be considerable, indicating that up to 50% of premature mortality from ischaemic heart disease may be amenable to health care.¹² To account for this we ran two sets of analyses, with and without ischaemic heart disease. Where we included ischaemic heart disease we assumed 50% of deaths under the age of 75 to be amenable to health care.

Based on age standardised death rates derived in this way we constructed rankings for the countries included in this analysis, supplemented with similar rankings for data on disability adjusted life expectancy for 1999, which were taken from the 2000 World Health Report.¹ Rankings were then compared for performance in terms of both disability adjusted life expectancy and amenable mortality.

Results

Figure 1 shows the rankings for the 19 countries included in our analysis based on disability adjusted life expectancy and on amenable mortality without ischaemic heart disease, showing that no country retained the same rank with both methods. Twelve of the countries moved more than two ranks. Major losses using amenable mortality were seen for Japan (from 1 to 13), Greece (7 to 12), and the United Kingdom (10 to 18). Large gains were seen for Canada (9 to 4), Norway (11 to 2), Finland (13 to 8), Germany (14 to 6), Denmark (17 to 10), and New Zealand (19 to 15). A few broad trends were observed in groups of countries that, at least in other respects, share particular characteristics. The Nordic countries ranked higher using amenable mortality, in several cases noticeably so. Southern European countries, whose high rankings have often been attributed to the known effects of a Mediterranean diet, had all fallen. The situation with the non-European countries is more mixed. Australia remained among the top five ranking countries with both measures whereas New Zealand, despite being placed four ranks higher with amenable mortality, remained in the bottom half of the ranking.

Fig 1.

Comparison of rankings based on disability adjusted life expectancy (1999) and standardised death rates (per 100 000; ages 0-74) from mortality amenable to health care (1998)

Figure 2 shows the rankings of the 19 countries for disability adjusted life expectancy compared with mortality amenable to health care with ischaemic heart disease. Rankings changed for all but two (Mediterranean) countries. Major losses were again seen for Greece (from 7 to 12) and particularly for the United Kingdom (10 to 19) but also for the Netherlands (8 to 11) and Australia (2 to 7). The Nordic countries generally showed substantial improvements in their rankings, especially Denmark (17 to 10) and Norway (11 to 5), but not Finland (13 to 15). With the exception of Greece, southern European countries retained either similar ranks to the report (Italy, Portugal) or improved (Spain, France). For the other countries the rankings based on amenable mortality with ischaemic heart disease were essentially similar to those based on amenable mortality without ischaemic heart disease. The only other exception was Japan, which, with the inclusion of ischaemic heart disease, remained among the top five ranking countries.

Fig 2.

Comparison of rankings based on disability adjusted life expectancy (1999) and standardised death rates (per 100 000; ages 0-74) from amenable mortality including ischaemic heart disease (50% of all deaths under age 75) (1998)

Discussion

Rankings based on mortality amenable to health care produce results different from those based on disability adjusted life expectancy, as used in the 2000 World Health Report. This highlights the problems involved in international comparisons.

The quest for accountability has generated an industry engaged in ranking performance in many different sectors. This approach developed from control systems in industrial management, where it has long been used with processes where the inputs and products are simple and unambiguous, and there are few if any extraneous factors. It is intuitively appealing, especially to politicians who are anxious to know how public funds are being spent. It is seen as a means to reduce a mass of complex information into a format that almost anyone can understand. Yet its apparent simplicity can be misleading, and many commentators have noted numerous technical problems, ranging from lack of validity to creation of perverse incentives as those involved change their practice or recording methods to achieve higher rankings, despite leading to worse performance.

We have looked at one of many possible issues related to health system rankings that has so far been unexplored. The hypothesis was that a measure of health attainment more closely linked to the health-care system would produce a systematically different ranking. This hypothesis was confirmed, with the Nordic countries doing better than in the WHO model of the 2000 report.

However we do not argue that amenable mortality should substitute disability adjusted life expectancy even if performance was to be ranked. Firstly, it is impossible to rank all countries by amenable mortality given the widespread absence of data by diagnosis. It should be noted, though, that lack of even total mortality for many countries was not seen as an obstacle by the authors of the 2000 World Health Report, who used regression modelling to impute figures for disability adjusted life expectancy. A recent study that looked at the method used in the report to generate data on disability adjusted life expectancy showed that, where actual data became available, the regression result produced a quite different value.¹⁷

Secondly, amenable mortality has itself some limitations. The diagnostic categories and the age range used involve some choices that are inevitably arbitrary. A major limitation is that, for many conditions, death is the final event in a complex chain of processes that involve issues related to underlying social and economic factors, lifestyles, and preventive and curative health care. Partitioning deaths among the categories is an inexact science. The example of ischaemic heart disease is instructive. Accumulating evidence suggests that advances in health care have contributed to the fall in mortality from ischaemic heart disease in many countries, yet it is equally clear that large international differences in mortality are caused primarily by factors outside the healthcare sector.¹⁶,^18-21 Thus our second analysis included only 50% of mortality from ischaemic heart disease. Obviously many different proportions could be used, from 40% up to 70%, and the choice may vary by country.¹⁶,²² However, it is important to note that the inclusion of ischaemic heart disease in our measure of healthcare outcomes again changes the rankings of countries compared with those of the 2000 World Health Report. This highlights the problems associated with rankings that are based on summary measures, as they can be sensitive to underlying definitions and concepts.

Furthermore, what is considered amenable to health care will change over time as new pharmaceuticals and management strategies are developed. Thus testicular cancer has now become potentially a largely curable disease, although the extent to which this is achieved by different healthcare systems varies.²³ In the future it is plausible that diseases such as prostatic cancer and AIDS could become amenable, with potentially important consequences for such rankings.

The upper age limit is also problematic. We included deaths only up to age 75, although it is also clear that advances in medical care are making an increasingly large contribution to survival of people at older ages. Conversely, there remains uncertainty about the validity of death certification at these ages in many countries, not least because of the problems created by comorbidity. We also only looked at rankings for both sexes combined, although a subsequent analysis showed that in some cases rankings differed by sex (data not shown), largely reflecting the relative importance of breast cancer as a cause of premature death in women in the country in question.

What is already known on this topic

The 2000 World Health Report compared the performance of 191 health systems, using health attainment, responsiveness, and fairness of financing

Overall health attainment used in the report, disability adjusted life expectancy, does not make it possible to separate the influence of health care from those of other sectors, such as agriculture (and thus diet), education, and fiscal policy

Calculation of mortality amenable to health care allows more specific attribution of levels of health attainment to health care

What this study adds

Rankings based on mortality amenable to health care produce results different from those based on disability adjusted life expectancy

Additional inclusion of ischaemic heart disease as an amenable cause of death produces rankings different from those in the WHO 2000 World Health Report

Rankings that are based on summary measures are sensitive to the definitions and concepts underlying them

So how can amenable mortality be used to understand the performance of a health system? We argue that it does have some value, but not in terms of the aggregate values (although arguably the rankings produced by amenable mortality have greater face validity than those using disability adjusted life expectancy). Instead, it enables comparison of the elements that make up the overall figure, permitting investigators to dig down to look at specific policies and learn from different experiences. In some cases these differences are already recognised, examples including advances in early detection and treatment of stomach cancer in Japan and of melanoma in Australia.²⁴,²⁵ Of course, whether these policies can or should be transferred between countries depends on the burden of disease involved and other contextual factors.

It now seems unlikely that the exercise to rank performance of health systems will be repeated, but this does not mean that the quest to identify improved measures of performance will be abandoned. What our study shows is that rankings based on overall health attainment and health attainment that can, however imperfectly, be more closely linked to health care, produce different results.