Comparative efficiency of national health systems: cross national econometric analysis

Abstract

Objective

To improve the evidence base for health policy by devising a method to measure and monitor the performance of health systems.

Design

Estimation of the relation between levels of population health and the inputs used to produce health.

Setting

191 countries.

Main outcome measure

Health system efficiency (performance).

Results

Estimated efficiency varied from nearly fully efficient to nearly fully inefficient. Countries with a history of civil conflict or high prevalence of HIV and AIDS were less efficient. Performance increased with health expenditure per capita.

Conclusions

Increasing the resources for health systems is critical to improving health in poor countries, but important gains can be made in most countries by using existing resources more efficiently.

What is already known on this topic

Evidence on the effectiveness of health system reforms is scarce

Studies have not used a consistent framework for specifying goals or measuring outcomes

What this study adds

Countries with the best levels of health do not always have efficient health systems

Efficiency is related to expenditure on health per capita, especially at low expenditure

The methods of measuring performance provide a basis for identifying policies that improve health and for monitoring reforms

Introduction

Policymakers have long been concerned with improving the performance of health systems.¹,² Reforms have targeted financing (for example, social health insurance and user charges), provision (for example, managed care, autonomous hospitals), stewardship (for example, regulation of the private sector, health legislation), and resource development (for example, retraining of staff).¹,^3–5 The impact of these reforms is increasingly being studied,⁶,⁷ but for the results to be useful to policymakers across different settings, studies need a consistent framework for assessing performance and a measurable indicator.⁸

The World Health Report 2000 defined three intrinsic goals of health systems—improving health, increasing responsiveness to the legitimate demands of the population, and ensuring that financial burdens are distributed fairly.⁹ For health and responsiveness, systems should improve levels and reduce inequalities. The report published first attempts to measure the attainment of these goals by 191 countries and considered how well countries were performing given their available resources.⁹ This paper describes the methods used for measuring and monitoring performance of health systems. Since improving health is the defining goal of the health system, we report performance in terms of that goal. Data sources have been given elsewhere.¹⁰

Methods

Theory

Efficiency is defined as the ratio of the observed level of attainment of a goal to the maximum that could have been achieved with the observed resources. Normally, outputs are zero when inputs are zero. In health, however, health levels would not be zero if there were no health expenditures—that is, no health systems. So to measure the contribution of the health system we have to determine what it achieves in excess of what would be achieved in its absence (the minimum). Accordingly, we define performance as the current level of population health, in excess of the estimated minimum, compared with the maximum achievable level of health given the inputs. Because of the similarity between performance and efficiency, we use the terms interchangeably.

Neither the maximum (frontier) nor the minimum levels of health are observable, so they have to be estimated. Two strategies could be used for estimating the maximum. One involves defining feasible interventions, identifying their costs and outcomes, and choosing those that maximise health for the available resources. This approach has not been widely used¹¹,¹² because of data limitations but is currently being pursued by the World Health Organization.¹³

The second approach, which we have used here, estimates the maximum from a sample of observed inputs and outcomes. This approach requires the relation between outcomes (population health) and inputs to be specified. We estimated this relation with a form of regression analysis that shows how health levels vary with inputs. The country with the highest health level, after controlling for inputs, is the most efficient. The maximum is the level of health the most efficient country would have produced at each observed combination of inputs. Efficiency of other countries is measured with respect to the maximum. Inefficiencies might be from wastage or because the most cost effective set of programmes or interventions are not used. Further details of the method are given on the BMJ 's website.

Data

We estimated the efficiency of 191 countries from data for 1993-7. Population health was measured as healthy life expectancy (box). The health system input was health expenditure per capita measured in 1997 US dollars (adjusted for the cost of a generic basket of goods in different settings).

Healthy life expectancy

Healthy life expectancy builds on the concept of life expectancy. Life expectancy is adjusted to allow for the fact that people live part of their lives in less than full health. These states are given weights between 0 and 1 to reflect their severity compared with full health (valued at 1). In rich countries, between 7 and 10 years are typically spent living in less than full health. Partly because of a longer life span, women spend more time in poor health than men do. In poor countries, people may spend over 20 years of their expected life span in poor health. Taking into account these weights, ill health and its consequences reduce healthy life expectancy by between 5 and 11 years across 191 countries.

Levels of health are not solely affected by health systems.⁹ The most widely accepted other determinant is education, which is strongly associated with the health of both children and adults in developed and developing countries.¹⁴,¹⁵ Educated people translate information and health services into health more effectively than uneducated people do. We used a summary indicator of educational attainment—average years of schooling in the adult population.¹⁰

We did not include income per capita because income is highly correlated with both health expenditure and education and complicates statistical estimation. Moreover, income does not directly contribute to health but acts through factors such as education, housing, and food intake. Inclusion of the part of income acting through mechanisms other than health expenditure and education made little difference to our results—the rank order correlation of efficiency scores was >0.99.

We estimated the minimum achievable health in the absence of a health system from observations on 25 countries before the existence of a modern health system (average year, 1908). Health levels were correlated mainly with literacy. We estimated the minimum health level for 1997 on the basis of current literacy rates as though the 1908 relation still applied.¹⁶

We generated an uncertainty interval as well as a point estimate of healthy life expectancy for each country.⁹ For all countries, we randomly drew observations from the uncertainty distributions and estimated efficiency and rank, repeating the procedure 1000 times with slightly different results. The reported efficiency estimate is the mean score for that country, and the uncertainty interval represents the range in which estimates fell, omitting the bottom and top 10%. Rank was based on mean efficiency, and rank uncertainty intervals were generated in a similar manner.

Results

Table 1 gives the coefficient estimates used in the regression equation to determine efficiency. We investigated numerous specifications of the regression equation, but they gave stable estimates of efficiency and rank.

Table 1.

Coefficient estimates used in calculation of maximum health, logged variables, 191 WHO member countries, 1993-7

Coefficient estimate	Median	Mean	Uncertainty interval (95%)
Health expenditure	0.0089	0.0089	0.0087 to 0.0090
Average years of schooling	0.0630	0.0630	0.0588 to 0.0673
Square of average years of schooling	0.0217	0.0217	0.0203 to 0.0232
Constant	4.0260	4.0269	4.0159 to 4.0397

Table 2 shows the efficiency and ranks for the highest and lowest 10 performers and the United Kingdom. Estimated efficiency varies from 0.08 to nearly 1, implying that although some countries may be close to their potential, others are not reaching anywhere near maximum levels of health. Figure 1 depicts the efficiency for all countries; the full results are available on the BMJ 's website.⁹

Table 2.

Efficiency (performance) score and 80% uncertainty intervals, highest and lowest ranking 10 countries and United Kingdom, 1993-7

Rank	Uncertainty interval	Country	Performance	Uncertainty interval
1	1 to 5	Oman	0.992	0.975 to 1.000
2	1 to 4	Malta	0.989	0.968 to 1.000
3	2 to 7	Italy	0.976	0.957 to 0.994
4	2 to 7	France	0.974	0.953 to 0.994
5	2 to 7	San Marino	0.971	0.949 to 0.988
6	3 to 8	Spain	0.968	0.948 to 0.989
7	4 to 9	Andorra	0.964	0.942 to 0.980
8	3 to 12	Jamaica	0.956	0.928 to 0.986
9	7 to 11	Japan	0.945	0.926 to 0.963
10	8 to 15	Saudi Arabia	0.936	0.915 to 0.959
24	21 to 28	United Kingdom	0.883	0.866 to 0.900
182	181 to 185	South Africa	0.232	0.209 to 0.251
183	181 to 185	Sierra Leone	0.230	0.213 to 0.247
184	181 to 186	Swaziland	0.229	0.205 to 0.255
185	182 to 187	Democratic Republic of the Congo	0.217	0.198 to 0.235
186	183 to 188	Lesotho	0.211	0.187 to 0.236
187	186 to 188	Malawi	0.196	0.181 to 0.211
188	187 to 189	Botswana	0.183	0.172 to 0.194
189	185 to 189	Namibia	0.183	0.152 to 0.214
190	190 to 190	Zambia	0.112	0.095 to 0.129
191	191 to 191	Zimbabwe	0.080	0.057 to 0.103

Figure 1.

Global distribution of health system efficiency (performance) in maximising population health

Figure 2 shows that efficiency is positively related to health expenditure per capita, especially at low expenditure. Performance sharply increases with expenditure up to about $80 (£53) per capita a year.

Figure 2.

Efficiency (performance) in maximising population health versus health expenditure per capita, with 80% confidence intervals

Discussion

Perceptions about the relative performance of health systems in different countries have been based on anecdote or case studies. For example, Sri Lanka and China are believed to have been efficient in producing health,¹⁷,¹⁸ but our results show that both perform less well than other countries at similar levels of development. On the other hand, Oman performs extremely well—perhaps because it has reduced child mortality from 310 to 18 per 1000 live births over the past 40 years.¹⁹

Our efficiency scores compare current population health levels with the maximum possible for observed levels of health expenditure and education in a country. This does not mean that 100% efficiency can be reached immediately. There will be time lags between some actions and their outcomes, and efficiency in many low performing countries is hampered by civil unrest or a high prevalence of HIV and AIDS (fig 1). Healthy life expectancy is reduced by up to 15 years in African countries with the highest prevalence of HIV, clearly restricting the ability of these systems to reach full efficiency in the short term.

Validity of findings

Although other non-health variables affect health (housing quality, environmental conditions, etc), relevant indicators are difficult to find or estimate for many countries. In addition, many are highly correlated with educational attainment, which we used because it functions as a broad measure of non-health inputs.

Omission of non-health variables reduces the estimates of efficiency. On the other hand, the measurement strategy biases estimates upward. The fact that five countries have efficiency scores >0.97 does not mean they can improve performance by nearly 3%. It means they could improve by 3% compared with the most efficient country, but we have no way of estimating the potential of the highest performer to become more efficient. Microlevel studies suggest the potential is there nevertheless.

Reasons for inefficiency

We found that efficiency is positively related to health expenditure per capita. Performance increased greatly with expenditure up to about $80 per capita a year, suggesting it is difficult for systems to be efficient at low expenditure. There seems to be a minimum level of health expenditure below which the system simply cannot work well. We estimate it would cost just over $6bn a year (<0.3% of global annual health expenditure) to increase health spending to this threshold in the 41 countries with lowest expenditures.

Despite the need to increase funds in poor countries, there is enough variation in efficiency at all levels of expenditure to suggest that using current resources better could improve health considerably. Reducing wastage is one way, but the studies of Tengs and Murray et al show that allocation of resources is also important.¹²,²⁰ They argued that health in the United States and sub-Saharan Africa could be greatly improved by reallocating available resources from interventions that are not cost effective to those that are more cost effective but not fully implemented.

Another possible reason for inefficiency is that goals other than health may be deemed important. The World Health Report 2000 recognised that countries may also wish to reduce inequalities or increase the responsiveness of the system.⁹ The efficiency of health systems in achieving all defined goals has been explored elsewhere.²¹ The analysis produced some changes in rank—for example, France was estimated to have had the most efficient system overall—but, in general, countries efficient in producing health are also efficient in producing other goals.²¹

Future research

Our conclusions are, of course, tentative. The quality of data across countries varies greatly, and only some of this is accounted for in our uncertainty analysis. Our main objective was to show that the attainment and efficiency of health systems can be measured and compared across countries and over time. Much can be done to improve the data and methods, and WHO is currently working on this with member countries and academic experts. We believe this is critical work for health policymakers considering reforms. Without the ability to measure the inputs and outputs of health systems, they cannot know if the reforms achieve their objectives.

Editorial by McKee