Health Care System Reforms in Developing Countries

Abstract

This article proposes a critical but non-systematic review of recent health care system reforms in developing countries. The literature reports mixed results as to whether reforms improve the financial protection of the poor or not. We discuss the reasons for these differences by comparing three representative countries: Mexico, Vietnam, and China. First, the design of the health care system reform, as well as the summary of its evaluation, is briefly described for each country. Then, the discussion is developed along two lines: policy design and evaluation methodology. The review suggests that i) background differences, such as social development, poverty level, and population health should be considered when taking other countries as a model; ii) although demand-side reforms can be improved, more attention should be paid to supply-side reforms; and iii) the findings of empirical evaluation might be biased due to the evaluation design, the choice of outcome, data quality, and evaluation methodology, which should be borne in mind when designing health care system reforms.

Significance for public health

There has been a growing awareness that improving population health is of significant importance to the development of a society, particularly for developing countries where the health care system is underdeveloped, fragile, or vulnerable. In the past 10 years, health care system reform has been under way in a number of developing countries with a special focus on the poor or disadvantaged groups, who face financial barriers to have access to health care services and are exposed to financial risk due to illness. Both of these can result in the medical poverty trap. In this article we sum up relevant empirical evidence and learn from past experience in order to improve future health policy-making. Knowing what really works will allow the design of better health care systems in the future, which in turn will improve people’s health.

Introduction

The 2000 World Health Report pointed out three fundamental objectives of health systems, namely improving the health of the populations they serve, responding to people’s expectations, and providing financial protection against costs due to illness. In addition, it stressed the poor’s need for financial protection being as great as or greater than well-off people’s, since even small absolute risks may give birth to catastrophic consequences for the poor.¹ In recent years, many lower- to middle-income countries, including China, Vietnam, India, Colombia, Mexico, Thailand, Kenya, Ghana, and Zambia, have been reforming their health systems to promote universal access to health care, and improving equity of health, quality of health service, and fairness of financing. Most of the reforms appeal to social health insurance as the main approach to improve health care systems so as to protect the poor. Empirical results have shown that some of these health reforms do provide financial protection to the poor but some do not. Mexico, Vietnam, and China are chosen here as representatives of health care system reforms to allow for discussion and comparison. Mexico has been regarded as one of the most successful cases of health care system reform in developing countries. In contrast, empirical evidence criticises the effort of the Chinese government, which has poured a sum of public finance into reforms and has planned to budget more. Vietnam, besides its similarities with China as for social and demographic background, has been implementing the compulsory social health insurance programme to the identified poor, which provides an opportunity to compare the effectiveness of public finance allocation. All three countries launched social health insurance for the uninsured poor – Seguro Popular (SP) in Mexico, Health Care Fund for the Poor (HCFP) in Vietnam, and New Cooperative Medical Scheme (NCMS) in China – around 9 years ago, thereby leaving sufficient time for policy evaluation and publication.

The paper is organised as follows. The following section describes the health care system reforms in Mexico, Vietnam, and China, and summarises their evaluation results. Then, we compare the three reforms in order to discuss the reasons why they differ one another. These are described along two lines: policy design and evaluation methodology. Policy design is further elaborated from background differences, the design of health care system reform, the eligibility of social health insurance for the poor, the benefit package of social health insurance, and the effects of insurance on health-seeking behaviour. Evaluation methodology covers the choice of outcome variables, data quality, and evaluation methodology. The last section draws conclusions.

An overview of health care system reforms

Mexico

The 2000 World Health Report stated that catastrophic health expenditure in Mexico is one of the major problems in the country.¹ At that time, more than half Mexican citizens were potentially exposed to health-related financial risk, which was caused by social inequalities in the developmental process but also contributed to deteriorate the existing social inequalities.² Since 2004, Mexico has been implementing a major health system reform and has established the System of Social Protection in Health (SSPH) to overcome the health-related problems of the uninsured. This was done according to the idea that health care is a social right rather than a commodity or a privilege.³ An empirical study conducted in 2002 showed excessive health-related spending for the poorest rural families in Mexico, most of which was attributed to outpatient care and medication.⁴

Seguro Popular (SP), the main innovation of the reform, aims at providing financial protection associated with illness to those who are not covered by any other public health schemes.^a Anyone who has not been receiving benefit from social security is eligible, regardless of employment status. SP is a subsidised voluntary insurance that offers free access to an explicit set of health care services at the delivery point. The three sources of contribution are: federal government, state government, and families. Families are supposed to contribute by paying a certain premium determined by family income. However, almost no family has virtually made premium contributions.^b In 2005, slightly more than 1 billion USD were invested in SP, three fourths of which was contributed by the federal government and the rest by the state governments.⁵ In addition to the necessary stewardship reform on the organisational structure, Mexico has built up a comprehensive monitoring and evaluation system to support the policy design and implementation of SP, such as publishing a benchmark report on an annual basis and conducting a longitudinal survey from 2005 purposely designed for the purpose of evaluation. The issues related to the provider payment system are not relevant since all the health interventions and drugs defined in the benefit package are access-free. As an example, in 2006, 266 unique health interventions and 312 medicines were access-free. It is worth noting that, before the reform, the uninsured population did have access to health facilities run by the states on payment of a user fee, which brought about the high proportion of out-of-pocket expenditure. Moreover, the uninsured may have suffered extra out-of-pocket expenditures due to the shortages of drugs resulting from budgetary limitations.³ SP made progress in covering more people, more interventions, and more conditions, with better quality.

A study on a 15-year trend regarding the evolution of catastrophic and impoverishing health expenditure suggests that the reduction in out-of-pocket health expenditure and catastrophic health expenditure by households is related to the expansion of SP, although no causality conclusion can be drawn given the available data. Moreover, the study found that some key components of SP - insuring the poorest quintiles, covering medications and ambulatory care, and including a package of catastrophic expenditures - are effective strategies to reduce catastrophic and impoverishing health expenditure.⁶ A causal effect study analysed the impact of SP on the incidence of catastrophic health expenditure and out-of-pocket health expenditure among poor households using non-experimental data from three different data sources, and reported significant reduction of households’ expenditures on medicines and outpatient care, regardless of the data sources. However, the effect on catastrophic health expenditure differs from data resources.⁷ A matched-pair cluster-randomised experiment has confirmed the effectiveness of SP in reducing overall catastrophic and out-of-pocket expenditure, especially for the poorest individuals. Contrary to other observational studies, there was no favourable evidence on medication spending, health outcomes, and health care utilisation, which may be attributed to the relatively short duration of treatment (10 months).^c In other words, a change might take place over a longer period,⁸ given the time lag existing between launching a new health policy and changing the health-related behaviour of those treated.

Vietnam

Like other developing countries in Asia, out-of-pocket expenditure is the dominant source of health financing in Vietnam.⁹ Health care system reform in Vietnam focuses on the promotion of social insurance, which can be dated back to 1992. By the end of 2002, a compulsory social health insurance programme targeting all poor households and selected disadvantaged groups called HCFP, had been formally set up.¹⁰ The eligible population embraces the poor and some socially protected groups, such as people of merit, the elderly, and war dioxin victims.^d All the eligible can be clearly figured out except for the so-called poor, who are virtually identified by existing lists of other government programmes as well as household surveys.¹¹ It is worth noting that self-employed workers, informal sector workers, and dependents of CHI members cannot benefit from this programme.^e The HCFP is 100% funded by public finance. Specifically, the central and provincial governments were to contribute 2.5 USD and 0.84 USD, respectively, per beneficiary per year, though in practice few provinces have done so.¹² The programme covered around 15 million people by 2009, ¹⁰ which resulted in a government investment of about 50 million USD. The insured can benefit from a relatively broad - covering outpatient as well as inpatient services at all health care levels, laboratory exams, x-ray, and so on - but unspecified benefit package.¹¹,¹³ Fee-for-service (FFS) is used for both outpatient care and inpatient care, which may cause the supply-induced demand for health care in the context of the social insurance. In other words, it is highly possible that physicians are inappropriately incentivised to over-treat patients. Although some alternative provider payment methods have been suggested and experimented with, such as capitation, they have not been rolled out in practice. In 2003, Vietnam Social Security (VSS) was founded to administer all social insurance programmes, including collecting premiums, issuing health insurance cards, and reimbursing service providers.

The literature evaluating the impact of HCFP is limited as well as mixed. By comparing households inside and outside the programme and employing propensity score matching on a trimmed cross-section of data, Wagstaff found that HCFP may be well targeted to Vietnamese poor since it had reduced the risk of catastrophic out-of-pocket expenditure but affected the average out-of-pocket expenditure incospicously.¹² However, the same author applied triple differencing with matching to a richer dataset consisting of two rounds of data prior to the programme’s implementation (2002 and 2004) and one round after (2006). The result is robust to the bias caused by unobserved heterogeneity. He concluded that HCFP had a considerable positive impact on the reduction of out-of-pocket health expenditure but not on health care utilisation.¹⁰ In contrast, Axelson et al. applied propensity score matching with both single differences and double differences to preintervention (2002) and post-intervention data (2004) from the Vietnam Household Living Standards Surveys.¹⁴ The authors reported a small, positive impact on health care utilisation, but a strong negative impact on out-of-pocket health expenditure.

China

In 2003, the Chinese government started a co-payment voluntary insurance system subsidised by the central government and provincial governments (NCMS), which aims at preventing rural population from being impoverished by catastrophic health expenditure. All rural residents are eligible for NCMS. Given its voluntary nature, participation is required at the household level to avoid adverse selection. According to statistical data provided by the Ministry of Health of China, by the end of 2009 NCMS had already covered more than 0.83 billion people (94% of the target population).

Although the minimum standard is regulated by the central government, detailed schemes – such as family contribution, local government subsidy, and benefit package – are designed at the discretion of the local governments according to their peculiarities. Hence, this system brings about considerable heterogeneity across regions. At the onset of NCMS, the minimum required contribution per person per year was 4.8 USD. This sum was evenly split among central government, local governments, and households. In 2006, the central and local governments increased their minimum contribution to 3.2 USD, while the family contribution remained unchanged.¹⁵ Therefore, the total government contribution was approximately 2.6 billion USD.^f Obviously enough, the benefit package could not be sufficiently generous in many regions due to limited financing. In other words, some services were not covered or were only partially covered, deductibles and coinsurance rates were high, while ceilings were low.¹⁷ Moreover, although all counties covered inpatient care, outpatient care did not receive enough attention. In 2009, the Chinese government committed an additional 125 billion USD investment to the health care sector over the next 3 years.

Several papers using different techniques have studied the impact of this reform on welfare and universal coverage. Sun et al. used a subsample of households which had suffered catastrophic health expenditure during 2004 to measure the impact of the NCMS by counterfactual analysis. To do this, the authors compared households’ catastrophic payments before and after NCMS reimbursements. Their study revealed that, despite a reduction in payments, the majority of the households’ expenditures remained catastrophic.¹⁸ Zhang et al. took a random sample of NCMS enrolees who had obtained reimbursement to apply counterfactual analysis. They found that NCMS could partially reduce catastrophic health expenditure and the reduction was in favour of the poor.¹⁹ A significant shortcoming of these counterfactual studies is that they did not consider the change of patients’ health care-seeking behaviour and providers’ demand-inducing behaviour after the implementation of the health insurance scheme. Wagstaff et al., combining difference-in-difference with propensity score matching, showed that introducing NCMS did not appear to reduce out-of-pocket health expenditures, whereas Wu reached an opposite conclusion by employing a Tobit model. The difference in these results may come from the estimation method used, the sample, and the information set.¹⁷,²⁰ Lei and Lin applied an individual fixed effect model, instrumental variable method, and propensity score matching with difference-in-difference estimation, respectively, to three-wave panel data and found no evidence that NCMS participation could relieve the financial burden, as measured by out-of-pocket expenditures among patients. On the contrary, the results of Babiarz et al. are in line with those of Wu.^20-22

Finally, social experiments have been conducted to detect potential problems and explore the possible improvement of NCMS. A social experiment called Rural Mutual Health Care (RMHC) was conducted in 2006 and it i) provided first-dollar coverage for primary care, hospital services, and drugs with a similar premium but a lower ceiling; ii) changed the provider payment method for village doctors from FFS to salary plus performance-based bonus; and iii) introduced bulk purchasing for drugs. Yip and Hsiao compared NCMS with RMHC using a static simulation model and suggested that RMHC was more effective at reducing medical impoverishment than NCMS.²³ However, their study neglected the behavioural responses of patients and providers under the different benefit package design. Besides, data were based on the sample which voluntarily enrolled in RMHC, whose health expenditure distribution might be higher than that of the general population. In addition, the World Bank financed a health system reform pilot in Gansu (China), which was composed of supply-side interventions aiming at improving the effectiveness and quality of care as well as demand-side interventions aiming at expanding health insurance and providing financial support to the poor. By combining differences-indifferences with propensity score matching, Wagstaff and Yu reported that the health system reform pilot did reduce out-of-pocket health expenditure, as well as the incidence of catastrophic health expenditure and impoverishment.²⁴

The reasons for the difference

In this section we attempt to explain why the results of health care system reform vary from Mexico, to Vietnam, and China. Our discussion will focus on two aspects: policy design and evaluation methodology.

Policy design

The different outcomes observed may derive from several problems related to the way policies were designed. In this article we focus on the following: i) background differences, such as social development, poverty level, and population health; ii) the design of health care system reform, including the intervention policy on the demand side as well as the supply side, and policy implementation and monitoring; and iii) target population, design of the benefit package, and theoretical foundation of social health insurance when focusing on the demand side of the social insurance reform.

Background differences

Background differences across countries may be one the most important causes of policy differences. Namely, importing the successful experience of other countries without taking background factors into account may not bring about the expected results. Table 1 compares some indicators for social development, poverty level, and population health of Mexico, Vietnam, and China. In terms of social development, Mexico was on average far better off than China and Vietnam. Gross domestic product (GDP) per capita in Mexico was more than five times that of China and even more than ten times that of Vietnam. Approximately 4% of Mexicans were living below the poverty line – only one seventh and one tenth the proportion of Chinese and Vietnamese, respectively. However, Mexico had the most serious income inequality as among households the poorest (10%) represented 1.8% of total consumption while the richest (10%) consumed nearly 40%. As far as population health is concerned, birth rate in China was much lower than in the other countries due to the one-child policy. In Mexico per capita health expenditure was much higher than in the other countries, which is consistent with its economic condition. Despite this, public health expenditure as a percentage of total health expenditure and total health expenditure as a percentage of GDP did not differ too much on another across the three countries. Besides, all the three countries heavily relied on private out-of-pocket expenditure to finance health care before introducing the health system reform.

Table 1.

Social development, poverty level, and population health (in 2003, unless otherwise specified). Data source and indicators: The World Bank Database. Available from: http://data.worldbank.org/indicator.

	Mexico	Vietnam	China
Population
Population, total (millions)	103.9	80.5	1288.4
Urban population (% of total, 2005)	76	26	40
Population ages 0-14 (% of total)	32	29	23
Population ages 15-64 (% of total)	62	65	69
Economy and poverty
GDP per capita (USD)	6740	492	1274
Income share held by highest 10% (2002)	39.4	30.3	31.7
Income share held by lowest 10% (2002)	1.8	3.3	2.3
PPP (% of population, 2002)	3.9	40.1	28.4
Population health
Birth rate, crude (per 1000 people)	22	17	12
Death rate, crude (per 1000 people)	5	5	6
Life expectancy at birth, total (years)	75	73	72
Improved sanitation facilities (% of population with access)	78	62	51
Improved water source, urban (% of urban population with access)	95	95	98
Improved water source, rural (% of rural population with access)	81	78	75
Health expenditure
Health expenditure per capita (current USD)	392	26	61
Health expenditure, public (% of total health expenditure)	44.2	31.4	36.2
Health expenditure, total (% of GDP)	5.8	5.3	4.8
Out-of-pocket health expenditure (% of private expenditure on health)	94.7	89.6	87.6

GDP, gross domestic product; USD, US dollar, PPP, poverty headcount ratio at 1.25 USD a day.

To sum up, these three countries were not on the same level when the health system reforms were launched for Mexico was much better as for economy and social development at that time. Therefore, although Mexico has demonstrated its great achievement in health care system reform, we should be cautious not to apply its experience in a dogmatic way. For instance, if poor people are extremely poor, more resources are required to make a significant improvement.

Design of the health care system reforms

Most of the health care system reforms in developing countries focus on demand-side intervention, mainly realised by the introduction or extension of social health insurance. The cases of Mexico, Vietnam, and China all fall into this category. As for the provider payment method, FFS is still widely used in practice. FFS basically incentivises health care providers to over-treat patients in order to pursue their own profit. Simultaneously, as for the demand side, the poor are much more likely to be over-treated when being insured, owing to the indirect elevation of their financial affordability. The interaction between supply side and demand side may enlarge the so-called supply-induced demand, which has been empirically evidenced. Most evaluation studies have been in favour of Mexico’s SP, with regard to its financial protection of the enrolees. One significant difference between SP and the other reforms (NCMS and HCFP) lies in the former offering free access to an explicit set of health care services at the delivery point. Obviously, this means that it is impossible to incur supply-introduced demand for the care covered in the benefit package. Besides, the common ground of two successful social experiments in China is that the supply-side intervention was well embedded in the demand-side intervention.²³,²⁴ These evidences suggest the importance of building up a right and balanced incentive structure in the health care system. Specifically, more attention should be paid to the design of the supply-side policy.

Implementation as well as evaluation play an essential role in the health care system reforms. Hence, they should be integrated into the design of the health care system which Mexico has explicitly planned to enlarge the scope of monitoring and evaluation, and which is financially supported by the regular budget from the Ministry of Health. Longitudinal surveys have been conducted since 2005 for the purpose of measuring the impacts of SP on health conditions, effective coverage, health-system responsiveness, and financial protection.³ Ekman et al. pointed out that the capacity of policy evaluation should be further strengthened and developed in Vietnam to pave the way for a successful health care system reform.¹¹ As already described, in China, NCMS has a decentralising design and implements local governments. These features put NCMS at a disadvantage in conducting a valid monitoring and evaluation. Patently enough, the county-specific policy must result in the lack of terms of comparison. Therefore, there is no strong incentive for the local government to conduct evaluation and improve implementation.

The eligibility of social health insurance for the poor

Among the three health care reforms, Chinese NCMS and Mexican SP belong to the universal programme in that all members of a given population are eligible to receive programme benefits: all rural residents in China are eligible for NCMS, while all those who are not covered by any other public health schemes in Mexico are eligible for SP. However, Vietnamese HCFP falls into the category of a targeted programme, which restricts benefits to some identified subgroups of a given population. In the case of HCFP, only the identified poor and some socially protected population groups are eligible.

Generally speaking, government subsidy plays a considerable and crucial role in promoting social health insurance in developing countries. In theory, by targeting limited resources to those identified as having the greatest needs, health care programmes improve equity in a more efficient way. However, to have a comprehensive account of the situation, we need to recall that: i) NCMS is a voluntary programme with a contribution requirement from the enrolees; ii) HCFP is a compulsory programme without any contribution requirement from the eligible; and iii) SP is a voluntary programme with a contribution requirement from family but, in practice, almost no families made premium contributions. Seemingly, HCFP as well as SP should be more effective and favourable to the poor than NCMS since some of the poor may not be able to benefit from NCMS due to the obligation of contribution. From another angle, although HCFP and SP thoroughly broke the financial barrier to health care utilisation, they may encounter shortcomings such as the limited scope of coverage.

Where the intervention targets some specific group, it raises an important issue called identification. In most cases of HCFP implementation, the official poverty line was employed as an instrument to identify the poor. This may bring about at least two problems. First, by assuming that the monetarily deprived are those the government wants to target, the arbitrary attribution of the poverty line leads to random assignment of HCFP coverage around the threshold. Indeed, the households just below or above the threshold have similar characteristics but opposite coverage status. Second, it is debatable whether we should narrowly define the monetarily poor as the poor. Nowadays, there is a growing literature discussing multidimensional poverty measurement. Although a consensus about how to measure poverty multidimensionally has not been reached yet, it has been generally accepted that the multidimensional perspective should be taken when measuring poverty.^25-28

The benefit package of social health insurance

One of the key components of social health insurance is the size of the benefit package, that is, which health care services are included, whether the included health care services are completely access-free, and if not, to what extent patients share the costs. There is an ongoing discussion about which health care services should be covered.²⁹ Some literature or policy documents propose to base this on cost-effectiveness or cost-utility analysis.³⁰ Other authors argue that the government should not try a vain attempt to provide everything for everyone with its limited public finance and health resources, particularly in developing countries.³¹ Among these authors, Baltussen has proposed a step-wise approach to decide the targeting and prioritisation of public spending.³² There is also some literature focusing on addressing the significance of financial protection in the benefit package design.³³ On the whole, these debates imply a trade-off between efficiency and equity in the policy design. In practice, these approaches do not mutually exclude each other, which means they can be applied in a consolidated way and tailored according to the specific circumstances. In this respect, SP, HCFP, and NCMS vary significantly. SP has an explicit benefit package consisting of 249 basic and 17 costly interventions, all of which are free of charge at the delivery point. Out-of-pocket health expenditure is mainly required when taking interventions that are not included in the benefit package. In contrast, HCFP and NCMS involve patient cost-sharing. HCFP has a broad but undefined benefit package covering outpatient and inpatient services and even some high-tech treatments, which might be financially unsustainable in the long run even with patient cost-sharing. Ekman advised providing a more focused benefit package with affordable and cost-effective interventions.³⁴ NCMS is much more favourable to inpatient services than to prevention and primary care. This system provides patients with the wrong incentive to over-consume hospital care and under-consume basic primary health care. Moreover, as provincial and county governments are primarily responsible for the design of the NCMS benefit package, the design is very likely to be ineffective and unscientific.²³ A comprehensive account of the key facts about social health insurance are provided below in Table 2.

Table 2.

Key facts about social health insurance.

	Mexico	Vietnam	China
Enrolment	Voluntary	Compulsory	Voluntary
Eligible population	Anyone who has not been receiving benefit from social security	The poor and some socially protected groups identified by governments	All rural residents
Contribution requirement from family	Theoretically yes, but practically no	No	Yes
Scope of benefit package	Explicit benefit package: 249 basic and 17 costly interventions	Broad but undefined benefit package; Covering outpatient and inpatient services and even some high-tech treatments	More favourable to inpatient ervices than to prevention and sprimary care; designed by local governments
Patient costing sharing	No	Yes	Yes

The effects of insurance on health-seeking behaviour

The effects of social health insurance heavily depend on how the benefit package interacts with the severity/cost of disease as well as with patients’ income. We can illustrate this interaction by employing a partial rather than a general equilibrium model, i.e. only considering the possible reaction of an individual to the introduction of a social insurance scheme without taking into account that the change of individuals’ health-seeking behaviour might affect the price of the health care (more broadly, the financing system of social health insurance). More specifically, we consider a utility-maximisation individual living in two periods: the current period when they fall ill (period one) and the following period (period two). Their income is discounted in the sick period(s) and only the total disposable net income of medical spending directly contributes to their utility. However, the probability of becoming healthy in the subsequent period only increases with the medical spending in the previous period. Put it differently, the medical care matters indirectly in their utility maximisation but with a time lag. Therefore, they spend on medical care in period one but not at all in period two, regardless of their health condition. We may conclude this after assuming that the government made participation in subsidised social health insurance compulsory and everyone paid a periodic flat premium in return for partial reimbursement once any medical spending occurred.

Given this simple but not too impractical model, introducing social health insurance may not necessarily improve the health conditions and/or income of the poor. The intuition behind this model is that, for all individuals, the first effect of launching a compulsory health insurance is to reduce their disposable income due to the premium charge. Then, the benefits in return depend on how generous the insurance programme is. It goes without saying that what matters is to what extent the government is willing to invest in or subsidise and that the introduction of social health insurance might create what the literature refers to as a medical poverty trap. In the appendix we will show how the poor could be worse off under some circumstances.

The poverty trap is defined as the self-reinforcing mechanism which causes poverty to persist.³⁵ Many factors can lead to the poverty trap, such as limited access to credit and capital markets, corrupt governance, poor education systems, lack of public health care, or poor infrastructure.³⁶ The term medical poverty trap has been coined to describe the negatively dynamic relationship between ill-health and poverty. Four possible consequences resulting from the medical poverty trap are: untreated morbidity, reduced access to care, long-term impoverishment, and irrational use of drugs.³⁷ Figure 1 presents the pathway of the medical poverty trap. The scheme implies that poor patients may be trapped regardless of their health care seeking. Indeed, health care seeking notwithstanding, poor patients have to cope with the effect of the reduced disposable income for other consumptions, which in turn increases poverty. This cycle may not be broken by means of introducing social health insurance. If the benefit package is not sufficiently generous for the household income and health care expenditure and if there is any obligatory contribution, the poor could be pushed into an even deeper poverty trap. On the contrary, if the benefit package is sufficiently generous, the poor could seek care without suffering an unaffordable or over-burdened cost. It is worth noting that although some models have been developed to describe the poverty trap caused by various factors, such as human capital externalities and child labour,^38-40limited progress has been made in modelling the medical poverty trap, which means that this component lacks theoretical foundation.

Figure 1.

Pathway of medical poverty trap.

Evaluation methodology

Given that most studies evaluating social health insurance reform are empirical works, the selection of outcome variables, the availability and quality of the data, and the choice of empirical methodology are of high relevance to the evaluation and to the future policy design.

Outcome variables

Catastrophic health expenditure,⁵,⁶,¹²,¹⁸,¹⁹ or out-of-pocket health expenditure, has been often used as the financial outcome variable in the literature.³,¹⁰,¹⁷,²⁰,²² However, the definition of catastrophic health expenditure itself has a few limitations which may mislead policy makers. According to the common definition, a household’s health expenditure is considered to be catastrophic if the ratio between the household’s out-of-pocket health expenditure and its disposable income exceeds a certain threshold. A portion of 30% or 40% of capacity to pay,^g or 10% of total income, has been widely employed in practice.⁴¹,⁴² These definitions are unable to capture the need of the extremely poor households which did not seek care or of those households which did not get sufficient care. Another limitation is that they only focus on the potential financial risk caused by out-of-pocket health expenditure in a short run. The introduction of social health insurance improves the health care utilisation of the poor, which in turn may increase the share of out-of-pocket health expenditure. If this increase exceeds a certain predetermined arbitrary threshold, it is classified as falling into the ‘catastrophic’ group, and consequently results in an unfavourable evaluation with regard to financial protection. Nevertheless, in the long run, the poor may be better off when they are enrolled in the social health insurance thanks to the positive correlation between health and income. Moreover, as for the same out-of-pocket health expenditure share, relatively rich people could be much better off compared with the poor since the rich cope with difficulties in a more flexible way.

The ultimate goal of promoting health care system reform or social health insurance is neither to lower the financial risk arising from out-of-pocket health expenditure nor to increase health care utilisation. Ruger states that clearly wealth is not the good we are seeking and human flourishing is the end of all social activity.⁴³ Evaluating the impact of health policy only considering certain unidimensional outcome variables might be injudicious. Instead, it might be better to jointly consider and evaluate the multiple impacts of social health insurance within a multidimensional well-being framework. This framework should encompass not only monetary-related but also health-related dimensions, since, as previously discussed, they dynamically influence each other but jointly contribute to the overall well-being.^h

Data

Chapman and Boothroyd argue that the quality of data used in analysis has been frequently overlooked.⁴⁶ This issue may be more serious in the setting of developing countries due to the limitations of external researchers, such as insufficient experience or knowledge of local conditions. Besides, survey data are mainly employed in health care system evaluation. Consequently, the influence of survey errors (e.g., frame deficiencies, deficient sampling process, inadequate interviewing and interviewers, respondents, missing data, coding, etc.) becomes inescapable.⁴⁷ Without ensuring the quality of the data, findings can be biasing or misleading, regardless of how sophisticated the methodology. Consequently, this may result in irrational decisions of policy makers. Most of the evaluation is virtually based on the data from general household surveys. Mexico is an exception as it has been innovatively collecting longitudinal data to evaluate the design and the implementation of SP, thus providing well-tailored information so as to benefit further policy design.

Methodology

Evidence-based policy evaluation aims at deriving causal inferences, i.e. at demonstrating the association between two events, where the second is the consequence of the first. However, at a given time an individual is either exposed to the intervention or not. Thus, we can evaluate the average impact of a policy by comparing the outcomes of two groups, where only one is exposed to the intervention. In this case, inferences about causal effects can be drawn only if, without any intervention, these two groups would have similar outcomes, which is rarely met in reality. Otherwise, the overall difference between the groups consists of the impact of the intervention as well as the selection bias.⁴⁸ Randomised experiments are seen as the gold standard for drawing causal inference. Their essential advantage is that, theoretically, they allow to create a control group having the same distributions of both observed and unobserved characteristics as the treatment group so that the selection bias problem can be naturally solved. Nevertheless, experimental evaluation is at a disadvantage against non-experimental evaluation in terms of cost. In addition, its validity and reliability could be threatened by some political issues, the inconformity between assigned treatment and received treatment, the poor commitment of treatment groups and control groups, and so on. King and his colleagues have proposed an approach to design a randomised experiment that is politically robust to evaluate the impact of SP. An experiment is politically robust when an evaluation is able to survive even if the experiment is destroyed to some extent due to unexpected political disruptions.⁴⁹ Needless to say that, all other things being equal, the evaluation relying on politically acceptable randomised experiment could provide more robust results.

To this day, the majority of the impact evaluations employ non-experimental data. A wide variety of estimation methodologies have been proposed and discussed in the literature to overcome the selection bias problem, among which difference-in-difference, matching, and the combination of these two are those most widely used. Difference-indifference, a quasi-experimental technique measuring the effect of a treatment at a given period, has been extensively applied in health policy evaluation. The difference-in-difference estimator represents the difference regarding an outcome before and after the treatment between the treatment and the control group. This technique implies the requirement of panel data or repeated cross-section data as well as the specification of treatment and control groups. The strictest assumptionⁱ of the difference-in-difference method lies in the parallel trend assumption: all the time-varying effects are common to both the treatment and the control group, and this can be investigated if multiple period data are available.⁵⁰ Many studies enhance the comparability of treatment and control groups by combining the difference-in-difference approach with matching. However, matching provides a general approach to overcome the observed differences between treatment and control with the assumption that there is no unobserved difference, which seems implausible.

Empirical evidences are various as to whether policy or interventions can be reliably evaluated without conducting a randomised experiment. In terms of widely used difference-in-difference approaches, a study on educational intervention argued that non-experimental results were similar to experimental ones when long series of pre-data were applied.⁵¹ Another study suggested that they could only reduce the bias to some extent but not remove it all.⁵² Furthermore, another comparative study in settings of welfare, job training, and employment service failed to find any approach that could remove the bias considearbly.⁵³ Therefore, more well-designed comparative studies should be encouraged to assess the size and prevalence of selection biases arising from using non-experimental data and provide concrete guidance on how to choose a most robust methodology and interpret causality properly.

Conclusions

In this article we discussed the health care system reforms in developing countries paying special attention to their financial impacts on the poor. To tackle this issue, three representative models were chosen, namley Mexican SP, Vietnamese HCFP, and Chinese NCMS. By comparing these three models we could make some interesting remarks. It is reasonable to comment on the single aspects of a policy design, whereas it is unreasonable to judge the policy design as a whole. This is true beacuse governments always face various country-specific constraints and trade-offs. Taking our models as an example, we should be cautious not to apply the Mexican experience dogmatically, although Mexico has achieved a very good health care system reform. As far as evaluation-related issues are concerned, the selection of outcome variables, the availability and quality of the data, and the choice of empirical methodology are of high relevance. As observed, choosing the estimation methodology and drawing causal inferences should be done cautiously, in particular when non-experimental data are employed. Morevoer, it is of significant importance to check to what extent the underlying assumptions can be satisfied. Simply applying the popular methodology in the literature without checking its suitability and validity under specific circumstances is not recommendable. Otherwise, findings will be biased and will possibly mislead future policy designs. Most of the ongoing health care system reforms focus on demand-side intervention, which primarily work on the scope and depth of social health insurance. Nevertheless, both economic theory and empirical evidence have suggested that health insurance itself may have limited influence on reducing patients’ financial risk. This is the result of the interaction between patients’ incentive to increase the level of health care demand and providers’ incentive to increase the level of demand inducement. Currently, most of the supply-side interventions are still at the stage of small-scale social experiment. However, it is important to give more attention to supply-side intervention (e.g. payment systems) which plays a crucial role in leading health care system reform to the desired success.