Strategies that promote high quality care in Indonesia

Abstract

Objectives

To investigate factors predicting the quality of care received using a nationally representative dataset from Indonesia.

Data Sources

The study combines two surveys in 13 provinces: a household survey of 2451 women who delivered a live birth in 1992-1998, and a facility survey that measured quality available from outpatient providers.

Study design

Multivariate regressions are used to explain the quality of care received. Explanatory variables are high facility quality, maternal education, household wealth, ethnicity, and insurance.

Data collection methods

Facility quality available is measured by adherence to prenatal protocols using a clinical case scenario. Quality received is measured by maternal reports about routine prenatal services received.

Principle findings

High facility quality predicts an increase in quality received. Although poor households have access to the same or higher quality care compared with the least poor, the poor receive lower levels of quality. In remote regions, quality received rises with increasing levels of maternal education and household wealth.

Conclusions

Improving health provider knowledge, and increasing household financial resources and information could redress inequalities in quality received among the poor and least educated.

Introduction

Many developing countries have invested in large-scale health delivery systems to expand access to basic health services. With the exception of a few very poor countries, such investments have led to widespread access to basic services. However, growing evidence points to inadequacies in the quality of care provided. Cross-national research has demonstrated widespread application of ineffective or outmoded curative care practices [1]; inadequate or inappropriate tertiary care for seriously ill children [2]; and variations in the provision of basic prenatal care procedures [3]. Within specific countries, practice variations have been demonstrated by clinical setting and condition [4-7].

Such variation has important health implications. Research in South African hospitals attributed one-half of deaths among severely ill children to failure in adherence to clinical guidelines [8]. In Mexico, case control studies have demonstrated relationships between poor adherence to standard treatment guidelines and preterm delivery [9] and perinatal mortality [10]. In part because of poor prescribing practices and low patient adherence, extensively drug resistant tuberculosis has developed, which is resistant to all second-line drug therapies [11]. The latter example reinforces the importance of involving patients and consumers in strategies to increase quality.

Interventions to improve quality include training, organizational reform, financial incentives for health providers or patients, and patient information, among others. In more developed countries, quality improvement initiatives have emphasized organizational and financing strategies in recognition that health providers and consumers respond to incentives [12-13]. In less developed settings, quality improvement interventions have primarily focused on in-service training to ensure up-to-date knowledge of protocols and practices. Skills are a major determinant of quality, particularly in settings characterized by weak initial or continuing education programs, licensing, and certification. However, where evaluated, the results of in-service training programs have been modest [14]. An evaluation of the Integrated Management of Childhood Illnesses (IMCI) concluded that the impact of training interventions is limited in the absence of policies that support health worker performance [15]. Studies in China, Brazil, and South Africa have highlighted the effects of financial incentives and insurance on practice variation [16-18]. Moreover, consumers and patients have an important role in quality improvement approaches – particularly in settings where the demand for health services is weak. In such settings with low utilization, supply-side interventions could be expected to have only a limited impact on health [19]. There is a need to examine the effectiveness of different options to help inform policy choices.

This study uses an Indonesian dataset to examine the factors associated with the quality of care received. The Indonesian government expanded its network of public health facilities under the principle of universal access in the 1980s. Today, there are more than 27,000 health centers and sub-centers. Maintaining high quality care across this network has proven difficult because of limited financial and human resources, particularly in the remote Outer Java-Bali region [20-21].

Methods

Data

The study uses household and community data from the 1997 Indonesian Family Life Survey (IFLS). The survey stratified on provinces and urban and rural regions, and 321 enumeration areas were randomly selected from a nationally representative sampling frame. The 1997 survey collected information from 7629 households across 13 provinces. The completion rate was 94 percent; 2 percent of households refused, and the remaining were not found [22]. The sample represented 83 percent of the Indonesian population. Details of the IFLS are documented elsewhere [23].

The facility survey aimed to capture a representative sample of ambulatory health providers. Similar to other low-income settings with weak systems of licensing and registration, there exists no complete list of public and private providers from which to draw a sample. Therefore, the sampling frame was generated from information collected in the household survey about knowledge of health care providers that offered outpatient services in the community. The selection among eligible providers was based on a random probability sample with the probability of selection proportionate to the frequency a given location was mentioned by households. A total of 2751 public and private health providers were interviewed, and the response rate was 99 percent. High response rates could be attributed to up to 3 follow-up visits. Public providers included public health centers and sub-centers; private providers included solo MDs, nurses, and midwives, and group clinics. A description of the sample has been published previously [21].

Technical quality assessments from the facility survey were based on clinical case scenarios. One health care provider per facility was presented with a case scenario, and he/she was asked a series of questions about activities performed during history taking, the physical examination, diagnostics, and follow-up. The interviewer marked those responses mentioned spontaneously from a pretested list of criteria corresponding with clinical guidelines. The scenarios used were pilot-tested before implementation to ensure minimal measurement error, and the method has been validated in other settings [24].

In the household survey, the fertility module collected detailed information from 5702 women aged 15 to 49 years, among which 3173 women experienced a pregnancy between 1992 and 1998 and reported about healthcare received. For this analysis, we omitted women with twins, and those who were pregnant at the time of the survey or experienced a negative birth outcome. Women who were pregnant might not have had an opportunity to obtain a full course of care, and those with negative outcomes might have received a higher level of care. Among women who experienced more than one pregnancy between 1992 and 1998, we included details about prenatal care for the most recent pregnancy. With these omissions, the analyses include 2451 women, and 92.0 percent (n=2255) of these women sought prenatal care and reported about procedures received.

Data processing

Using the data collected from the clinical case scenarios, 19 criteria were identified that correspond with clinical guidelines for routine pregnancies [25]. The criteria are summed up to create quality scores, and the raw scores are expressed as the sum of the criteria as a proportion of the total. The scores are standardized with a mean of zero and a standard deviation of one, and variations in quality are expressed in standard deviation units. The facility-based scores are referred to as quality available. In multivariate analyses, high facility quality is expressed as a dummy variable where “one” equals a quality score greater than one standard deviation above the mean.

Using data collected from the fertility module, we use reports about prenatal care procedures received. Women who sought prenatal care reported about whether they received prenatal services routinely conducted during prenatal consultations. These procedures were measurement of blood pressure, weight, height, and uterine height; testing for anemia; examination as indicated by listening to fetal heartbeat; and receipt of tetanus toxoid and iron supplements. We developed a composite index, which represents the sum of positive responses as proportion of the total. Similar to the facility scores, we standardize the index to a mean of zero and standard deviation of one, and the results are expressed both as a proportion of the total and as standard deviations from the mean. Maternal reports of procedures received are referred to as quality received.

The facility quality scores were matched to the household data, by using information about geographic location and maternal reports of public or private clinical setting. For women that reported more than one source of care, we determined whether the primary source was public or private. Where more than one provider was available in a given community, we generated a mean score by clinical setting.

Additional socioeconomic variables from the household survey include household wealth, maternal education, ethnicity, and insurance coverage. Household wealth was measured using a consumption module that collected detailed data from each household about weekly, monthly, and annual spending on food and non-food products. Household expenditures are considered more reliable than income in capturing wealth among populations with informal or seasonal employment. Expenditures are summed up and divided by the number of people in the household; wealth is expressed in quartiles, with “one” representing lowest household monthly expenditures and “four,” the highest. Maternal educational levels are expressed categorically as some primary education or less, completed primary level, some secondary education, and completed secondary level or higher. Ethnicity is measured by whether the household spoke Indonesian during the interview. Insurance status identifies households with any type of public or private insurance coverage.

Characteristics from the fertility module include maternal risk factors that could affect the intensity of care received. They include prior negative birth outcomes, the number of previous births, gestational age, and maternal age in years. Maternal age and the number of prior pregnancies are included as continuous variables and squared. Gestational age is expressed in weeks and accounts for preterm births and a reduced opportunity for receiving care. We also generated dummy variables for child year of birth to control for possible maternal recall bias.

Java-Bali and Outer Java-Bali are the main two geographic regions in Indonesia. Outer Java-Bali is the more remote region of Indonesia, and includes Sumatra, Kalimantan, and the Eastern Islands. Java-Bali and Outer Java-Bali are used by the central government for allocating public resources.

The analyses

The first two graphs present the proportion of women that sought prenatal care and those that sought public care. Utilization is presented by household wealth, maternal education, and region. Second, we compare the raw and standardized quality scores by clinical setting and household wealth. Quality available (facility-based quality scores) and quality received (maternal reports of procedures received) are compared as standardized scores.

Maternal reports may reflect systematic differences in quality based on individual risk factors. We correct for this bias by estimating the adjusted mean procedures received [21]. The adjusted means are generated using community fixed effects regressions predicting the quality index of prenatal care procedures received on a set of risk factors. These include previous negative birth outcomes, the number of previous pregnancies, maternal age, gestational age, and child year of birth. Missing values for maternal age are imputed as the mean value for the sample. The facility-based scores and the unadjusted and adjusted maternal reports of quality received are presented by wealth quartiles and separately for Java-Bali and Outer Java-Bali.

Third, logistic regression models are used to explain quality received as measured by maternal reports, as a function of high facility quality (>1 SD), maternal education, household wealth, ethnicity, and insurance status. The regressions control for maternal risk factors and child year of birth. The regressions are estimated using the sample as a whole, and separately for Java-Bali and Outer Java-Bali. The analyses adjust for over-sampling and correct for the cluster survey design. Significance levels are reported at conventional levels.

Results

Coverage of prenatal care is high, with 93 and 88 percent of women having accessed prenatal care in Java-Bali and Outer Java-Bali, respectively. The poorest and least educated groups in Outer Java-Bali are the least likely to get care. Some 58 and 60 percent of poor women in Java-Bali and Outer Java-Bali use public care (Graph 1). However, there are no significant differences by educational level for women in Outer Java-Bali, and 45 percent of women with secondary education or more obtain public care. On average, utilization of public facilities is higher in Outer Java-Bali.

Graph 1. Percent of women that got care, by public or private source, household wealth quartiles, and region±.

± Public sources of care include public health centers, auxiliary health centers, or hospitals. Outer Java-Bali represents communities outside of the islands of Java and Bali, including Sumatra, Kalimantan, Sulawesi, and Nusa Tenggara.

The raw quality scores average 45 percent [95% CI: 43.05, 46.67] (Table 1, Panel A). Public facilities offer significantly higher quality compared with private facilities. Adjusted maternal reports do not differ significantly by clinical setting (Panel B).

Table 1. Quality scores and 95 percent confidence intervals (CI) for prenatal care quality available and quality received by public and private clinical settings^±.

Quality measures	Public	Private	All settings
Panel A. Quality available: facility scores
Raw scores	46.38 [44.30,48.46]	43.52** [41.27,45.77]	44.86 [43.05,46.67]
Standardized scores	0.19 [0.05,0.33]	0.00** [-0.16,0.15]
Panel B. Quality received: adjusted maternal reports
Raw scores	68.72 [63.99,73.45]	67.70 [63.30,72.09]	68.19 [67.42,68.96]
Standardized scores	0.13 [-0.07,0.33]	0.09 [-0.10,0.27]

^±Outpatient clinical settings. Quality scores measured as the number of criteria mentioned in the clinical case scenario as a proportion of the total. Facility scores based on 19 criteria from health provider interviews. Maternal reports based on 8 criteria from household interviews. Significance noted at *p<0.10; **p<.05; and ***p<.01

Significant differences exist in quality available by household wealth in Java-Bali, with the poorest households having access to 0.40 SD higher quality care in the community compared with the least poor (Table 2, Panel A). However, the poor actually receive lower levels of quality. In Outer Java-Bali (Panel B), facility quality is significantly lower. Although there is no wealth difference in quality available by wealth, the poor in Outer Java-Bali also receive significantly lower quality, and the difference is more than one-half of a standard deviation unit (0.56 SD). On average (panel C), quality available is slightly pro-poor, and quality received is significantly lower for the poor. While the poorest in Outer Java-Bali receive the lowest levels of quality, the least poor receive similar levels of quality in either region.

Table 2. Standardized quality available compared with quality received for prenatal care in Java-Bali and Outer Java-Bali by household wealth quartiles ^±.

	Household wealth quartiles
Quality scores and region	1	2	3	4	Average
	Panel A. Java-Bali
	Quality available	0.42 [0.21,0.63]	0.30 [0.11,0.48]	0.14 [-0.06,0.34]	0.02*** [-0.21,0.25]	0.24 [0.08,0.39]
Quality received	0.15 [0.06,0.25]	0.28 [0.19,0.38]	0.30 [0.19,0.41]	0.30** [0.07,0.24]	0.22 [0.18,0.25]
Panel B. Outer Java-Bali
Quality available	-0.40 [-0.55,-0.24]	-0.35 [-0.48,-0.22]	-0.31 [-0.46,-0.16]	-0.30 [-0.55,-0.04]	-0.35 [-0.46,-0.23]
Quality received	-0.31 [-0.44,-0.19]	-0.09 [-0.22,0.03]	-0.01 [-0.14,0.12]	0.24*** [0.07,0.41]	-0.10 [-0.16,-0.04]
Panel C. Both regions
Quality available	0.17 [0.00,0.33]	0.14 [0.00,0.29]	0.02 [-0.13,0.18]	-0.04* [-0.23,0.15]
Quality received	-0.02 [-0.11,-0.06]	0.06 [-0.01,0.13]	0.17 [0.09,0.24]	0.27*** [0.17,0.37]

^±Household wealth quartiles range from lowest (1) to highest (4). Quality available represents facility scores, and quality received represents maternal reports. Significance noted at *p<0.10; **p<.05; and ***p<.01 based on linear trend tests of significance.

High facility quality (>1 SD) predicts increases in quality received for the sample as a whole for women in Java-Bali (Table 3). High facility quality predicts a 5.8 point increase in quality received on average (95% CI: 0.028, 0.0884), and a 4.9 point increase in Java-Bali (95% CI: 0.020, 0.078). For women in Java-Bali, increasing levels of maternal education is associated with higher quality. The coefficients for moderately high household wealth and insurance status are significant at the 10 percent level. In Outer Java-Bali, the most important predictors of quality received are maternal education and household wealth. Finishing a secondary education and living in the least poor household is associated with 12.8 and 12.0 points higher quality. High facility quality is not significantly associated with quality received in Outer Java-Bali. Maternal risk factors are not significant predictors of quality received.

Table 3. Models explaining choice of public care and the quality of care received, as a function of high facility quality, maternal education, household wealth, insurance, and ethnicity ^±.

	Quality of care received (standardized)
Explanatory variables	All observations	Java-Bali	Outer Java-Bali
High facility quality	0.058*** [0.02]	0.049*** [0.01]	0.024 [0.04]
Maternal education (no education omitted)
Finished primary	0.059*** [0.01]	0.049*** [0.01]	0.066*** [0.02]
Some secondary	0.083*** [0.01]	0.070*** [0.02]	0.106*** [0.02]
Finished secondary or more	0.084*** [0.02]	0.061*** [0.02]	0.124*** [0.03]
Household wealth quartiles (poorest omitted)
2	0.020* [0.01]	0.003 [0.01]	0.047** [0.02]
3	0.038*** [0.01]	0.028* [0.02]	0.055** [0.02]
4 (least poor)	0.060*** [0.01]	0.027 [0.02]	0.120*** [0.03]
Spoke Indonesian	-0.004 [0.01]	0.009 [0.01]	-0.007 [0.02]
Insurance	0.018 [0.01]	0.023* [0.01]	0.007 [0.02]
Constant	0.490*** [0.11]	0.599*** [0.13]	0.249 [0.21]
Observations	2255	1417	838

^±All regressions include maternal risk factors (prior negative birth outcome, number of prior births, gestational age, maternal age), and none of these coefficients are significant in these regressions. In addition, regressions include dummy variables for child year of birth and maternal missing age values. Significance noted at *p<0.10; **p<.05; and ***p<.01.

Discussion and conclusions

This study uses case scenarios to assess knowledge and decision-making, and control for variation in patient characteristics. Although the criteria do not comprehensively cover all elements of care, the scenarios were designed by local practitioners that identified criteria considered important in this setting. Maternal reports of prenatal procedures are used to measure quality received. Maternal reports focus on a small number of basic procedures compared with the scenarios that evaluate health provider practice and decision-making more comprehensively. This could explain why the raw scores for procedures received are higher than the facility reports of quality available. Sources of health care not included in the survey (i.e., hospitals, community-based health workers) are unlikely to bias the results because these providers represent a small proportion of utilization for prenatal care. We use cross-sectional data, and our results are descriptive.

Access to basic care was high, and 92 percent of women got care on average. The poorest and least educated women were less likely to seek care but more likely to use public care when they did so. Across both regions, public facilities represent an important source of prenatal care, which is of significantly higher quality compared with private alternatives. Utilization of public facilities across all wealth quartiles is higher in Outer Java-Bali, reflecting greater reliance on public care in more remote regions. The government plays an important role in ensuring the provision of basic services such as prenatal care, particularly in remote regions and for poor households.

Based on facility reports, health providers demonstrated relatively low knowledge (<50 percent) of the basic care criteria, and private health care providers were less knowledgeable compared with public providers. High facility quality predicts increases in quality received for the sample as a whole and for women in Java-Bali. Increasing the knowledge and capacity of healthcare providers could have an important impact on quality received.

Training is an important strategy where quality deficiencies result from a lack of skills. In other settings, interactive continuing education linked to re-certification or re-licensing programs have been demonstrated effective [26]. Promoting continuing education as a prerequisite for license renewal would provide incentives for maintaining clinical skills in both public and private clinical settings.

However, sustaining new skills and changes in behavior remains a challenge. Organizational and financing structures provide few incentives for improved performance in Indonesia. Several countries are testing pay for performance initiatives to improve quality and coverage. Performance-based payment schemes (also known as target payments) establish performance indicators and targets and give health providers – as individuals or groups – financial incentives to achieve these targets. Bonus payments in the US and UK have led to increases in immunization, smoking cessation, and cervical cancer screening [27-31]. Overall, however, results have been mixed [32-33]. Such schemes need to be monitored closely to determine their cost-effectiveness and impact on health outcomes – in addition to possible negative effects such as increased fragmentation of care and declines in quality or coverage for conditions not covered under the contract.

Facility quality is significantly lower in remote Outer Java-Bali compared with Java-Bali. These differences reflect to some extent the health and human resource allocation systems based on population criteria [34]. Efforts have been made to develop the right balance of financial and non-wage incentives for deployment; however, little work has been done in examining alternatives to population-based deployment. Upgrading quality in remote regions requires more information about public resource levels and disease burden to make specific recommendations about strategies relative to the costs for sparsely populated and culturally diverse communities. In addition, the public system, in particular, does not reward high performance. Developing appropriate staffing levels, incentives for the deployment of qualified staff to remote regions, and promoting high performance are all key issues, particularly that a large proportion of people in remote regions rely on public sources of care.

It is remarkable that we find no wealth differences in access to quality available. Poor households have access to the same or higher level of quality compared with the least poor. This suggests that the government has had success in making quality services available for the poor. However, the poor actually receive lower levels of quality, and the difference is particularly large in Outer Java-Bali. In addition, the highest wealth quartiles receive similar levels of quality regardless of region. Similar disparities by wealth in the receipt of maternal and child health procedures have been reported in cross-country analyses [35].

Such differences within a given clinical setting could be the result of tiered systems of quality based on ability to pay. In Mexico, the poor get fewer procedures and diagnostics because they are less able to afford the additional out of pockets costs required to follow through with referrals [4]. Given that the cost of basic services was free or nearly free at this time in Indonesia, informal user fees may have contributed to different levels of care received. In addition, differential treatment by race and sex by the same health care provider has been documented in other settings [4, 36]. The reasons for differential treatment in Indonesia could include socioeconomic differences and individual discrimination.

Demand-side household financial incentives could be an appropriate strategy to increase quality through empowering individuals to be more active health care consumers. Indonesia is piloting a conditional cash transfer program (CCT), in which cash benefits are provided to poor households as an incentive to obtain health services. In Mexico, the CCT program was successful in increasing quality of care received through increased consumer demand among the poorest and most marginalized groups [37]. In this program, cash was provided conditional on obtaining specific health services from accredited facilities, as well as health education to inform beneficiaries about the optimal content of this care. This resulted in increased demand for care that complied with quality standards. This experience suggests that CCTs combined with household education and facility accreditation programs could be effective in promoting quality care that is linked to better health outcomes.

Should maternal education represent an information effect, strategies such as quality reports or patient education about the content of care should be emphasized. US experience suggests that quality reporting could be effective if the performance indicators reflect consumer priorities, accurately adjust for case-mix, and can be modified by better practice [38]. Quality reporting could also promote accountability and transparency among health providers [39]. Other methods to empower patients include satisfaction surveys, formal systems of accepting and responding to complaints, and publicly available health information [40]. The use of targeted mass media has also been demonstrated as effective in influencing utilization patterns [41].

Insurance status has a moderately positive impact on women in Java-Bali. This effect could be related to the fee-for-service reimbursement system. The level of insurance coverage was approximately 22 percent in 1997, and this proportion has increased with the introduction of private and social health insurance, and special insurance programs for the poor.

Cost escalation is a concern, particularly with increasing insurance coverage. In addition, increasing the purchasing power of the poor through demand-side transfers could raise health care prices. Health care costs could be controlled through an active purchaser acting in the interests of consumers. This could improve adherence to quality standards, reduce costs related to unnecessary care, and also help overcome the difficultly in consumers judging technical quality. Purchasing linked with accreditation programs that measure progress against best practices (rather than minimum standards) could encourage quality improvement over time. Several other strategies to control costs could include implementing and evaluating mixed provider payment mechanisms to replace fee-for-services, promoting generic medicines and rational drug use, and implementing health technology assessments to evaluate cost-effectiveness prior to introducing new technology.

Improving quality has become a policy priority in low-income settings. In this study, most households have access to basic care. Quality, however, varies by region, clinical setting, education, and household wealth. This research suggests that strategies to promote high quality include improvements in provider knowledge and capacity, household information, education, and resources for health, and insurance. In the more remote regions of Outer Java-Bali, strategies to increase household information and demand could help to redress inequalities in quality received by household wealth. This study is relevant to other settings where quality improvement initiatives are being evaluated to inform policies that protect and promote community health.