Contracting for health and curative care use in Afghanistan between 2004 and 2005

Abstract

Afghanistan has used several approaches to contracting as part of its national strategy to increase access to basic health services. This study compares changes in the utilization of outpatient curative services from 2004 to 2005 between the different approaches for contracting-out services to non-governmental service providers, contracting-in technical assistance at public sector facilities, and public sector facilities that did not use contracting.

We find that both contracting-in and contracting-out approaches are associated with substantial double difference increases in service use from 2004 to 2005 compared with non-contracted facilities. The double difference increase in contracting-out facilities for outpatient visits is 29% (P < 0.01), while outpatient visits from female patients increased 41% (P < 0.01), use by the poorest quintile increased 68% (P < 0.01) and use by children aged under 5 years increased 27% (P < 0.05). Comparing the individual contracting-out approaches, we find similar increases in outpatient visits when contracts are managed directly by the Ministry of Public Health compared with when contracts are managed by an experienced international non-profit organization. Finally, contracting-in facilities show even larger increases in all the measures of utilization other than visits from children under 5.

Although there are minor differences in the results between contracting-out approaches, these differences cannot be attributed to a specific contracting-out approach because of factors limiting the comparability of the groups. It is nonetheless clear that the government was able to manage contracts effectively despite early concerns about their lack of experience, and that contracting has helped to improve utilization of basic health services.

KEY MESSAGES.

• Large-scale contracting for health services in Afghanistan has been associated with substantial increases in curative care use.

• Curative care at contracted facilities is reaching the poor and female patients.

Introduction

Contracting-out of primary health service delivery to private providers is being advocated as a potentially effective way of rapidly expanding coverage of critical health services in developing countries (Loevinsohn et al. 2005). By one estimate, between 30 and 40 developing countries have contracted out a health service at some point (Liu et al. 2004). The bulk of previous experience with large-scale contracting-out for primary health services has come from middle-income countries. Recently, the Democratic Republic of Congo, South Sudan, Bangladesh, Cambodia and Afghanistan—all low-income countries—have contracted-out health services on a large scale.

Principal-Agent theory suggests that contracting-out can improve access to health services by aligning the incentives of health providers with policy-desired outcomes as expressed in contracts. The way in which contracts are specified and performance is monitored are posited as critical determinants of their potential effects (Liu et al. 2008). Appropriate contract incentives, in combination with management autonomy that allows providers to take context-appropriate decisions, are expected to improve use of important health services. However, contract specification, management and monitoring are rarely reported in the literature on contracting-out of primary health services in developing countries.

Performance-based contracts explicitly link contract payments to performance measured in contract-specified terms. In general, the limited evidence available from low-income countries suggests that performance-based contracts have more positive results than contracts that are not performance based (Liu et al. 2008).

The financial and managerial capacity of the purchaser is another important determinant of the effectiveness of contracting as a strategy to increase service use (Mills 1998; Liu et al. 2007). Whether a government division or public sector entity can monitor and manage contracts for primary health services is potentially relevant to the ownership of contracting efforts, their sustainability and health sector stewardship. The evidence on this from low-income countries is limited to one study which found very large increases in health service coverage (Bloom et al. 2006).

In low-income countries, little is known about how well contracting-out approaches perform when they are scaled up beyond pilot settings. Early concerns about contracting-out in Afghanistan, for instance, included whether contracted NGOs would have the capacity to deliver a wide range of services on a large scale and whether the Ministry of Public Health (MoPH) would be able to manage and monitor contracts (Strong et al. 2005; Siddiqi et al. 2006). These and other practical concerns about the technical and managerial capacity of providers to deliver services on a large scale, and the capacity of purchasers to effectively monitor health services, can only be addressed by examining the experience with large-scale contracting in non-pilot settings.

Recent experience from Afghanistan presents a unique opportunity to examine the results of large-scale contracting for health services. Not only has contracting been implemented on a very large scale, four different approaches to contracting have been implemented: contracting-in and three models of contracting-out. Differences in contract scale, performance incentives and contract monitoring make it possible to examine improvements in outpatient curative health service use in the light of these differences. The purpose of this paper is to compare changes in the use of curative outpatient services between the different approaches to contracting and public sector facilities that did not use contracting in Afghanistan between 2004 and 2005.

Contracting for primary health services in Afghanistan

In 2002 Afghanistan emerged from almost three decades of conflict with a devastated health infrastructure, some of the highest mortality rates in the world and a primarily infectious burden of disease. Health services at this point were provided mostly by non-governmental organizations (NGOs) (Sondorp 2004). By the middle of 2003, the Afghanistan MoPH, in consultation with other stakeholders, took the decision to deliver a basic package of health services through a mix of international and Afghan NGOs (Ministry of Health 2003; Strong et al. 2005). By 2005, the proportion of contracts awarded to Afghan NGOs was fairly substantial and varied between 38% and 50% by donor partner. The MoPH and its partners also decided to provide the basic package of health services through the public sector with financial support from a major donor and technical assistance contracted as necessary. As of 2006, 77% of the Afghan population lived in a district where health services are provided through a contracting approach (Palmer et al. 2006).

Table 1 presents key features of the different approaches to contracting in Afghanistan between 2004 and 2005. The three approaches to contracting-out differ in important ways in terms of:

1. Scale of contract (province-wide or sub-provincial cluster of districts);

2. Performance-based payments: the extent to which payment is linked to objectively measured standardized performance criteria and whether it has a bonus component;

3. Contract management responsibility: who manages contracts;

4. Monitoring processes that are independent of service providers and donors;

5. NGO capacity building.

Table 1.

Features of approaches to contracting in Afghanistan (2004–05)

	Contracting-out approach 1 (CO-1)	Contracting-out approach 2 (CO-2)	Contracting-out approach 3 (CO-3)	Contracting-in (CI)
Contract scale	Province (except Badghis & Balkh provinces)	Cluster of districts	Province or cluster of districts	Province
Average per capita amounta	US$4.30	US$4.72a	US$3.87	US$4.55
Contract management agency	Grants and Contracts Management Unit in the Ministry of Public Health	International non-profit organization	Donor agency	Strengthening Mechanism unit in the Ministry of Public Health
Payments	Lump sum contract. 10% on signing the contract. 10% when inception report submitted. 20% every 6 months thereafter. 20% at end of the contract. Residual contract amount remains with the NGO.	Line item budget. Monthly reimbursement for expenses based on submitted invoices subject to a cap based on past spending. Special approval needed if request exceeds cap.	Line item budget with 80% of the total or first year budget disbursed in advance depending on total budget and duration. Final payment disbursed on submission of final report by NGO. Residual contract amount reverts to the donor.	Quarterly requests for funds based on schedule of activities laid down in work plan.
Performance incentive payments	1% of contract value every 6 months for 10% point increase on third-party evaluation Balanced Score Card. 5% of contract value at the end of the contract for a 50% point improvement on Balanced Score Card and household survey.	No, but payments are subject to satisfactory performance.	No, but payments are subject to satisfactory performance.	Yes, based on score in third-party evaluation calculated Balanced Score Card and household survey.
Technical monitoring & evaluation by contract management agency	Third party evaluation surveysb: 6-monthly and annual facility surveys, household survey in 2004 and 2006. Grant & Contracts Management Unit (GCMU) monitoring visits: announced monitoring visits.	NGO conducted baseline and final household surveys: provide information on 10 basic indicators. Field monitoring visits by non-profit organization staff both with and without MoPH staff. Quarterly reviews: with individual NGOs to evaluate progress and solve problems. NGO roundtables: examine strategic performance issues for a group of NGOsb.	Third-party evaluation surveys: 6-monthly (abridged) and annual facility surveys. Household survey in 2004 and 2006. NGO specified logical framework indicators are the primary focus of monitoring. Monitoring visits from donor consultants in Afghanistan. Annual monitoring mission by international donor team. Evaluation report by NGO.	Third-party evaluation surveys: 6-monthly (abridged) and annual facility surveys. Household surveys in 2004 and 2006. Monitoring visits by Strengthening Mechanism unit staff.
Capacity building of the NGO	Meetings to provide information on new procedures and guidelines. Workshops to brief NGOs on and discuss provincial-level performance. Feedback to individual NGOs based on monitoring visits.	Training: Technical, financial and other training. Workshops and meetings: Management workshops NGO roundtables Quarterly reviews. Quality improvement initiatives that give NGOs quality improvement tools. Targeted assistance for NGOs performing sub-optimally. General support from provincial project office.	NGOs provided with written information on guidelines and other necessary details. Workshops to brief NGOs on and discuss provincial-level performance.	Not applicable.

^aThese are the average per capita contract amounts committed to NGOs. The figures do not include the cost of managing and monitoring contracts. (Source: Strong et al. 2005).

^bCO-2 group facilities are included in third-party evaluation surveys, and this data is available to CO-2 managers for monitoring purposes. However, managers in this group assess performance primarily as mentioned above.

These contract features have important implications for the incentives that NGOs face under the different contracting approaches and also for the NGOs’ ability and autonomy to respond to these incentives.

Contracting-out approach 1 (CO-1) is characterized by: province-wide lump sum contracts (except in two CO-1 provinces that have sub-provincial contracts) with performance bonuses linked to monitoring of health services performed by an independent group; a high degree of autonomy to contracted NGOs; and relatively limited capacity building of the contracted NGOs. A dedicated unit within the MoPH is responsible for managing the contracts.

Under the second contracting-out approach (CO-2) each individual contract is sub-provincial and covers a cluster of districts. Only CO-2 contracted NGOs are provided with extensive capacity building support. They are also closely monitored both technically and financially using a system of monthly reimbursements based on a line item budget. Both monitoring and support are provided through an international non-profit organization that manages service delivery contracts.

Contracting-out approach 3 (CO-3) includes both sub-provincial contracts that cover a cluster of districts and province-wide contracts. Although NGOs contracted under this approach are subject to a line item budget, the bulk of it (80% of the total or of the first year budget) is disbursed in advance. NGOs are directly accountable to the donor that funds these contracts, and the in-country donor mission monitors NGOs’ performance. Contracting-out approaches 2 and 3 do not give contracted NGOs any performance bonuses.

In the contracting-in (CI) areas, the public sector delivers the basic package of health services and drugs are procured through government channels. However, management and some support services are contracted. For instance, technical assistance (like training for community health workers or midwives) is contracted from NGOs. CI facilities are managed and monitored not by government employees but by individually contracted staff based in Kabul. CI facilities are eligible for performance bonuses.

The remaining public sector facilities are operated by the MoPH. About 80% of these facilities reported receiving some kind of support from an NGO when surveyed in 2004 and 2005. However, this support is not within the context of any coordinated contracting effort and varies greatly between facilities.

Methods

Data description

This study relies on data generated from a national facility survey (the National Health Services Performance Assessment, or NHSPA), donor and MoPH records, and the routine information reporting system. The NHSPA has been undertaken in June to September each year since 2004, and comprises the independent assessment of basic health services including public sector and NGO providers. It is described in detail elsewhere (Peters et al. 2007).

Only provinces covered in both 2004 and 2005 are retained in this analysis. Furthermore, surveyed facilities that did not have outpatient visit records for at least 1 month preceding the survey are also excluded from the final sample for analysis. The final sample includes 343 and 576 facilities in 2004 and 2005, respectively (Table 2). Forty-seven per cent of the contracted-out facilities in the sample for 2005 are in a province where service delivery is contracted-out to an international NGO. This analysis also uses information on users’ wealth status from 8012 exit interviews in 2004 and 2005.

Table 2.

Facilities surveyed

	2004 facilities n (%)	2005 facilities n (%)	Total facilities n (%)
Study groups
Contracting-out  approach 1	63 (18.4%)	106 (18.4%)	169 (18.4%)
Contracting-out  approach 2	84 (24.5%)	168 (29.2%)	252 (27.4%)
Contracting-out  approach 3	51 (14.9%)	119 (20.7%)	170 (18.5%)
Contracting-in	30 (8.7%)	63 (10.9%)	93 (10.1%)
Reference	115 (33.5%)	120 (20.8%)	235 (25.6%)
Facility type
Basic Health Centres	180 (52.5%)	331 (57.5%)	511 (55.6%)
Comprehensive Health  Centres	136 (39.6%)	206 (35.8%)	342 (37.2%)
District Hospitals	27 (7.9%)	39 (6.7%)	66 (7.2%)
TOTAL	343 (100%)	576 (100%)	919 (100%)

The 2004 NHSPA includes a survey of households in the catchment area of each surveyed health facility (within 1.5 hours walking distance from each surveyed facility). In total, 12 507 households were surveyed.

The Health Management Information System (HMIS) constitutes an additional data source for this study. Data on outpatient visits and facility catchment area populations collected during the facility survey were sourced from each facility's HMIS records. Regular reporting to the HMIS is part of the formal contractual performance obligations for all contracted NGOs and also for contracted-in public sector facilities.

Three surveyed facilities per province were randomly selected for re-survey by an individual who had not been part of the team of surveyors for data quality assurance purposes. If data discrepancies were deemed to be beyond reasonable bounds, the facilities surveyed by that team were re-surveyed. All survey data, except for the catchment area household survey, were double entered.

Analysis methods

Contracted facilities were identified using donor and MoPH records. The non-contracted group includes all remaining public sector facilities that did not receive funds from any of the major donors between NHSPA 2004 and 2005.

Study outcome variables include new outpatient visits per 1000 (catchment area population) per year, new female outpatient visits per 1000 (female catchment area population) per year, new outpatient visits from the poorest 20% of the population per 1000 (poorest 20% catchment area population) per year and new under-5 outpatient visits per 1000 (under-5 catchment area population) per year. The number of outpatient visits in the most recently completed month preceding the survey has been expressed as the number of annual visits per 1000 individuals in the catchment area for ease of comparability. A ‘new outpatient’ is a first-time visitor to the health facility for a specific health problem or a patient who has returned because of a change in symptoms or worsening of the problem. ‘New female outpatients’ includes new female outpatients aged under 5 years.

The proportion of females in each province in 2003 (Afghanistan Statistical Yearbook 2003, Central Statistics Office) is used to estimate the number of females in each catchment area under the assumption that females are evenly distributed among the catchment areas on average. Given the lack of province-level estimates of the proportion of children under 5 years for the 2004 to 2005 period, the proportion of under-5 children is assumed to be 20%. This is in keeping with the assumption the MoPH makes for planning purposes.

We use Principal Components Analysis (PCA) methods in each province to create a wealth index (Filmer et al. 2001) based on asset ownership, housing characteristics and primary source of income for each household surveyed in 2004. Province-specific percentile cut-offs for the poorest 20% are identified after examining the wealth index distribution. We consider province-specific cut-offs to be more appropriate since nationally defined cut-offs would bias findings against facilities in wealthier provinces. We then applied the province-specific cut-offs and weights from the PCA to construct a wealth index for exit survey patients who were asked identical questions about assets, housing and income. Finally, we multiplied the proportion of exit survey patients who belong to the poorest quintile by the total number of new outpatients in the HMIS to estimate the number of outpatient visits from the poorest fifth at each facility.

A nationally defined cut-off for the poorest 20% is used in the PCA when comparing the proportion of poor households in facility catchment areas to ascertain systematic differences in wealth in 2004 among the study categories. We use a nationally defined cut-off here because the purpose was to compare poverty levels in the catchment areas of the study groups.

Districts or provinces were not randomized to contracting-out group (CO-1, CO-2 and CO-3). Instead, the MoPH and donors agreed to a ‘lead donor’ concept with specific donors responsible for distinct geographic areas (Strong et al. 2005). Donors had some flexibility in choosing individual areas of operation with the guiding principle that each donor's presence would be spread out across the country and that all contracted facilities would provide the same minimum package of health care. However, CI provinces were chosen specifically for their proximity to Kabul city. (See Figure 1 for the distribution of contracting groups in Afghanistan 2004–05.)

Figure 1.

Geographic distribution of contracting groups in Afghanistan (2004–2005)

To address the problems that derive from non-random allocation, the analysis examines double differences (DD) to assess changes in service use in contracted facilities relative to non-contracted facilities. Double differences compare the change since baseline for each contracted group relative to non-contracted facilities. This reduces threats to validity from unobserved and observed factors that are constant over time and unique to a specific group, and from time-variant factors that are common to all groups (Meyer 1995; Bertrand et al. 2002).

Assuming a simplified scenario with one contracting group and one reference group (non-contracted public sector facilities) for illustrative purposes, the regression model used to estimate contracting effects includes dummy variables for contracting group membership, year and health facility type. The regression coefficient for the year–contracting group interaction presents the double difference in outcome between a contracting group and the non-contracted group.

Duration of exposure to contracting, user fee implementation and facility type were all initially included as independent variables in the analysis. Of these variables, only facility type was retained in the final models because it was consistently statistically significant.

We also conducted a sub-analysis restricted to contracted-out facilities. Facility type and type of NGO (international or Afghan, with the latter defined as NGOs that work in Afghanistan only or also in refugee camps close to the border) were retained as independent variables in the sub-analysis.

Bootstrapping, an empirical method for estimating variance based on repeated sampling of the available data, is used to calculate the standard errors (Mooney et al. 1993). The bootstrapped standard errors are calculated by re-sampling (with replacement) from the sample of health facilities within the strata created by province and facility type in each year.

All regression models were checked for departures from important regression assumptions. This paper presents findings from regression analyses which exclude high influence observations since we are concerned with describing typical gains in contracted facilities.

An additional regression analysis was conducted to check whether facilities with missing outpatient visit records scored systematically lower on technical quality of care than facilities that had outpatient visit records. The outcome variable for this analysis is a Staffing and Service Capacity index which assesses the availability of doctors, nurses and midwives, and facility capacity to provide health services in terms of equipment, general protocols, laboratory tests and delivery services. The regression coefficient for the dummy variable indicating whether a facility has missing outpatient records describes the association between missing data and technical quality of care. If the coefficient is statistically significant this suggests that facilities with missing outpatient records have systematically different quality scores from facilities without missing outpatient visit records, holding facility type, year and contracting group constant.

Findings

There are no statistically significant differences (P < 0.05) among the five study groups in mean new under-5 outpatient visits per 1000 per year for 2004. CI facilities have fewer new visits on average in 2004 for the remaining study outcomes. CO-3 facilities have, on average, fewer new visits from the poorest fifth compared with CO-2 facilities (Table 3).

Table 3.

New outpatient visits per 1000 per year in 2004, by study group

Study group	New outpatient visits (OPVs) n (95% CI)	New female OPVs n (95% CI)	New OPVs by poorest 20% n (95% CI)	New under-5 OPVs n (95% CI)
Contracting-out 1	431.9	464.9	367.1	571.9
	55	57	54	61
	(336.2–593.9)	(335.4–753.5)	(167.1–832.3)	(414.6–819.7)
Contracting-out 2	367.7	385.4	397.0	594.1
	79	83	77	81
	(303.0–444.3)	(300.9–468.0)	(310.7–496.7)	(421.7–762.4)
Contracting-out 3	387.9	474.2	227.7	771.3
	51	51	51	49
	(336.3–445.5)	(402.7–558.0)	(164.3–292.4)	(593.5–949.1)
All contracted-out	392.4	432.9	340.7	632.5
	185	191	182	191
	(344.1–452.0)	(375.4–515.8)	(260.8–472.5)	(526.9–744.4)
Contracted-in (CI)	278.5	315.8	68.6	625.9
	30	30	30	28
	(240.2–315.9)	(264.9–374.4)	(16.8–155.0)	(511.3–735.9)
Non-contracted	368.1	377.3	349.0	511.2
	105	115	104	114
	(322.6–411.8)	(328.8–425.8)	(256.6–467.8)	(426.9–608.0)

When comparing the unadjusted changes between 2004 and 2005 by study group (Table 4), the change since 2004 for all outcomes—except new under-5 outpatient visits—is greater in each of the contracting groups than in the non-contracted group.

Table 4.

Results: comparison with non-contracted public sector facilities (Reference: non-contracted facilities)

Group	Unadjusted double differencea	Adjusted double differencea	Adjusted double difference as %b
New outpatient visits per 1000 per year
All contracting-out groups	108.7 (P < 0.01)	93.1 (P < 0.01)	29.3
Contracting-out 1	40.9 (n.s.)	43.3 (n.s.)	12.9
Contracting-out 2	114.1 (P < 0.01)	101.5 (P < 0.05)	30.3
Contracting-out 3	230.2 (P < 0.01)	221.3 (P < 0.01)	65.9
Contracting-in	310.2 (P < 0.01)	280.7 (P < 0.01)	83.6
New female outpatient visits per 1000 per year
All contracting-out groups	141.5 (P < 0.01)	127.7 (P < 0.01)	41.0
Contracting-out 1	96.4 (P < 0.05)	103.0 (P < 0.05)	31.7
Contracting-out 2	135.5 (P < 0.05)	133.8 (P < 0.01)	41.2
Contracting-out 3	250.6 (P < 0.01)	240.1 (P < 0.01)	73.9
Contracting-in	327.6 (P < 0.01)	329.9 (P < 0.01)	101.5
New outpatient visits from poorest 20% per 1000 per year
All contracting-out groups	95.2 (P < 0.01)	130.5 (P < 0.01)	67.9
Contracting-out 1	128.1 (P < 0.05)	114.6 (P < 0.05)	56.1
Contracting-out 2	108.2 (P < 0.05)	107.7 (P < 0.05)	52.7
Contracting-out 3	175.7 (P < 0.01)	202.8 (P < 0.01)	99.3
Contracting-in	244.9 (P < 0.01)	229.3 (P < 0.01)	112.3
New under-5 outpatient visits per 1000 per year
All contracting-out groups	111.5 (P < 0.05)	111.8 (P < 0.10)	26.9
Contracting-out 1	27.8 (n.s.)	−23.7 (n.s.)	−5.2
Contracting-out 2	96.9 (n.s.)	52.9 (n.s.)	11.7
Contracting-out 3	309.9 (P < 0.01)	296.2 (P < 0.01)	65.5
Contracting-in	137.8 (P < 0.10)	96.6 (n.s.)	21.4

^aDouble difference without influential points. Double differences are adjusted for facility type.

^bAdjusted double difference as a percentage of the adjusted average number of new outpatient visits in the non-contracted group at baseline.

Comparing all contracted-out facilities to non-contracted facilities, the unadjusted mean increase over time relative to the non-contracted group has been substantial. The observed double differences are 109 (P < 0.01) for new outpatient visits per 1000 per year, 142 (P < 0.01) for new female outpatient visits, 95 (P < 0.01) for new outpatient visits from the poorest 20%, and 112 (P < 0.05) for new under-5 outpatient visits. Expressed as percentage changes over the corresponding adjusted average number of visits in non-contracted facilities at baseline, the double differences associated with contracting-out are large: 29%, 41%, 68% and 27%, respectively, for new outpatient visits, new female outpatient visits, new outpatient visits from the poorest 20%, and new under-5 outpatient visits.

However, the unadjusted double differences among the individual contracting groups and the non-contracted group are not statistically significant for all the outcome variables examined. Only CO-2, CO-3 and CI facilities show statistically significant unadjusted relative increases over time in new outpatient visits per 1000 compared with non-contracted facilities. All the individual contracting-out facilities and CI facilities show statistically significantly larger increases over time relative to non-contracted facilities in new female outpatient visits and new outpatient visits from the poorest fifth of the population. Only the CO-3 and CI facilities show larger unadjusted increases between 2004 and 2005 compared with non-contracted facilities in new under-5 outpatient visits.

After adjusting for facility type, the average change since baseline remains larger in all contracted-out facilities than in non-contracted facilities (Table 4). The pattern of double differences is largely similar to the unadjusted double differences. However, double differences in new under-5 outpatient visits between CI and non-contracted facilities are no longer statistically significant (P < 0.10).

The sub-analysis of contracted-out facilities (Table 5) finds no statistically significant double differences in any of the outcomes between CO-2 and CO-1 facilities after adjusting for facility type and type of contracted NGO (Afghan or international). CO-3 facilities show a statistically significantly greater increase since 2004 in new outpatient visits (198, P < 0.01) and new under-5 outpatient visits (334, P < 0.01) compared with CO-1 facilities. No other statistically significant differences were apparent.

Table 5.

Results: comparison with contracting-out approach 1 group (CO-1)

Group	Adjusted double differencea	Double difference as %b
New outpatient visits per 1000 per year
Contracting-out 2	54.5 (n.s.)	14.9
Contracting-out 3	198.1 (P < 0.01)	54.1
New female outpatient visits per 1000 per year
Contracting-out 2	20.7 (n.s.)	5.3
Contracting-out 3	118.2 (n.s.)	30.2
New outpatient visits from poorest 20% per 1000 per year
Contracting-out 2	−37.8 (n.s.)	−17.1
Contracting-out 3	67.0 (n.s.)	30.2
New under-5 outpatient visits per 1000 per year
Contracting-out 2	21.0 (n.s.)	5.1
Contracting-out 3	333.6 (P < 0.01)	81.7

^aAdjusted double difference without influential points. Double differences are adjusted for facility type and type of contracted NGO.

^bAdjusted double difference as a percentage of the adjusted average number of new outpatient visits in the non-contracted group at baseline.

Discussion

The study findings echo those of other studies on the association between contracting-out (all approaches combined) and service use. In Afghanistan, contracting-out is associated not only with an overall increase in service use but also with an increase in use by the poor, female patients and children under-5. Since these are stated policy priorities, it is clear that contracting-out has been an effective approach to achieving the MoPH's public health policy goals for service utilization.

Comparisons between the groups that include all the contracted-out facilities and non-contracted facilities in 2004 show that the two study groups are not significantly different in terms of study outcome variables or levels of wealth as indicated by asset ownership in facility catchment areas. This suggests that the association between contracting-out and service use in this analysis is unlikely to be caused by differences in household wealth or because of differences in utilization in 2004.

Many concerns were voiced by stakeholders at the outset of the contracting process. These included very low per capita allocations to contracted NGOs and the possibility that NGOs may not have the capacity to deliver services on a large scale (Strong et al. 2005). Estimates of the annual per capita costs of delivering a minimum package of primary health services for low-income countries (Tollman et al. 2006) vary from US$15 (World Bank 1993) to US$36–40 (World Health Organization 2001). Average annual per capita contract allocations to contracted NGOs were considerably lower than both these figures, at US$3.87–4.72.

The improvements evident in service utilization need to be understood in the context of the Afghan health system in 2003. The complete devastation of the Afghan health system over the years of conflict meant that the challenges facing contracted NGOs and their partners were massive. In the period immediately following the overthrow of the Taliban, the number of health facilities and health personnel in the country was small and unevenly distributed (MSH et al. 2002).

Facilities in the sample from the CO-1, CO-2 and CO-3 groups had been exposed to their respective contracting approaches for a mean of approximately 16, 16 and 13 months, respectively. In spite of the short duration of implementation, the average adjusted double difference is 46% (or the median of the three double differences is 41%). This compares favourably with average double differences documented in Cambodia and Bangladesh (Loevinsohn et al. 2005) where the time period between baseline and final evaluation was 30 months. The progress that is visible is therefore indicative of the larger potential of contracting-out to increase curative care utilization in Afghanistan.

Service utilization has improved very substantially for all the outcomes considered here in facilities where services are provided by the Afghan MoPH with management contracted-in (CI group). This is particularly impressive since the MoPH was not a major source of health services through the decades of conflict. After the years of devastation, procurement, financial management and other systems in the MoPH had to be built to support service delivery.

However, it is debatable whether this increase could be replicated to the same extent if the contracting-in approach were implemented in other areas within Afghanistan. Unlike other contracting groups, the CI provinces were selected for inclusion in the CI group because of their proximity to Kabul and they are also substantially wealthier. Ten per cent of households in CI facilities’ catchment areas belonged to the poorest 20% of the population in 2004 compared with 22% in non-contracted facilities’ catchment areas (P < 0.05).

Another concern is that contracting-in had not commenced when the 2004 survey was conducted. If increases in outpatient visits are the largest immediately after contracting begins—since the starting point is lower—the double differences could be biased upwards compared with the individual contracting-out groups where contracting had already commenced by the 2004 survey.

The three approaches to contracting-out in Afghanistan share the same institutional environment but differ in important ways with reference to other determinants of the effectiveness of contracting-out. These differences include the characteristics of the contractual relationship, purchaser and contract management entity, and the autonomy providers have. However, our findings do not conclusively suggest that any one approach to contracting-out is more effective than any other in increasing outpatient curative care visits.

CO-3 facilities show statistically significantly greater increases on all the outcomes studied here than non-contracted facilities. CO-3 facilities also demonstrated statistically significantly larger increases than CO-1 and CO-2 facilities on some service use outcomes examined. However, a statistically significantly (P < 0.05) smaller proportion of households in CO-3 areas (7%) belong to the poorest 20% of the national population than in CO-1 (25%), CO-2 (30%) and non-contracted (22%) areas. Greater household wealth could therefore have contributed to the larger relative increases.

The study findings allay some early concerns about the CO-1 approach. These include reservations about NGOs’ ability to provide province-wide services with the limited capacity-building support (see Table 1) provided to them under the CO-1 contracts (Strong et al. 2005). NGO technical capacity constraints have clearly not been serious enough to preempt the large improvements in new female outpatient visits and new outpatient visits from the poorest 20% evident under CO-1 contracts. Alternatively, NGOs may have been able to purchase the technical support they needed to meet contract obligations.

Another concern regarding the feasibility of contracting is that ministries of health may lack the capacity to manage contracts. A recent study describes the Afghan political, bureaucratic and legal framework for contracting as weak, and emphasizes the MoPH's lack of prior experience with purchasing health services (Siddiqi et al. 2006). However, accounts of the contracting process from Afghanistan suggest that the MoPH has taken monitoring of NGO performance under CO-1 contracts very seriously (Palmer et al. 2006; Waldman et al. 2006; Peters et al. 2007). Third-party evaluation data were used to determine whether contracts should be renewed or terminated when the first round of contracts expired (Hansen et al. 2008). In 2006, a poorly performing CO-1 NGO had its contract terminated when it failed to make improvements despite repeated warnings.

Service use improvements between 2004 and 2005 were very similar in CO-1 and CO-2 facilities. Furthermore, alternative explanations do not account for the similarity in improvements. CO-1 areas and CO-2 areas have relatively similar poverty levels. The proportion of households below the 20^th national wealth percentile in CO-1 areas is not statistically significantly different compared with CO-2 areas (25% compared with 30%). The service use outcomes studied here are also not significantly different (P < 0.05) between the two areas for 2004.

However, narrative reports from CO-2 contracted NGO managers indicated that they found the support received from the contract management agency very useful in meeting contractual obligations. Providing this type of support requires very high technical capacity in the entity managing contracts and is more resource intensive (the costs of this support are not reflected in the per capita contract amounts). The available data indicate that overhead costs for CO-2 contracts are close to 39% of the total value of CO-2 service delivery contracts (Strong et al. 2005). This is certainly an over-estimate of the transaction costs of CO-2 contracts as it includes substantial technical assistance provided to the MoPH in rebuilding the health system.

On the other hand, although the CO-1 approach makes less intensive technical demands on the contract management agency, it relies more heavily on financial performance incentives to improve performance. The cost of independently monitoring performance to determine performance bonuses is a general concern. In the Afghan context, however, available evidence suggests that these costs are very reasonable. Third-party evaluation monitoring costs are estimated at 2.5% of the cost of delivering the Basic Package of Health Services (Peters et al. 2007). These data are used by the MoPH primarily for the stewardship of the health sector as a whole (Peters et al. 2007) and to improve decision-making (Hansen et al. 2008). In addition, contract managers use the Balanced Scorecard constructed from these data to monitor contracted NGOs’ performance. Data on the transaction costs of CO-1 contracts are not available. However, the MoPH reports contract management costs that work out to about 3% of total CO-1 and CI contract value (Strong et al. 2005). It is unclear whether this estimate includes the cost of the Strengthening Mechanism unit.

Study limitations

The findings from this study are subject to some limitations. The lack of data on funding for non-contracted facilities presents a challenge to interpreting the study results. Non-contracted facilities did not receive systematic support from any of the major donors, although close to 80% surveyed reported receiving some sort of support from NGOs.

The absence of data on funding received by non-contracted facilities also implies that separating the effects of the institutional arrangements of contracting from the financial resources injected is not possible when making comparisons with non-contracted facilities. On the other hand, contracted facilities received fairly similar per capita allocations.

Since this is a facility-based study it was not possible to assess whether an observed increase in use was because of an actual increase or was an artifact of individuals substituting publicly funded services for privately funded and provided services. However, available private sector services that are not provided by non-profits are likely to be delivered by less than fully qualified providers. Moreover, increasing the number of contacts with publicly financed facilities provides opportunities to improve coverage of preventive services with currently low coverage levels, like antenatal care and skilled birth attendance. The increases in outpatient visits should therefore be viewed as unambiguously beneficial. The value of increasing new outpatient visits is underscored by the predominantly infectious burden of disease in Afghanistan. In 2004, acute respiratory infections, diarrhoea and vaccine preventable diseases were responsible for close to 60% of child deaths. Tuberculosis was a major health problem accounting for approximately 15 000 adult deaths each year and malaria was a health risk for an estimated 15% of the population (Ministry of Health 2004).

This analysis relies on information from the health management information system (HMIS). Exploratory analysis of the data found a high proportion of missing values in the outcome variables (ranging from 12% to 31%). An important concern stemming from this is that facilities that have missing outpatient visit records are relatively poorly run facilities with lower quality services. We compared facilities with missing and non-missing outpatient visit records on an index of staffing and service capacity. This analysis found no association between missing outpatient records and structural quality (P > 0.10). Therefore there is no evidence suggesting that poor facility functioning is the cause of missing outpatient visit records.

The high proportion of missing outpatient visit records is not entirely unexpected given that both facilities and the HMIS were being set up during the 2004–05 period. Missing values may reflect the incremental health sector reconstruction process, rather than poor facility performance.

This analysis does not control for insecurity due to continued conflict and crime. However, the survey sampling methodology ensures that the most insecure provinces are not surveyed. Furthermore, the provinces covered by each CO group are scattered across the country. All these factors in combination increase the likelihood that surveyed contracted-out and non-contracted facilities were not differentially affected by insecurity.

Conclusions

This is the first study to present data on the implications of contracting for curative care use in Afghanistan. The study finds a substantially greater improvement in the use of outpatient curative care in facilities where contracting for services has occurred compared with facilities where no contracting has occurred, indicating that the policy to contract for health services in Afghanistan has been successful in this important area. These improvements have clear public health benefits in a country with a large burden of infectious disease (Ministry of Health 2003) and where creating new institutions is important, and offer the promise of further enhancing the coverage of other health services. Despite these improvements in new outpatient visits (+142% to +227%), substantial further improvements are needed. To illustrate this point, whereas the number of new outpatient visits per capita in 2005 ranged from 0.5 to 0.8 in Afghanistan, the corresponding figure is 3.0 in Pakistan in 2000 (World Development Indicators). Whereas this study examines the utilization of health services, further research planned will examine the effects of contracting on quality of care.