The cost-effectiveness of using performance-based financing to deliver the basic package of health services in Afghanistan

Abstract

Performance-based financing (PBF) is a mechanism to improve the quality and the utilisation of health benefit packages. There is a dearth of economic evaluations of PBF in the ‘real world’. Afghanistan implemented PBF between 2010 and 2015 and evaluated the programme using a pragmatic cluster-randomised control trial. We conducted a cost-effectiveness analysis of the PBF programme in Afghanistan, compared with the standard of care, from the provider payer’s perspective. The incremental cost-effectiveness ratio of PBF compared with the standard of care was US$1242 per disability-adjusted life year averted; not cost-effective when compared with an opportunity cost threshold of US$349. Incentive payments were the main contributor to PBF financial cost (70%) followed by data verification (23%), staff time (5%) and administration (2%). The unit cost per case of antenatal care (ANC), skilled birth attendance (SBA) and postnatal care (PNC) services in the standard of care was US$0.96 (95% CI 0.92–1.0), US$4.8 (95% CI 4.1–6.3) and US$1.3 (95% CI 1.2–1.4), respectively, whereas the cost of ANC, SBA and PNC services per case in PBF areas were US$4.72 (95% CI 4.68–5.7), US$48.5 (95% CI 48.0–52.5) and US$5.4 (95% CI 5.1–5.9), respectively. To conclude, our study found that PBF, as implemented in the Afghan context, was not the best use of funds to strengthen the delivery of maternal and child health services. The cost-effectiveness of alternative PBF designs needs to be appraised before using PBF at scale to support health benefit packages. PBF needs to be considered in the context of funding the range of constraints that inhibit health service performance improvement.

Key questions.

What is already known?

• In the last decade, there has been extensive investment in and implementation of performance-based financing (PBF) in low-income and middle-income countries, specifically post-conflict settings, as an approach to expand and improve service delivery.

• To date, there have only been two studies reporting on the cost-effectiveness of PBF for health services, with inconclusive findings.

What are the new findings?

• This study found that PBF, as implemented in the Afghanistan context, was not the most cost-effective use of scarce funds within the health sector budget constraint.

What do the new findings imply?

• Our research supports the growing evidence highlighting the importance of considering efficiency in the design of the PBF incentive structure, the capacity at the facility level to respond to incentives and the relative importance of services in terms of health outcomes.

• The success of PBF as a health system strengthening strategy ultimately remains dependent on a sound understanding of context-specific constraints and may require complementary health system investments.

Introduction

Performance-based financing (PBF) has received considerable policy attention in recent years in low-income and middle-income countries (LMICs), as a means to improve health system performance as part of Universal Health Coverage.¹ PBF provides incentives to service providers (facilities and workers) when they achieve predefined performance targets.² A critical concern is whether the overall costs of PBF, including transaction costs of setting up the payment and information systems required, have a greater impact than other direct forms of funding health services (such as budgets). Despite extensive implementation of PBF programmes to improve maternal and child health (MCH) services, to date, there are only two studies of the impact of PBF in LMICs.^{3 4} Borghi et al³ found that the PBF programme in Tanzania was not cost-effective, whereas Zeng et al⁴ concluded that PBF was a cost-effective intervention in Zambia.

The Ministry of Public Health (MoPH) of Afghanistan implemented PBF to fund its Basic Package of Health Services (BPHS) between 2010 and 2015 aiming to strengthen the performance of MCH services. The BPHS provides uniform preventive and curative primary health services. The main elements of the BPHS are maternal and newborn care, child health and immunisation, public nutrition, communicable diseases treatment, mental health, disability and physical rehabilitation services and regular supply of essential drugs.⁵

This study examines the ‘real world’ cost-effectiveness of PBF in Afghanistan ex-post, using data from a pragmatic randomised control trial, using a decision-analytic model, that estimates the cost per disability-adjusted life year (DALYs) averted. To our knowledge, this is the first cost-effectiveness study of PBF in a fragile conflict-affected state.

We aim to contribute to the broad evidence base informing LMICs on whether PBF can extend and improve the performance of health benefits packages in a cost-effective way.

Methods

Study setting

Over the past four decades, Afghanistan has experienced political instability, civil war and pervasive conflict.⁶ A new democratic government was established in December 2001. In 2003, the BPHS was introduced to provide primary healthcare services, specifically MCH services, to the population. BPHS were contracted to non-state providers (NSPs) in 31 provinces, whereas the MoPH managed the BPHS in the remaining three provinces through direct implementation called Ministry of Public Health Strengthening Mechanism (MoPH-SM).⁷ PBF was implemented in nine provinces supported by NSPs and two provinces supported by MoPH-SM, covering 463 BPHS health facilities out of 1892 nationally in 2010.⁸

Intervention description

PBF provided financial incentives to health workers based on the increased production of MCH services (antenatal care (ANC), skilled birth attendance (SBA) and postnatal care (PNC)) above the baseline for each BPHS health facility. These incentives were paid in addition to routine salaries and funding of the health facility. Services were monitored using data from the Health Management Information System (HMIS) and verified through household visits, comparing reported visits with those noted in the health facility registers. If the community validation rate exceeded 80%, then the health facility was entitled to a performance payment. The payments were weighted according to the quality of care assessed by a quarterly score on the national monitoring checklist.⁹ The indicators and level of performance payments are provided in the online supplemental table S3.

PBF was evaluated through a large-scale pragmatic cluster randomised trial, details of which are reported elsewhere.⁵ All facilities within each province were stratified by type of facility and then matched based on the outpatient utilisation rate. Within each matched pair, health facilities were randomly assigned to control and treatment groups. The treatment group received PBF in addition to routine funding, whereas the control group received only their routine salaries.⁹ The evaluation of PBF involved two household surveys conducted at baseline in 2010 and end-line in 2015 in the catchment area of a sample of treatment and control health facilities. The impact evaluation found that, on average, PBF improved the utilisation of services in all the payment triggering indicators; however, no statistically significant differences were found between study arms. In addition, there was an overall performance score, which was an average of the scores on the 20 indicators from five domains, which were client and community, human resources, physical capacity, service provision and management systems. Each indicator was given equal weighting to develop the composite score that ranged from 0 to 100. Facilities exposed to PBF achieved a statistically higher index score compared with the control group on quality of care.¹⁰

Economic evaluation framework

We compared the PBF ‘treatment’ with the standard of care ‘control’ for the population of Afghanistan, assessing cost-effectiveness using incremental cost per DALYs averted. We used a time horizon of 1 year from the start of pregnancy for a hypothetical cohort of women attending BPHS in Afghanistan between 2010 and 2015. We used a decision tree model to estimate DALYs averted by PBF for both mothers and neonates as the trial reported intermediate health outcomes (ANC, SBA and PNC). We parameterised the model with primary cost data, service data from the trial and secondary data. We assessed cost-effectiveness from a provider perspective (MoPH, donor), as data on patient incurred costs were not available.

Model design

We adapted a validated model called the ‘Maternal Health Policy Model’ to evaluate the costs and health outcomes of the PBF intervention in Afghanistan (online supplemental figure S2). This model simulates the natural history, events and service utilisation related to pregnancy and childbirth, including ANC, delivery, abortion, complications and PNC.¹¹ Adapting this model, we developed two decision trees, one for pregnant mothers and one for newborns, to predict incremental costs and health outcomes (online supplemental figure S3). The decision trees were parameterised on the probability of care-seeking and events occurring and associated costs collected during the trial. Within the maternal decision tree, pregnant women have the option to use or not use ANC services, to proceed to delivery with or without a skilled birth attendant or have an abortion, to incur potential complications of pregnancy (ie, haemorrhage, obstructed labour, sepsis, hypertensive disorders, fistula, anaemia and infertility) and to use or not use PNC services. In the neonatal decision tree, newborns may receive PNC and may develop complications (ie, low birth weight, neonatal sepsis and birth asphyxia).

Cost parameter estimation

Estimating the cost of implementing the PBF intervention

We estimated the financial and economic costs of implementing PBF, using primary cost data from Afghanistan. Financial costs included the PBF managers’ salary, incentive payments, data verification and PBF project administration cost. Economic cost added the financial cost with the opportunity costs of items that were used for the purpose of the PBF implementation at the national and provincial level by the MoPH and NSPs (shared staff, building, equipment and transport costs), but were not directly paid for by the PBF scheme. Financial cost data were obtained from PBF project accounts and financial reports. Where PBF costs were shared with other interventions and activities, we allocated costs to PBF using a variety of allocation factors, further details can be found in the online supplemental appendix section on costs. The cost of personnel whose salaries were not solely funded by the PBF project was allocated based on the proportion of their time spent on PBF-related activities. For shared building space, we allocated according to the percentage of floor space used for the PBF intervention. The cost of transportation was estimated based on the consumption of fuel used for the PBF compared with other activities. As the MoPH owned buildings, we used estimates of rent of equivalent building spaces to determine building prices. All (non-building) capital costs were annuitised using a 3% discount rate and life span of 5 years.

Estimating the unit cost of providing services incentivised by PBF

We estimated the financial costs of the services supported by PBF and those in the standard of care. We estimate the unit costs of services. We therefore first estimated unit costs of service delivery in terms of staff time, medical supplies, drugs and capital items. We conducted primary data collection, using microcosting methods (see the online supplemental appendix for more details) in a random sample of 25 health facilities from the 463 BPHS health facilities where the PBF intervention was implemented to estimate the unit costs of services. These costs, minus PBF incentives and support, were used to estimate service costs in the standard of care, as we had no access to standard care facilities. Unit costs were estimated using a micro-costing method including the costs of salaries, drugs, equipment and building. Interviews were undertaken with all staff (206) at sampled health facilities to determine the proportion of staff time spent on each service and the proportional use of equipment for each service. The percentage of floor space used for each service was measured. The average costs of drugs and supplies used were calculated using the list of prescribed medicine for each service and the pharmacy register book at each of the health facilities.

Estimating incremental costs of the PBF

The total cost of services was estimated in the decision-analytic model by adding the unit cost for each of the services (for PBF including PBF costs) received by the cohort in the decision-analytic model. Prices in local currency were converted to US dollars to allow comparisons between countries (US$1 to 58 Afghani in 2015). We used secondary data from the literature to estimate unit costs of maternal and newborn-related complications.¹¹ All costs were inflated and adjusted to 2015 US dollars.

Service utilisation and outcomes

Given the PBF impact evaluation shows an increase in the utilisation of services and significant improvement in the quality of care in the PBF intervention (treatment) compared with the standard of care (control); we used utilisation rates of ANC, SBA and PNC services derived from the PBF impact evaluation¹⁰ to estimate the relative effect of PBF intervention on the utilisation of these services. We also sourced other data from the literature.^11–15All parameters used in the decision-analytic model are presented in table 1.

Table 1.

Model parameters

Parameter	Value	Lower	Upper	Distribution	Source
Maternal mortality rate per 100 000 live birth*	396	396	396		24
Neonatal mortality rate per 1000 live birth*	22.00	18.70	25.68		25
Life expectancy at birth*	64.20	64.20	64.20		26
Life expectancy at age 30–34*	41.00	41.00	41.00		26
Average age of pregnant women (years)*	32.00	32.00	32.00		26
Pregnant population in PBF provinces*	79 504	77 388	81 619		25
ANC rate, control group	0.50	0.50	0.50	Beta	10
SBA rate, control group	0.50	0.50	0.50	Beta	10
PNC rate, control group	0.50	0.50	0.50	Beta	10
ANC rate, treatment group	0.60	0.52	0.68	Beta	10
SBA rate, treatment group	0.52	0.50	0.54	Beta	10
PNC rate, treatment group	0.55	0.51	0.60	Beta	10
Unit cost of ANC, control group	0.96	0.92	1.00	Gamma	Primary data
Unit cost of delivery with SBA, control group	4.76	2.69	4.08	Gamma	Primary data
Unit cost of PNC, control group	1.28	1.19	1.45	Gamma	Primary data
Unit cost of ANC, treatment group	4.72	4.7	5.7	Gamma	Primary data
Unit cost of delivery with SBA, treatment group	48.48	48.0	52.5	Gamma	Primary data
Unit cost of PNC, treatment group	5.38	5.1	5.9	Gamma	Primary data
Management of maternal haemorrhage cost*	0.11	0.05	0.23	Gamma	11
Management of obstructed labour cost*	69.33	34.67	173.33	Gamma	11
Treatment of maternal sepsis cost*	37.46	18.73	93.64	Gamma	11
Management of hypertensive disorders Cost*	57.31	28.65	143.28	Gamma	11
Management of abortion cost*	45.98	31.54	79.42	Gamma	11
Safe abortion cost*	31.96	15.98	47.94	Gamma	11
Management of unsafe abortion cost*	60.00	30.00	90.00	Gamma	11
Management of low birth cost*	8.91	6.40	8.67	Gamma	11
Management of sepsis cost	21.31	18.20	22.83	Gamma	11
Management of birth asphyxia cost*	6.34	7.65	5.57	Gamma	11
Haemorrhage incidence*	0.11	0.05	0.23	Beta	11
Hypertensive disorder incidence*	0.03	0.01	0.05	Beta	11
Abortion incidence*	0.105	0.084	0.096	Beta	27
Unsafe abortion incidence*	0.04	0.02	0.10	Beta	11
Obstructed labour incidence*	0.06	0.03	0.07	Beta	11
Severe anaemia incidence*	0.09	0.08	0.09	Beta	11
Maternal sepsis incidence*	0.05	0.04	0.06	Beta	11
Fistula incidence*	0.02	0.02	0.04	Beta	11
infertility incidence*	0.09	0.08	0.09	Beta	11
Low birthweight incidence*	0.17	0.12	0.05	Beta	26
Neonatal sepsis/infection incidence*	0.02	0.02	0.01	Beta	14
Birth asphyxia incidence*	0.03	0.02	0.01	Beta	14
Obstructed labour/fistula duration*	32.0	28.0	36.0		Expert Opinion
Maternal haemorrhage/severe anaemia duration*	0.50	0.50	0.50		Expert opinion
Maternal sepsis/infertility duration*	17.00	17.00	17.00		Expert opinion
Low birthweight duration*	0.06	0.04	0.08		Expert opinion
Neonatal sepsis/all infection duration*	0.04	0.02	0.06		Expert opinion
Birth asphyxia duration*	0.19	0.01	0.04		Expert opinion
Maternal haemorrhage/severe anaemia disability weight*	0.093	0.090	0.093		28
Obstructed labour/fistula disability weight*	0.43	0.43	0.43		28
Maternal sepsis/infertility disability weight*	0.18	0.18	0.18		28
Low birthweight disability weight*	0.11	0.11	0.11		28
Neonatal sepsis/infection disability weight*	0.62	0.62	0.62		28
Birth asphyxia disability weight*	0.37	0.34	0.38		28
Quality Index Score, control group	54.9	53.5	56.3		10
Quality Index Score, treatment group	56.7	55.4	58.0		10

*The same parameter is used for both control and treatment groups.

ANC, antenatal care; PBF, performance-based financing; PNC, postnatal care; SBA, skilled birth attendance.

We estimated the incremental cost-effectiveness ratio (ICER) defined as the incremental cost of PBF per DALY averted compared with the standard of care. The ICER was assessed against the US$349 per capita threshold estimated by Wood et al’s for Afghanistan,¹⁶ that is higher than 0.5 of gross domestic product per capita in 2015 (US$570), as no more recent estimates were available. Given that 1 DALY averted equals 1 year of healthy life lived, we translated the total number of DALYs averted by the intervention to an annual reduction in maternal and neonatal deaths. We used a 3% discount rate on cost and DALYs averted in our primary analysis. We undertook a series of one-way sensitivity analysis across key model parameters varying each parameter at a time up to ±30% of the base case value. In addition, we used probabilistic sensitivity analysis (Monte Carlo situation of 1000 times) to assess parameter uncertainty and produce a 95% CI around the ICER.

Patient and public involvement

Patient and the public were not involved in the design and other stages of this study. However, we plan to disseminate the findings through publication and presentations in conferences and public events.

Results

The total incremental financial and economic provider costs of the PBF programme were US$10 677 465 and US$11 896 380, respectively, during the 6-year life of the scheme, as shown in table 2. Incentive payments were the main contributor to economic costs (63%) followed by HMIS data verification (21%), administration (10%) and staff time (6%). The online supplemental table S4 shows the PBF annual financial programme cost and the online supplemental table S5 presents the annual breakdown of the administration costs.

Table 2.

The cost of implementing the PBF programme over a 6-year period (US$ 2015)

Cost centre	Financial cost	Percentage	Economic cost	Percentage
Salary	522 957	5	772 118	6
Incentives	7 481 266	70	7 481 266	63
Verification	2 475 952	23	2 475 952	21
Administration	197 290	2	1 167 043	10
Total (US$)	10 677 465	100	11 896 380	100

PBF, performance-based financing.

Table 3 presents the average annual health facility cost of providing ANC, SBA and PNC services in the PBF and control areas. The average annual costs for PBF facilities of providing ANC, SBA and PNC services were US$20 607 (95% CI 14 359–26,818) compared with US$2168 (95% CI 2073–2438) in control facilities. Table 4 shows the unit cost of services in the PBF and control groups. The estimated unit costs per case receiving ANC, SBA and PNC services in the control group were US$0.96 (95% CI 0.92–1.0), US$4.76 (95% CI 4.1–6.3) and US$1.3 (95% CI 1.2–1.4), respectively, whereas the costs of ANC, SBA and PNC services per case in the treatment group were US$4.72 (95% CI 3.6–5.8), US$48.5 (95% CI 48.0–52.5) and US$5.4 (95% CI 5.1–5.9), respectively. The costs of incentives and data verification were the main driver for the higher unit costs in the PBF facilities. The annual per capita cost of the PBF programme was US$0.4 (95% CI 0.2–0.6) in the study areas as presented in the online supplemental table S6.

Table 3.

Average cost of PBF indicators in control and treatment groups per health facility per year (US$ 2015)

Cost centres	ANC cost (US$)		SBA cost (US$)		PNC cost (US$)		Cost per HF (US$)
	Control	Treatment	Control	Treatment	Control	Treatment	Control	Treatment
Salary	607	867	460	658	419	598	1486	2123
Incentive	0	4345	0	10 170	0	2353	0	16 868
Data verification	0	214	0	214	0	214	0	642
Building	150	214	243	348	108	154	501	716
Equipment	32	46	44	63	25	36	101	145
Drug	22	32	15	22	12	17	49	71
Transport	0.8	1.2	29	41	0	0	30	43
Cost per health facility	812	5719	792	11 515	564	3373	2168	20 607

ANC, antenatal care; PBF, performance-based financing; PNC, postnatal care; SBA, skilled birth attendance.

Table 4.

The unit cost of delivering selected maternal and neonatal health services for facilities in control and treatment groups (US$ 2015)

Services	Control	Treatment
Antenatal care	US$0.96 (95% CI 0.6–1.5)	US$4.7 (95% CI 3.6–5.8)
Skilled birth attendance	US$4.8 (95% CI 4.1–6.3)	US$48.0 (95% CI 31.0–67.8)
Postnatal care	US$1.3 (95% CI 0.8–2.1)	US$5.4 (95% CI 4.3–6.5)

The incremental cost of the PBF programme per DALY averted was US$1241 (table 5). The probabilistic sensitivity analysis at US$1228 (95% CI 1118–1337) found that 0% of simulations the ICER lay below the opportunity cost threshold (not cost-effective). In total, 13 028 incremental DALYs (5658 incremental maternal DALYs and 7370 incremental neonatal DALYs) were averted that corresponds to an incremental 253 deaths averted (138 maternal and 115 neonatal) between 2010 and 2015 (equivalent to an annual reduction in maternal mortality of 29 deaths per 100 000 live births and neonatal mortality of 0.24 deaths per 1000 live births), across the 11 provinces of Afghanistan, with a total population of 4.06 million living in the coverage area of PBF facilities. Figure 1 presents the PBF cost-effectiveness acceptability curve, figure 2 the cost-effectiveness plane and online supplemental tables S7 and S8 show the results of one-way and two-way sensitivity analyses.

Table 5.

Annual costs and outcomes for mothers and newborns in PBF and control areas and the incremental cost-effectiveness of the PBF programme (US$ 2015)

	Mother	Newborn	Total
Standard of care (control) group
Annual cost per person (US$) [1]	9.87	2.12	12.32
Annual DALYs averted [2]	0.18	1.07	1.25
PBF treatment group
Annual cost per person (US$) [3]	34.51	11.39	45.77
Annual DALYs averted [4]	0.2	1.08	1.28
Incremental cost (US$) [3-1]	24.64	9.27	33.91
Incremental DALY averted [4-2]	0.012	0.015	0.027
ICER (US$)	2,077	599.8	1,241

DALY, disability-adjusted life years; ICER, incremental cost-effectiveness ratio; PBF, performance-based financing.

Figure 1.

PBF cost-effectiveness acceptability curve. GDP, gross domestic product; PBF, performance-basedfinancing; WTP, willingness to pay.

Figure 2.

Cost-effectiveness plane of the PBF treatment group compared with the control group. DALY, disability-adjusted life years; ICER, incremental cost-effectiveness ratio; PBF, performance-based financing.

Discussion

The focus on maternal and neonatal health during pregnancy, childbirth and postpartum period is proved to be effective in reducing maternal and neonatal morbidity and mortality.^{17 18} Afghanistan implemented a PBF intervention on a large scale aiming to improve MCH services. The PBF intervention was evaluated through a pragmatic cluster randomised trial. We developed a decision-analytic model of the care pathways, cost and outcomes for pregnant women and newborns to estimate the incremental cost-effectiveness of the PBF scheme relative to the standard of care. Although the PBF intervention resulted in an improvement in the quality of MCH services in the PBF group, our study found that this initiative was unlikely to be cost-effective from a provider payer’s perspective. The finding supports the study from Tanzania that the PBF intervention was not cost-effective (which did not measure DALYs averted), despite improvements in utilisation rates,³ but our study finding contradicts the only previous robust study from Zambia in which the PBF intervention was found to be cost-effective.⁴

Our study found that the costs of implementing PBF were substantial compared with those in the non-PBF control districts. PBF cost was primarily driven by the payment of incentives and data verification linked to incentive payment (together accounting for 93% of the total financial cost and 84% of the total economic cost). The proportion of incentives contrasts with Zambia and Tanzania. In Zambia, incentives and verification accounted for around 50% of the total cost.⁴ Borghi et al¹⁹ reported a high administration cost (financial 63% and economic 78%) in Tanzania when implementing a PBF intervention.

There are a number of possible explanations for the lack of cost-effectiveness of PBF to support the basic package for MCH services in Afghanistan. First, the theory of change that supported the design of PBF in Afghanistan posited that high levels of financial incentives would motivate healthcare workers to improve the quality of care and subsequently increase demand for MCH services.⁵ Although incentives may influence providers, availability of resources such as sufficient healthcare workers, equipment, essential drugs and supplies, and effective referral systems are essential to ensure the quality provision of services.²⁰ However, neither the PBF programme was able to overcome these systemic resource constraints, nor the health facilities had financial autonomy to procure them locally.²¹ In Zambia, health workers had significant autonomy in addressing the shortage of essential inputs in health facilities.⁴

Second, the level of incentive is critical to PBF efficiency.²² When too low, an incentive might fail to result in behaviour change, whereas when too high, an incentive consumes resources unnecessarily. In Afghanistan’s PBF scheme, the incentive payment for delivery with SBA was set at US$37 per case. In contrast, the incentives for ANC and PNC were set much lower, at US$2.8 per case. The SBA incentive made 72% of the overall cost of the PBF incentives, and it consumed 51% of the PBF total financial cost, whereas the ANC and PNC incentives constituted only 17% and 9% of the PBF total financial cost. Although the substantial effect of SBA on proper management of delivery and prevention of pregnancy complications is critical, both ANC and PNC are also important. ANC has a positive effect on the identification of pre-eclampsia/eclampsia and prevention of anaemia,²³ and PNC plays a crucial role in early identification and appropriate referral of maternal and newborn complications, family planning and promotion of healthy behaviours for mother and newborn. Both are designed to encourage a sustained relationship between health services and the mother during pregnancy. It is also important to note that 66% of global maternal mortality happens in the postpartum period, and the first 24 hours after delivery is crucial given 45% of deaths occur in this time.¹⁷ Therefore, although the incentives set may have reflected a level of workload, it is essential to also consider the value of services in terms of their contribution to health outcomes, and not comparatively disincentivise those services with low or no additional payments.

Third, even though HMIS data verification is a crucial element of PBF interventions, this ‘transaction cost’ can be relatively high. In our case, the verification was 23% of the total financial cost and 21% of the overall economic cost of the PBF intervention in Afghanistan. This cost could be reduced by testing different modalities. For example, Zambia replaced the monthly verification of all health facilities with a risk-based model in which health facilities were assessed based on the expected risk of misrepresenting data.⁴

Our study has some limitations. The main limitation is that we had to estimate health service costs in the non-PBF sites, using micro-costing from the PBF sites. This was done based on the assumption that underlying service costs and expenditures from other sources would be balanced between intervention and control sites due to the randomised design and similar funding to all sites from other funders (including government payment of staff salaries). Although there may have been some fungibility at the facility level away from MCH services in PBF sites, biasing our PDF site upwards, this will have been minimal and only apply to non-salary items. Further, the PBF impact evaluation shows that NSPs, in implementing the PBF programme, demonstrated different capacities that might have affected both cost and effectiveness. However, aside from the control design, we did not have additional data to explore the impact of these differences on cost-effectiveness. On the effect side, there may have been a spillover effect from the PBF group into the control group due to the location of both facilities in the same province and the movement of staff and the population across facilities. Control health facilities were likely aware of PBF and tried to compete with treatment health facilities on performance. An additional limitation is that the estimation of staff time spent on each service and proportional use of equipment for each service was based on self-reported interviews with health facility staff that may be prone to bias. We also had to source some data regarding maternal and neonatal-related complications from outside the study.^11–15 These parameter limitations were addressed by applying a wide-range sensitivity analysis.

To conclude, our study found that PBF, as implemented in the Afghan context, was not the best use of funds to strengthen the delivery of MCH services. It is likely that the amounts provided were too low for some services, there was insufficient flexibility in using those resources to address service performance constraints and data verification was not economically efficient. Further research into the efficiency and cost-effectiveness of PBF schemes with different designs in different settings is important to ensure that PBF improves performance and inform how best to strategically purchase health benefit packages in LMICs in order to make progress towards UHC.