Good while it lasted? Estimating the long-term and withdrawal effects of results-based financing in Malawi on maternal care utilisation using routine data

Abstract

Objectives

This study aimed to evaluate the effect of introduction and subsequent withdrawal of the Results-based Financing for Maternal and Newborn Health Initiative (RBF4MNH) in Malawi on utilisation of facility-based childbirths, antenatal care (ANC) and postnatal care (PNC).

Design

A controlled interrupted time series design was used with secondary data from the Malawian Health Management Information System.

Setting

Healthcare facilities at all levels identified as providing maternity services in four intervention districts and 20 non-intervention districts in Malawi.

Participants

Routinely collected, secondary data of total monthly service utilisation of facility-based childbirths, ANC and PNC services.

Interventions

The intervention is the RBF4MNH initiative, introduced by the Malawian government in 2013 to improve maternal and infant health outcomes and withdrawn in 2018 after ceasing of donor funding.

Outcome measures

Differences in total volume and trends of utilisation of facility-based childbirths, ANC and PNC services, compared between intervention versus non-intervention districts, for the study period of 90 consecutive months.

Results

No significant effect was observed, on utilisation trends for any of the three services during the first 2.5 years of intervention. In the following 2.5 years after full implementation, we observed a small positive increase for facility-based childbirths (+0.62 childbirths/month/facility) and decrease for PNC (−0.55 consultations/month/facility) trends of utilisation respectively. After withdrawal, facility-based childbirths and ANC consultations dropped both in immediate volume after removal (−10.84 childbirths/facility and −20.66 consultations/facility, respectively), and in trends of utilisation over time (−0.27 childbirths/month/facility and -1.38 consultations/month/facility, respectively). PNC utilisation levels seemed unaffected in intervention districts against a decline in the rest of the country.

Conclusions

Concurrent with wider literature, our results suggest that effects of complex health financing interventions, such as RBF4MNH, can take a long time to be seen. They might not be sustained beyond the implementation period if measures are not adopted to reform existing health financing structures.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• Use of routine secondary data allowed for investigation of long-term impact of the Results-based Financing for Maternal and Newborn Health Initiative over 7.5 years, including withdrawal effects.

• Although we used imputation strategies, sensitivity analysis, and liaised with local Health Management Information System management staff about missing data, our results might still be subject to bias due to data quality issues.

• We were limited by a narrowly defined indicator of postnatal care, which excludes any care provided while the women are still hospitalised after childbirth and could have skewed our findings.

Introduction

Over the last few decades, there has been a concerted global effort to reduce maternal mortality ratio (MMR) and end preventable deaths of newborns and children under 5 years of age.^{1 2} The global MMR estimated in 2017 was 211 maternal deaths/100 000 live births, which reduced by 38% since 2010.¹ Despite this improvement, 86% of all maternal deaths occurred in low-income and middle-income countries (LMICs) in sub-Saharan Africa, reflecting the inequalities in income and access to healthcare services in these countries.² Among various approaches to improve access to quality healthcare in LMICs, results-based financing (RBF) has caught traction over the last 15 years. RBF includes supply-side and demand-side interventions remunerating institutions and/or individuals for attaining predefined performance standards or behaviours.³ On the supply side, RBF enhances financial and managerial autonomy at decentralised levels. Performance contracts between healthcare providers and usually the government stipulate payments conditional on meeting defined targets. On the demand side, RBF aims at enhancing service utilisation or other behaviours through direct rewards such as conditional cash transfers (CCTs).³

A recent systematic review of 59 studies across 25 LMICs on the effectiveness of RBF, most focusing on maternal and child health (MCH), found that while RBF might have improved certain aspects of quality of care for targeted services, its effects on utilisation of services are mixed, both within and across interventions and countries.³ In contrast to a large body of evidence on effectiveness of introducing RBF to low-resource settings, although limited in follow-up, we know of only three studies assessing impact of removing RBF once implemented.^4–6 Of these, only one assessed the impact of RBF withdrawal on the utilisation of health services, finding no withdrawal effect in Mali.⁴ However, RBF showed no impact in the first place, and the intervention was withdrawn after only 16 months, calling into question the generalisability of this Malian experience. The dearth of evidence is particularly relevant as many RBF pilot schemes are in flux currently, with decisions on scale-up or closing outstanding. Understanding what happens when interventions are withdrawn is essential for the decision-making process.

Our study contributes towards filling this lacuna. Specifically, we evaluated the effect of the introduction and subsequent withdrawal of the Results-based Financing for Maternal and Newborn Health Initiative (RBF4MNH) in Malawi, implemented from 2013 to 2018, on the utilisation of three MCH services, facility-based childbirths, antenatal care (ANC) and postnatal care (PNC).

Intervention design

Prior to the intervention, Malawi had one of the poorest health outcomes for mothers and newborns globally, with maternal mortality at 675 deaths/100 000 live births and neonatal mortality at 31 deaths/1000 live births in 2010.⁷ In addition, the 2011 emergency obstetric care (EmOC) Needs Assessment recorded only 47% of facilities offering comprehensive services. This was attributed to widespread workforce shortages including nurses, midwives and doctors, alongside poor motivation, and low skills and knowledge levels.⁸ The RBF4MNH Initiative aimed at reducing maternal mortality by increasing the number and quality of institutional childbirths. It was introduced in four non-randomly selected districts, that is, those with the poorest maternal health outcomes, out of the country’s 28, namely: Dedza, Mchinji, Ntcheu and Balaka.⁸

On the supply side, RBF4MNH aimed at strengthening the already existing EmOC infrastructure. It comprised financial incentives conditional on performance improvements to district health management teams and 18 EmOC facilities initially.^{8 9} An additional 10 and 5 EmOC facilities were included in October 2014 and in October 2015, respectively, to cover all 33 facilities across 4 districts. Performance indicators at the facility-level focused on content of care, and on procurement and maintenance of equipment at the DHMT level.⁹ Service utilisation targets, specifically increase in number of facility-based childbirths, were also incentivised, although constituting only a smaller proportion of the overall incentive envelope.⁹ Of the rewards earned by the facility, 30% had to be spent on facility-wide initiatives, whereas the remaining 70% could be either distributed to the staff as bonus payments or also invested into the health facility.^{9 10} On average, staff bonus payments amounted to about 6% of salary, with wide variations by facility and cadre.¹⁰ On the demand side, CCTs (approximately €6) were offered to pregnant women from the catchment area of an RBF4MNH facility arriving for childbirth.⁹ These were intended to partially reimburse for costs associated with seeking childbirth and postpartum services, such as transport and subsistence costs during the health facility stay, paid out in three parts, on arrival, after childbirth and after a 48-hour stay at the facility.⁹

After 5 years of implementation, RBF4MNH delivered its final round of payments in March 2018 and closed officially in June 2018, after donor funding ended.^{8 9}

Theory of change in relation to maternal care utilisation

RBF4MNH aimed at directing women’s childbirth utilisation patterns towards EmOC facilities and away from non-EmOC ones, to improve the quality of institutional deliveries and thereby prevent maternal and neonatal deaths. A primary data-based impact evaluation, comparing RBF4MNH to non-RBF4MNH facilities within the intervention districts, provided no evidence for a substantial redirection of demand within the first 2 years of implementation,⁹ but suggested that this might, at least in part, be due to the heavy involvement of the DHMTs, resulting in district-wide improvements, beyond just the intervention facilities and their catchment population.^10–12

We expected an overall increase in facility-based childbirths at district level, driven by the combination of supply-side incentives and CCTs. On the supply side, prior research has demonstrated the potential of RBF for affecting, both positively and negatively, health worker motivation and effort through a range of mechanisms.^13–15 Effort, in this context, might entail both improvements in technical and interpersonal quality of care as well as explicit activities to motivate women to attend the health facility for childbirth. In Malawi, qualitative research has indicated a range of positive motivational effects of RBF4MNH for health workers, particularly through improvements in the working environment, but also a number of demotivating effects associated with the financial incentives.¹⁰ On the demand side, we expected that the intervention’s potential to lower pre-existing financial and quality of care-related barriers to facility-based childbirth for women, such as transport and in-hospital stay expenses, lack of midwives, insufficient equipment to be used during delivery,^{16 17} would increase women’s inclination to give birth in a health facility. As for health workers, qualitative research largely confirmed both for the specific case of RBF4MNH.^{8 12} Against already high utilisation levels of childbirth care prior to implementation (around 90% in the intervention districts),¹¹ we expected resulting increases in facility-based childbirth utilisation to be rather small. We did not expect substantial changes in non-incentivised services such as ANC and PNC, although the intervention bears potential for both positive and negative externalities through overall quality improvement or a focus on childbirths taking attention away from other services, respectively.

Assumptions on the effects of RBF withdrawal also related to both the supply and demand side. On the supply side, prior research on the effects of withdrawing RBF on health worker motivation,^{5 6} and theory regarding the adverse effect of paying for performance on intrinsic motivation,¹⁸ suggest that health workers might reduce efforts if no longer paid for tasks or services for which they had previously been explicitly remunerated. We also expected that the reduced cash inflow to facilities following the withdrawal of RBF would reverse some of the RBF-induced improvements in working conditions, negatively affecting health worker motivation and effort, and resulting in reduced quality of care. On the demand side, withdrawing the intervention might, therefore, reinstate previous quality of care-related and financial barriers to health service use. Further, motivation of patients might have been eroded by the CCTs, and CCT removal may have consequently resulted in reduced motivation to attend a health facility for childbirth, beyond mere ability to pay. Taken together, we expected that the withdrawal of RBF4MH would lead to a drop in service volume especially for childbirths, and to a lesser extent also for ANC and PNC.

Methods

Study design and outcome variable

We used a controlled interrupted time series (ITS) design. In line with the above argument on district-wide effects, we compared the four intervention districts to control districts using monthly, facility-disaggregated data on the number of childbirths and ANC/PNC consultations provided by the Malawian Health Management Information System (HMIS) from July 2012 to December 2019, a period of 90 consecutive months using three interruption time points; namely introduction of RBF4MNH in April 2013, the completed roll-out of programme by October 2015, and end of the last performance incentive round in March 2018. We used intervals of 3 months to account for time lags in roll-out.

As control districts, we identified 20 of the 24 non-intervention districts. We excluded four districts (Likoma, Lilongwe, Mzimba North and South, and Zomba) due to a lack of a priori comparability, with three being large urban centres and one a small remote island, all characterised by very different service provision structures, utilisation patterns and service volumes. For the remaining 20 control districts, inspection of preintervention utilisation trends for each of the three indicators demonstrated that they were sufficiently comparable to the four intervention districts.

As per the theory of change, we analysed volume of facility-based childbirths, ANC and PNC as outcome indicators. The HMIS indicator reflects total counts per month per facility of (1) childbirths in facility irrespective of mode of delivery, (2) ANC consultations irrespective of timing and number of visits or ANC content provided and (3) PNC consultations within 2 weeks after childbirth irrespective of timing and number of visit or PNC content provided. However, the indicator excludes PNC services received by a woman while still hospitalised following childbirth.

Data preparation

An issue with HMIS data in many settings is variability with respect to data quality and information completeness.¹⁹ In Malawi, however, HMIS data on key maternal care indicators was found to be of sufficiently high quality by our team in prior analyses^{20 21} and by others,^{22 23} with acceptable gaps and inaccuracies. Investigations included, for instance, statistical comparison of entries in electronic records to physical registers.²² For this study, we first conducted a statistical assessment of data quality, and then involved HMIS management staff to ascertain possible causes of gaps or seeming inaccuracies in data, and to decide on data cleaning and imputation strategies, including sensitivity analyses, as described below.

Data were extracted into Microsoft Office Excel spreadsheets from DHIS2, the web platform on which HMIS is hosted, for the three indicators for the time period of July 2012 to December 2019. We omitted facilities that either reported no data and/or all zeroes for more than 85 out of the total 90 months, assuming that these facilities were unable to reliably provide MCH services. 493, 445 and 454 facilities provided ANC, childbirth and PNC services, respectively. We further excluded any facility if more than 30% of data points were missing or more than 10% of data points were outside of the mean±2 SD range for the facility. For ANC, we retained 459 facilities (84 intervention, 375 control); for childbirths 414 facilities (79 intervention, 335 control) and for PNC 378 facilities (79 intervention, 299 control). We inspected any remaining missing data points (ANC 14.8%, childbirths 8.1%, PNC 12.6%), which appeared to be largely missing at random and similar across intervention and control districts. We imputed these using single imputation, taking the average of 6 months, three prior and post, for each missing observation. To ascertain that imputation had not inadvertently introduced bias, we conducted all analyses both with raw and imputed data, without any significant differences, and therefore, only present results of the imputed data set.

Data analysis

We averaged monthly service utilisation data at district level and then aggregated across all intervention and all control districts, to analyse immediate effect and change in time trends within and between preintervention, intervention phase 1 (ie, after the introduction of RF4MNH), intervention phase 2 (ie, after full roll-out in October 2015) and the postintervention period (after removal). As we observed slight seasonal patterns in the data, with higher utilisation during the months of June–September (cool-dry winter season), we adjusted our model for seasonality. The ITS regression model:

$${\displaystyle y_t=\begin{array}{l}{\beta }_0+{\beta }_1{T}_t+{\beta }_{2}z+{\beta }_{3}z{T}_t+{\beta }_4{x}_1+{\beta }_5{x}_1{T}_t+{\beta }_{6}z{x}_2+{\beta }_{7}z{x}_1{T}_t+{\beta }_8{x}_2\\ +{\beta }_9{x}_2{T}_t+{\beta }_{10}z{x}_2+{\beta }_{11}z{x}_2{T}_t+{\beta }_{12}{x}_3+{\beta }_{13}{x}_3{T}_t+{\beta }_{14}z{x}_3+{\beta }_{15}z{x}_3{T}_t+{\in }_t\end{array}}$$

where $y_t$ represents total count for each service measured monthly at time t, $T_t$ is the time elapsed since start of study period, $x_1$ , $x_2$ and $x_3$ are dummy variables representing the study periods pertaining to each implementation change ( $x_1=0$ preintervention; $x_1=1$ first intervention phase; $x_2=1$ second intervention phase; and $x_3=1$ postintervention), z is a dummy variable representing treatment group (0=control, 1=RBF), ${\beta }_0$ and ${\beta }_1$ are the baseline level and trend difference in the control group, ${\beta }_2$ and ${\beta }_3$ represent the difference in level and trend between the control and intervention group in the preintervention phase, ${\displaystyle {\beta }_4,{\beta }_5}$ , ${\beta }_6$ and ${\beta }_7$ in the first intervention phase, ${\displaystyle {\beta }_8,{\beta }_9}$ , ${\beta }_{10}$ and ${\beta }_{11}$ in the second intervention phase and ${\displaystyle {\beta }_{12},{\beta }_{13}}$ , ${\beta }_{14}$ and ${\beta }_{15}$ in the postintervention phase. The model estimated the respective coefficients using generalised least squares-fitted linear regression using an autoregressive moving average process.

R V.4.0.2 was used to conduct all analysis.

Patient and public involvement

Patients and the public were not involved in this research, which was exclusively based on secondary data, but we involved key implementation and funding stakeholders in the data cleaning and preparation stage as well as in the interpretation of results.

Results

Sample distribution and characteristics are shown in table 1. The time series plots and ITS regression model estimates for mean monthly utilisation of facility-based childbirth, ANC and PNC are shown in figure 1 and tables 2–4, respectively.

Table 1.

Sample distribution and characteristics in July 2012

	Intervention	Control	Total
Total no of districts	4	20	24
ANC
Total no of health facilities	84	375	459
Mean (SD) monthly services per facility	221.08 (14.30)*	175.35 (6.88)*	198.21 (25.51)
Facility-based delivery
Total no of health facilities	79	335	414
Mean (SD) monthly services per facility	91.20 (6.38)*	70.44 (4.68)*	80.82 (11.81)
PNC
Total no of health facilities	79	299	378
Mean (SD) monthly services per facility	39.13 (4.89)	38.79 (3.27)	38.96 (4.15)

*Strong evidence of difference in means between groups at 99% CI.

ANC, antenatal care; PNC, postnatal care.

Figure 1.

Time trends by treatment group for (A) facility-based childbirths, (B) antenatal care and (C) postnatal care. Dots represent utilisation per month averaged across all facilities within intervention and control districts, respectively; lines represent predicted utilisation based on segmented linear regression model accounted for seasonality and autocorrelation.

Table 2.

Predicted effect of RBF4MNH implementation and withdrawal on facility-based childbirth service utilisation

Phase	Model parameter	Estimated utilisation	95% CI
Preintervention	Control level	76.26†	67.64 to 84.88
	Control monthly trend	−1.44	−3.09 to 0.21
	Difference in level between control and intervention	29.31†	17.30 to 41.33
	Difference in monthly trend between control and intervention	−0.38	−2.70 to 1.93
Intervention phase 1 (after introduction of RBF)	Level change in control	8.13	−4.38 to 20.65
	Trend change in control	1.49	−0.18 to 3.17
	Difference in level change between control and intervention	2.63	−14.91 to 20.19
	Difference in trend change between control and intervention	−0.08	−2.44 to 2.27
Intervention phase 2 (after full roll-out)	Level change in control	−5.50	−12.71 to 1.70
	Trend change in control	0.26	−0.16 to 0.69
	Difference in level change between control and intervention	7.74	−2.43 to 17.93
	Difference in trend change between control and intervention	0.62*	0.01 to 1.23
Postintervention (after removal of RBF)	Level change in control	−0.34	−8.09 to 7.40
	Trend change in control	−0.10	−0.65 to 0.45
	Difference in level change between control and intervention	−10.84*	−21.79 to 0.10
	Difference in trend change between control and intervention	−0.27*	−1.06 to 0.50

*Strong evidence of difference between groups at 95% CI.

†Strong evidence of difference between groups at 99% CI.

RBF, results-based financing; RBF4MNH, Results-based Financing for Maternal and Newborn Health Initiative.

Table 3.

Predicted effect of RBF4MNH implementation and withdrawal on ANC utilisation

Phase	Model parameter	Estimated utilisation	95% CI
Preintervention	Control level	173.24†	158.09 to 188.40
	Control monthly trend	0.20	−2.83 to 3.25
	Difference in level between control and intervention	46.37†	25.11 to 67.64
	Difference in monthly trend between control and intervention	−2.02	−6.33 to 2.27
Intervention phase 1 (after introduction of RBF)	Level change in control	−5.20	−26.20 to 15.79
	Trend change in control	0.07	−2.93 to 3.07
	Difference in level change between control and intervention	7.11	−22.56, 36.78
	Difference in trend change between control and intervention	2.73	−1.51 to 6.97
Intervention phase 2 (after full roll-out)	Level change in control	−15.45†	-21.82 to 9.09
	Trend change in control	0.38	0.01 to 0.76
	Difference in level change between control and intervention	−5.78	−14.78 to 3.20
	Difference in trend change between control and intervention	0.09	−0.44 to 0.62
Postintervention (after removal of RBF)	Level change in control	−6.30	−13.61 to 0.99
	Trend change in control	−0.67*	−1.20 to 0.14
	Difference in level change between control and intervention	−20.66†	−30.99 to 10.33
	Difference in trend change between control and intervention	−1.38†	−2.12 to 0.63

*Strong evidence of difference between groups at 95% CI.

†Strong evidence of difference between groups at 99% CI.

ANC, antenatal care; RBF, results-based financing; RBF4MNH, Results-based Financing for Maternal and Newborn Health Initiative.

Table 4.

Predicted effect of RBF4MNH implementation and withdrawal on PNC utilisation

Phase	Model parameter	Estimated utilisation	95% CI
Preintervention	Control level	44.41†	40.14 to 48.68
	Control monthly trend	−1.51†	−2.34 to 0.68
	Difference in level between control and intervention	−1.61	−7.59 to 4.37
	Difference in monthly trend between control and intervention	1.11	−0.05 to 2.28
Intervention phase 1 (after introduction of RBF)	Level change in control	8.06*	1.84 to 14.28
	Trend change in control	1.59†	0.75 to 2.44
	Difference in level change between control and intervention	−5.98	−14.73 to 2.76
	Difference in trend change between control and intervention	−0.96	−2.15 to 0.22
Intervention phase 2 (after full roll-out)	Level change in control	−2.95	−6.31 to 0.40
	Trend change in control	0.23*	0.03 to 0.43
	Difference in level change between control and intervention	−5.30*	−10.05 to 0.55
	Difference in trend change between control and intervention	−0.55†	−0.84 to 0.27
Postintervention (after removal of RBF)	Level change in control	−1.43	−5.06 to 2.19
	Trend change in control	−0.48†	−0.74 to 0.22
	Difference in level change between control and intervention	−0.26	−5.39 to 4.87
	Difference in trend change between control and intervention	0.49*	0.12 to 0.86

*Strong evidence of difference between groups at 95% CI.

†Strong evidence of difference between groups at 99% CI.

PNC, postnatal care; RBF, results-based financing ; RBF4MNH, Results-based Financing for Maternal and Newborn Health Initiative.

During intervention phase 1, childbirth utilisation slightly decreased for intervention districts while staying stable in control districts. However, ITS estimates indicate differences in service utilisation had not changed in a statistically significant way between preintervention and intervention phase 1. For ANC and PNC, we observed a slight increase in service utilisation in intervention districts while control districts remained relatively stable. ITS estimates indicate changes in utilisation trends from the preintervention phase to intervention phase 1 were not statistically significant. During the second intervention phase (ie, once the programme was fully implemented), we observed parallel positive utilisation trends for childbirth and ANC, while PNC utilisation declined in intervention districts, but increased in control districts. This resulted in a significant positive reversal in the childbirth utilisation trend between phases 1 and 2 (+0.62 childbirths/month/facility). For PNC utilisation, this resulted in a negative interruption effect (−5.3 consultations/month/facility) as well as a significant negative reversal in the trend of PNC utilisation between phases 1 and 2 (−0.55 consultations/month/facility).

After removal of RBF, we observed unique trends for each utilisation indicator: Childbirth utilisation showed parallel upward trends, but less steep in intervention districts compared with the previous period; ANC utilisation flattened in control districts but declined in intervention districts; PNC utilisation trends showed a parallel decline after programme removal. This resulted in a negative reversal of the trends for both childbirth (−0.27 childbirths/month/facility) and ANC utilisation (−1.38 consultations/month/facility) between phase 2 and the postintervention phase. We further estimated a negative interruption effect on both childbirth (−10.84 childbirths/facility) and ANC utilisation (−20.66 consultations/facility). For PNC utilisation, RBF4MNH removal reverted the earlier negative trend into a positive trend with similar effect size (+0.49 consultations/month/facility).

Discussion

Our study aimed to make a unique contribution to the RBF literature by investigating long-term impacts of RBF4MNH in Malawi and withdrawal after a 5-year implementation period. We investigated three key indicators of maternal care continuum: facility-based childbirth utilisation, the focus of RBF4MNH, as well as ANC and PNC utilisation, not directly incentivised but potentially affected by positive or negative externalities given their close relationship with childbirth services.

Impact of RBF4MNH implementation

Our district-level assessment did not identify any substantial overall impact of RBF4MNH on childbirth utilisation during the first 2.5 years of implementation. Nevertheless, during the second intervention phase after full roll-out of the programme, we observed a small ‘late’ positive effect on facility-based childbirth utilisation, which was directly incentivised by RBF4MNH. This indicates that complex interventions such as RBF4MNH may take time to unfold effects, and that long-term impact evaluations are crucial to understanding their full potential.

We found no substantial effect on ANC and PNC utilisation during the first intervention phase. During the second intervention phase, PNC utilisation in control districts showed a substantial upward trend which exceeded the relatively unchanged trend observed in RBF4MNH districts. While the unchanged trend observed in intervention districts aligns with our expectations from the primary impact evaluation,^{8 11} we suspect that the increasing PNC utilisation trend in control districts is caused by changes in timing of service utilisation and related HMIS reporting. Specifically, within the Malawian context, promotion of PNC has been generally poor and primarily focused on the first 48 hours,²³ rather than within the first 6 weeks after childbirth as recommended by WHO.²⁴ The only HMIS indicator pertaining to PNC currently in use aggregates all outpatient PNC consultations by women within 2 weeks after giving birth, excluding both PNC services while the woman is still in hospital as well as after the 2-week period postchildbirth. RBF4MNH incentivised through the CCTs a 48-hour postpartum stay during which women received initial PNC, not captured in the HMIS indicator we used in our study. The seeming negative impact of RBF4MNH on PNC is, therefore, likely not reflective of a decrease in postnatal check-ups per se. Rather, we assume it to be due to the timing of care—with early PNC being shifted from outpatient return visits to inpatient postpartum care—and related registration in HMIS.

Impact of RBF4MNH withdrawal

Our results support concerns about potential unintended negative effects of removing incentives. We found that in comparison to control districts, counts of childbirth and ANC utilisation in RBF4MNH districts dropped immediately after withdrawal of the intervention and further declined over the following 21 months of observation. Removal of the CCTs might have reinstated pre-existing economic and quality of care-related barriers. Further, the absence of performance payments might have lowered both individual workers’ motivation and services’ ability to maintain high-quality provision, as well as women’s motivation to give birth in a health facility in light of potentially decreased quality of care and lack of reimbursement to which they had since become accustomed.

We would like to note that while figure 1 may suggest that service utilisation after withdrawal not only reversed, but even dropped slightly below preintervention levels, it is important to consider the overall decline in fertility rates in Malawi during our study period, from a national total fertility rate of 5.7 in 2010 to 4.4 in 2015/2016.²⁵ Therefore, with fewer births, there was lesser need of MCH services, and thus lower service provision levels in health facilities. Our results should be interpreted in this context of overall decline of fewer babies being born, and additional population-based research would be necessary to understand to what extent gains made during implementation were reversed.

For PNC, in intervention districts, the negative trend observed in the second implementation period continued into the postintervention period, against a reversal of the positive trend in control districts. Particularly considering the registration issue discussed above, further research is necessary to fully understand this finding. However, it appears that unlike for childbirths and ANC, withdrawal of RBF4MNH did not lead to a drop-in outpatient service utilisation within the first 2 weeks postchildbirth.

Our findings in relation to the wider RBF literature

Our findings regarding the impact of RBF4MNH implementation resonate findings in the wider body of RBF impact evaluations, although most are without a demand-side component. A recent Cochrane review on paying for performance to improve delivery of health services in LMICs indicated that impact on the utilisation of directly incentivised services such institutional childbirths attended by skilled personnel tended to be positive, whereas impact on services not directly incentivised such as ANC tended to be neutral and impact on PNC utilisation tended to be negative, although from mixed quality evidence.³ This is concurrent with our findings discussed above. Other evaluations of similar health financing schemes in Burundi also showed similar effects,²⁶ the authors hypothesising in similar lines as our theory of change, specifically that direct financial rewards for facility-based childbirths may have led to providers making special effort to convince and sensitise women to give birth at a hospital.²⁷ For indirectly or non-incentivised services such as ANC, the relatively lower financial incentive would warrant much less effort.²⁶ Additionally, use of ANC services may be perceived as more ‘optional’ and is subject to cultural and behavioural norms such as knowledge of benefit of ANC or immunisations.²⁶ Further community-specific research and interventions are required to influence these health-seeking behaviours. For PNC utilisation, authors evaluating a performance based financing (PBF) scheme in Nigeria also found no impact, hypothesising that PNC is difficult to monitor as it requires repeated action by the healthcare provider, often in various settings.²⁸

Health financing strategies such as RBF have been seen as a tool to reform the health sector and lead countries towards universal healthcare. However, as our results show, these schemes may take a long time to unfold effects, may be somewhat ineffective and may have negative consequences once removed. As many RBF projects have or are soon ending, it will be valuable to replicate this study in other settings to better understand the extent to which RBF might have adverse long-term effects if not integrated into routine care structures and health service management and complemented with measures to counteract or mitigate potential unintended negative effects.

Methodological considerations

Our study is primarily limited by the sole use of secondary data and associated data quality issues. First, while we mitigated potential bias due to missing data, by using imputation strategies, sensitivity analysis, and liaising with local HMIS management staff about gaps in the data, we cannot fully exclude that our imputation strategy impacted the results obtained. Second, intervention districts were non-randomly selected by the Malawian Ministry of Health at the time of implementation of RBF4MNH. While we undertook efforts to ascertain that intervention districts were reasonably similar to the districts used as controls, we cannot fully exclude dissimilarities, both a priori and developing over the course of the study period, which might have impacted our results. In particular, while RBF4MNH districts did not receive additional MCH-related interventions in the implementation period, many concurrent interventions were implemented in control districts across the country. Although primarily focused on family planning and quality of care rather than perinatal service utilisation, we acknowledge that our estimates represent RBF4MNH effects against the reality of a plethora of other interventions rather than against no intervention. Following the withdrawal of RBF4MNH, the only notable intervention in RBF4MNH districts pertained to improving quality of newborn care and was implemented nationally, thereby equally affecting control districts. Finally, in estimating RBF4MNH impact on PNC, we were limited by the narrowly defined HMIS indicator which only records PNC visits within 2 weeks postchildbirth and which excludes any care provided while the women are still hospitalised after childbirth. In future revisions of the HMIS in Malawi and beyond, refining and expanding the PNC indicator might be a worthwhile effort to allow better understanding of PNC utilisation.

Conclusion

Concomitant with prior research, we found no immediate impact of RBF on maternal care utilisation after introduction of the intervention but observed an increase in utilisation of directly incentivised services after full roll-out of the programme. RBF withdrawal might result in reversal of gains, although the exact mechanisms remain unclear from our study and the little other available literature. While more research on sustainability of RBF is urgently necessary, our findings support prior calls for planning ahead for discontinuation and investing in measures to enhance sustainability of intervention gains.

Data availability statement

Data may be obtained from a third party and are not publicly available. We do not have permissions to share the national data as used in this study, however, this can be requested from the Malawian Ministry of Health.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

As only anonymised routine data were used, no approval by an ethics review board was necessary. The Malawian Ministry of Health authorised use of the data for the purpose of this study.