Performance-based financing in low- and middle-income countries: still more questions than answers

Performance-based financing is promoted as a promising strategy for improving health service delivery and helping to reach the Millennium Development Goals.^1,2 But what is the evidence supporting its use?

Performance-based financing may be defined as “the transfer of money or material goods conditional on taking a measurable action or achieving a predetermined performance target”.² We recently conducted a Cochrane review of the available evidence on the effectiveness of performance-based financing in low- and middle- income countries.³ Nine studies, of which fewer than half had been published in scientific journals, fulfilled our inclusion criteria: one randomized controlled trial (RCT) and two interrupted time series conducted in Asia, and six controlled before–after studies conducted in Africa. Only two outcomes related to health care utilization – institutional deliveries and antenatal care – were assessed in more than one trial. Inconsistent results across studies made summarizing and interpreting the evidence difficult.

The most rigorous African study reported a moderate increase in institutional deliveries, from around 35% to 42% (i.e. 7 percentage points; 95% confidence interval, CI: 1–14).⁴ Findings from studies in Burundi,⁵ the Democratic Republic of the Congo⁶ and Rwanda⁷ showed disparate findings: one reported a significant increase in institutional deliveries, another found little or no change and the third showed a significant decrease. For these findings the risk of bias is high, partly because intervention and control areas were not randomly allocated and the same people who implemented the programmes also evaluated them. Two additional studies reporting on institutional deliveries were programme evaluations with a substantial risk of bias due to questionable data quality. The results on antenatal care attendance were also heterogeneous.

These findings clearly show that no general conclusion can be drawn regarding the likely impact of performance-based financing in low-and middle-income countries. For one thing, most of the studies found through the review were methodologically weak and had poor internal validity. Furthermore, since the impact of complex interventions such as performance-based financing depends largely on the context in which they are implemented, results may vary. Finally, the studies differed substantially in the way in which the performance-based financing scheme was designed and implemented.

In our view, minimizing the risk of producing biased results in future performance-based financing evaluations is essential. Since performance-based financing schemes are not likely to have a large impact, effect estimates need to be as reliable and precise as possible to avoid missing real effects or being misled by seemingly positive findings. Thus, trials should be conducted in settings where a sufficiently large number of facilities or geographical areas can be randomized.

Sometimes RCTs are too expensive, time-consuming or impractical. The most recent study of performance-based financing, from Rwanda, illustrates the practical setbacks researchers sometimes face. The study was set up as a well-designed RCT with districts randomly assigned to performance-based financing intervention and non-intervention groups,⁴ but right before the trial began, the government redefined the administrative district boundaries and several of the districts in the control group were shifted to the intervention group. This may have weakened the validity of the study findings.

If conducting an RCT is not feasible, other robust evaluation designs should be considered. One option is an interrupted time series, in which outcome data are collected at regular intervals during baseline and post-intervention periods (e.g. for one year before and after). This type of study, however, normally requires access to reasonably good routine data from hospital records or other reliable sources not always available in low-resource settings.

Rather than focus exclusively on targeted indicators, future performance-based financing evaluations should consider untargeted effects and systems effects by starting with a conceptual framework of how the intervention might work and what the important outcome measures are. Also, study protocols and evaluation plans should be prepared and published as early as possible, preferably before programme implementation. It is of concern that, in a search conducted on 3 April 2012, we found no ongoing performance-based financing trials from low- and middle-income countries in the International Clinical Trials Registry Platform. We hope that more and better evaluations will emerge in future updates of our Cochrane review to enable us to formulate forthright conclusions about the effects of performance-based financing in low- and middle-income countries.