Effects of performance based financing on facility autonomy and accountability: Evidence from Zambia

Highlights

• •PBF can enhance financial and managerial autonomy, and accountability in service delivery.
• •Additional financing is a necessary but insufficient for enhancing autonomy and accountability.
• •PBF promoted good governance by re-enforcing existing structures and mechanisms.
• •The magnitude of change across the implementing entities can be different.

Abstract

Several low and lower- middle income countries have been using Performance-Based Financing (PBF) to motivate health workers to increase the quantity and quality of health services. Studies have demonstrated that PBF can contribute to improved health service delivery and health outcomes, but there is limited evidence on the mechanisms through which PBF can necessitate changes in the health system. Using difference-in-difference and synthetic control analytical approaches, we investigated the effect of PBF on autonomy and accountability at service delivery level using data from a 3-arm cluster randomised trial in Zambia. The arms consisted of PBF where financing is linked to outputs in terms of quality and quantity (intervention 1), input financing where funding is fully provided to finance all required inputs regardless of performance (intervention 2), and the current standard of care where there is input financing but with possible challenges in funding (pure control). The results show an increase in autonomy at PBF sites compared to sites in the pure control arm and an increase in accountability at PBF sites compared to sites in both the input-financing and pure control arms. On the other hand, there were no effects on autonomy and accountability in the input-financing sites compared to the pure control sites. The study concludes that PBF can improve financial and managerial autonomy and accountability, which are important for improving health service delivery. However, within the PBF districts, the magnitude of change was different, implying that management and leadership styles matter. Future research could examine whether personal attributes, managerial capacities of the facility managers, and the operating environment have an effect on autonomy and accountability.

1. Introduction

During the past decade, several low and lower-middle income countries (LLMIC) have introduced performance-based financing (PBF) mechanisms as a means to improve the motivation of health providers, to enhance accountability in the provision of health services, and ultimately to increase the quantity and quality of healthcare [1], [2], [3]. Such decisions have been motivated by the need to improve health outcomes, particularly reproductive, maternal, neonatal, and child health (RMNCH) outcomes. In developing countries, notable barriers that affect utilisation and quality of care are: low availability, imbalanced staff mix, and sub-optimal performance of health workers; inadequate funding to health facilities; low managerial capacities; and lack of performance incentives and autonomy to manage health facilities [1], [4], [5], [6].

PBF has been advocated as a key transformative approach to health financing, with the potential to strengthen health systems, and improve health outcomes [4], compared to the dominant financing system where funding of health services is focussed on what inputs and activities are needed to deliver health services. These input-based financing models have been criticised for their focus on input and not output and hence not providing sufficient incentive to deliver more and better health services, unlike PBF. However, literature shows varied effects of PBF on health outputs and quality, with some studies showing that PBF can improve health outputs and quality [7], [8], [9], while others show adverse or no effects [10]. Critics of PBF argue that it may lead to undesirable effects such as dilution and crowding out of workers intrinsic motivation, neglect of non-numerated aspects of work (gaming), financial drain to the economy as it is costly to establish a system that continuously monitors the quantity and perceived quality of health sector performance; and dependence on financial incentives [11], [12], [13]. Consequently, Paul et al., [14] contend that implementers of PBF provide minimal attention to systemwide and long-term effects, which can be damaging to the health system.

Despite the potential successes and challenges with PBF, the contextual factors or mechanisms through which PBF improves health outcomes is still unclear [5], [15]. One school of thought is that PBF can facilitate improvements in the motivation of health workers, which can lead to an increase in accountability and performance of the health system. On health worker performance, the impact of PBF on human resources for health has been widely investigated, with studies showing that PBF improves health worker motivation, job satisfaction and reduces attrition [1], [16], all of which enable the health system to achieve its goals[12], [17].

However, there are limited or no studies on whether PBF can increase autonomy and accountability at the level of health facilities. Mayumana et al. [5] and Fritshe et al. [11] argue that when PBF is associated with increased facility autonomy and accountability, the objectives of the health systems are likely to be met. A strand of literature finds that PBF can play an important role in the enhancement of autonomy of health facilities and in promoting accountability through the separation of roles according to the main functions in the health system [13]. Since local health workers and managers understand local conditions better, it is postulated that giving them the flexibility and freedom to manage resources effectively can lead to improved accountability and service delivery. Apart from autonomy, accountability is achieved when resources are managed transparently without misuse, and when all parties are satisfied with the results or value of the investments. Generally, accountability entails the procedures and processes by which one party justifies and takes responsibility for its activities [18] and includes the imposition of sanctions for failure to abide by appropriate action [19].

The pathways through which PBF affects autonomy and accountability and how it facilitates improvements in service delivery and health outcomes are presented in Fig. 1.

Fig. 1.

Autonomy and accountability mechanisms.

In this paper, we contribute to the literature by presenting evidence on the causal effects of PBF on facility autonomy and accountability in Zambia using a cluster randomised controlled trial. Our study extends the analysis by Mayumana et al. [5] by examining the effects of PBF on both autonomy and accountability and assessing the differential effects by type of funding modality. Specifically, this study seeks to address the following question: Does PBF increase autonomy and accountability at the health facility level? Are there differential effects between input financing and PBF mechanisms?

Like many other LLMICs, resource constraints have limited Zambia’s ability to finance healthcare fully [20] fully. In addition to low funding, the available resources are not efficiently utilised, leading to gaps in service delivery and poor health outcomes [21]. Motivated by the idea of improving efficiency, the Zambian Government, with support from the World Bank, implemented a PBF project between 2012 and 2014 in 11 districts out of the 72 districts at that time, countrywide (See Annex A1). To test the impact of PBF, there was a need to compare PBF to not only to the current standard of care, input financing, which is termed “business as usual” but also to an input financing model, that were financed so that the failure of input financing is not attributed to poor financing of inputs.

The PBF project also had an impact and process evaluation components. One of the three (3) objectives of the impact evaluation was to assess the causal effect of the Zambia PBF project on the population health indicators of interest. Although the evaluation showed that both PBF and the enhanced input financing increased utilisation of MCH services, PBF had an additional effect of increasing the quality of MCH services [4]. This study extends the analysis by examining the effects of PBF on facility autonomy and accountability.

2. Methods

2.1. Study design

The study uses data from the impact and process evaluations of the Zambia health PBF project, which were conducted between October 2011 and January 2015. This was a 3-arm cluster randomised control trial consisting of a PBF intervention arm, an input-financing arm and a pure control (business-as-usual) arm. Districts were selected as units of randomisation and randomly assigned to one of the three study arms. Thirty districts were selected based on administrative and population outcomes data and randomly allocated to each of the three arms, with a few more districts in Northern and Southern (see Annex A2). Inputs were assigned to the three arms as follows: (a) Health centres in the PBF districts received Emergency Obstetric and Neonatal Care (EmONC) equipment and monetary grants. The EmONC equipment was distributed to all the health centres immediately after the project commenced. The PBF grants were disbursed directly into the health centre bank accounts each quarter after verification of performance; (b) Health centres in the input-financing districts received EmONC equipment exactly as in the PBF arm, and money (not conditional on results) equivalent to the performance payments in PBF districts; and (c) Health centres in the pure control arm received nothing and continued with the status quo. The health centres on the PBF scheme were allowed to use 40% of their PBF grants for investments or recurrent expenditures at the health centres and 60% for staff incentives or bonuses. To increase and sustain productivity, health centres on the PBF scheme were penalised for delayed transmission of invoices, discrepancies in reported and verified data, and if there was a reduction in performance on non-incentivised indicators.

2.2. Data and indicators of interest

This study uses both quantitative and qualitative data that was collected during the impact and process evaluations. Quantitative data was collected from 176 health facilities at baseline (October-November 2011) and 210 health facilities at end line (November 2014-January 2015). Some of the data collected include information on infrastructure, availability of basic drugs and equipment, planning and administration, budgeting, and financial and performance management. The key indicators which were used to measure autonomy focussed on the perceived autonomy of the health centre-in-charge in planning and budgeting; and use of financial, material and human resources (See Annex A3). The responses were recorded on a Likert scale ranging from 1 (least autonomy) to 5 (maximum autonomy). Responses were then converted to a binary scale were scores of 4 and 5 were coded as 1 and zero otherwise. In addition, a standardised autonomy index, using the mean and standard deviation for the individual indicators, was constructed. The key indicators that were used to measure accountability were: availability of coordination structures at the health centres (i.e. Health Centre Committees [HCCs]); frequency of HCC and health centre staff meetings; minutes of meetings; existence of a work plan and budget; number of supervisory or technical visits to the health centres; number of supervisory or technical visits by the health centre to community health workers; and frequency of internal and external health centre staff performance assessments (see Annex A4). In addition, a standardised accountability index, using the mean and standard deviation for the individual indicators, was constructed.

Other than the quantitative data, in-depth interviews with key stakeholders at the District Health Office (DHO) and health centres were also conducted. The purpose of these interviews was to review the perceptions of service providers on the effect of PBF on autonomy and accountability. The stakeholders were purposively selected based on their formal and informal role in the implementation of PBF, delivery of health services at the health centres, and experience in managing community-level activities. The actual people who were interviewed were staff at the DHOs (x9) and health centres (x18).

2.3. Empirical strategy

2.3.1. Quantitative data analysis

We use a difference-in difference (DiD) linear regression model to estimate the effect of PBF on autonomy and accountability as captured by the indicators outlined in the previous section. The model is specified as:

$$Y_{\mathit{idt}}={\beta }_0+{\beta }_1{\mathit{Treat}}_d+{\beta }_{2}Post+{\beta }_3({\mathit{Treat}}_d\ast Post)+{\gamma }_d+\sum _{r=1}^{R-1}{\mathit{districts}}_r+{\epsilon }_{\mathit{idt}}$$

where $Y_{\mathit{idt}}$ is the outcome of interest for facility $i$ in district$d$ at time $t$.${\mathit{Treat}}_d$is a dummy variable, indicating treatment status. In one estimation, ${\mathit{Treat}}_d$ takes the value of 1 if the district d where the facility is located is PBF (Intervention 1) and 0 if pure control, while in another estimation the dummy variable is 1 if the district in which the facility is located is input financing (intervention 2) and 0 if not. Apart from comparing PBF and input financing to the pure control, we will also compare PBF to input financing. Thus, we estimate the following models; 1) PBF vs. input-financing group; 2) PBF vs. pure control group; 3) input-financing group vs. pure control group.

$\mathit{Post}$ is a dummy variable denoted as 1 for the end line observations collected after the PBF intervention and 0 otherwise. The coefficient ${\beta }_3$captures the DiD treatment effect; $\gamma$ is the facility fixed effects, to control for facility-level unobserved time-invariant characteristics. To control for regional specific fixed effects, we also include the variable ${\mathit{district}}_r$ where $r$ is the number of matched districts and ${\epsilon }_{\mathit{idt}}$is the error term. To account for autocorrelation and ensure that we obtain correct standard errors—since the treatment varied at district level—we cluster the standard errors at the district level [22]. Clustering standard errors at the district level produces the correct standard errors as would be if the analysis is conducted at the district and not facility level [23].

The estimates from the DiD model are derived on the assumption that trends in autonomy and accountability measured in districts that received PBF and those that did not were moving in parallel. If this assumption is not true, then our results are not valid. Unfortunately, we do not have a way of checking the validity of this assumption since we only have one pre-treatment period. However, Chansa et al. [24], in the evaluation of the effect of PBF on the utilisation and quality of MCH services, shows that the parallel trends assumption holds for MCH indicators using administrative data for the period prior to the implementation of PBF.

For robustness, we also use the synthetic control approach [25]. The approach requires constructing a counterfactual for the intervention facilities by taking a weighted average of the available control facilities, and a higher weight is given to control units that are similar to the intervention sites. The synthetic control group or twin is generated to ensure that the levels of the outcomes prior to the intervention are the same as those of the intervention group so that it can be used as a counterfactual post-implementation of the intervention. Choosing the control facilities to include in the pool for all potential control units is crucial for the synthetic control approach. The basic idea is that the counterfactual outcomes for each synthetic treated district is constructed as a weighted average of the outcomes of control districts. If some control facilities are contaminated by the intervention, but are comparable to those in the intervention sites, the effect of the intervention is underestimated (See Annex A5 for a more technical explanation).

2.3.2. Qualitative data analysis

Digital recorders were used to record interviews with individuals from the DHOs and health centres. The information was later transcribed and analysed deductively. As suggested by Zhang and Wildemuth [26], deductive analysis is useful for examining an existing concept. For our study, this was an ideal analytical approach since the study was building on an existing body of work by Mayumana et al. [5]. In line with a deductive analytical approach, views with similar meaning within and across the interviews were assigned to each of the key themes for the study and then analysed. This process also allowed for the retention of some of the actual expressions or views from the interviewees, as presented in the results section.

2.4. Results

In this section, we present the findings of the effects of PBF on autonomy and accountability from the DiD, qualitative and synthetic control analyses.

2.4.1. Baseline characteristics

Table 1 shows the baseline characteristics of health facilities. On average, more than 93% of the health facilities in each of the intervention sites had a health centre committee, and the health facilities in each site had an average of four health centre committee meetings every 12 months. Overall, there were no differences in covariates, accountability as well as autonomy measures at baseline in the PBF, input-financing (C1) and control sites (C2), except for a few indicators (see Annex A6-A7). We control for these baseline differences by including baseline covariates in the main regression models.

Table 1.

Descriptive statistics across district and by intervention, means.

Indicator	Status at baseline
	PBF(n = 78)	C1 (n = 44)	C2 (n = 45)	Total (N = 167)
Accountability Measures
Overall Accountability Index	−0.065 (0.746)	−0.054 (0.548)	0.169 (1.581)	0.000 (1.000)
Single measures of Accountability
Facility has a Health Center Committee	0.939 (0.241)	0.978 (0.147)	0.978 (0.147)	0.960 (0.197)
Number of Health Center Committee meetings held in the last 12 months	3.917 (2.306)	4.591 (3.164)	4.772 (3.124)	4.338 (2.806)
Facility has written records of the health centre committee meetings	0.935 (0.248)	0.978 (0.149)	0.978 (0.149)	0.958 (0.201)
Facility budget developed for the current financial year	0.771 (0.423)	0.860 (0.351)	0.805 (0.401)	0.805 0.397
Facility has a work plan for the current financial year	0.851 (0.358)	0.791 (0.412)	0.889 (0.319)	0.843 0.365
Number of health facility staff meetings held in the last 3 months	2.873 (2.927)	2.864 (1.995)	2.821 (2.594)	2.857 (2.591)
Number of supervisory or technical visits made to the health facility by the health centre committee in the past 3 months	2.043 (3.544)	1.205 (1.211)	1.474 (2.165)	1.658 (2.729)
Number of supervisory or technical support visits by a district hospital representative in the last 3 months	1.061 (1.842)	1.739 (3.460)	1.533 (5.930)	1.364 (3.717)
Number of supervisory visits by the District Health Office (DHO) in the last 3 months	1.432 (1.816)	1.848 (1.460)	3.600 (13.226)	2.110 (6.924)
Number of visits made by the health facility to community health workers for supervision or technical support in the last 3 months	4.185 (0.940)	3.587 (6.210)	4.489 (10.92)	4.1 (8.594)
Number of internal staff performance assessment in the last 12 months	2.272 (3.718)	2.318 (3.056)	2.867 (7.232)	2.441 4.754
Number of external staff performance assessment in the last 12 months	1.877 (1.812)	1.956 (1.882)	2.178 (1.874)	1.977 1.841
Number of external facility performance assessment in the last 12 months	1.925 (2.151)	1.667 (1.000)	1.844 (1.906)	1.835 (1.839)
Autonomy Measures
Overall Autonomy Index	−0.037 (1.050)	−0.132 (1.028)	0.192 (0.866)	0.000 (1.000)
Single measures of Autonomy
Able to allocate my facility budget	0.769 (0.424)	0.681 (0.471)	0.800 (0.404)	0.754 (0.431)
Able to assign tasks and activities to staff	0.872 (0.336)	0.863 (0.347)	0.822 (0.387)	0.856 (0.352)
DHO supports my decisions and actions for doing a better job in my facility	0.871 (0.336)	0.727 (0.450)	0.889 (0.318)	0.838 (0.369)
Choice over whom I allocate tasks to	0.769 (0.424)	0.727 (0.451)	0.822 (0.387)	0.077 (0.421)
Choice over what services are provided in the facility	0.603 (0.493)	0.613 (0.493)	0.756 (0.435)	0.647 (0.479)
Enough authority to procure (drugs, supplies, services etc.)	0.717 (0.453)	0.795 (0.408)	0.800 (0.405)	0.760 (0.428)
Policies and procedures for doing things are clear to me	0.910 (0.288)	0.977 (0.151)	1.000 (0.000)	0.952 (0.214)
Policies and procedures for doing things are useful tools for the challenges faced	0.923 (0.268)	0.954 (0.211)	1.000 (0.000)	0.952 (0.214)
DHO provides adequate feedback to me about my job and the performance of my facility	0.744 (0.439)	0.682 (0.471)	0.667 (0.477)	0.707 (0.457)
Infrastructure index	−0.133 (0.931)	0.217 (1.057)	0.010 (1.049)	0.000 (1.000)
Total number of patients	1395.6 (1952.7)	1163.8 (1842.2)	1258.9 (1599.0)	1295.03 (1827.49)
Facility type	1.159 (0.400)	1.109 (0.315)	1.130 (0.400)	1.138 (0.378)

Note: Standard deviation in parenthesis.

2.4.2. Effect of PBF on accountability

Overall, accountability (as represented by the accountability index) increased in the PBF sites as compared to input-financing sites (0.508, p = 0.011), and in the PBF sites as compared to pure control sites (0.406, p = 0.016) even after adjusting for baseline conditions (Table 2). There is no statistically significant difference in overall accountability between the input-financing sites and the pure control sites. With regard to the single measures of internal and external accountability, PBF facilities performed better than both the input-financing and pure control sites in most of the areas. Compared to the input-financing sites, PBF facilities had a higher number of HCC meeting (1.086, p = 0.093), they received more supervisory or technical visits from the district hospitals (0.835, p = 0.061), and they got a higher number of external staff performance assessments (1.433, p = 0.039). However, the effects on these three measures of accountability did not persist after controlling for baseline conditions. For internal staff performance assessment, there was a statistically significant increase in the PBF sites compared to input-financing sites (3.487, p = 0.002); and the effect persisted even after controlling for baseline conditions.

Table 2.

Effect of PBF intervention on accountability-facility level analysis.

Outcome	PBF vs. C1				PBF vs. C2				C1 vs. C2
	Effect	p value	Effect	p value	Effect	p value	Effect	p value	Effect	p value	Effect	p value
Accountability Index	0.508**	0.011	0.557***	0.006	0.406**	0.016	0.605**	0.012	0.194	0.513	0.468	0.267
Facility has a Health Centre Committee	0.039	0.274	0.009	0.821	0.041*	0.098	0.002	0.926	0.036	0.354	−0.005	0.893
Number of Health Centre Committee meetings held in the last 12 months	1.086*	0.093	1.345	0.124	0.579	0.126	0.539	0.269	−0.034	0.964	−0.211	0.802
Facility has written records of the health centre committee meetings	0.064	0.143	0.012	0.825	0.030	0.160	0.002	0.931	−0.001	0.985	−0.008	0.814
Facility budget developed for the current financial year	0.086	0.476	0.040	0.743	0.038	0.531	0.013	0.824	0.025	0.750	0.046	0.700
Facility has a work plan for the current financial year	−0.102	0.245	−0.148	0.126	0.044	0.382	0.017	0.779	0.153	0.127	0.135	0.217
Number of health facility staff meetings held in the last 3 months	−0.103	0.828	0.243	0.675	−0.167	0.572	0.011	0.978	−0.312	0.519	−0.394	0.551
Number of supervisory or technical visits made to the health facility by the health centre committee in the past 3 months	−0.488	0.210	−0.634	0.414	−0.071	0.794	−0.062	0.887	0.099	0.848	−0.009	0.990
Number of supervisory or technical support visits by a district hospital representative in the last 3 months	0.835*	0.061	0.590	0.363	0.310	0.552	0.546	0.483	−0.175	0.846	0.497	0.701
Number of supervisory visits by the DHO in the last 3 months	0.330	0.537	0.845	0.162	1.192	0.275	2.032	0.230	1.670	0.389	2.455	0.396
Number of visits made by the health facility to community health workers for supervision or technical support in the last 3 months	0.065	0.957	0.432	0.565	−0.040	0.973	2.244*	0.088	−0.693	0.756	3.158	0.195
Number of internal staff performance assessment in the last 12 months	3.487***	0.002	4.468***	0.002	2.106***	0.004	2.681**	0.032	0.676	0.544	0.963	0.601
Number of external staff performance assessment in the last 12 months	1.433**	0.039	0.362	0.579	1.091***	0.004	0.812**	0.043	0.543	0.255	0.818	0.228
Number of external facility performance assessment in the last 12 months	1.125	0.377	0.622	0.508	1.286**	0.022	1.115**	0.016	1.249*	0.070	1.264	0.110
Controls	NO		YES		NO		YES		NO		YES
N	256		158		256		152		184		118

Note: Impact estimates adjusted for district pair matching with standard errors clustered at district level: C1: input-financing, C2: pure control; *p < 0.1 **p < 0.05 ***p < 0.01.

Control variables included are: infrastructure index, health service utilisation and facility type.

Compared to the pure control sites, PBF facilities had more supervision or technical visits to community health workers (2.244, p = 0.088) after controlling for baseline conditions. Similarly, internal staff performance assessments (2.106, p = 0.004), external staff performance assessments (1.091, p = 0.004), and external facility performance assessments (1.286, p = 0.022) increased in the PBF sites relative to the pure control sites, with effects persisting even after controlling for baseline conditions. No statistically significant effects were observed in most of the single measures of internal and external accountability between the input-financing sites and the pure control sites. The only exception was with external facility performance assessment where input-financing sites performed better than the pure control sites (1.249, p = 0.070). However, this effect was attenuated after controlling for baseline conditions.

2.4.3. Subnational heterogeneous effects of PBF on accountability

Results in Table 3 show that PBF increased accountability in seven out of the 10 districts that were being evaluated. Notably, accountability improved in districts from all the provinces of Zambia with the exception of districts in Central and Southern provinces.

Table 3.

District level analysis for accountability– DiD model.

PANEL A
District	Change in Accountability Index	p value
Mumbwa	0.954	0.611
Lufwanyama	1.081*	0.086
Lundazi	1.109*	0.056
Mwense	1.125*	0.054
Isoka	1.688*	0.075
Mporokoso	1.694***	0.004
Mufumbwe	2.147**	0.010
Gwembe	0.767	0.266
Siavonga	0.509	0.415
Senanga	1.332*	0.075
Overall Effect	0.443***	0.004

*p < 0.1 **p < 0.05 ***p < 0.01.

2.4.4. Perspectives of service providers and supervisors on accountability

From a qualitative perspective, interviews with staff from the DHOs and health centres suggests that PBF had contributed to improved internal accountability. This was achieved by increased interaction and knowledge sharing among the health centre staff and between the health centre staff and staff at the DHOs. This was attributed to the fact that performance at the health centre was influenced by the level of interaction among the health staff and external relations with the DHOs. Similarly, staff at the DHOs indicated that increased dialogue and a fruitful relationship between the DHO and health centres was an obligation of the PBF as this was stipulated in the individual staff performance assessment forms.

All the things we do in PBF are the things, which the Ministry of Health talks about …so everyone knows that this is supposed to be done. The PBF has [only] come to strengthen some of these things (Interview with staff at a health centre).

The DHO used to visit us once per month, but these days they visit us quite frequently …I think and feel that they are trying to make sure that we are doing our work (Interview with staff at a health centre).

… if they have any question, any queries or if there is anything that they do not understand about the project they are free to come to our offices. If they can’t make it; most of us are a phone call away. All the facilities know that they can call me …and most of these facilities do. Where I am not clear as well, I will say ‘okay fine; let me just get in touch with staff at the province and I get back to you; which I always do (Interview with staff at the DHO).

2.4.5. Effect of PBF on autonomy

Overall, there was a statistically significant increase in autonomy (as represented by the autonomy index) (0.291, p = 0.019) in the PBF sites compared to the pure control sites. Even after adjusting for baseline conditions, the effect persisted (0.379; p = 0.014) (see Table 4). The single measures of autonomy such as: authority to procure goods and services (0.126; p = 0.72), clarity of policies and procedures for doing things (0.070; p = 0.018), and usefulness of available tools to deal with existing challenges (0.040; p = 0.011) improved in the PBF compared to the pure control sites. These positive effects persisted even after controlling for baseline conditions. However, there were no statistically significant differences between the PBF and input-financing sites for the autonomy index and single measures of autonomy. Although we observe a statistically significant difference in the overall autonomy index in the input-financing sites compared to the pure control (0.510; p = 0.084), this effect is attenuated after controlling for baseline conditions. Furthermore, the single autonomy measures related to choice over allocation of tasks and choice over what services are provided in the facility were statistically significant in the input-financing sites as compared to the pure control sites before and after adjusting for baseline conditions.

Table 4.

Effects of PBF on autonomy-facility level analysis.

Outcome	PBF vs. C1				PBF vs. C2				C1 vs. C2
	Effect	p value	Effect	p value	Effect	p value	Effect	p value	Effect	p value	Effect	p value
Autonomy Index	0.072	0.844	0.270	0.451	0.291**	0.019	0.379**	0.014	0.510*	0.084	0.469	0.121
Able to allocate my facility budget	0.039	0.762	0.050	0.744	0.095	0.128	0.112	0.110	0.146	0.248	0.159	0.243
Able to assign tasks and activities to staff	0.018	0.822	0.044	0.702	−0.008	0.809	−0.043	0.341	−0.014	0.835	−0.095	0.251
DHO supports my decisions and actions for doing a better job in my facility	−0.186	0.113	−0.079	0.572	−0.030	0.292	0.044	0.334	0.092	0.330	0.112	0.347
Choice over whom I allocate tasks to	−0.070	0.549	−0.080	0.482	0.041	0.483	0.059	0.302	0.138*	0.078	0.151*	0.064
Choice over what services are provided in the facility	0.022	0.877	−0.044	0.828	0.142**	0.033	0.135	0.176	0.268**	0.038	0.308*	0.098
Enough authority to procure (drugs, supplies, services etc.)	0.146	0.457	0.245	0.212	0.126*	0.072	0.166*	0.061	0.128	0.396	0.133	0.464
Policies and procedures for doing things are clear to me	0.095	0.176	0.142	0.130	0.070**	0.018	0.106**	0.022	0.044	0.408	0.065	0.323
Policies and procedures for doing things are useful tools for the challenges faced	0.110	0.119	0.121	0.251	0.040**	0.011	0.039*	0.090	−0.012	0.829	−0.009	0.891
DHO provides adequate feedback to me about my job and the performance of my facility	−0.054	0.694	0.052	0.710	−0.015	0.731	−0.019	0.761	0.015	0.902	−0.092	0.478
Controls	NO		YES		NO		YES		NO		YES
N	246		154		247		149		179		117

Note: Impact estimates adjusted for district pair matching with standard errors clustered at district level: C1: input-financing, C2: pure control; *p < 0.1 **p < 0.05 ***p < 0.01 .Control variables included are: infrastructure index, health service utilisation and facility type.

2.4.6. Subnational heterogeneous effects of PBF on autonomy

The estimated improvement in autonomy in PBF districts compared to the pure control districts was almost entirely driven by t districts located in Luapula, Northern, and Muchinga provinces. None of the districts in the other parts of Zambia experienced improvement in autonomy (Table 5).

Table 5.

District level analysis for autonomy – DiD model.

District	Change in autonomy index	p value
Mumbwa	0.392	0.460
Lufwanyama	−0.190	0.736
Lundazi	0.222	0.646
Mwense	0.964*	0.064
Isoka	1.575*	0.055
Mporokoso	1.202**	0.018
Mufumbwe	0.915	0.202
Gwembe	−0.107	0.863
Siavonga	0.129	0.807
Senanga	0.391	0.596
Overall Effect	0.284**	0.020

*p < 0.1 **p < 0.05 ***p < 0.01.

2.4.7. Perspectives of service providers and supervisors on autonomy

During the qualitative interviews in the PBF intervention districts, health workers reported that the PBF program allowed health centres to make their own financial and managerial decisions, without seeking permission from the DHOs office. For instance, health workers indicated that the DHO did not have influence on the monthly individual staff performance assessments and this empowered the health centres to manage themselves. With the implementation of PBF, the health centres were able to meet and plan as needed, develop and revise business plans and allocate resources to priority areas with the ultimate goal of improving the quantity and quality of health services. The decisions they made on areas that needed improvement were based on the assessments from the quantity audits and quarterly quality assessments. Below are the responses from respondents from DHOs and health centres.

“They are autonomous in terms of decision-making; the use of funds; they have opened their own bank accounts; they have their own signatories…they are managing the funds and are able to withdraw at their own time (Interview with staff from the DHO).

We are working like managers whereby the money is sent straight to our accounts. We are able to withdraw that money, sit with our members, and plan based on the business plan. In short, in the PBF process, we are managers in our own right (Interview with staff at a health centre).

When we make decisions at this health center, we sit as a group and when those from the district (DHO) come, they will just go through what we have already done and merely give advice where necessary (Interview with staff at a health centre).

From the time we were given the cheque books, they said …if you want to withdraw some money; …you can come and withdraw and use it for whatever activity you want (Interview with staff from the health centre).

2.5. Synthetic control analysis

An alternative approach to estimating the effects, which is robust to the parallel trends assumption, is the synthetic control approach. The results from the synthetic control analysis are generally consistent with those from the DiD analysis. Specifically, the results in Table 6 show that improvements in autonomy were driven by two districts in Northern and Muchinga provinces (Panel A). However, though the overall estimate from the synthetic control approach is close to the overall estimate from the DiD analysis, it is not statistically significant.

Table 6.

Synthetic control district level analysis for autonomy and accountability.

Panel A
District	Change in autonomy index	p value
Mumbwa	0.207	0.600
Lufwanyama	0.203	0.600
Lundazi	−0.006	0.900
Mwense	0.607	0.200
Isoka	1.386***	0.000
Mporokoso	0.749*	0.099
Mufumbwe	0.232	0.600
Gwembe	0.133	0.700
Siavonga	−0.113	0.700
Senanga	−0.219	0.599
Overall Effect	0.317	0.58
Panel B
District	Change in accountability index	P value
Mumbwa	0.684	0.110
Lufwanyama	0.751*	0.100
Lundazi	1.024*	0.100
Mwense	0.705*	0.100
Isoka	1.199***	0.000
Mporokoso	1.744***	0.000
Mufumbwe	1.508***	0.000
Gwembe	0.002	1.000
Siavonga	−0.024	1.000
Senanga	0.387	0.500
Overall Effect	0.763*	0.09

*p < 0.1 **p < 0.05 ***p < 0.01.

Findings from the synthetic control analysis on accountability (Table 6, Panel B) are also generally consistent with those from the DiD analysis. The results show an overall improvement in accountability in districts from all the provinces with the exception of districts in Southern, Central, and Western provinces.

3. Discussion

This study sought to investigate the causal effect of PBF on autonomy and accountability at health centres in Zambia. The study uses a difference-in-difference and synthetic control analysis to evaluate causal effects based on nine single measures of financial and managerial autonomy and an overall autonomy index; and thirteen single measures of internal and external accountability and an overall accountability index. Qualitative data from a process evaluation complemented data from the quantitative component. On financial and managerial autonomy, results from the quantitative component show no differences between the PBF and input-financing sites on both the single and overall measures of autonomy. However, the PBF sites performed better than the pure control sites on the overall autonomy index and four of the nine single financial and managerial autonomy measures. These are the freedom to decide the services provided at the health centres, authority to procure goods and services, clarity of roles and responsibilities, and use of the existing tools and procedures to resolve challenges at the health centres. On the other hand, as compared to the pure control sites, input-financing sites had a positive effect on the overall autonomy index and on two individual measures of managerial autonomy, i.e. freedom to decide the services provided at the health centres and allocation of tasks to members of staff.

In a nutshell, the results show a positive effect of PBF on financial and managerial autonomy at health centres compared with the pure control sites and no effect relative to input-financing sites. On the other hand, input-financing sites have a relatively smaller positive effect on managerial autonomy but no effect on financial autonomy when compared with the pure control sites. These results are similar to Eichler and Levine [27], who showed that health centres on PBF enjoy more flexibility in spending money than cost-based reimbursement approaches, which require a detailed justification of expenditures. In Tanzania, Mayumana et al. [5] show that PBF increased financial autonomy and responsiveness from managers by making it possible for them to attend to broken equipment and drug stock-outs, and this contributed to an increase in service delivery. In Rwanda, clarification of roles and responsibilities among the various parties through a PBF program helped increase staff efficiency and strengthen existing quality assurance mechanisms [28]. In our study, interviews with staff in the PBF sites also revealed that PBF was a valued intervention by staff at both the DHOs and health centres. In particular, staff at the health centres were happy that PBF had enabled them to work as managers and that they were able to perform some tasks they never used to perform before PBF was initiated.

Our study also unveils the importance of additional financing in enhancing managerial autonomy in a health system with some mechanisms to promote independent decision-making. As observed in the results, there is no difference in financial and managerial autonomy between PBF and input-financing sites; but there is a positive effect on managerial autonomy in input-financing sites as compared with pure control sites. In addition, across all the study groups (PBF, input-financing, pure control) there were no differences in some of the single measures of financial and managerial autonomy i.e. ability to allocate the budget and support and feedback from the DHO on the performance of the staff and the health centres. This is because the bottom-up planning process that has been in use in the Zambia public health system since the 1990s allows health centre managers (in-charges) to come up with annual work plans and budgets based on the needs of the health centres and communities [29], and there is some degree of flexibility in the use of the monthly operational grants that the Government provides. Secondly, the quarterly performance assessments and technical support visits that DHOs already provide and Provincial Health Offices (PHOs) in all public facilities also explains why there is no effect on some of the measures of managerial autonomy. Therefore, with the additional financing, health centre managers in the PBF and input-financing groups re-enforced the autonomy that already exists in the Zambia health system.

In addition to the above explanation, it is also important to note that not all of the studies that have examined the impact of PBF on financial and managerial autonomy had an input-financing arm. Comparing the effect of PBF on autonomy only to a pure control group overlooks the effect of additional financing. Thus, while a study by Zeng et al. [30] suggests that incentives that were provided in a PBF program in Haiti offered health facilities greater autonomy to allocate their budgets, the positive effect could have been due to additional funding rather than the PBF itself. Moreover, results from the full impact evaluation report on the Zambia PBF—where quantitative results from our study were generated —show that the money disbursed to the input-financing sites was only 56 percent of what the PBF sites received [4]. Therefore, it is plausible that if the input-financing sites had received and spent all the funding, they could have performed at the same level as PBF or possibly better. However, Friedman et al.’s [4] findings also suggest that the PBF sites were more efficient in absorbing or utilising disbursed funds—which could be attributed to better financial and managerial autonomy. This underscores the point that channelling money directly to health centres, as was the case in the PBF sites, is more efficient and favourable for financial and managerial autonomy than channelling the money to health centres through the DHOs.

Regarding accountability, the results show that overall accountability improved in PBF facilities compared to both the input-financing and pure control sites. PBF sites also performed better than both the input-financing and pure control sites in four out of the thirteen internal and external accountability measures. These are frequency in holding HCC meetings, supervisory or technical support visits by district hospitals, visits to community health workers, internal and external staff performance assessments, and external health centre performance assessments. In comparison to the pure control sites, the input-financing sites only performed better in external health centre performance assessment. On the other hand, there is no difference in internal and external accountability across all the study groups (PBF, input-financing, pure control) on some of the single measures of accountability such as: having a HCC, record of HCC meetings, availability of work plans and budgets, frequency of health centre staff meetings, supervisory or technical support visits by the HCC, and supervisory or technical support visits by the DHO. These accountability measures have been in place since the 1990s—way before the PBF was implemented in 2012. As highlighted by the Ministry of Health [31], all government health centres are required to have a functional neighbourhood health committee and HCC. An annual bottom-up planning and budgeting system and quarterly performance assessments and technical support visits, which DHOs and PHOs provide, have also been in place since the 1990s [31].

However, even though the above accountability structures and mechanisms have always existed, overall and for some single accountability measures, the PBF sites performed better than the input-financing sites. Consequently, it is vital to note that money alone is not sufficient to improve performance and achieve better results. Over the years, it has become clear that good governance is important in obtaining a good return to investments in the health sector, including quality healthcare [32], [33], [34], [35]. Brinkerhoff [36] adds that providing greater autonomy to individual facilities, facility accreditation, and facility-based contracting can increase financial and performance accountability. In Zambia, community groups such as neighbourhood health committees and HCCs have been used since the 1990s to enhance accountability and transparency in health service delivery [31]. However, the level of engagement has been low, particularly after the abolition of user fees in 2006. In our study, the evidence shows that PBF strengthened these governance structures, as more HCC meetings were held and more visits were made to community health workers. While a detailed analysis on the quality of community engagement is not available, some studies show that increased community participation often plays a key role in the management and delivery of health services [37].

The PBF also increased technical support visits to health centres by district hospitals and internal and external performance assessments. The changes explain why the PBF in Zambia increased utilisation, structural quality, and client satisfaction [24]. As Lewis [32] observes, productivity and performance can increase where incentives for strong performance exist. Information from interviews with staff at the health centres and DHOs also supports the finding that PBF improved internal accountability and performance by enhancing interaction, relationships and knowledge sharing among the health centre staff and between the health centre staff and DHOs. This finding supports the results from a study that was conducted in Tanzania, where PBF contributed to improvements in internal accountability by enhancing relations and communication between stakeholders [5]. However, the differences in effects of the PBF on autonomy and accountability among the districts where PBF was being implemented suggests that managerial practices and leadership are key factors in the successful implementation of PBF [37]. The key management and leadership qualities that were observed in the PBF districts wthat had high scores on autonomy and accountability include (i) strong organisational and personal commitment, (ii) trust, team building, and consultative decision-making, (iii) proactive resolution of implementation challenges, (iv) transparency in the distribution of PBF staff incentives, and (v) high participation of neighbourhood health committees and safe motherhood action groups in the implementation and review of activities in the health centre action plans.

Notwithstanding the results from the study, there are some limitations. Firstly, the two main variables of interest (autonomy and accountability) were assessed by using nine and 13 indicators, respectively, and indexes that may not fully explain all the attributes of autonomy and accountability. To overcome this problem, the definition and assessment of autonomy and accountability in the study were linked to how the PBF was designed and implemented in Zambia; and widely accepted standards for measuring autonomy as provided by Kramer et al., [38] and for measuring accountability as provided by Brinkerhoff [36]. Secondly, the test for parallel trends assumption was not undertaken because there was only one pre-treatment period. However, our evaluation used a 3-arm quasi-randomised model, which is a better method for assessing the variables of interest in the study. In addition, the synthetic control approach was used as a robustness check, and the results are consistent with those from the DiD analysis.

4. Conclusion

Based on the results and discussion presented above, we conclude that PBF can improve financial and managerial autonomy and accountability at health centres. Autonomy and accountability go hand in hand, as autonomy is necessary for the achievement of greater accountability. The study also demonstrates that additional financing is a necessary but insufficient requirement for enhancing autonomy and accountability. This is because the Zambia PBF project promoted good governance by re-enforcing existing structures and mechanisms for autonomy and accountability. Notwithstanding the above, within the PBF districts, the magnitude of change differed across the districts, suggesting that management and leadership styles are critical. Possible enabling factors include: (i) strong organisational and personal commitment; (ii) trust, team building, and consultative decision-making; (iii) proactive resolution of implementation challenges; (iv) transparency in the distribution of PBF staff incentives; and (v) high community participation. Therefore, future research on the effect of PBF on autonomy and accountability should take these factors into consideration.

Funding

This work was supported by the African Economic Research Consortium (AERC). The findings and conclusions in this paper are those of the authors and do not necessarily reflect the position of the AERC.

Ethical approval

Ethical clearance for the impact evaluation was obtained from the Humanities and Social Science Ethics Committee at the University of Zambia.

CRediT authorship contribution statement

Chitalu Miriam Chama-Chiliba: Conceptualization, Formal analysis, Methodology, Writing – original draft, Writing – review & editing, Funding acquisition. Peter Hangoma: Formal analysis, Methodology, Writing – review & editing, Funding acquisition. Collins Chansa: Conceptualization, Writing – original draft, Writing – review & editing. Mulenga Chonzi Mulenga: Formal analysis, Writing – review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following are the Supplementary data to this article: