Using causal loop diagrams to explore the maternal and child health system response to payment for performance in Zambia, and it’s generalisability across settings

Abstract

This study investigates the generalisability of pathways depicted by causal loop diagrams (CLDs) in payment for performance (P4P) schemes by adapting and validating the Tanzanian CLDs to the Zambian context. Specifically, it explores whether the health system pathways represented by CLDs, are consistent across different settings and how variations in programme design and local context influence these pathways. Using a five-stage approach, the study adapted the Tanzanian CLDs to reflect the Zambian P4P programme context, validating them through stakeholder interviews, workshops, and secondary qualitative data. The findings show that while the overarching pathways influencing P4P outcomes are similar, differences in programme design and contextual factors shape their intensity and impact. Notably, Zambia’s higher facility autonomy and stronger trust in the health system contributed to greater health worker motivation and service delivery compared to Tanzania. Programme design features such as safeguards for non-incentivised services and adequate funding for facility investment with provider autonomy influenced performance outcomes. Additionally, contextual factors such as trust in the system mitigated programme delays, while decentralised procurement systems enhanced P4P effectiveness. These findings highlight the need for context-specific adaptation when implementing P4P programmes. The study advances the application of CLDs for cross-country health system analysis, highlighting both their potential and limitations in comparing health interventions across diverse settings.

Highlights

• •Differences in P4P response attributed to programme design and local context.
• •Design features and contextual factors influence P4P response.
• •Contextual factors: trust in the system, mitigates programme delays.
• •Decentralisation of procurement systems enhances P4P effects.
• •The study highlights CLDs' potential and limitations for cross-country health comparisons.

1. Introduction

Payment for performance (P4P)— also referred to as results-based financing or performance-based financing— has been widely implemented in several countries in sub-Saharan Africa since the early 2000s to address persistent challenges in maternal and child health (MCH) outcomes. Despite increased service utilisation, many health systems in low and middle income countries (LMICs) continue to struggle with poor MCH outcomes (Das et al., 2016, Diaconu et al., 2021, James et al., 2020 Kovacs et al., 2020a; Diaconu et al., 2022). P4P schemes, which combine financial incentives with key reforms to promote transparency, accountability and health system strengthening, have been proposed to improve patient outcomes (Diaconu et al., 2021). Previous studies have extensively analysed the effects of P4P on various aspects of health systems, including health outcomes, service quality, provider motivation, and facility autonomy (Binyaruka et al., 2018, Binyaruka and Borghi, 2017, Chama-Chiliba et al., 2022, Diaconu et al., 2021, Friedman et al., 2016, Mayumana et al., 2017, Shen et al., 2017, Russo et al., 2024). However, the inherent complexity and variability in P4P programme design across settings have led to mixed results, with significant heterogeneity in impacts observed across LMICs (Diaconu et al., 2021, Kovacs et al., 2020b, Singh et al., 2021, Russo et al., 2024). Traditional evaluation methods, such as regression analysis often fail to capture the dynamic and complex nature of health systems, leading to potential biases in understanding the true effects of P4P interventions (Borghi and Chalabi, 2017, Sterman, 2000).

Systems thinking methods are gaining traction within health systems research to support evaluation of policy impact on health system outcomes (Baugh Littlejohns et al., 2021, Cassidy et al., 2022). Causal loop diagrams (CLDs) for example are a tool that can be used to help identify and visualise structural drivers for health systems behaviour, allowing for a more nuanced understanding of how complex interventions, such as P4P, influence system dynamics overtime (Adam, 2014, Peters, 2014). CLDs have been previously used to evaluate pathways to impact for P4P on key health system outcomes in Afghanistan, Uganda, Tanzania, and used as a complementary tool for a realist review into how P4P programmes work in LMICs.

Although CLDs have been used to model health system interactions within single settings and increasingly used in health system research, their applicability for cross-country comparisons remains underexplored. There is little evidence on whether CLDs developed in one setting can be generalised to other contexts in terms of the pathways they depict, and consequently whether the research and policy implications derived from their analysis are generalisable. While CLDs inherently reflect specific contexts and systems in which they are developed, the generalisability of policy pathways to impact remains an open question. Traditionally, CLDs have been used to explore dynamics within a single system or setting, providing visual and analytical representation of how different variables interact to drive outcomes (Peters, 2014). For example, CLDs have been used to model the interactions within health systems at a district or national level, focussing on a specific intervention or policy. Previous studies using CLDs to study health system behaviour and policy impact have reported limited generalisability of findings due to contextual differences within countries between rural and urban settings (Paina et al., 2014) and between different countries (Glenn et al., 2020). Evidence generation into what extent CLDs, and the research and policy implications derived from their analysis, are generalisable to other contexts is needed to explore the further utility of multi-year research in single country settings. If it is possible to generalise results and implications to comparable settings, for example, to use CLDs developed in a previous study as a foundation for developing a CLD in another context, this may reduce the overall resource and time burden attributed to their development and analysis.

This study seeks to extend the traditional use of CLDs by investigating their generalisability througha cross-country comparison of pathways to impact for a health systems initiative.- Building on Cassidy et al., (2021), which used CLDs to explore the impact of P4P in Tanzania, this study investigates whether the pathways to impact identified via CLDs developed for a P4P programme in Tanzania are representative of a comparable P4P programme that took place in another setting, Zambia. By adapting and validating the Tanzanian CLDs to the Zambian context, we aim to understand how these system pathways are consistent or vary across these two distinct settings. The key research question is: ‘To what extent are the pathways depicted by CLDs developed in Tanzania for a P4P programme generalisable to the Zambian context, and what factors influence any observed differences?’ We hypothesise that while the overarching pathways may be similar, differences in programme design and local context will lead to variations in the intensity and expression of these pathways.

This study contributes to the literature on P4P, health systems and systems thinking by providing insights into the conditions under which CLD pathways may or may not be generalisable across different settings. By focussing on the pathways or dynamics, the study offers a more targeted analysis that can inform the design and implementation of more effective P4P programmes in diverse contexts. Additionally, this study makes cross-country comparisons of CLDs, extending their application and utility from a single-setting analysis, contributing experience and evidence into what extent CLDs developed in a single context, and the research and policy implications derived from their analysis, are generalisable to other health or country settings.

1.1. Study setting

Zambia is a low-income country with a GDP per capita of US$1120.6 (Current US$)) and a population of 19.6 million (World Bank, 2023a; Zambia Statistics Agency, 2022). In 2016, per capita total health expenditure was US$59, below the LMIC average of US$82 (Ministry of Health-Zambia, 2018). Health financing is primarily sourced from general tax revenue (35 %), donor funding (38 %), out-of-pocket payments (12 %), and employer-based medical schemes and other unspecified sources accounting for approximately 5 % of total health spending (Ministry of Health-Zambia, 2018). Government facilities provide 85 % of health services, with the remainder delivered by private, not-for-profit, and for-profit institutions (Ministry of Health, 2017). Health services are organised at three levels: district (health posts, health centres, and first-level hospitals), provincial level (general hospitals) and national level (tertiary and specialised hospitals) (Ministry of Health, 2017). In 2012, the government abolished user fees for all services at public primary health care level facilities. In 2019, social health insurance (SHI) was introduced, covering 24 % of the population by 2022, primarily formal sector employees (Government of Zambia, 2018, National Health Insurance Management Authority, 2022). Decentralisation of health systems remains a priority (Ministry of Health-Zambia, 2023).

Tanzania, a lower middle-income country with a GDP per capita- of US$1135.5 (Current US$) and a population of 61.5 million, had a per capita health expenditure of US$40 in 2019/20 (World Bank, 2023b, Ministry of Health-Tanzania, 2022). Health financing is derived from government taxation (22 %), donor support (34 %), household out-of-pocket payments (32 %), and health insurance contributions (12 %) (Ministry of Health-Tanzania, 2022). Despite efforts, health insurance coverage remains low at 15 %, with most uninsured individuals among the poor and informal workers. Tanzania’s health system is decentralised across 26 administrative regions in the mainland and 184 district councils, with over 70 % of health facilities publicly owned.

1.2. Intervention design

Zambia initiated a P4P pilot in 2008, expanding it from one district to ten across nine provinces by 2014, and further to 61 districts in five provinces by 2019 (Friedman et al., 2016, World Bank, 2020). Tanzania launched its P4P pilot in 2011 in the Pwani region, later scaling it to nine regions by 2021 (Binyaruka et al., 2015). The P4P pilot in Tanzania received funding from the Government of Norway, and the World Bank, while the Health Results Innovation Trust Fund (HRITF) funded the programme in Zambia. This paper focuses on these pilot programmes.

Table 1 summarises the intervention design of the pilot P4P scheme in Zambia and Tanzania (See Appendix A for detail). In Zambia, P4P incentives were distributed quarterly based on performance across nine MCH indicators, using a fee-for-service model. Facilities allocated 40 % of earned funds for operational activities and 60 % for staff incentives, leading to improvements in service delivery, health worker retention, and facility autonomy (Chama-Chiliba et al., 2022, Friedman et al., 2016, Shen et al., 2017).

Table 1.

Typology for design of P4P schemes in the pilot P4P in Tanzania and Zambia.

SCHEME DESIGN	TANZANIA	ZAMBIA
A. GENERAL CHARACTERISTICS
Name of programme	Payment for Performance (P4P) programme	Results Based Financing (RBF)
Funder	Norwegian Ministry of Foreign Affairs	World Bank (Health Results Innovation Trust Fund)
Purchaser	National Health Insurance Fund	Provincial RBF Steering Committees
Time period	2011 – 2015	2012 – 2014
Scale	Pwani region (7 districts)	9 provinces (10 districts)
Regulator	Pilot Management Team - (Clinton Health Access Initiative (CHAI) and Ministry of Health	Ministry of Health
Fund holder	National Health Insurance Fund	Ministry of Health
B. WHAT IS INCENTIVISED?
Measures used to assess performance		9 MNCH indicators for facilities, 11 indicators for district managers based on facility performance and a fixed amount for supervision (Unit price, US$).
C. WHO IS INCENTIVISED?
Performance measured and receives payment	•Health workers, district and regional managers •Public and non-public facilities (hospitals, health centres and dispensaries) providing MCH services	•Health workers, district managers •Public and not-for profit health centres and health posts providing MCH services
D. PAYMENT ATTRIBUTES
Frequency	Six monthly	Quarterly
Size	Health workers- About 10 % of official government salary as monetary incentive	Health workers- About 10 % of official government salary as monetary incentive
Payments coupled with salary	Monetary incentive payments were separate from salary	Monetary incentive payments were separate from salary
Lag time between reporting and payment	3 months	45 days after end of quarter
Rewards vs. penalties	Financial rewards provided	Financial rewards provided
Use of money	▪25 % for investments at facility 75 % for staff monetary incentives (health centres and dispensaries) ▪10 % for investments at facility and 90 % for staff monetary incentives (hospitals) ▪District/regional level staff	▪40 % for facility investments and demand creation and 60 % for staff monetary incentives (health centres) ▪DHMT and District hospitals
E. BASIS FOR PAYMENT
Each action vs. threshold of performance	Single target and multiple threshold targets	Fee for service –threshold graduated payment
Type of ranking (tournament, absolute, relative etc)	Absolute and relative performance	Incentives based on improvements in providers performance
Payment adjustment (equity, quality etc)	None	Yes
F. GAMING SAFEGUARDS
Performance audits	Internal and external verification	▪Internal and external verification
Penalties	None	▪Yes

Sources: Framework adapted from Kovacs et al. (2020a)

The Tanzanian P4P design differed by using coverage targets rather than a fee-for-service model. Incentives were also extended to district and regional managers. Evaluation results indicated improvements in institutional deliveries, malaria treatment during antenatal care, drug availability, and health worker performance (Anselmi et al., 2017, Binyaruka et al., 2015, Binyaruka and Borghi, 2017, Mayumana et al., 2017).

2. Methods

2.1. Overview of approach

This study employed a five-stage approach to adapting and validating the CLDs developed for a Tanzanian P4P programme to the Zambian context. The process was designed to ensure that both programme design and local contextual factors were carefully considered and that the pathways of the CLDs were accurately captured and validated.

2.2. Description of the five-step approach

2.2.1. Step 1: initial Tanzanian CLD development

The Tanzanian CLDs were developed through a three-step process that involved creating individual CLDs to represent stakeholder understanding of how P4P affected the Tanzania health system as detailed in Cassidy et al., (2021). Through a step-by-step process, the individual CLDs were combined to create a single mental model or visual representation of the P4P pathways to impact the Tanzania health system and its subsystems. Key stakeholders validated the combined CLD structure to ensure that it represented the Tanzania experience of the P4P programme. Two categories of performance targets were identified: 1) number of women and children who receive incentivised services and 2): submission of routine health facility data by providers. Furthermore, three core mechanisms responsible for Tanzanian providers’ ability to achieve or fail to reach targets were identified as: 1) changes in the supply of services, 2) changes in facility reporting, 3) and changes in demand for services.

2.2.2. Step 2: expert interviews and CLD adaptation

In the second stage, we consulted experts familiar with the P4P programmes in both countries. These consultations were essential for adapting the Tanzanian CLDs to reflect the Zambian context. The initial step in determining whether the Tanzanian CLD was suitable for adaptation to the Zambian setting was to identify the similarities and differences in the P4P programmes between the two countries. A comparison of the P4P programmes was conducted based on key programme attributes, as outlined in Table 1. After establishing that the P4P programme designs in the two countries were generally similar in terms of goals and outcomes, with a few notable differences such as the payment attributes and basis for payment, the next step involved adapting the Tanzania CLDs generated in the first step to suit the Zambian context.

2.2.3. Step 3: stakeholder engagement

The third stage involved interviews and workshops with key stakeholders, including national, provincial and district administrators, health workers, facility managers, representatives from the funding agency and policymakers from Zambia who had been involved in the pilot P4P programme in Zambia. During these interviews and workshops, participants reviewed and provided feedback on the adapted CLDs, to ensure that they accurately reflected the realities of the Zambian health system and the P4P mechanisms. Based on their input, we assessed and eliminated P4P mechanisms that were non-existent or not applicable to the Zambian context from the original Tanzanian CLDs. Conversely, we included other relevant or applicable mechanisms unique to the Zambian setting. For instance, we maintained existing relationships in the CLDs if Zambian stakeholder descriptions fully or largely supported them. Nonetheless, if the stakeholder’s description constituted a substantial change to the Tanzanian CLD, such as removing variables or causal links, introducing new variables and relationships, we made further adjustments to the CLDs.

We created a CLD to demonstrate the similarities and differences between the Tanzania and Zambia P4P programmes (See Appendix B). The arrows on the CLDs indicate the direction of causal relationships. A positive polarity sign indicates that the variables move in the same direction, while a negative sign indicates that they move in opposite directions. The CLDs also demonstrate time delays (the lag between an action and its impact on the system) in causal effects between two variables with double lines through arrows and feedback loops, which can be reinforcing (producing an amplified, spiralling behaviour) or balancing (presence of one or more variables prevents the loop from exhibiting reinforced, spiralling behaviour) (Cassidy et al., 2022, Sterman, 2000, Tomoaia-Cotisel et al., 2017). We used a colour scheme to differentiate the CLD elements (variables or causal links) that were retained or modified to create the Zambian CLD. We also noted potential differences in the intensity of the relationships, whereby the relationship is the same, but it is more or less intense/strong in one setting than the other.

Similar to the Tanzanian CLD (Cassidy et al., 2021), the Zambia CLD maps pathways to outcomes in terms of the number of women and children who receive incentivised services and providers' submission of routine health facility data. Three key mechanisms are responsible for provider performance: changes in supply, changes in demand, and changes in facility reporting.

2.2.4. Step 4: validation of the Zambia CLDs

In the fourth stage, we aimed to ensure the validity of the Zambian CLDs using secondary qualitative data that were collected during the process evaluation of the pilot P4P in Zambia. This evaluation was conducted in 2013 and focused on nine health centres from three districts and three provinces, selected based on their performance on three key indicators. The qualitative data included 42 interviews, with various stakeholders, such as the district medical officers, health workers, health centre committee members, and parents who participated in the focus group discussions. By using these additional data, we validated the information gathered from key informants and experts at different levels of the health care system. This allowed us to adjust the CLDs based on a triangulation of evidence from multiple sources, including primary and secondary data.

2.2.5. Step 5: final refinement and analysis

The final stage involved refining the CLDs using the validation data and analysing the generalisability of the pathways. This included comparing the dynamics between the Tanzanian and Zambian CLDs to identify factors driving observed differences. The project team held a series of meetings to conduct a detailed analysis cross-country analysis, developing a framework to highlight the similarities and differences in the P4P programme. The goal was to uncover discrepancies between the P4P designs and the country-specific contexts that could explain variations in the CLDs.

At this stage, the team identified key leverage points where interventions could optimise system performance, based on how key variables influenced others in the CLDs. Presenting the CLDs revealed that two elements newly identified in the Zambian CLD were also relevant in Tanzania, and these elements were marked with dashed arrows for clarity in the Zambian CLD.

2.3. Sampling and participant selection

Purposive sampling was used to select participants with in-depth knowledge of the P4P programmes in Zambia. At the national level, participants include researchers from academia, policymakers from the Ministry of Health and representatives from funding agencies. Sub-national representatives included district and provincial officials overseeing the pilot P4P programme implementation, as well as health facility managers and frontline health workers from selected facilities involved in the programme

At sub-national level, stakeholders were selected from two provinces with contrasting poverty levels— North-western, which has the lowest incidence of poverty, and Western, which has the highest incidence. Within these provinces, participants were selected from districts involved in the pilot P4P, including the provincial capital and one additional district. Health workers from two health facilities and district managers in each P4P district who were involved in the pilot P4P were also interviewed.

To mitigate selection bias, we ensured a diverse representation of stakeholders across different levels of the health system. Recall bias was reduced through triangulation of interview data with secondary data from the P4P process evaluation.

2.4. Data collection and analysis

Data collection involved interviews and workshops with 33 stakeholders, including academia involved in the P4P programme evaluation, national-level stakeholders, provincial and district managers, and health facility managers as shown in Table 2. Workshops at the district and facility level provided a broader range of district managers, and health worker experiences. A secondary data review of qualitative data from the P4P process evaluation, including interviews and focus group discussions conducted during the programme process evaluation, complemented the primary data.

Table 2.

Stakeholders involved in the validation of the adapted Zambia CLD.

Level	Stakeholder type	Number of interviews
National	Academia	6
	Funding agency and MOH headquarters	4
Province	Provincial-level MOH	5
District	District-level MOH	6
Health facility	Health facility managers and health workers	12
Total participants		33

Qualitative data were coded to identify key themes and pathways, which were mapped onto the CLDs. The generalisability of the pathways of the Tanzanian CLDs to the Zambian context was assessed based on the following attributes: 1) the alignment between P4P programme designs in the two countries; 2) the influence of contextual factors such as trust in the health system and facility autonomy; 3) the consistency of pathways between the Tanzanian and Zambian CLDs.

Our methodology aligns with Stopps et al. (2021), which used confirmatory/disconfirmatory workshops to identify differences between settings. The focus of the Stopps paper is grounded in a single setting (UK) with reflection on new additions as a result of consulting stakeholders in a second setting (Canada). However, these new additions are highlighted in the second CLD, contextual differences between settings and differences in effect (i.e., time delays) are not visualised in the CLD but verbally described in the accompanying text.

3. Results

3.1. Changes in supply-side factors influencing service delivery

Fig. 1 illustrates (A) the mechanisms involved in the supply of services during the P4P programme in Zambia and (B) the original Tanzania CLD from Cassidy et al. (2021). Specifically, it shows the reinforcing loop-feedback loop where an increase in one variable leads to further increases in the same direction- of ‘motivation-service delivery’ (Figs. 1A,1B, R1), which highlights the positive impact of incentive payments on health providers, motivating them to exert effort towards providing incentivised services. The CLDs indicate that an increase in the amount of health worker incentive payments resulted in increased motivation to provide incentivised services in both countries. Notably, in Zambia, the increased motivation resulted in staff voluntarily working longer hours, including after-hours and off-duty hours, to provide incentivised health services to women and children. This was not observed in Tanzania:

Fig. 1.

Changes in the supply of services during the P4P. Panel A: Zambia CLD. Note to Fig. 1) Black-Element common to both Zambia and Tanzania’s P4P programmes. 2) Grey-Elements specific to Tanzania. 3) Blue-Newly identified elements specific to Zambia. 4) Green dashed line- Newly identified elements in Zambia that are also relevant for Tanzania. 5) Green solid line-Indicates same intensity of effects in both Zambia and Tanzania. 6) Red- Represents the removal or addition of delays in the system. Abbreviations: District Health Director (DHD), District Health Office (DHO), Medical Stores Limited (MSL), Payment for performance (P4P). Panel B: Tanzania CLD. Source: Adapted from Cassidy et al#, (2021), figure reproduced in black and white. Abbreviations: Council Health Management Team (CHMT), Community Health Fund (CHF), District Executive Director (DED), Health Facility Governing Committee (HGFC), Medical Stores Department (MSD), Payment for performance (P4P).

“…they [health workers] were willing to work longer hours, willing to work even during awkward hours, and willing to attend to more clients or patients” (District level stakeholder, 2013); and “Sometimes, if one knocked off at 17 hours and had gone somewhere such as to the [maize] field and then was called and informed that there was a patient, one did not hesitate and went to attend to the patient despite being tired” (Facility level stakeholder, 2021).

The delay in receipt of incentive payments frustrated health workers in Tanzania, reducing motivation. In Zambia, the delays had minimal influence on workers’ morale as they were confident that payments would be made, despite the delays. Health facilities in both countries had bank accounts that facilitated the payment of incentives to health providers and enhanced financial autonomy. However, the facility bank accounts in Tanzania were also used to deposit other financial resources.

The impact of increased health worker motivation to deliver incentivised services had diverse implications for the delivery of non-incentivised services. In Tanzania, concerns were raised that staff attention and time would be diverted away from non-incentivised services, particularly in smaller facilities with inadequate staff. The P4P programme in Zambia included sanctions for reduced performance in relation to non-incentivised services, while such measures to safeguard the performance of non-incentivised indicators were absent in the Tanzanian P4P programme.

“The contractual agreement between the facility and the district managers in Zambia stipulated that if the performance in the non-incentivised indicators declined by 20 percent or more relative to the baseline, the facility would be sanctioned or removed from the P4P programme,” (National level stakeholder, 2021).

The ‘acknowledgement loop’ (Fig. 1A,B, R2) shows the impact of facility and district-level supervisor recognition of health worker efforts in motivating workers. In both Zambia and Tanzania, the increase in the number of women and children receiving incentivised services led to an increase in the acknowledgement of health workers’ efforts by facility managers. However, the extent of the acknowledgement in Zambia was lower compared to Tanzania:

“It is difficult to say that the facility in-charge acknowledged everyone individually because they were performing and contributing to the output” (Provincial level stakeholder, 2021).

The level of health worker motivation was also influenced by the type and quality of supportive supervision by district managers (Fig. 1A,B, R3). This factor had the potential to either demotivate or motivate the health workers in both countries:

“Among the confounding factors was the capacity of the management team at the district health office to motivate the facility personnel. You will note that districts that had effective managers tended to perform a lot better than those that didn't have good leadership. Leadership was one of the key aspects that influenced performance” (National level stakeholder, 2013).

The “motivation-district provision” loop (Fig. 1A,B, R3) illustrates that the motivation of district teams to perform supervisory visits in both countries was linked to the P4P design. In Zambia, similar to Tanzania, district managers were motivated by the expected P4P incentive payment, which was tied to the performance of facilities in their district. In Tanzania, the district managers received incentive payments for performance based on drug availability and overall facility performance (Fig. 1B, R7). In Zambia, however, the district manager’s incentive payment consisted of two parts: a fixed amount for supervision and another tied to facility performance (Fig. 1A,R7). The provision of the fixed amount ensured that district managers undertook supervisory visits and other activities contributing to effective P4P implementation, regardless of facility performance.

‘…within that funding there are components for the supervision needed to be done and equally, funding from RBF [P4P] …where we have the operational fund and the incentives. This operational fund was guaranteed for the districts to perform supervision…for quantity audits, assessments and technical support. The incentive part was earned based on the district’s performance framework” (National level stakeholder, 2021).

The ‘motivation-medical commodities’ loop shows that health workers’ motivation to deliver incentivised services leads to the timely requisition of medical commodities in both countries (Fig. 1A,B, R4). This ensures that there are adequate medical commodities for service delivery, with some incentivised services being linked to the use of medical commodities (Fig. 1A,B, B1). Outside of P4P, facilities routinely order drugs through the district office from the central level (Medical Stores Limited/Medical Stores Department), for redistribution to the facilities, with some expected delays in the processing of drug requisitions and delivery of orders, which could result in a failure to achieve targets or increase utilisation (Fig. 1A,B, B2).

The P4P programme incentivised drug availability in similar ways in both countries. In Tanzania, it was an incentivised target for district managers, while in Zambia, district managers were responsible for managing the district pharmacy to ensure the availability of drugs, among other related activities. While drug availability was not directly incentivised, it was important to ensure drug availability at the facility level as it was linked to quality assessments which were incentivised:

‘under quality assessment, …there was a direct relationship between having essential drugs in stock and the quality of services you have to provide. There were certain drugs that they looked for. Without those drugs in stock, for example, amoxicillin, or liquid formulation, you would not be able to provide certain essential incentivised services’ (District level stakeholder, 2021).

The increase in the facility budget from incentive payments enabled providers in both countries to procure basic essential medical commodities outside of the routine drug delivery system (Fig. 1A,B, R6) in cases where the delivery of medical commodities was delayed or depleted (Fig. 1A,B, R8). Other funding sources that could be used to purchase medical commodities included the district and health facility operational grants in Zambia and government funding, health basket funds, and community contributions to the community health fund (CHF) in Tanzania (Fig. 1A,B, R9). In Zambia, the district managers would procure medical commodities on behalf of more remote health facilities or those with less densely populated catchment populations that could not afford transport costs to procure drugs. A notable stop-gap feature to mitigate drug stockouts in Zambia is that the district team would redistribute drugs from facilities that had excess supplies to those that had stockouts, a strategy that was not observed in Tanzania. District managers in Tanzania would provide medicines and medical equipment to health facilities, where needed as a stop-gap measure (Fig. 1A,B, R5).

The increased facility budget due to incentive payments also provided additional resources to improve the facility environment by purchasing cleaning supplies, mattresses, paint and other materials in both countries (Fig. 1A, B, R10). In Zambia, some health facilities used the funds to build maternity wards, renovate old structures or construct mother’s shelters to encourage women who stayed in distant places to deliver from the facility:

“We were able to provide ‘domestos’ [detergent]. At the same time, the facility had no maternity [ward]. That's how we started looking at extending the structure and used RBF [P4P] money,” (Facility level stakeholder).

The ‘ability-retention loop’ (Fig. 1A, B,R11), shows the link between health worker motivation, worker retention, availability and type of health workers and the ability to deliver services. In Zambia, incentive payments and staff accommodation were important factors in retaining qualified health workers. This led to higher retention rates of qualified staff in health facilities, which in turn resulted in increased numbers and improved cadre composition of health workers. Additionally, the hiring of new staff also contributed to an increase in the number of health workers and improvements in the cadre composition at health facilities. In both countries, district managers had the authority to reallocate staff within districts (Fig. 1A,B, B4) to ensure that P4P health facilities were operated by qualified staff:

‘… a health facility needed to be run by trained health personnel. We saw the redistribution of staff to these rural facilities’ (National level stakeholder, 2021).

Similar to Tanzania, where the district managers could request funding and permits for new staff from the national or central level, staff recruitment in Zambia was conducted at the central level, subject to fund availability (Fig. 1A,B, B5). However, some P4P facilities in Zambia used a portion of their incentive payment for operational costs to employ qualified temporary relief staff from within their communities (Fig. 1A, R8.1) as a stop-gap measure while awaiting the staff allocation from the district. This practice was not observed in Tanzania:

‘Hiring of new staff was centrally done by Ministry of Health (Government). However, health facilities, [using incentives payment] hired specialised or trained staff to provide incentivised services’ (Facility level stakeholder, 2021).

3.2. Changes to facility reporting

The mechanisms that led to changes in facility reporting during the P4P programme in Zambia are shown in Fig. 2, Panel A, while the original CLD from the Tanzania P4P programme is presented in panel B. The incentive payment motivated health workers to exert effort towards reporting in both countries, resulting in the timely submission of complete routine health facility data (Fig. 2A,B, R12). In both countries, the failure to submit complete health data would affect the incentive payment amount:

Fig. 2.

Changes to facility reporting during the P4P. Panel A: Zambia CLD. Note to Fig. 1) Black-Element common to both Zambia and Tanzania’s P4P programmes. 2) Grey-Elements specific to Tanzania. 3) Blue-Newly identified elements specific to Zambia. 4) Green dashed line- Newly identified elements in Zambia that are also relevant for Tanzania. 5) Green solid line-Indicates same intensity of effects in both Zambia and Tanzania. 6) Red- Represents the removal or addition of delays in the system. Abbreviations: District Health Director (DHD), District Health Officer (DHO), Payment for performance (P4P). Panel B: Tanzania CLD. Source: Adapted from Cassidy et al#, (2021), figure reproduced in black and white. Abbreviations: Council Health Management Team (CHMT), District Executive Director (DED), Payment for performance (P4P), Pilot Management Team (PMT).

The facility knew that even if they had seen a lot of patients, writing a report was a must. Failure to send the report would affect the facility. …They knew that if they didn't submit within the time limit, they would not get the P4P bonuses … During P4P, almost everyone was keen to submit [their report] on time (District level stakeholder, 2021).

The reporting requirements under P4P were found to be time-consuming and dependent on adequate numbers and cadre of health workers (Fig. 2A,B, R13), who had a good understanding of the reporting system to ensure timely and complete submission of routine health facility data (Fig. 2A,B, R14). However, turnover of staff in Tanzania negatively affected the understanding of the reporting system in health facilities. In contrast, in Zambia, almost every health facility staff member had an idea about the reporting system, and new staff members were taught about the system by existing staff. Consequently, the absence of one staff member did not necessarily leave a gap in the remaining staff’s understanding of the reporting system in Zambia:

“P4P training was comprehensive, running through all the different facets, through all the information systems. Those that were trained, were taken through all the processes from data generation through the reporting process up to when they sent it to the District Health office’ (Provincial level stakeholder, 2021).

P4P not only incentivised reporting but also promoted teamwork and regular review meetings, which facilitated information sharing and allowed support staff to train newly hired qualified staff. This training, whether done formally or informally at the national or facility level, contributed to an increased understanding of the data reporting system. Additionally, health workers in Zambia were already familiar with the information reporting system as it aligned with the government’s regular reporting, with only a few extra indicators added under P4P. However, in Tanzania, the reporting system under P4P was new, and introduced at a similar time to the District Health Information System (DHIS2), meaning that data generation took time as staff familiarised themselves with the new system.

‘The reporting of P4P was in line with government reporting except that we were trying to emphasise on the definition of those indicators. It added on to what was already known …in terms of reporting those indicators, there were one or two indicators that we had introduced for P4P, which were not being reported to the DHIS2 but most of the indicators were those that they were already reporting on in line with Ministry of Health guidelines” (National level stakeholder, 2021).

The ‘support-reporting’ loop (Fig. 2A,B, R15) illustrates the link between supportive supervision by the district team and the ability of health workers to submit complete and timely reports. In Zambia, for example, the district team not only provided guidance on reporting but also collected reports from the health facilities that were unable to submit them. The ability of district managers to perform these supervisory functions related to the P4P programme for the facilities was linked to having received training as part of the programme package. The ‘temptation to misreport’ loop (Fig. 2A,B, R16) represents the temptation to report service delivery at higher levels than was achieved in an attempt to receive a higher incentive payment. In both countries, misreporting was deterred through verification visits. In Zambia the potential loss or decrease in incentive payments for those found non-compliant, and potential suspension for those suspected during verification visits further deterred misreporting.

‘… at the beginning, others tried to do that, but they stopped because when we saw that, we introduced checks and balances on the ground which were very effective … There was a penalty where you could even lose the incentive you were trying to gain. The temptation was there but after seeing the enforcement of the penalty, people started reporting the correct data’ (Provincial level stakeholder, 2021).

3.3. Changes in demand for services

Fig. 3 illustrates the pathways that resulted in changes in demand for services in Zambia during the P4P programme in Zambia (panel A), and the original CLD from Tanzania (panel B). The ‘interaction-quality’ loop (Fig. 3A, B, R17) indicates that kindness and respect of health workers towards patients during P4P had a positive impact on the patients' perceived quality of services. In Zambia, similar to Tanzania, health workers made an extra effort to improve their interactions with patients, such as extending consultation hours and being more friendly, which led to an increase in perceived service quality and improved utilisation of MCH services.

Fig. 3.

Changes in the demand of services during the P4P. Panel A: Zambia CLD. Note to Fig. 1) Black-Element common to both Zambia and Tanzania’s P4P programmes. 2) Grey-Elements specific to Tanzania. 3) Blue-Newly identified elements specific to Zambia. 4) Green dashed line- Newly identified elements in Zambia that are also relevant for Tanzania. 5) Green solid line-Indicates same intensity of effects in both Zambia and Tanzania. 6) Red- Represents the removal or addition of delays in the system. Abbreviations: Community Health Workers (CHWs). Panel B: Tanzania CLD. Source: Adapted from Cassidy et al#, (2021), figure reproduced in black and white. Abbreviations: Council Health Management Team (CHMT), Community Health Fund (CHF), Community Health Workers (CHWs), District Executive Director (DED), Health Facility Governing Committee (HGFC).

Outreach activities played a critical role in creating demand for providing mobile MCH services in both countries (Fig. 3A,B, R18). However, the delivery of outreach services depended on the availability of sufficient health workers and funding (Fig. 3A,B, R20). Due to P4P, both Zambia and Tanzania had increased human and financial resources, which facilitated an increase in outreach activities, resulting in greater MCH service uptake and improved community awareness of the services available at health facilities (Fig. 3A,B, R19). In Zambia, the P4P programme enabled an increase in the number of outreach activities through the hiring of additional staff and the use of payment incentives to acquire transport, such as motorbikes, to reach remote areas for delivering outreach services:

‘We only had five [outreach stations] but with the RBF [P4P] programme now, the number increased from five to nine outreach posts because of the number of health workers that had increased (Facility level stakeholder, 2021).

Health workers in Zambia were eager to undertake outreach activities, enabling them to deliver some incentivised services (Fig. 3A, R24). This was not the case in Tanzania, because incentivised indicators in Tanzania were tied with facility-based care and outreach was instead used as a platform to stimulate facility-based services.

Community health workers (CHWs) in Zambia and Tanzania played a critical role in demand and awareness creation activities for MCH services. In both countries, traditional birth attendants were informally incentivised by health providers to refer mothers to deliver at the health facility (Fig. 3A,B, R21). Similarly, other community health workers engaged in MCH activities were informally incentivised (Fig. 3A, R25) through mechanisms such as the provision of bicycles, and payment of token amounts to improve service delivery (Fig. 3A, R26). In Zambia, there were localised initiatives that were not part of the formal pilot P4P design, such as incentivising women to encourage the use of MCH services (Fig. 3A, R23). In some P4P health facilities in Zambia, mothers who brought their children for immunisation services would be incentivised by health workers. Similar strategies were observed in Tanzania particularly for women giving birth at the health facility.

There are facilities where when a woman delivers, they would give them baby clothes and then sometimes when you bring your child for under 5 maybe they will be given salt, chitenges [cloth]…that motivated other mothers to come for the services (District level stakeholder, 2021).

One of the key mechanisms for demand creation under the P4P programme was the enhancement in the facility environment (Fig. 3A,B, R22) and availability of drugs and supplies (Fig. 3A, B,B7), which led to improved community perceptions regarding the quality of the facility and services offered. Health facilities in both countries used the increased budget from the incentive payments to purchase cleaning supplies, mattresses and other items that improved the facility environment. However, there were some differences between Tanzania and Zambia in terms of facility-level investment in infrastructure and basic equipment. This was partly due to differences in procurement systems and the allocation of incentives. In Tanzania, health workers opted for small investments such as renovation but not construction, and procurement of drugs and supplies but not equipment as since procurement of equipment was managed centrally. In Zambia, in some cases, an increased facility budget facilitated the construction or renovation of structures such as maternity wards or mothers’ shelters to facilitate institutional deliveries.

4. Discussion

We adapted and validated the CLDs developed for the Tanzanian P4P programme to the Zambian context to evaluate the generalisability of the pathways to health system outcomes. We examined factors influencing performance differences, focussing on the P4P programme design, mechanisms and contextual factors underpinning these. While the overarching pathways influencing P4P outcomes in both countries were similar, variations in the intensity of the P4P response in supply, demand, and facility reporting were observed. These differences are primarily attributed to variations in programme design and contextual factors unique to each setting.

On the supply side, incentive payments to health providers increased motivation, enhanced supportive supervision, and improved the procurement of medical commodities in both Tanzania and Zambia. However, the impact of these incentives on staff motivation differed between the two countries. In Zambia, higher levels of trust in the system and safeguards against reduced performance in non-incentivised services contributed to sustained motivation and lower absenteeism rates among health workers (Shen et al., 2017, Friedman et al., 2016). The health workers in Zambia were confident that payments would be made despite the delays, indicating that trust in the system may influence health worker response. In contrast, delays in incentive payments in Tanzania led to frustration and decreased motivation, highlighting the importance of timely and reliable disbursement of funds (Binyaruka et al., 2015, Binyaruka and Borghi, 2017).

The structure of the P4P programmes and the degree of autonomy granted to health facilities influenced their ability to use funds for infrastructure improvements and staffing. In Zambia, allocating a larger share of P4P funds for facility operations and a more decentralised procurement system allowed for greater flexibility in infrastructure improvements and hiring additional staff (Friedman et al., 2016). This, in turn, resulted in a greater impact on service delivery outcomes, such as increased institutional deliveries, compared to Tanzania, where facility autonomy was more restricted (Friedman et al., 2016, Binyaruka et al., 2015). Although tied to different indicators, the incentivisation of district managers in both countries resulted in enhanced supervision—a critical variable affecting provider ability to deliver services through acknowledgement and drug provision— as reported elsewhere (Binyaruka and Borghi, 2017; Friedman et al., 2016).

The P4P programme influenced timely and complete reporting of routine health facility data in both countries. Whereas misreporting was deterred through verification visits in both countries, in Zambia, the potential loss or decrease in incentive payments for non-compliant staff further deterred misreporting (Friedman et al., 2016). Staff turnover in Tanzania negatively affected the understanding of the reporting system in health facilities, while in Zambia, most health facility staff were familiar with the system. On the demand side, P4P influenced patient-perceived service quality and community awareness in both countries, with community health workers, outreach services, and traditional birth attendants playing a critical role in demand and awareness creation activities for MCH services (Binyaruka and Borghi, 2017, Cassidy et al., 2021). The post-pilot phase in both countries included a community incentive package, acknowledging the role of these actors in service delivery and demand creation.

The findings from this study have several implications for the design and implementation of P4P programmes in LMICs. In Zambia, facility budgets emerged as a crucial factor in enhancing the health system. Coupled with financial autonomy, incentive payments allowed facilities to invest in human resources and infrastructure, such as maternity shelters and delivery beds, which attracted women from remote areas and increased facility deliveries. Evidence indicates that P4P enhances facility managerial autonomy, and improvements in indicators are often a result of funding rather than performance-related conditionalities (Diaconu et al., 2021).

Despite the potential benefits of P4P programmes, they can also be complicated and costly due to the rigorous verification processes (Walque and Kandpal, 2022). Some argue for alternative approaches, such as direct facility financing (DFF), which may achieve similar results at lower cost (Khanna et al., 2021, Walque and Kandpal, 2022). The Zambia P4P programme impact evaluation show that while P4P was more costly than input-based models, it delivered better health system outcomes (Chama-Chiliba et al., 2022, Friedman et al., 2016). Regardless of the financing model, a strong public financial management system is essential (Barroy et al., 2022). Further, the active purchasing element triggered by P4P programmes can be used to design bigger and nationwide schemes. In Zambia, elements from the P4P programme were used to inform the design of the social health insurance programme that was operationalised in 2019 (World Bank, 2020).

A key question to address is whether adapting existing CLD pathways is a viable alternative to building a CLD from scratch. The approach offers significant advantages, especially when the research scope— such as the time frame of interest, the boundary of the issue, and the level of system aggregation— remains consistent, as described by Cassidy et al. (2022). In this study, the P4P pilot programme spanned two years, and involved national, regional, and primary care level actors, with the boundaries defined by health facilities in both Zambia and Tanzania. The integration of primary and secondary data sources—such as key informant interviews and focus group discussions—enabled a comprehensive analysis of the generalisability of CLD pathways. Assessing whether existing CLDs can be adapted across contexts is particularly relevant given the substantial time and resource burden involved in developing new CLDs. Our findings suggest that while adaption is feasible, contextual differences require careful consideration. Stakeholder engagement in refining these pathways, can increase stakeholder buy-in and the uptake of findings by policymakers (Zimmerman et al., 2016).

The study has limitations. Focussing on the pilot phase introduces potential recall bias during validation, and the process itself is time-intensive, requiring multiple verification stages. Further, stakeholder interviews maybe subject to recall bias and social desirability bias, particularly in discussions on programme effectiveness. The participants were directly involved in the P4P programmes, potentially leading to overemphasis on positive outcomes or underreporting challenges. To mitigate this, we triangulated interview data with secondary qualitative data from the process evaluations. While the CLD pathways highlight similarities and differences in the P4P between Zambia and Tanzania, precise presentation techniques are necessary for clarity. The use of qualitative data limits the ability to quantify the strength of the relationships between the variables and only allows for the consideration of the nature and direction of the relationship. It is possible that the magnitude of the cause-effect relationship differed between countries. Quantifying the relationship in a simulation model is the subject of a future paper. Although we explored the generalisability of CLD pathways across Zambia and Tanzania, cross-country adaptation of CLDs remains challenging due to differences in programme design and contextual factors. The findings should be interpreted with caution, recognising that certain pathways maybe specific to the contexts under study rather than universally generalisable.

5. Conclusion

The study explored the generalisability of pathways depicted by CLDs in the P4P schemes by adapting and validating Tanzanian CLDs to the Zambian context. Our findings demonstrate that while the overarching pathways influencing P4P outcomes share similarities across these two contexts, variations in the intensity and impact of these pathways were observed. These differences were primarily driven by variations in programme design and local contextual factors, such as facility autonomy and trust in the health system.

The analysis showed that Zambia’s higher level of facility autonomy and stronger trust in the system contributed to more pronounced improvements in health worker motivation, service delivery, and overall health outcomes compared to Tanzania. These findings highlight the importance of context-specific adaptation when designing and implementing P4P programmes, as a one-size-fits-all approach may not yield the desired outcomes in different settings. The study also highlights the utility of CLDs as a tool for understanding health system dynamics and conducting cross-country comparisons of health interventions. However, the generalisability of CLD pathways should be approached with caution, as local factors significantly influence how pathways manifest in practice. Future research should continue to explore the conditions under which CLD pathways can be reliably transferred from one setting to another and investigate the potential for using CLDs in cross-country comparisons of health interventions. Policymakers and programme designers should consider local context and system-specific characteristics when adapting P4P programmes to ensure their effectiveness and sustainability.

CRediT authorship contribution statement

Chama-Chiliba Chitalu Miriam: Writing – review & editing, Writing – original draft, Visualization, Validation, Software, Resources, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Cassidy Rachel: Writing – review & editing, Writing – original draft, Validation, Software, Resources, Methodology, Investigation, Formal analysis, Conceptualization. Sachingongu Nkenda: Writing – original draft, Validation, Resources, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Chansa Collins: Writing – review & editing, Writing – original draft, Validation, Resources. Binyaruka Peter: Writing – review & editing, Writing – original draft, Validation, Resources. Borghi Josephine: Writing – review & editing, Writing – original draft, Validation, Supervision, Methodology, Funding acquisition, Formal analysis, Conceptualization. Semwanga Agnes Rwashana: Validation, Software, Resources.

Funding

This work was funded by the Medical Research Council under the Health Systems Research Initiative grant (MR/R013454/1). Study approval was obtained from the Observational Research Ethics Committee at the London School of Hygiene and Tropical Medicine (LSHTM Ethics Ref: 16139-2), the Institutional Review Board at Ifakara Health Institute (IHI/IRB/No:15-2019), the National Institute for Medical Research (NIMR/HQ/R.8a/Vol.IX/3154) in Tanzania, Ministry of Health in Zambia (MH/1001/23/10), ethics review from ERES Converge (Ref. No. 2020 Dec009), and the National Health Research Authority in Zambia (NHRA000015/26/01/2021).

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary material

Supplementary material

Data availability

The analysed qualitative data is not publicly available as it cannot be sufficiently anonymised. Some interviews with key stakeholders may reveal their identity through the responses, making confidentiality impossible to ensure.