Activity-based contracting for optimization of the mass distribution of insecticide-treated nets in the Democratic Republic of Congo: pilot implementation in Kwilu province

Abstract

Background

Promoting the use of insecticide-treated mosquito nets (ITNs) is one of the main strategies for reducing malaria-related morbidity. An innovative activity-based contracting (ABC) approach has been implemented in Kwilu Province, Democratic Republic of Congo to optimize ITN mass distribution campaigns, with payments based on contractually defined programmatic outcomes for key campaign activities following independent verification of results.

Methods

This internal evaluation was carried out using a mixed methods approach combining qualitative and quantitative document and content analysis from a series of three workshops: validation workshops for campaign results at provincial level for the 2021 and 2022 campaigns; internal evaluation workshop for the Kwilu campaign as part of the ABC approach organized by “Santé pour tous en milieu rural” (SANRU) with its sub-contractors; and national campaign evaluation workshop organized by the National Malaria Control Program.

Results

The pilot campaign with the ABC approach in Kwilu has demonstrated better results than campaigns conducted using the standard, non-ABC, approach: better household coverage (99.9% vs. 97.3%) and improved compliance with ITN allocation to households based on the household size (98. 9% vs. 84.7%); lower loss of ITNs (0.3% vs. 0.5%) with immediate penalties for lost ITNs in the province under the ABC approach; shorter campaign lead times (14 vs. 28 weeks from the start of training to the launch of distribution). This last point is crucial, as it is likely to generate efficiencies and contribute to ensuring timely ITN replacement campaigns.

Conclusion

The challenges encountered and the lessons learned in the implementation of the pilot ABC approach in Kwilu could guide future distribution campaigns in the DRC and other African countries that would like to engage in distribution campaigns based on performance-based incentive contracts.

Background

Numerous performance incentive systems have been implemented over the past few decades under a variety of names—pay for performance (P4P), performance-based contracting (PBC), performance-based financing (PBF), and results-based financing (RBF)—but all boil down to rewarding the delivery of specific services to encourage greater coverage, better quality, or improved health outcomes [1]. The World Health Organization (WHO) defines PBF as “a form of payment to service providers in which financial incentives are provided only to health care providers (and not to beneficiaries) when they achieve pre-determined, verified performance targets, often defined in terms of process or outcome indicators, adjusted by some measure of quality” [2]. More recently, however, PBF has been presented as a package of reforms that goes beyond simple payments to providers and includes separation of functions between purchaser, providers, and auditors; increased autonomy for health care providers; strengthened monitoring activities; and community participation [2]. Thus, beyond improving health service delivery indicators relating to incentive payments, PBF also has the potential to increase awareness of strategic purchasing, improve governance and institutional arrangements, and strengthen strategic purchasing functions [2].

Vector control using insecticide-treated nets (ITNs) is a key malaria prevention strategy that has contributed to a decrease in malaria worldwide [3, 4]. ITN access and use have greatly increased in sub-Saharan Africa in the last decade, resulting in a significant decrease in malaria morbidity and mortality in Africa, despite the emergence of vector resistance to insecticides [5–8]. Mass distribution campaigns (MDC) are the best way to rapidly increase coverage of at-risk populations with ITNs to achieve impact with continued ITN distribution through various other channels to fill gaps and ensure key vulnerable populations have access to non-campaign ITNs [3, 9].

Numerous operational difficulties and constraints that hinder the achievement of the universal coverage objective set for ITN distribution campaigns in the DRC and other countries have been noted over time: (i) a significant and unpredictable fluctuation in the population between demographic data used for planning and data from the health zones (HZ) from microplanning and household registration, due to the unavailability of reliable census data, the last of which dates back to 1984 in the DRC; (ii) a propensity of heads of households to exaggerate the number of people living in households in order to benefit from more ITNs, leading to stockouts during distribution; (iii) conflicting agendas where provinces targeted by ITN distribution campaigns are engaged in parallel with other mass activities such as immunization, as well as the response to COVID-19 considered by the Ministry of Public Health (MPH) to be higher priority; (iv) households not reached either because of difficulties in access or lack of motivation on the part of the campaign personnel in the revisiting of missed households; (v) disruptions in the provision of malaria prevention and management services since the occurrence of the COVID-19 pandemic [10–13].

In addition to these operational difficulties, the Additional Safeguards Policy (ASP), which has been in use since 2004, is a “Global Fund risk management tool” that leads the Global Fund to Fight Tuberculosis, AIDS and Malaria (GFTAM) Secretariat to apply mainly two categories of measures: [1] approval of implementation modalities and [2] more restrictive modalities relating to the flow of funds [14]. The ASP has led to the implementation of restrictive procedures, including eligibility of expenditures based on compliance with procurement and financial management procedures, right down to the most peripheral levels of the health system, resulting in recurrent delays in campaigns lasting up to one year, with three to five months taken up by local procurement alone [15].

On the other hand, fragile governance, lack of access to health services, and human-made and natural crises have led the GFTAM to classify the DRC as a “Challenging Operating Environment” (COE) country [16]. The three major strategies recommended by the GFTAM Board to improve the effectiveness of interventions in this context are innovation, increased flexibility and partnerships [16]. Thus, an innovative multi-partner performance incentive approach was considered to optimize ITN mass distribution campaigns through payments based on performance for key ITN mass distribution campaign activities in accordance with defined contractual terms. This approach, which is related to P4P/PBC/PBF/RBF, has been called “activity-based contracting” (ABC).

Many countries in sub-Saharan Africa have implemented PBF to improve the performance of their health systems [2, 17–22]. Several studies from low-income countries have shown that PBF had better outcomes for improving health services than the traditional “input” approaches which are characterized by centralized planning and the distribution of inputs such as salaries, essential drugs, medical equipment [17]. Before the ABC pilot in Kwilu Province, ITN MCDs were carried out using traditional “input” approaches, here referred to as the “standard approach”.

The DRC also has a long history of PBF in the health sector [23–27]. However, the ABC pilot in Kwilu Province marked the first time the Global Fund approved the adoption of an intervention-focused results based contracting approach in the DRC, although an ABC approach to incentive payments conditioned on activity results is defined in its Grant Budgeting Guidelines [28].

This ABC approach is a variant of PBF. Two aspects differentiate the usual PBF from the ABC approach: (i) in contrast to ABC approach, in the PBF approach no advances are given, the services are purchased after they have been provided and (ii) in the PBF, payments are made on basis of independent verification of the achievement of required performance on the basis of process indicators, whereas in the ABC approach, payments are made on basis of independent verification of the achievement of required performance on the basis of outcome or coverage indicators.

In this article, we present the results of an internal evaluation of the planning and implementation process of a pilot mass ITN distribution campaign using the ABC approach in Kwilu Province and compare the pilot results to the results obtained in provinces that have implemented campaigns using the standard approach (e.g. payments being not conditioned on performance defined in pre-determined contractual results).

Materials and Methods

Study design

This internal evaluation was carried out using a mixed methods approach combining qualitative and quantitative document and content analysis from a series of three workshops: (i) workshops to validate the campaign results in each province (all eight implemented) between the end of 2021 and July 2022; (ii) internal evaluation workshop for the Kwilu campaign under the ABC approach organized from August 25 to 27, 2022 in Kinshasa by “Santé pour tous en milieu rural” (SANRU) with its sub-contractors; and (iii) a workshop organized by the National Malaria Control Programme (NMCP) from September 26 to 30, 2022 in Lubumbashi for the evaluation of all campaigns in 2021 and the first half of 2022 with validated data.

During the data validation workshops at the provincial level for each of the campaigns, the Heads of the Provincial Health Divisions (H-PHD) brought together two delegates from each of the health zones (HZ) under their jurisdiction [the Chief Medical Officer (CMO) and his or her Nurse Supervisor (NS)], to present the results obtained in their HZ. A validation panel consisted of the Provincial Health Division Management Team (PHD-MT), including the NMCP Provincial Coordinator, accompanied by delegates from the central NMCP and Technical and Financial Partners (TFPs) supporting the campaign.

During the Kinshasa workshop, SANRU, GFTAM’s Principal Recipient (PR), brought together stakeholders subcontracted by itself or directly by GFTAM for ABC technical assistance, for a self-assessment. Participants at the workshop included the following: (i) IMA World Health (IMA), Sub-Recipient (SR) of SANRU, responsible for the technical, logistical and financial support for the implementation of the campaign in Kwilu Province and the management of the ABC contract with the HZs in Kwilu; (ii) the National Coordination of “Coordination Nationale du Renforcement du Système Communautaire” (CNRSC), subcontracted by SANRU and responsible for the independent verification of household coverage; (iii) the University of Kisangani (UNIKIS), subcontracted by SANRU and responsible for assessing the conformity of the coverage provided by the CNRSC and for the independent verification of the quality of the training of the household registration and distribution teams by the HZ; (iv) the “Fonds de Développement des Services de Santé” (FDDS), a Sub-Recipient of SANRU responsible for supporting routine malaria control activities (SRR), here responsible for independent verification of ITN inventories in the HZ; (v) Palladium, contracted by GFTAM and responsible for providing technical assistance to the various parties and keeping a record of the difficulties encountered during the implementation of the ABC approach from the conceptualization phase to the finalization of payments in the Kwilu Province pilot. Each party presented its contractual results, strengths and weaknesses, and areas for improvement, which were then discussed by all workshop participants.

The logbook of problems encountered, which was continually updated thanks to Palladium’s technical assistance, consisted of a matrix with twelve columns and as many rows as the problems were identified as the Kwilu CDM was implemented. The 12 columns are: (i) domain of the ABC campaign; (ii) difficulty encountered; (iii) severity; (iv) causes identified; (v) changes proposed; (vi) proposed time of application of the change; (vii) level of achievement; (viii) implementation strategy; (ix) resources required; (x) actor proposing the change; (xi) actor responsible for implementing the change; (xii) risk associated with implementing the change.

The lines were filled in as and when difficulties were identified during the implementation of the successive stages of the CDM in Kwilu province. These lines would refer to the different “domains of CDM” categorized into: (i) micro-planning, (ii) training, (iii) logistics and procurement, (iv) disbursements, (v) household enumeration and ITN distribution, (vi) data quality assessment; (vii) independent verification of deliverables.

The Lubumbashi workshop was organized by the NMCP with the participation of the H-PHDs, each accompanied by the NMCP Provincial Coordinator in their province for the presentation of results, strengths and weaknesses as well as lessons learned during the campaign in their province. Delegates from the central NMCP and TFPs that supported the campaign presented the results of their support and their impact on the campaign process and results. Each presentation was followed by time for discussion to capture lessons learned that could improve future campaigns. On the sidelines of this workshop, there was a presentation of the results of SANRU’s internal evaluation of the pilot ABC approach in Kwilu Province for discussion. In addition, a special panel was organized by the Director of the NMCP to consolidate the main findings and highlight strengths and areas for improvement for future campaigns under the ABC approach.

In all workshops, the elements presented revolved around: (i) the planning and implementation of the campaigns in terms of compliance with the timeline and the causes of deviations from the NMCP standards; (ii) the coverage results presented by the HZ, hosted in the SANRU server, cross-checked via the databases resulting from the consolidation of the daily summary sheets kept by the HZ management teams (HZ-MT); (iii) the results of independent audits conducted by the CNRSC; (iv) the major findings of MPH management supervision and TFPs monitoring missions; and (v) ITN logistical data submitted by the HZ, hosted on the SANRU server, cross-checked using logistical tools duly signed by authorized stakeholders.

The roles and responsibilities of the various CDM stakeholders and a brief description of the tasks of the key players in the ABC approach are presented below.

Scope of the study

The study focuses on mass ITN distribution campaigns organized in 10 provinces during 2021 and the first half of 2022 under GFTAM funding: Bas-Uélé, Haut-Uélé, Ituri, Kinshasa, Kwilu, Maniema, North Kivu, Sankuru, South Kivu and Tshopo (Fig. 1). Of these 10 campaigns, those in North and South Kivu which had not held provincial campaign data validation workshops prior to the national evaluation workshop were excluded from the study. Of the eight GFTAM-funded provinces included in the study, only Kwilu Province was selected as the pilot province to implement the ABC approach; the other seven provinces conducted campaigns using the standard approach. These provinces included eight HZs in Bas-Uélé, 13 in Haut-Uélé, 36 in Ituri, 35 in Kinshasa, 24 in Kwilu, 18 in Maniema, 16 in Sankuru and 23 in Tshopo.

Fig. 1.

Mapping of mass ITN campaigns in 2021 and first half of 2022

Conceptual framework of the ABC approach

To address the recurrent challenges of the standard campaign implementation approach in the DRC, a paradigm shift from expenditure eligibility at the HZ level based on compliance with procurement and financial management procedures to expenditure eligibility based on programmatic deliverables required for a high-quality campaign was proposed and piloted in Kwilu Province in 2021 (Fig. 2) [15]. A high-quality campaign was defined as one that (i) is completed within 36 months of the previous campaign distribution, including five months between the start of the training of the PHD-MTs and the launch of the distribution, (ii) has coverage exceeding 90% of households having received at least one ITN, in order to expect more than 80% utilization, and (iii) has households served with the correct number of ITNs according to household size on the basis of the ITN allocation key defined by the NMCP.

Fig. 2.

Paradigm shift to an ABC approach in campaign implementation.

(Adapted from the NMCP Manual for Planning and Implementing ITN Distribution Campaigns Using the ABC Approach) [15]

The ABC approach was based on the principle of contracting HZs on the basis of key activity results and the principle of separation of functions as recommended in the PBF approach: (i) the delivery function performed by the HZ teams; (ii) the verification function performed by independent agencies subcontracted by the GFTAM’s PR SANRU [CNRSC, a Scientific Research Institution (SRI) and an SRR]; (iii) the payment or purchasing function performed by SANRU’s SR IMA in charge of implementing the campaigns at the provincial level; and (iv) the regulatory function performed by the MPH through the General Secretariat for Health (GSH) and the NMCP, consisting of the issuing of standards and directives for implementing the campaigns, relayed by the Provincial Health Division (PHD) (Fig. 3).

Fig. 3.

Conceptual framework of the ABC approach

Service delivery

The main service expected of the HZs was the delivery of ITNs to households in accordance with the standards defined by the NMCP, with household ITN coverage as a contract deliverable (Contract Deliverable 1). After validation of the microplans, the process of which is described below, the operations of counting households and delivery of ITNs to households really took place at HA level under the coordination of the NTs. The various stages of household enumeration and ITN distribution consisted of (i) recruiting community workers (trio of CHWs responsible for enumerating households and distributing ITNs, CHWs responsible for sensitizing and social mobilization, and village chiefs responsible for storing ITNs in the villages), carried out by the TNs and validated by the HZs CMO; (ii) training of TNs and C-HDACs by the HZ-MT approximately one month before the start of ITN distribution, followed by training of community actors by TNs 1 to 3 days before the start of distribution, depending on the category of actor; (iii) sensitization of the population (Three days before the enumeration coupled with distribution, the social mobilisers of the community relays were trained to begin sensitization with megaphones two days before the start of the enumeration coupled with distribution. They continued to raise awareness according to the roadmap of the trinomials of CHWs responsible for household enumeration and ITN distribution. They were also sensitizing heads of household during enumeration visits coupled with ITN distribution. In addition, all available communication channels were used by the HZ-MTs and TNs, with particular emphasis on public criers); (iv) preparation of operations (24 h before the start of ITN distribution, at the level of each HA, the TNs drew up a map of the deployment of the distribution teams and the itinerary of the social mobilisers. Then TNs prepared the various tools to be given to each distribution team, i.e. an android telephone configured with the ODK form, a powerbank, leaflets explaining the importance of using ITNs to be given to each head of household, markers for marking the households visited, a hip bag per team and the number of registration forms required depending on the villages and households expected, Personal Protective Equipment to prevent transmission of COVID19 and T-shirts for each actor bearing messages on the importance of using ITNs); (v) deployment of actors (The actors in charge of enumeration coupled with distribution (CHW Trinomials) with a signed assignment form from the CMO of HZ, especially in urban areas, were deployed according to the cartography and itinerary previously established by the TN); (vi) household enumeration and ITN distribution took place over 7 days, followed by 5 days or more for the recovery of households missed by the CHW teams responsible for household enumeration and ITN distribution (we have previously described in detail the process of household registration coupled with ITN distribution) [10, 12]; (vii) Monitoring and internal evaluation of the MCD (During the distribution of ITNs, the CHW trinomials responsible for counting and delivering ITNs were supervised daily by the TNs at a rate of at least 5 teams per day. The TNs and their teams were supervised by a member of the HZ-MT who was supported by a member of the PHD-MT. At the end of each working day, the TN, assisted by an IMA independent supervisor (IS), reviewed the data collected on the telephones, cross-checked it with the data recorded on the registration forms before transmitting the ODK data online and sending the data from the registration forms, compiled in daily compilation sheets, to the HZCO. At the end of the activity, a final triangulation between the paper data, the ODK data and the data from the daily ITN deployment sheets was carried out by the TN with the assistance of the IMA IS in the presence of the distribution teams and the C-HDACs for the validation of the household enumeration and ITN distribution data before the final validated data was sent to the HZ-MT. On a day-to-day basis, the HZ-MT supported the TNs in their work through supervision, daily feedback based on ODK data and paper data previously analyzed and interpreted by the HZ-MT, and during daily meetings of the local coordination committee. At the end of the activity, after receipt of the data validated by the TNs, the HZ-MT brought together all the TNs for a 2-day meeting for a final triangulation between the paper data, the ODK data and the data from the ITN daily deployment sheets to make any final corrections if necessary.) [15]

To ensure proper delivery of ITNs, Community Health Workers (CHWs) had to be properly trained by the HZ-MT to ensure quality distribution, resulting in the “Quality of Training” contract deliverable (Contract Deliverable 2). Of all the training sessions organized to strengthen the capacities of actors involved in household enumeration and delivery of ITNs to households, the output of the training session for three CHWs responsible for enumeration and the delivery of ITNs was selected as one of the contractual deliverables in the ABC approach. The training took place over 2 days in the health centers, where all the CHWs responsible for counting households and distributing ITNs were brought together under the facilitation of the TN. During the 1 st day, they were briefed on (i) the importance of ITNs in combating malaria; (ii) the practicalities of distributing ITNs according to household size; (iii) mapping the enumeration and distribution area from the 1 st to the last day; (iv) the procedures for collecting data on the ODK form on the telephone and recording data on the registration form; (v) the synthesis and reconciliation of demographic and ITN management data at the end of the day; (vi) elements of communication in support of ITN distribution and use. Once the theoretical part of the training had been completed, the TN organized for the CHWs to visit households in the field for a simulated enumeration on Day 2. After this simulation field visit, each CHW was then asked to carry out a correct registration of at least 5 households in a maximum of 1 hour as a post training evaluation test. All the investigating CHWs in a ZS must pass the test to validate the deliverable on trained personnel [15].

Given the importance and cost of ITNs, it was agreed that ITN tracking would be strict with a balance inventoried at the end of the distribution (Contract Deliverable 3). Rather than a centralized strategy (in two phases: (i) importing ITNs into the country and storing them at a centralized location, followed by (ii) distributing them in the country from this centralized location), the DRC has opted for a decentralized strategy characterized by two distinct activities: macro-logistics consisting of (i) the importation of ITNs into the country and their delivery to the lowest accessible storage point (in this case, at the level of the HZCO) and (ii) micro-logistics involving the transport of ITNs from these various storage locations to the villages via transitory storage at the level of the HAs (AMP). This last strategy is compatible with the ABC contract, which makes the HZs responsible for all micro-logistics operations that take place at the optional level, i.e. from the HZCO to the households. The various phases of this micro-logistics were carried out at the HZ level, at the HA level and at the community level in three sequences: reception - storage - delivery to the lower level.

I. At the HZ level:

1. Reception of ITNs: the transporter delivered the ITNs to the HZ warehouse where, under the supervision of the HZ-CMO, an HZ-MT member analyzed the documentation accompanying the ITNs using the ITN distribution plan before organizing the unloading of the ITNs and the signing of the proofs of delivery (POD) drawn up by SANRU and the transporter. The ZS Storekeeper then prepared the reception document (RD), which he signed with the transporter. The HZ-MT then prepared the Receipt Report. At the end of this process, the IMA Tracking Agent scanned the RD and forwarded them to his superiors.

2. ITN warehousing: under the supervision of the HZ-MT, the ZS Storekeeper completed the stock sheet using information from the RD, and kept the keys to the warehouses in accordance with their procedures.

3. Delivery of ITNs to the health center: the ZS Storekeeper prepared the delivery form in accordance with the pre-established deployment plan, which was countersigned by the HZ-CMO. He then drew up the delivery form for each destination, to be countersigned by the carrier once the ITNs had been received. He delivered the ITNs and updated his stock sheet. The transporter then collected the parcels after countersigning the delivery note, of which the storekeeper kept one copy and the transporter took two, one for the HA and the other for his use for payment purposes.

II. At the HA level:

1. Reception of ITNs: the transporter delivered the ITNs to the HA’s storage site and signed the POD. The TN analyzed the documents accompanying the ITNs using its microplan, the POD and the quantities of ITNs counted with the transporter before unloading them, signing the RD with the transporter and the receipt report with the PRESICODESA. At the end of this process, the IMA IS scanned the RD and forwarded them to its hierarchy.

2. ITN warehousing: Under the supervision of the TN, the HA Storekeeper filled in the stock sheet using information from the RD from the TN, and kept the keys to the warehouses.

3. Delivery of ITNs to the villages: the HA Storekeeper prepared the delivery form according to the pre-established distribution plan countersigned by the IT. He then drew up the POD for each destination to be countersigned by the transporter before delivering the quantities to him and updating his stock sheet. The transporter then picked up the parcels after countersigning the POD, of which the storekeeper kept one copy, and the transporter took two, one for the manager of the village or secondary storage site and the other for his use for payment purposes.

III. At Community level:

1. Reception of ITNs: the transporter delivered the ITNs to the village chief at the ITN storage site. The village chief analyzed the POD and counted the bundles of ITNs before unloading them and signing the POD. Finally, he completed the daily deployment form.

2. ITN storage: the village chief filled out the daily deployment form using information from the POD and the visual count and kept the keys to the site.

3. Delivery of ITNs to CHWs for distribution to households: the village chief delivered ITNs to the CHWs trinomials on a daily basis. At the end of the day, he recorded the balance of his stock on the daily deployment form.

At the end of the household enumeration and ITN distribution activities, all the village chiefs who have stored the ITNs returned any balances to the health center, where the TN draws up a discharge for them. The TNs then brought all the balances back to the HZCO when they travelled to validate the data, as mentioned above. The storekeeper at the HZ received all the ITN balances brought back by the TNs and drew up discharges for them. He then enters all these quantities of returned ITNs in the HZ stock record pending independent verification of the ITN inventories described below.

Verification function

It was carried out by three different stakeholders, CNRSC for deliverable 1, UNIKIS for deliverable 2 and SRR FDSS for deliverable 3. The validation procedures for these deliverables are described below, in the “data validation” section.

Purchasing function

This function was carried out by IMA, SANRU’s SR. Payment terms were specified in the contracts signed between IMA and the HZs. We provide more details on this matter in the “Contracting procedures” section.

Regulatory function

1. Decision-makers at the national level (SG, PNLP) defined the priorities based on the complex task of balancing political priorities with technical rationale during de the process of designing the ABC approach, coordinating the implementations of MDC of ITN through macroplanning until the evaluation closing workshop for all MDC of a period (usually a year), assuring support of key implementation steps to the provincial level.

1. Design phase: The process of designing the ABC approach has been long and laborious. As early as the last year of implementation of the grantt cycle 5 of GFTAM (2018–2020 grant), SANRU PR and its SR IMA initiated discussions with the GFTAM on the consequences of complex implementation contexts and weak governance, particularly at the peripheral level. These two organisations had proposed applying fixed costs to the various activities and simplifying the justification procedures at the ZS level. As this simplistic approach did not meet with the GFTAM’s policies, GFTAM contacted SANRU and IMA to develop an ABC approach, for which new guidelines were available from the GFTAM, but needed to be made operational. Starting at the beginning of the 2nd quarter of 2021, a series of 2 meetings per week have been set up between the GFTAM, the PR SANRU and the SR IMA. At the end of this quarter, Palladium was contracted by the GFTAM to provide technical support in formalising the exchanges as the model developed. From the 3rd quarter of 2021, the NMCP was involved in the discussions to ensure that the priorities of the MPH and the technical requirements of the NMCP were considered. In the 4th quarter, the SG was consulted to ensure ownership at the highest level of the MPH admnistration. In December, an exchange mission led by the SG office and the Director of the NMCP, in collaboration with SANRU and IMA, travelled to Bandundu town, the Kwilu provincial capital, for consultative meetings with the political and administrative authorities at provincial level and in the Bandudu urban health zone, then in the urban health zone of Bandudu and the rural health zone of Vanga, reaching the level of the health authorities and a few villages in the 2 health zones. At the end of this process, the documentation and tools needed to start up the Kwilu pilot were drawn up, mainly (i) the ABC campaign manual defining the procedures and roles of all the stakeholders, accompanied by implementation tools at different levels of the health pyramid and (ii) the ABC contract between IMA and the health zones and its appendices. All of this documentation was validated at a series of workshops coordinated by the NMCP, which then assumed the regulatory function through the mechanisms described below.

2. Coordination of implementation phase: The 2 meetings a week attended by the GFTAM, SANRU, IMA and Palladium mentioned above continued, involving the verification stakeholders as soon as they were contracted by SANRU. During these meetings, the logbook of difficulties encountered mentioned above was filled in to identify corrective actions to be implemented for the Kwilu pilot or for the subsequent extension phases. At the same time, a national technical co-ordination committee (NTC) bringing together the partners, under the leadership of the NMCP, met once a week to seek synergies and harmonize joint work between the partners, from the village level up to the national level of the organizations concerned. The difficulties identified in the above-mentioned logbook were presented at these NTC meetings so that decisions could be taken to solve the problems encountered.

3. Support to key steps at the provincial level: This support was provided throughout the key stages of the preparation and implementation of the MDCs, in particular by: (i) the macro-planning workshop organized by the central NMCP; (ii) the facilitation of training workshops for PHD-MT, HZ-MT and HZ data managers; (iii) support for the micro-plan validation workshop, (iv) supervision of household enumeration and ITN distribution activities and (v) support for the data validation workshop and validation of performances.

2. At the provincial level, the PHD-MT assured the implementation of national health policy and conducted quality reviews. In fact, this involved: (i) countersigning contracts between the IMA and the HZ, to ensure that the PHD fulfilled its supervisory compliance role; (i) participating to the macro-planning workshop organized by the central NMCP; (ii) organizing training workshops for HZ-MT and HZ data managers and supporting training of TN and C-HDAC as well as community actors training; (iii) organizing for the micro-plan validation workshop, (iv) supervising household enumeration and ITN distribution activities, support for the data validation workshop and validation of performances; (v) supporting the data validation workshop and validation of performances at peripheral level and (vi) organizing the data validation workshop and validation of performances at the provincial level.

Roles and responsibilities of stakeholders in the ABC approach

Table 1 describes the roles and responsibilities of campaign stakeholders under the ABC approach as described in the NMCP manual for planning and implementing ITN distribution campaigns under the ABC approach [15].

Table 1.

Roles and responsibilities of stakeholders in the ABC approach

Domain	Stakeholder	Roles and responsibilities
Regulation	MPH	Sectoral policy guidelines
	SG	• Monitoring compliance between the strategies developed by the NMCP and the policies of the MPH as well as compliance with government administrative regulations; • Validation of the campaign planning and implementation manual
	NMCP	• Development of strategies and definition of standards for planning and implementing ITN distribution campaigns; • Training of stakeholders; • Supervision; • Validation of macroplan, microplan and registration/distribution results; • Organization of the meetings of the National Technical Committee (NTC)
	PHD	• Supervision of the implementation of ITN distribution campaign in the HZs; • Translation of guidelines and standards into operational instructions via bulletins and memos to the HZ-MTs; • Countersigning of contracts between the SR IMA and the HZs to endorse their effectiveness
Service	HZ-MT	• Contract signing; • Organization of training activities, microplanning for the HZs, deployment of ITNs and other inputs to health areas (HAs) and villages; • Dispatch of phones and power banks to the HAs and pick up at the end of the distribution; • Registration and ITN distribution to households; • Data validation and reporting; • Organization of collection of remaining ITNs; • Submission of ABC deliverables to the SR IMA
	Titular Nurse (TN)	• Organization, at the HA level, of the training of distribution agents, mobilizers and village/street chiefs; • Development of the HA microplan; • Deployment of ITNs from the HC to the villages/streets; • Receipt and dispatching of telephones and power banks to distribution agents and their return to the Health Zone Central Office; • Proximity supervision; • Distribution data validation and reporting; • Routing of leftover ITNs to the Health Zone Central Office
	Village chief	• Proposal of the long list of distribution stakeholders to the TN which will retain the required number meeting the profile; • ITN storage and custody; • Daily ITN allocation to distribution teams; • Participation in the daily inventory reconciliation; • Mobilization of the Community Animation Cells (CAC)/Community Based Organizations (CBO) and of the village/community
	CHW	• Distribution by teams of three (one in charge of registering the household on the telephone (CHW registrar = Team leader), one in charge of handing over the ITNs to the head of the household and marking the tally sheet (CHW distributor) and one in charge of transporting the ITNs (CHW transporter); • Community outreach
Verification	CNRSC	• Organization of the verification of 5% of households to assess the actual level of ITN coverage which will be used as the final coverage to be retained for payment; • Determination of the accuracy of the distribution by comparing the campaign standards for household ITN allocation with the number received by households based on household size; • Proposal of corrective actions in case of omission or inadequacy observed in the field; • Contribution to the mobilization of the community; • Raising awareness of the use of ITNs among the households visited
	SRI	• Provides correction of post-test results (registration of households within the required time) to determine quality of training (≥ 1/3 of CHWs correctly registered households); • Performs independent monitoring by lot quality assurance sampling (LQAS) as a cross-check to the 5% verification of households to certify its validity
	SRR	• Carrying out the physical inventory of the balance at the Health Zone Central Office by means of a report co-signed with the Health Zone Medical Director and the zonal supervisors of IMA
Payment	IMA	• Executes payment for the HZ activities based on the contract deliverables through the performance validation report based on the Open Data Kit (ODK) administrative coverage data, the 5% verification report, the post-test correction report and the ITN inventory report; • Follows up on implementation (effectiveness of activities, compliance with standards) and provides logistical and technological support (functional phone with ODK forms, power banks, bar code stickers for ITN traceability) and technical support (support for training, review of completed forms before online submission, support for cleaning and analysis of server data)
	SANRU	• Makes financial resources available to IMA for the implementation of activities in the province and the HZs; • Performs fixed price analysis to be used to determine the costs of contracts to be signed between the HZs and IMA; • Deploys ITNs from the country’s entry points to the Health Zone Central Office; • Monitors implementation (effectiveness of activities, compliance with standards) and provides technical support to IMA in the implementation of activities
Coordination	National Coordination Committee	• Coordination of all stakeholders involved in ITN distribution campaigns led by the NMCP
	Provincial Coordination Committee	• Provincial coordination of all stakeholders involved in ITN distribution campaigns led by the Governor
	Local Coordination Committee	• Local coordination of all partners involved in ITN distribution campaigns led by the Territory Administrator/Sector Leader
Financing	GFTAM Country Team	• Financial support for the implementation of activities through the PR (operational costs); • Follow-up on the implementation of the activities
	Against Malaria Foundation (AMF)	• Financial support for the implementation of activities through the PR (ITN procurement); • Follow-up on the implementation of the activities
Governance	General Health Inspection (GHI)/(Provincial Health Inspection (PHI)	• Preventive inspection missions to ensure compliance with the country’s administrative and financial management standards and procedures; • Investigations into allegations of fraud or misappropriation
	Congolese National Police (CNP)	• Securing people and inputs for the campaign; • Missions to recover diverted ITNs for return to the PR; • Investigations into allegations of fraud or misappropriation
	Local Fund Agent (LFA)	• Field audit missions to ensure compliance with procedures and detect management deviations in order to propose corrective measures
	Chairman, Health Area Development Committee (C-HADC)	• Contributing to the development of microplans at the grassroots level (HA) to ensure that the needs of the entire community are addressed; • Countersignature of the list of distribution stakeholders to certify compliance with the selection criteria; • Contribution to the management of inputs from reception to the daily triangulation of stocks at the end of the day
Technical assistance	Alliance for Malaria Prevention (AMP)	• Technical assistance in the development and implementation of campaign management manuals and tools
	Palladium	• Technical assistance in the development and implementation of ABC manuals and tools
	GFTAM Fraud and Risk Assessment (FRA) Team	• Guidance and technical assistance in the development and implementation of campaign management manuals and tools using the ABC approach

Contract deliverables

Table 2 defines the three contract deliverables and the evidence of their achievement to be provided by the HZs prior to payment.

Table 2.

Three contract deliverables and evidence of their achievement

Deliverables	% of total contract price	How the deliverable is defined	Proof of delivery
Deliverable 1: Households covered	75	Per household registered and allocated the correct number of ITNs Payment of a unit price/household = portion of the contract price/number of households with the number of ITNs available according to the validated microplan (or adjusted during the campaign) A. A health zone that achieves more than 80% of households receiving ITNs, i.e. between 80 and 100%, will be paid proportionally to the households served after validation of the coverage survey B. A health zone that achieves less than 80% coverage must do mop up to improve its coverage The ITN allocation key (e.g. number of nets to be provided to a household based on number of people) must be respected in at least 85% of covered households to trigger full payment of the deliverable if verified coverage is ≥ 95%	Data stored on the server: data recorded by the CHWs on the telephones with ODK handed over daily to the independent supervisors in charge of verification and online transmission
Deliverable 2: Staff trained	5	100% of the required number of CHW registrars (equivalent to one third of the training participants, which is quantified according to the validated microplan) will have to pass the post-test (correct registration of five households in one hour maximum in rural areas and six in urban areas) using ODK at the end of the training	Households registered in the phone at the end of the training by the CHW registrar, who will return the ODK phone to the independent supervisor at the end of the training for quality control of the data collected and online submission IMA Central will give the server access to an SRI for cross-checking and transmission of results within one week
Deliverable 3: ITNs inventoried	20	ITN loss = (A) Total ITNs delivered to Health Zones—[(B) # of ITNs distributed + (C) # of remaining ITNs returned to SANRU by the HZ-MT in good condition] 1. If the ITN loss remains below the 1% threshold, then the HZ will access the entire portion of the contract related to that deliverable 2. If the HZ exceeds the 1% loss threshold, then the total number of ITNs lost will be multiplied by: a. The unit price of an ITN at the local market for all destroyed but physically inventoried ITNs, b. The unit price of an ITN at the local market × 2 for all non-inventoried ITNs	A: Voucher for delivery/receipt to HZ B: Number of ITNs distributed to households validated and hosted on the server C: Physical inventory of the balance at the HZ by the SRR of SANRU by means of a report co-signed with the HZ’s CMO and the zonal supervisors of IMA Report of damaged products

Campaign planning and implementation process

The implementation of the mass ITN distribution pilot using the ABC approach was preceded by a design phase that lasted from mid-2021 to mid-February 2022, ending with the adjustment of the Kwilu macroplan based on prices of services independently evaluated by an agency contracted by GFTAM. The design phase included all stakeholders, including the NMCP, GFTAM, PR, SR, AMP and Palladium, and meetings took place through in-person and virtual workshops and conference calls. In general, for both standard and ABC campaigns, it is after the macroplanning workshop that the campaigns enter their implementation phase, which includes the following main steps: training of stakeholders, microplanning, household registration coupled with ITN distribution, followed by data validation. We have previously described the process of planning and implementing campaigns in the DRC by describing the changes that were made following the COVID-19 pandemic [10, 12]. Here, we only include the particularities of the campaigns using the ABC approach.

Macroplanning

It concerns the large-scale planning of MDC of ITN. In the DRC, it is organized each year to (i) define the overall objectives of the campaign in terms of the provinces to be covered; (ii) define the overall strategy of the campaign and ensure that the objectives can be achieved on a national scale; (iii) determine the resources required, mainly on the basis of the population as counted by the CHWs during the previous MCD in each targeted province, adjusted on the basis of the annual population growth rate provided by the National Institute of Statistics; and (iv) set up coordination mechanisms for the various stakeholders involved in the campaign.

At this stage, the PHD stakeholders produce a campaign implementation plan for the province and its appendices (a provisional budget, a communication plan, a logistics plan, an indicative timeline and a risk management plan). This macroplanning takes place in the last quarter of the year preceding the year of implementation. In fact, early budgeting is essential to convince partners to fill any gaps and establish mechanisms for the timely disbursement of funds.

As part of the ABC approach, there were two additional appendices that do not exist in the provinces under the eligibility-based management of expenditures based on compliance with procurement and financial management procedures: (i) a price analysis matrix for the provision of services (room rental, catering, accommodation, transport, storage, handling, security) carried out by the PR and approved by the donor (instead of prices obtained after a procurement process) and (ii) an initial ABC contract between the SR and the HZs based on activities against the three deliverables.

These deliverables were defined as follows:

1. Deliverable 1 (Household Coverage): 75% of the total cost of the CDM determined during macro-planning will be paid to the HZ in pro-rata to households enumerated and served with ITNs after independent verification of the proportion of households registered by CHWs (Household Coverage) and served with the required number of ITNs according to household size as recommended by the NMCP (Coverage Compliance). This deliverable has a minimum requirement, namely that any HZ that does not achieve at least 80% coverage based on data collected by CHWs, is not subject to independent verification to trigger payment and is called upon to continue household sweeps. The deliverable also includes an incentive, i.e., any HZ that achieves at least 95% household coverage and at least 85% coverage compliance will receive the full 75% portion of its total CDM cost related to this deliverable.

2. Deliverable 2 (Quality of CHW Training): a number equivalent to the quantitative need for CHW interviewers must pass the post-test during the training of all CHWs who will be called upon to work in trios, i.e. 1 CHW interviewer, 1 CHW distributor and 1 CHW ITN carrier (we have previously described in detail the roles of each of these three types of CHW) [10, 12]. Only the HZ with at least a 33% success rate among trained CHWs, based on independent verification, will be paid the full (all-or-nothing) 5% of its total CDM cost related to this deliverable.

3. Deliverable 3 (Inventory of ITNs): 20% of the total cost of the CDM will be paid to the HZ based on the proportion of ITNs tracked by an independent verification.

The validation procedures for these deliverables are described below.

Training of stakeholders

The training of teams composed of three CHWs (one in charge of household registration using a smartphone, a second in charge of distributing ITNs to the heads of households and filling out a tally sheet, and a third in charge of transporting the bales of ITNs) was critical to the quality of the household registration combined with ITN distribution. This training was verified under the ABC approach by a test consisting of the registration of five households in rural areas and six in urban areas by each CHW in a maximum of one hour. This test took place on the second day of the training.

Microplanning

The microplan concerns the more local and operational planning of ITN MDC. It is organized province by province, starting with the health areas, then the HZs, before finalizing the microplans at a provincial workshop. The aim is to define the concrete actions to be put in place to achieve the objectives set out in the macroplan; in this case (i) to take into account the discrepancies between the populations estimated in the microplan and those expected in the HAs, based on population movements linked to insecurity, economic or climatic reasons, and distribution logistics; (ii) to readjust ITN and other input requirements on the basis of this more precise population, and the logistics of deploying them to villages or avenues, taking into account any accessibility or storage difficulties; (iii) determine human resource requirements, taking into account any difficulties of geographical or cultural accessibility for certain categories of the population; (iv) identify channels and methods for raising the population’s awareness, taking into account the existence of any groups resistant to the use of ITNs for cultural or religious reasons, as well as those known to be diverting use to other forms (fishing, garden protection, wedding dressmaking, etc.).

After validation of the microplans, possible quantitative adjustments to populations and accessibility could impact implementation costs. In the ABC province, an amendment to the contract was signed between the parties following the microplanning to account for 100% of the readjusted budget of the campaign.

Household registration coupled with ITN distribution

This step is performed in the same way in the province under the ABC approach as in the provinces under the standard approach. The expected difference is a better quality of training to obtain a good Deliverable 2, greater motivation of the stakeholders to find the last household to obtain a good Deliverable 1 and a more robust follow-up and rigorous management of inputs to present a good Deliverable 3.

Validation of data

In the context of the ABC approach, this was the validation of the three contract deliverables. It was conducted in the following manner:

1. Deliverable 1 (Household Coverage): Once the HZ downloaded and cleaned the data from the IMA ODK server, it then proceeded to do a comparative analysis against the tally sheets. Any discrepancies noted were corrected and the final estimated coverage data from the HZ-MT was shared with the PHD for the final provincial validation workshop. The “5% verification” report was delivered to the HZ on the 10th day following the end of the household registration coupled with ITN distribution at the HZ data validation workshop to allow the HZ to be informed of the verified coverage that will serve as the basis for payment. Finally, a validation report on Deliverable 1 was signed by all the stakeholders of the data validation workshop at the HZ level (HZ’s CMO, SR IMA delegate, CNRSC Coordinator, SANRU’s SRR delegate and provincial supervisor representing the H-PHD). This report included the coverage obtained by the HZ, the coverage verified by the CNRSC, and the amounts to be paid based on the verified coverage to complete the equivalent of 75% of the contract amount (Deliverable 1 portion).

2. Deliverable 2 (Quality of CHW Training): As described in Table 2, the number of CHWs who correctly registered the required number of households was to match the need for the number of CHWs responsible for the registration (use of the devices) based on the number of teams expected for each health area to enable the HZ to access the portion of the contract (5%) related to Deliverable 2.

3. Deliverable 3 (Inventory of ITNs): A physical inventory of the balance of ITNs at the HZ was undertaken by SANRU’s SRR axis supervisor and a report co-signed with the HZ’s CMO and the IMA zonal supervisors. A comparison of ITN receipt data (receipt slip, report of receipt), ITN distribution to households (cleaned ODK server), and ITN inventories at the end of distribution (physical inventory, report of damaged products) was used to calculate the amount to be paid as described in Table 1. This amount was to complete the 20% of the contract amount (portion related to Deliverable 3).

Contracting procedures

The ABC contract was signed based on the provisional budget resulting from the provincial macroplanning and validated by the central NMCP. This contract signing took place the day after the HZ-MT training was completed to allow the HZ CMOs to commit to the ABC approach with full knowledge of the terms and conditions. The signatory parties were (i) the HZ represented by the CMO who signed as service provider, responsible for organizing the mass ITN distribution campaign activities at the HZ level and providing the contract deliverables; (ii) IMA, SR of SANRU, the contract manager responsible for performance monitoring and payment of the mass ITN distribution campaign activities at the HZ level based on the contract deliverables; and (iii) the H-PHD who signed the contract as the regulator to ensure its approval as well as his/her commitment to supervise the proper implementation of mass ITN distribution campaign activities organized in the health zone and to provide the contract deliverables.

This ABC contract was subsequently supplemented by an amendment after the microplans were validated to readjust the budgets according to the validated HZ microplans. Figure 4 shows the dates of signing and proportions of disbursements to the HZs.

Fig. 4.

Contracting and payment terms of the ABC contract

Payments due to the HZ were made to the HZ bank account in the following manner: (i) Disbursement 1 of 15% of the total budget within 10 calendar days of signing the contract, allowing preparatory activities for the campaign to begin; (ii) Disbursement 2 after validation of the microplans in an amount equal to 40% of the total budget revised by the amendment, less the amount of Disbursement 1, within 10 calendar days of signing the amendment; (iii) Disbursement 3 corresponding to the balance of the budget, less any unvalidated deliverables, paid to the health zone once the three deliverables have been acquired by the SR IMA after validation of the performance of the HZ and after calculation of the final amounts still due (Fig. 4).

Indicators for comparing the two approaches

The ABC approach has been put in place to help remove the bottlenecks encountered in the implementation of the standard campaigns and to optimize the campaigns with (i) better coverage, (ii) better adherence to deadlines, and (iii) better management (better traceability of inputs, reduction of ineligible expenses, etc.). In this paper, we conducted a comparison of ITN coverage, campaign preparation times, and ITN reconciliation and accountability.

Household coverage

As described above, in the province under the ABC approach, only coverage verified by CNRSC was included in the payment for Deliverable 1. Household coverage was assessed in two ways: [1] ITN coverage corresponding to the proportion of households having received at least one ITN among all the households surveyed and [2] compliance of the coverage which was based on the proportion of households having received the required number of ITNs according to the number of people living in the household as recommended by the NMCP [15].

This verification was done in two steps, an “in process” verification of 2.5% of randomly sampled households during the household registration and ITN distribution and an “end process” verification of 2.5% of randomly sampled households at the end of the campaign.

For all provinces under ABC or under the standard approach, all HZ were covered by the verification. In each HZ, all HAs were also involved in the verification. In each HA, the full list of villages (rural) or avenues (urban) was obtained from the HA management teams according to their microplans, with their estimated populations and the corresponding number of households based on the average household size of the province from the last Multiple Indicator Cluster Survey (MICS) [28]. The populations used are those from previous registrations adjusted on the basis of the annual population growth rate provided by the National Institute of Statistics and on the fluctuation factors presented by the HZ-MT and cross-checked with the PMT. Similarly, the expected number of households is obtained by dividing the estimated population by the average household size provided by the latest National Statistics Institute survey for the province [29]. On this basis, 5% of the population was calculated to obtain the total number of households to be surveyed.

We adapted our sampling plan according to the WHO vaccination coverage cluster surveys. This involves a 2-stage cluster survey to estimate coverage at the HZ level, by random sampling of villages in each HZ carried out by the AMF and systematic sampling of households in villages selected by CNRSC interviewers [30]. In this manual, the effective sample sizes proposed in Table B-1 are aligned with the sample size calculation equations on page 35 of Fleiss, Levin, and Paik (2003), Statistical Methods for Rates and Proportions, 3rd edition, John Wiley & Sons, Inc, Hoboken, New Jersey. We chose a size of 354 households per HZ, corresponding to an expected coverage of 95% (minimum coverage required as one of the two criteria for full payment of the portion of the total cost of the CDM related to deliverable 1) and a precision of ± 3°% (maximum precision in the table). In order to take into account the clustering terms of design effects (DEFF), the WHO proposes 5 Intracluster correlation coefficients (ICC) as follow: (i) ICC = 0 for uniform coverage; (ii) ICC = 1/24 for very little variation in coverage; (iii) ICC = 1/6 as conservative choice for supplementary immunization activity surveys (close to the MDC of the ITN); (iv) ICC = 1/3 as conservative choice for routine immunization surveys; and (v) ICC = 1 in situations with some clusters 100% covered, all others 0%. We assumed a ICC = 1/24, corresponding to a very low little variation in coverage, given that MDC of ITN are carried out door-to-door with a minimal risk of missed households. By applying the formula “DEFF = 1 + (m—1) * CCI” where m = number of respondents per cluster, we obtained an DEFF of 2.1 for 25 respondents by cluster. This leads to a minimum number of households per HZ of 354 × 2 = 708 households per HZ to cover 2.5% of the population at a rate of 25 households per village. The number of 25 households per village per interviewer per day corresponds to a reasonable workload, particularly in rural areas.

Based on this convention of 25 households to be surveyed per village/street, the number of households to be surveyed in each HA was divided by 25 to obtain the number of villages to be selected. Subsequently, a list of villages/streets corresponding to 2.5% of the households to be surveyed in “in process” and the list of villages/streets corresponding to 2.5% of the households to be surveyed in “end process” was generated. This number of villages/streets was selected by simple random sampling. The entire procedure was carried out by AMF in London, which provided CNRSC with the list of villages/streets for each phase one week before the start of the process. CNRSC in turn revealed them to the interviewers 48 h before the start of the survey to avoid any interaction between the interviewers and the inhabitants of the selected villages.

The coverage obtained from the end process surveys was used for the comparisons presented in this article.

Household visits to be verified by the 2.5% “in process” verifiers began on the third day of the household registration coupled with ITN distribution and ended two days after the completion of the distribution. Each verifier visited 25 households per day per village/street in a HA using the data collection sheet and simple random sampling based on the survey of all households in the village. In the event that the selected village had fewer than 25 households, the verifier completed the remaining households in the nearest village that was not initially targeted for verification.

A second 2.5% “end process” check was performed from day 11 (third day after the end of the distribution). It used the same methodology for a shorter period (three days instead of seven) with twice as many verifiers.

To ensure the quality of the independent verification of coverage by CNRSC, SANRU subcontracted with a Scientific Research Institute for a crosscheck using lot quality assurance sampling. For the LQAS, the survey was conducted to cover 50% of the HZs in Kwilu Province (i.e., 12 HZ) for household coverage estimates. The HZ was the operational unit corresponding to the data collection area. In each data collection area (corresponding to an HZ), five supervision areas were selected by simple random sampling. In each supervision area, a sample of 95 households was selected by systematic random sampling. This resulted in a sample size of 1140 households. This sample provides acceptable precision with alpha and beta errors below 10%.

In addition, the same SRI completed quality control of CNRSC’s work in all 24 Kwilu HZs.

Campaign preparation time

For the purposes of this study, the time analyzed is the time between the first day of training of the PHD-MT and the first day of ITN distribution in the province. Indeed, the province is considered a target geographic unit for a given time specifically in relation to the other provinces programmed in the year and, in terms of the organization of health services, the province is responsible for organizing the delivery of health services in contrast to the national level which remains an essentially normative body. Thus, the start of the PHD-MT capacity building workshop was chosen as the trigger for the province’s preparation of the campaign. The NMCP standard is to start training PHD-MT four months before ITN distribution begins.

ITN loss rate

Evidence of this deliverable was provided to SANRU and to the PHD by the SANRU’s SRR via an inventory report co-signed with the CMO and IMA zonal supervisors as described in Table 1.

Results

Contract deliverables from the ABC pilot province of Kwilu

It was noted that all HZs exceeded 98% coverage (vs. 80% minimum requirement and 95% to trigger full payment) and 90% coverage in accordance with ITN allocation key based on household size (vs. 85% required to trigger full payment) (Table 3).

Table 3.

Household ITN coverage in the HZs of the Kwilu ABC pilot province after end process verification

No	Health zone	ITN Coverage	Compliance of the coverage	No	Health zone	ITN Coverage	Compliance of the coverage
1	Bandundu	99.7	98.3	13	Mungindu	100	98.9
2	Kikwit-North	99.6	98.4	14	Kingandu	100	99.5
3	Kikwit-South	99.71	97.5	15	Bulungu	100	100
4	Mosango	99.71	99.3	16	Kimputu	100	100
5	Lusanga	100	99.6	17	Moanza	100	99.3
6	Masi-manimba	99.9	98.9	18	Djuma	100	100
7	Yasa-Bonga	100	100	19	Gungu	100	100
8	Kikongo	100	99.9	20	Ipamu	99.8	99.7
9	Bagata	99.7	98.4	21	Koshibanda	99.8	100
10	Mokala	100	99.3	22	Mukedi	99.7	90.8
11	Idiofa	100	99.7	23	Sia	99.8	98.3
12	Pay Kongila	99.8	96.9	24	Vanga	99.8	99.9

Four HZs (Bagata, Gungu, Masimanimba and Moaza) did not reach the required number of CHWs passing the post-test (Table 4).

Table 4.

Proportion of CHWs passing the training post-test in the ABC pilot province of Kwilu

No	Health zone	Validated CHWs	CHWs trained* n(%)	CHWs who passed the test n(%)	CHWs with comprehensive filling n(%)	Number of CHWs required	Sufficient number of successes
1	Bagata	477	477(100)	123(26)	123(26)	159	No
2	Bandundu	717	717(100)	405(56)	405(56)	239	Yes
3	Bulungu	750	750(100)	285(38)	285(36)	250	Yes
4	Djuma	564	564(100)	318(56)	318(55)	188	Yes
5	Gungu	630	630(100)	207(33)	207(30)	210	No
6	Idiofa	750	750(100)	603(80)	603(68)	250	Yes
7	Ipamu	549	549(100)	267(49)	267(43)	183	Yes
8	Kikongo	489	342(70)	258(75)	258(72)	163	Yes
9	Kikwit North	615	615(100)	417(68)	417(68)	205	Yes
10	Kikwit South	678	678(100)	276(41)	276(41)	226	Yes
11	Kimputu	468	468(100)	273(58)	273(32)	156	Yes
12	Kingandu	279	279(100)	180(65)	180(60)	93	Yes
13	Koshibanda	540	540(100)	273(51)	273(51)	180	Yes
14	Lusanga	732	606(83)	444(73)	444(73)	244	Yes
15	Masimanimba	612	612(100)	195(32)	195(32)	204	No
16	Moaza	441	246(56)	51(21)	51(21)	147	No
17	Mokala	567	567(100)	216(38)	216(34)	189	Yes
18	Mosango	312	312(100)	210(67)	210(67)	104	Yes
19	Mukedi	513	513(100)	303(59)	303(50)	171	Yes
20	Mungindu	333	333(100)	234(70)	234(56)	111	Yes
21	Pay Kongila	387	387(100)	333(86)	333(86)	129	Yes
22	Sia	288	288(100)	219(76)	219(69)	96	Yes
23	Vanga	735	735(100)	393(53)	393(53)	245	Yes
24	Yasabonga	579	579(100)	396(68)	396(59)	193	Yes

^*Having participated in the complete training until the test is completed

Three HZs (Bulungu, Kimputu and Sia) exceeded the 1% loss threshold and will therefore be penalized by subtracting the number of ITNs lost x the local market price (Table 5).

Table 5.

Inventory of ITNs at the end of distribution to households in the ABC pilot province of Kwilu

No	Health zone	HZ’s stock received from SANRU	ODK consumption data	Overall theoretical stock	Physical inventory	Difference Stock Vs Inventory	Loss %
1	Bagata	126,304	122,587	3,717	2,100	1,617	1.30
2	Bandundu	233,627	223,911	9,716	9,258	458	0.20
3	Bulungu	207,336	193,523	13,813	13,427	386	0.20
4	Djuma	153,693	148,995	4,698	4,727	-29	0.00
5	Gungu	172,846	160,518	12,328	11,322	1,006	0.60
6	Idiofa	210,949	203,035	7,914	7,627	287	0.10
7	Ipamu	151,651	148,379	3,272	3,151	121	0.10
8	Kikongo	133,654	123,988	9,666	9,666	0	0.00
9	Kikwit North	210,258	207,159	3,099	1,476	1,623	0.77
10	Kikwit South	234,623	233,761	862	541	321	0.14
11	Kimputu	132,624	124,294	8,330	7,318	1,012	0.80
12	Kingandu	77,249	71,331	5,918	4,814	1,104	1.40
13	Koshibanda	146,086	142,655	3,431	3,431	0	0.00
14	Lusanga	201,884	194,191	7,693	6,964	729	0.36
15	Masimanimba	168,483	164,750	3,733	2,919	814	0.48
16	Moaza	121,542	118,943	2,599	2,547	52	0.00
17	Mokala	155,232	151,124	4,108	3,603	505	0.30
18	Mosango	86,464	83,370	3,094	2,696	398	0.46
19	Mukedi	142,802	140,929	1,873	852	1,021	0.70
20	Mungindu	91,317	87,479	3,838	3,508	330	0.40
21	Pay Kongila	106,212	100,235	5,977	5,874	103	0.10
22	Sia	77,550	70,125	7,425	5,801	1,624	2.10
23	Vanga	203,024	196,512	6,512	6,173	339	0.20
24	Yasabonga	159,250	152,075	7,175	7,172	3	0.00

Comparison of performance between the ABC pilot province of Kwilu and the provinces under the standard campaign approach

Table 6 shows that, notwithstanding the programmatic data presented by the PHDs, the independent verification by civil society (CNRSC) shows the best performance in terms of coverage in the PHD of Kwilu (99.9% in the province under the ABC approach vs. an average of 97.3% in the provinces under the standard approach). This is particularly evident when considering the quality of coverage in terms of the conformity of the number of ITNs received according to household size (98.9% in the province under the ABC approach vs. an average of 84.7% in the provinces under the standard approach).

Table 6.

Comparative ITN household coverage

Provinces	Approach	HZ Data			Verification data 5%
		Households micro-plan	Households served	Household coverage	Household coverage	Compliance of the coverage
BAS-UELE	Standard	293,086	303,050	103.4	96.0	81.0
HAUT-UELE	Standard	451,991	447,234	98.8	100	77.0
ITURI	Standard	1,354,747	1,177,216	86.6	97.4	95.1
KINSHASA	Standard	2,035,090	1,921,054	93.8	93.0	78.0
MANIEMA	Standard	621,497	588,177	98.0	96.4	88.5
SANKURU	Standard	517,858	475,344	91.8	98.5	91.4
TSHOPO	Standard	801,702	753,439	93.9	100	81.6
Subtotal	Standard	6 075 971	5 665 514	93.2	97,3	84,7
KWILU	ABC	1,254,257	1,157,980	92.9	99.9	98.9

With the exception of Bas-Uélé, Kwilu (14 weeks) had the best performance in terms of time management compared to all provinces under the standard approach (28.6 weeks on average) (Table 7).

Table 7.

Comparison of campaign preparation times between the provinces under the standard approach and Kwilu province

Provinces	Approach	Training		Launches		Differences
		Dates	Weeks	Dates	Weeks	Days	Weeks
BAS-UELE	Standard	02/02/2022	5	12/03/2022	11	38	6
HAUT-UELE	Standard	26/11/2020	48	14/04/2021	15	139	19
ITURI	Standard	01/10/2020	40	30/06/2021	26	272	38
KINSHASA	Standard	05/10/2020	41	18/03/2021	11	164	22
MANIEMA	Standard	05/08/2021	31	17/06/2022	24	316	45
SANKURU	Standard	12/10/2021	41	30/05/2022	22	230	33
TSHOPO	Standard	18/09/2020	38	11/06/2021	23	266	37
Subtotal	Standard		34,9		18,9	203,6	28,6
KWILU	ABC	15/04/2022	15	23/07/2022	29	99	14

Table 8 shows that the loss rates appear to be comparable (0.3% in the province under the ABC approach vs. 0.5% on average in the provinces under the standard approach).

Table 8.

Comparative ITN balances after campaigns

Provinces	Approach	ITNs received	ITNs distributed	% ITNs distributed	Balance	Lost ITNs	% ITNs lost
BAS-UELE	Standard	836,718	821,063	98.1	10,543	5,112	0.61
HAUT-UELE	Standard	1,131,667	1,127,792	99.7	2,981	894	0.08
ITURI	Standard	3,632,980	3,558,233	97.9	70,802	3,945	0.10
KINSHASA	Standard	6,861,760	6,813,354	99.3	36,818	11,588	0.17
MANIEMA	Standard	1,928,662	1,807,121	93.7	100,003	21,508	1.12
SANKURU	Standard	1,570,832	1,525,533	97.1	44,131	1,168	0.07
TSHOPO	Standard	2,313,149	2,245,687	97.1	41,504	25,958	1.12
Subtotal	Standard	18 275 768	17 898 783	97,6	306 782	70 173	0,5
KWILU	ABC	3,698,612	3,563,613	96.4	124,660	10,339	0.3

Discussion

In national health financing policies, reforms referred to as PBF or P4P or RBF aim to strengthen incentives to provide more or better-quality priority services [1, 2]. Performance-based incentives are not offered through a single mechanism, but as an additional payment method on top of the existing basic payment methods [1].

In the case studied, the pilot ABC approach in the DRC was designed for payment of mass ITN distribution campaign activities that take place at the HZ level to optimize ITN distribution campaigns for better household coverage (programmatic outcomes), both quantitatively and qualitatively. Table 1 shows that all HZs in Kwilu ABC province exceeded 98% independently verified coverage. During the same period, the other seven provinces that had conducted campaigns using the standard approach had coverage averaging around 97.3% compared to the average of 99.9% for the ABC pilot province of Kwilu (Table 6). Considering that for all the mass campaigns during the period under study, each 1% of household coverage corresponds to 73,302 households, or 373,842 people on the basis of an average of 5.1 inhabitants [29], the difference between ABC Campaign and standard campaign seems operationally significant.

More than quantitative coverage, the proportion of coverage compliance in terms of the number of ITNs to be allocated depending on the number of people living in the households showed a much larger gap, with 98.9% compliance in Kwilu province under the ABC approach vs. an average of 84.7% in the provinces under the standard approach (Table 8). These improvements in quantitative and qualitative coverage are in line with other PBF initiatives implemented in the DRC [24] and in other African countries [17, 24], despite the great heterogeneity of their implementation modalities [24].

PBF initiatives have often focused more on process and service quality indicators than on outcome indicators [24]. In the ABC approach, in addition to household coverage, there were also process drivers such as the success rate of CHWs in the training (Table 4) and the inventory of ITNs at the end of the distribution (Table 5). In the Kwilu pilot, these indicators have resulted in penalties that are likely to lead to increased attention that can, in turn, lead to incremental improvements over time. In fact, studies have shown that improvements in PBF initiatives, like most changes, occur gradually, presumably to allow time for providers to internalize the consequences of under-performance [24, 31]. In addition, research in the area of drinking and driving prevention has shown that sanctions, the perceived likelihood of being sanctioned, and drivers’ perception of the severity of punitive measures are associated with behavioral change related to driving under the influence [32].

With loss rates ranging from 0 to 1.4% (Table 5), it was noted that the comparison of the average ITN stock balances in the ABC pilot province of Kwilu with those of the other provinces, which used the standard approach, did not show any significant difference (Table 7). However, as with the quality of training indicator, it is plausible that the penalties incurred by Kwilu HZs for under-performance on ITN inventories could lead to improvements in the next Kwilu campaign or elsewhere where this approach is exported. Indeed, several theories of behavioral change, such as the Health Belief Model, consider the perception of risk or consequences associated with a behavior or disease as a trigger for action or at least the likelihood of a desired action or behavior [33]. However, studies have shown that reward incentives are more effective than punitive incentives [34]. Otherwise, for all the mass campaigns during the period under study, 0.1% of ITN loss corresponds to 21,974 ITNs, given that the purchase cost per PBO ITN was $2.68 on the Wambo.org platform, the PSM cost of an ITN delivered to the health zone was $2.40 and the distribution cost was $2.11 per ITN, i.e. an average operating cost of $7.19 per ITN distributed. Any loss of 0.1% corresponds to a loss of $157,993.06 for the country over the period under study. From an operational point of view, the difference in losses between standard and ABC campaigns is therefore not negligible.

A major handicap of standard campaigns was the long lead time in the preparatory phase. The ABC approach has shown that these delays can be considerably reduced (Table 8). These delays were attributed, among other things, to lengthy contracting processes that caused three to five months of delay. Bas-Uélé, the only province under the standard approach to have had short deadlines, is precisely the province under the standard approach where the HZs deliberately skipped these procedures. As a result, the ABC approach could be an important source of efficiencies, as shortening campaign timelines can save many categories of resources for most parties involved in campaign implementation.

Despite the encouraging results obtained in the pilot of the ABC approach in Kwilu, it should be noted that some authors believe that the evidence of the effectiveness and efficiency of PBF is still controversial [18]. PBF has been criticized for (i) often being implemented without taking into account the holistic context of health sector financing as a quick, fragmented and unsustainable solution [25, 26]; (ii) a failure to take equity into account [18]; and (iii) a lack of accountability on the part of the MPH [35].

This last point is crucial, as a study evaluating RBF programs in three African countries (Uganda, Zimbabwe and the DRC) found that accountability mechanisms were generally very weak or even non-existent in the DRC, which hindered the success and expansion of these programs [35]. In Kwilu, the PHD-MT was characterized by very dynamic leadership with very active participation, from the design phase of the approach through the evaluation phase. This is a province that is used to PBF initiatives, particularly through The Programme de Développement de Services de Santé (PDSS) mainly funded by the World Bank with contributions also from GFTAM, United Nations Children’s Fund (UNICEF), United Nations Fund for Population Activities (UNFPA), United States’ Agency for International Development (USAID) and Global Alliance on Vaccines Initiative (GAVI), with a relative stability of managers at both the provincial and operational levels [35–37]. The leadership aspects of the MPH should be considered in future ABC experiments in the DRC, as organizational change management is an important responsibility of the manager concerned with improving the performance of his or her organization [38, 39].

On the other hand, the General Health Inspectorate was not as involved at the beginning of the process and would have contributed little in this pilot phase. Yet recent research emphasizes the role of contextual factors related to the health system in general and governance in particular in the design and implementation of PBF initiatives [40].

The understanding of the new approach by field agents who had not signed an ABC contract such as TNs and CHWs would have been limited. This could be due to a loss of quality of the contents during the training sessions by people barely trained in this approach according to the NMCP standards, known as “cascade training”, which consists in successively training the personnel of a lower level by the personnel of the higher level. More assistance could be considered to support training at different levels by people who have mastered the approach for provinces in their first experience implementing the ABC approach. It may also be useful to consider simplifying the manuals for the HA level that is not directly contracted but is critical to field activities.

Conclusion

Providing incentives for increased campaign performance is an option that can improve ITN coverage in endemic countries at a time when the WHO is encouraging countries to innovate to reverse the delays in malaria elimination efforts that have been exacerbated in the last two years by COVID-19 [41]. The encouraging results of the Kwilu pilot should make it possible to replicate this model in other COE countries if scaling up this approach produces similar effects in other provinces of the DRC. The challenges encountered and the lessons learned in the implementation of the pilot ABC approach in Kwilu Province could guide future distribution campaigns in the DRC and other African countries that would like to engage in performance-based incentive contracts.

Abbreviations

ABC

Activity-based contracting

AMF

Against Malaria Foundation

AMP

Alliance for Malaria Prevention

ASP

Additional Safeguards Policy

CHW

Community Health Worker

CMO

Chief Medical Officer

C-HDAC

Chairman of the Health Area Development Committee

CNP

Congolese National Police

CNRSC

Coordination Nationale du Renforcement du Système Communautaire

DRC

Democratic Republic of Congo

FDSS

Fonds de Développement des Services de Santé

GFTAM

The Global Fund to Fight Tuberculosis, AIDS and Malaria

GHI

General Health Inspectorate

HA

Health Area

H-PHD

Heads of the Provincial Health Divisions

HZ

Health Zone

HZ-MT

HZ management teams

HZCO

Health zone central office

IMA

Interchurch Medical Assistance

IS

Independent supervisor

ITN

Insecticide-Treated mosquito Nets

LFA

Local Fund Agent

MDC

Mass Distribution Campaign

MICS

Multiple Indicator Cluster Survey

MPH

Minister of Public Health

NCCSS

National Coordination of Community Systems Strengthening

NMCP

National Malaria Control Programme

NS

Nurse Supervisor

P4P

Pay For Performance

PBC

Performance-Based Contracting

PBF

Performance-Based Financing

PBO

Piperonyl butoxide

PHD

Provincial Health Division

PHD-MT

Provincial Health Division Management Team

POD

Proof Of Delivery

RBF

Results-based financing

RD

Reception Document

SANRU

Santé pour tous en milieu rural (Primary health care in rural areas)

SG

Secretary General for Health

SR

Sub-Recipient

SRI

Scientific Research Institution

SRR

Sub-Recipient for Routine Activities

TFPs

Technical and Financial Partners

TN

Titular Nurse

UNIKIS

University of Kisangani

WHO

The World Health Organization

Authors’ contributions

LLJ designed the study, analysed the data and wrote the main manuscript text. LLJ, WA, EM, KNA, PI, NIC, MSE, TME, MAF, MLE, ML, MP and LKM contributed to the design of the study. LLJ, TME, MAF, MLE, LKM analysed data. WA, EM critically revised the manuscript for intellectual content. All authors reviewed the manuscript. All authors read and approved the final manuscript.

Funding

The ITNs MDC in the DRC referred to in this manuscript was funded by GFTAM and AMF to support the DRC government in the fight against malaria.

Data availability

The generated dataset is not publicly available to preserve the confidentiality of information relating to identifiable individuals. However, it is stored and available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

Data used in the study were obtained from various activity reports mentioned above, which presented aggregated data that did not include private human data. Authorization by an ethics committee was therefore not required.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.