Measuring effective coverage of curative child health services in rural Burkina Faso: a cross-sectional study

Abstract

Objective

To estimate both crude and effective curative health services coverage provided by rural health facilities to under 5-year-old (U5YO) children in Burkina Faso.

Methods

We surveyed 1298 child health providers and 1681 clinical cases across 494 primary-level health facilities, as well as 12 497 U5YO children across 7347households in the facilities’ catchment areas. Facilities were scored based on a set of indicators along three quality-of-care dimensions: management of common childhood diseases, management of severe childhood diseases and general service readiness. Linking service quality to service utilisation, we estimated both crude and effective coverage of U5YO children by these selected curative services.

Results

Measured performance quality among facilities was generally low with only 12.7% of facilities surveyed reaching our definition of high and 57.1% our definition of intermediate quality of care. The crude coverage was 69.5% while the effective coverages indicated that 5.3% and 44.6% of children reporting an illness episode received services of only high or high and intermediate quality, respectively.

Conclusion

Our study showed that the quality of U5YO child health services provided by primary-level health facilities in Burkina Faso was low, resulting in relatively ineffective population coverage. Poor adherence to clinical treatment guidelines combined with the lack of equipment and qualified clinical staff that performed U5YO consultations seemed to be contributors to the gap between crude and effective coverage.

Strengths and limitation of this study.

• Using multiple data sources (direct observation, vignettes, facility inventories) this study comprehensively assessed under 5-year-old child service performance of first-line health facilities.

• We conducted this study in around 500 primary-level health facilities and within 7000 households across six regions in Burkina Faso.

• While our performance score accounted for both inputs and process elements related to technical quality of care, we were not able to include elements of outcome quality.

• Our study does not provide any information on service coverage provided by higher levels of care such as district or regional hospitals.

Introduction

In spite of a recent decline in child mortality worldwide, sub-Saharan Africa (SSA) continues to be the region with the highest child mortality rates globally.¹ Most of these deaths occur among under 5-year-old (U5YO) children and are due to common infectious diseases (malaria, diarrhoea, pneumonia)—all of which are preventable and/or treatable by commonly available and cost-effective interventions.^{1 2}

Availability of and accessibility to effective child health services (CHS) are essential in reducing child mortality.^3–6 Child health interventions therefore need to ensure a combined focus on both access (ie, removal of financial, geographical or cultural barriers) to essential health services and high-standard quality of care provided by these services. While isolated removal of existing barriers to care may improve crude service coverage (ie, number of service users able to access available services),⁷ this may not result in an effective improvement of health outcomes especially if available service quality remains substandard.^{8 9} By assessing the maximum possible health gain an individual can receive from a given health service, the concept of ‘effective coverage’ (EC) therefore adjusts the commonly used crude coverage estimates by the quality of the actual services received by a service user.^{10 11}

EC has been increasingly used in the evaluation of maternal and child health programmes.^12–15 For instance, Nesbitt et al compared crude coverage and EC of pregnant women with facility-based obstetric services in Ghana and estimated that although 68% of the women studied had service access only 18% received high-quality care provided by a skilled birth attendant.¹⁶ Similarly, by comparing EC of young children receiving malaria-related care from formal and informal health providers across SSA countries, Galactionova et al found an enormous variance in estimates ranging from 8% to 72% depending on country.⁸

While in Burkina Faso U5YO service coverage has been previously assessed along crude coverage,^{17 18} this is the first study to our knowledge that tries to estimate both crude coverage and EC. We estimated both crude coverage and EC of U5YO children with CHS in Burkina Faso. Our focus hereby is on curative care (as opposed to preventive care such as vaccinations or nutrition supplementation) provided by primary-level health facilities.

Methods

Study setting

Burkina Faso is a low-income country¹⁹ located in West Africa. This landlocked country covers an area of 274 200 square kilometres with a population of about 18.4 million, of which about 18% are U5YO children.²⁰ In 2015, the neonatal mortality rate and the U5YO mortality rates were 26.2 and 88.5 per 1000 live births, respectively.²¹ Malaria, diarrhoea and acute respiratory infections are the leading causes of deaths in U5YO children.²² In Burkina Faso, the health system follows a three-level pyramidal structure (central, intermediate and peripheral).²³ At the peripheral or lower level, the Centres de santé et de promotion sociale (CSPS) function as the entry point to the health system. CSPS represent health centres that provide minimum preventive and curative services to the community. Each CSPS serves a catchment area of several villages or sectors and employs a minimum staff consisting of at least one nurse, one midwife and one outreach health worker (Agent Itinérant de Santé (AIS)). According to national quality assurance policies, both the nurse and midwife professionals have to be qualified to provide U5YO services.^{24 25} Curative care utilisation by U5YO children in 2010 was poor with only 50% of those children suffering from common infectious diseases (eg, malaria, diarrhoea, pneumonia) having sought care at a health facility.²⁶ As user fees were the main barrier to curative care utilisation, the government started a subsidisation programme offering free services for all U5YO children in 2016.^{27 28}

Study design and study participants

We used cross-sectional facility and household data from the baseline survey of a government-led evaluation of a nationwide performance-based financing programme conducted between October 2013 and February 2014.²⁹ Regions and districts included into the evaluation study have been purposely selected on the basis of low performance in identified maternal and child health indicators: (1) contraceptive prevalence rate; (2) assisted deliveries; (3) antenatal consultations; (4) postnatal consultations and (5) childhood vaccination coverage.

Facility sample

A total of 513 CSPS located in 24 districts across 6 out of 13 regions of the country were included, representing approximately 70% of all CSPS in these districts. We excluded 19 CSPS as they represented either recently opened facilities (less than 6 months in service) or did not provide general primary care services (eg, at high schools, colleges, garrisons or prisons), resulting in a final sample of 494 CSPS. About 91% of selected facilities were considered rural CSPS.

Individual provider sample

Across selected facilities, a total of 1298 individual providers was included. This sample represents the staff on duty on the day of the study visit at a given facility and included all CSPS employed staff cadres.

U5YO case sample

Across selected facilities, a total of 1681 cases of U5YO children presenting to the outpatient department on the day of the study visit were included following a convenience sampling approach. Only first-time presentations (ie, no follow-up visits) were included.

U5YO sample

Households were identified using a two-stage sampling technique. First, one village was randomly selected from all villages located within a given catchment area. Second, in each selected village households qualified for inclusion if at least one pregnant woman or a woman who gave birth within the previous 2 years was living in the household on the day of the survey. All eligible households per village were then listed and 15 of them randomly selected to be surveyed. This way we identified 7410 households, of which 60 households across four villages could not be surveyed for logistical reasons, while in three villages only 14 instead of 15 households were surveyed due to the limited number of eligible households. The resulting final sample therefore included only 7347 households.

Data collection

The survey instruments used in this study are based on the Health Results Innovation Trust Fund’s impact evaluation toolkit and adapted to the Burkina Faso context:^{29 30}

1. A facility inventory was conducted at each sampled facility assessing the availability of staff, infrastructure, equipment, drugs, supplies and consumables. Each facility head verbally completed a structured checklist and a research assistant verified availability and functionality of reported items. Inventory content was based on the service availability and readiness assessment framework.³¹

2. For each U5YO case, the patient-provider interaction during consultation was directly observed and recorded by a trained research assistant using a structured checklist.³² Checklist items were based on clinical activities outlined by the Integrated Management of Childhood Illness (IMCI) standards.³³ As the IMCI standards promote a generic approach to the initial health status assessment of a child regardless of the individual chief complaint, health workers’ adherence to this non-case-specific initial approach was observed in order to allow comparison between different cases.

3. A vignette-based knowledge assessment³⁴ including three different case scenarios was conducted with clinical staff to evaluate familiarity with specific IMCI standards as related to the case management of severely ill children (ie, dehydration, fever, respiratory distress). Each scenario represented a typical case relevant to IMCI^{30 33} and was adapted to the Burkina Faso context.³⁵ A trained research assistant recorded steps in clinical management suggested by the health worker on a structured checklist. Additional information related to a health professional’s qualification and IMCI training background, was also obtained.

4. A structured interview was conducted with the caregiver of each child in the U5YO population sample to collect information on any illness episodes and resultant care-seeking behaviour during the 4 weeks preceding the survey date.

Written informed consent was obtained from all study participants (ie, health workers, patients, caregivers).

Measures and analysis

EC is defined as the relationship between service utilisation conditional on true need and the service quality received^{11 12 36} and can be described as:

$${\displaystyle E{C}_{ij}=(Q_{ij}{U}_{ij}|N_{ij}=1)}$$

Where Q_ij is the EC of individual i with health service j; ${\displaystyle U_{ij}}$ is the expected quality-of-service j provided to individual i; ${\displaystyle N_{ij}}$ is the probability of individual i receiving service j; and ${\displaystyle N_{ij}}$ indicates all individuals i in true need of service j.

For this study we defined true need N as all U5YO reporting an illness episode during the past month. We defined utilisation U as U5YOs who actually sought care at the nearest facility. Our definition of utilisation conditional on true need followed the underpinnings by Shengelia et al.¹¹ Given the data available to us, we defined true need based on reported illness,^{37 38} while utilisation is a function of perceived need among those with reported true need.

Based on the Donabedian framework and the indices developed by Gouws et al to assess the quality of child healthcare, we defined quality Q as a facility or service-specific score composed of three quality dimensions:^{39 40}

1. Observed management of common childhood diseases (MCCD) consists of five process indicators related to health status review and four process indicators related to health status examination assessed by the case observation survey. Two of these process indicators (ie, ‘weight check’ and ‘temperature check’) are further linked to the availability of essential input elements assessed by the facility inventory (ie, ‘functional scale’ and ‘functional thermometer’). This dimension reflects the validated indices 1 and 2 (Integrated child assessment based on IMCI guidelines and facility readiness to deliver IMCI) developed by Gouws et al.

2. Theoretical management of severe childhood diseases (MSCD) is based on provider knowledge on appropriate first-line management processes of (1) severe dehydration in a 2-year-old (five process indicators), (2) breathing difficulties in a 1-year-old (three process indicators), and (3) lethargy in a newborn (three process indicators) assessed by the three vignettes. Seven of these process indicators are further linked to the availability of essential input elements assessed by the facility inventory. This dimension reflects indices 3 and 4 (capacity to manage severe illness using vignettes and capacity to manage severe illness given availability of essential drugs) developed by Gouws et al.

3. General service readiness is based on five structural indicators on availability of electricity, water, sanitation, patient transport and waiting rooms assessed by the facility inventory. This dimension reflects structural elements relevant to essential facility infrastructure based on the Donabedian framework.

Composite score generation included the following steps. Each indicator measuring inputs, or structures, was assigned a value of 1 if at least one unit of the observed item was available and functional at a given facility, otherwise 0. To account for the multiple case observations and vignettes conducted per facility, we averaged findings from multiple process measures at the facility level into a single facility-specific process measure, by assigning a value of 1 when a given process was observed in at least half of the observed instances and 0 if not. For those quality measures where process indicators could be linked to input indicators, we assigned a value of 1 only when both indicators were met, otherwise 0. Table 1A–C- provide an overview of the three quality dimensions including the respective process, input and structural indicators together with overall facility performance across all sampled CSPS facilities.

Table 1A.

Indicators and related performance of management of common childhood diseases (MCCD)

	Process indicators (based on direct observation)	Number (%) of all facilities with observed process	Input Indicators	Number (%) of all facilities with observed input	Overall facility performance Number (%) *
1.	Provider asks for at least two general danger signs†	191 (38.6)	N/A	N/A	191 (38.6)
2.	Provider asks for presence of fever	465 (94.1)	N/A	N/A	465 (94.1)
3.	Provider asks for presence of cough	412 (83.4)	N/A	N/A	412 (83.4)
4.	Provider asks for presence of diarrhoea	370 (74.9)	N/A	N/A	370 (74.9)
5.	Provider asks for presence of ear problems	128 (25.9)	N/A	N/A	128 (25.9)
6.	Provider checks child’s weight	366 (74.1)	Functional scale available	448 (90.7)	333 (67.4)
7.	Provider checks child’s temperature	477 (96.5)	Functional thermometer available	480 (97.1)	463 (93.7)
8.	Provider checks for signs of anaemia (conjunctivae, palms)	381 (77.1)	N/A	N/A	381 (77.1)
9.	Provider checks child’s current vaccination status	207 (41.9)	N/A	N/A	207 (41.9)

*In instances where both process and input indicators were applicable, overall facility performance was only counted once both indicators were performed.

†Per IMCI standard: four general danger signs need to be assessed (difficulties breast feeding/taking any food, considerable vomiting, lethargy, convulsions). Due to the generally low performance of this indicator if measured against this standard, we considered this process to be performed when at least two danger signs were reviewed.

IMCI, Integrated Management of Childhood Illness.

To further categorise facilities, we combined the resulting MCCD and MSCD performance scores with the characteristics of health professionals (ie, professional qualification and IMCI training background) providing U5YO consultations and responding to the vignettes. For each of the three quality dimensions, facilities were then grouped into one of three categories of performance quality (high, intermediate and low) based on the criteria shown in table 2.^{14 16} For facilities that met different criteria levels for each dimension, we assigned them to the lower level. For instance, if a facility performed a high performance quality score but did not met required staff characteristics, we assigned it to the intermediate level.

Table 1B.

Indicators and related performance of management of severe childhood diseases (MSCD)

	Process Indicators (based on vignettes)	Number (%) of all facilities with observed process	Input Indicators	Number (%) of all facilities with observed input	Overall facility performance Number (%) *
Vignette-based scenario 1: Viral illness with severe dehydration in a 2-year-old
1.	Provider starts rehydration (either intravenous or enteral fluids)	445 (90.1)	Isotonic fluid† or oral rehydration salt and nasogastric tube in stock	486 (98.3)	439 (88.8)
2.	Provider administers a dose of paracetamol to lower fever	190 (38.4)	Paracetamol suppository in stock	424 (85.8)	165 (33.4)
3.	Provider recognises viral origin of illness and withholds antibiotics until further evaluation	353 (71.4)	N/A	N/A	353 (71.4)
4.	Provider withholds malaria treatment until parasitaemia is confirmed	396 (80.1)	Malaria testing supplies in stock	203 (41.1)	162 (32.7)
5.	Provider admits patient for further observation and reassessment	128 (25.9)	N/A	N/A	128 (25.9)
Vignette-based scenario 2: Breathing difficulties in a 1-year-old with simple pneumonia
1.	Provider initiates antibiotic treatment at facility	380 (76.9)	Antibiotics in stock‡	486 (98.3)	376 (76.1)
2.	Provider administers a dose of paracetamol to lower fever	331 (67.0)	Paracetamol suppository in stock	424 (85.8)	293 (59.3)
3.	Patient is discharged with close follow-up plan (opposed to admission or referral)	70 (14.1)	N/A	N/A	70 (14.1)
Vignette-based scenario 3: Lethargic 1-month-old
1.	Provider administers a dose of injectable antibiotic	230 (46,5)	Ceftriaxone in stock	410 (83.0)	194 (39.2)
2.	Provider checks and controls hypoglycaemia	37 (7.5)	Dextrose solutions or dextrose containing intravenous fluids in stock	464 (93.9)	36 (7.2)
3.	Provider refers patient to higher level of care	388 (78.5)	N/A	N/A	388 (78.5)

*In instances where both process and input indicators were applicable, overall facility performance was only counted once both indicators were performed.

†Ringer’s lactate or normal saline.

‡Amoxicillin or cotrimoxazole or ceftriaxone.

To estimate EC, we defined EC as the proportion of all U5YO children in need who actually sought care at a facility categorised as least high or intermediate performance quality.

Table 1C.

Indicators and related performance of general service readiness

	Process indicators	Structural indicators	Overall facility performance Number (%)
1.	N/A	Functional electricity source available	412 (83.4)
2.	N/A	Functional water source and soap available in the consultation room	281 (56.8)
3.	N/A	Functional toilet facilities available	480 (97.2)
4.	N/A	Functional emergency vehicle available	115 (23.3)
5.	N/A	Patient waiting room available	406 (82.2)

Results

U5YO characteristics

Across the 7347 households surveyed, we identified and included at total of 12 497 U5YO children. Of these children, 614 (4.9%) experienced an illness episode during the 4 weeks prior to the survey date. Among these children, 463 (75.4%) had fever, 63 (10.2%) had diarrhoea, 20 (3.3%) had cough and 68 (11.1%) had other conditions.

Staffing

In our study, the clinical staff observed independently managing U5YO consultations at CSPS facilities included 64.1% nurses, 6.8% midwives and 29.1% AIS. Among health professionals responding to the vignettes, 74.1% were qualified to provide CHS and 32.7% reported to be trained in IMCI. In 66% of the studied CSPS, all observed U5YO consultations were performed by qualified health providers, but only in 42.5% of CSPS consultations were provided by a health professional trained in IMCI.

Quality-of-care functions

Table 1A shows the percentage of facilities meeting each of the listed MCCD indicators. With regard to symptom review (indicators 1–5), frequencies for overall performance were highest for routine fever (94.1%), cough (83.4%) and diarrhoea (74.9%) reviews but not for routine ear problems (25.9%) and danger signs (38.6%). With regard to patient examination (indicators 6–8), routine checks of body temperature and signs of anaemia were observed in 93.7% and 77.1% of CSPS, but body weight and vaccination status review were observed in only 67.4% and 41.9% of CSPS.

Table 1B presents the overall percentage of facilities meeting each of the listed MSCD indicators. In scenario 1, providers would have administered appropriate initial treatment (ie, immediate fluid resuscitation by intravenous or enteral route) and would have withheld immediate antibiotic administration given the viral cause of diarrhoea in the majority of CSPS (86% and 71.4%, respectively). In contrast, providers in only 32.7% of CSPS would have withheld malaria treatments until further proof of parasitaemia and in only 25.9% providers would have initiated indicated further care (ie, admission for further reassessment and monitoring). In scenario 2, providers in 76.1% of CSPS would have administered antibiotics, but only in 14.1% of CSPS, indicated further care (ie, outpatient treatment with close follow-up) would have been implemented. In scenario 3, although in the majority of CSPS (78.5%) providers would have referred the ill infant to a higher-level care facility, in only 39.2% and 7.2% of CSPS life-saving antibiotics and hypoglycaemia as potential cause of lethargy would have been adequately addressed, although the necessary drugs to do so were available in the majority of CSPS. Interestingly, for most combined indicators in table 1A,B high availability of input components (except isotonic fluid, malaria, scales) appeared not to be related to more frequent health worker performance in the respective related process.

Table 1C presents the overall percentage of facilities meeting general service readiness indicators. The majority of facilities met general infrastructural readiness. However, only about half of facilities had water and soap for handwashing directly accessible in the consultation rooms (56.8%), and only 23.3% could directly access a vehicle for emergency patient transport.

Overall quality-of-care categorisation

Applying the criteria outlined in table 2 to assign each CSPS to a performance quality category resulted in the distribution shown in figure 1. For the MCCD dimension, 80.4% of CSPS were categorised as meeting high or intermediate quality, while only 19.6% of CSPS fell into the low-quality category. A similar pattern was found for the general service readiness dimension with 84.6% of CSPS meeting high or intermediate performance quality. In contrast, only 49.4% of CSPS met high or intermediate MSCD quality, with more than half of facilities providing relatively poor management to children with critical health conditions. Taking all three dimensions together, 69.8% of CSPS met high or intermediate quality.

Table 2.

Categories of facility quality based on performance scores

Performance quality	Criteria MCCD*	Criteria MSCD†	Criteria general service readiness‡
High	Performance score§ ≥7 All observed cases attended by a qualified HCW¶ All observed cases attended by an HCW trained in IMCI	Performance score ≥8 All vignettes-based scenarios answered by at least two-thirds** of qualified HCWs¶ All vignettes-based scenarios answered by at least one HCW trained in IMCI	Performance score ≥4
Intermediate	Performance score 5–6 All observed cases attended by a qualified HCW¶	Performance score 6–7 All vignettes-based scenarios answered by at least two-thirds of qualified HCWs	Performance score =3
Low	Performance score <5	Performance score <6	Performance score <3

*Maximum possible score =9.

†Maximum possible score =11.

‡Maximum possible score =5 .

§Performance score: high (≥70% of the maximum possible score); intermediate (50%–69%); low (<50%).

¶Qualified HCW: According to the national policy of quality assurance, nurse and midwife/midwife assistant are qualified to perform U5YO curative consultations.

**We used the cut-off of two-thirds because the minimum requirement staff at the CSPS level is composed of two-thirds of qualified HCWs for U5YO curative consultations (one nurse, one midwife/midwife assistant) and one AIS.

AIS, Agent Itinérant de Santé; CSPS, Centres de santé et de promotion sociale; HCW, healthcare worker; IMCI, Integrated Management of Childhood Illness; MCCD, management of common childhood diseases; MSCD, management of severe childhood diseases; U5YO, under 5-year-old.

Figure 1.

Proportion of health facilities per performance quality category (n=494).

Crude coverage and EC for curative CHS

Out of the 614 children who experienced an illness episode, 427 (69.5 %) actually sought facility-based care (ie, crude coverage). Given that the majority of CSPS fell into the intermediate-quality category, we estimated EC for two scenarios: scenario A only considering facilities in the high-quality category and scenario B considering both high and intermediate performing facilities. For EC scenario A only 33 (5.3%) U5YO children received high-quality services; for scenario B 274 (44.6 %) U5YO children were effectively covered (see figure 2).

Figure 2.

Crude coverage and effective coverage for curative child health services (CHS). (A) Scenario 1, high quality only (B) Scenario 2, high and intermediate quality.

Discussion

Our study revealed two major findings regarding CHS provision in Burkina Faso. First, there are existing gaps between crude coverage and EC. Second, performance quality related to the management of ill children provided by CSPS in our study area is generally substandard and varies greatly between quality dimensions.

Our study found that only about two-thirds of ill U5YO children presented to a CSPS, which in our study is assumed to be equivalent to crude service coverage. At this point, we were unable to explore the reasons of not seeking care for those non-using children in our sample—for example, whether there are persisting access barriers or whether the child’s illness was treated at home or elsewhere outside the formal health system. Additional research will therefore be warranted to better understand the health-seeking behaviour of households caring for ill children not seeking care provided free through the CSPS system.

More disturbingly, we found the gap in EC to be considerably wide, especially when considering only high-quality facilities. These estimates might be biased to some extent, as we assumed every sick child to be taken to the CSPS closest to the household when estimating service use (available data did not allow for a more specific assessment). While this would not have affected our crude coverage estimation, it might have diminished the EC estimates in cases where caretakers actually bypassed the closest CSPS in favour of a more distant facility with better quality.⁴¹ However, our assumption is supported by the literature on primary healthcare utilisation in SSA and we trust that our EC estimates are sufficiently representative of the situation in Burkina Faso.^42–46

EC estimates are heavily influenced and can be easily modulated depending on the indicators selected to measure service quality. Although the process, input and structural indicators included in our quality score are informed by the work of other authors, they still can be considered selective or biased towards technical elements of the care delivery process.^{40 47} Still, we understand that for healthcare provision to be effective, evidence-based clinical protocols (such as IMCI) need to be adhered to and can therefore be considered the gold standard against which quality should be measured.

In doing so, we observed quite a few differences between the measured quality dimensions used in this study. While observed MCCD processes did not meet IMCI standards, it became nevertheless obvious that providers still follow an assessment approach that seems to be focused on or informed by the leading causes and symptoms among the U5YO population. The vignette-based assessment of MSCD processes revealed that providers generally adhere to treatment guidelines regarding the initial management of severely ill infants (except for the newborn case in scenario 3), but deviate from protocol when making definitive care decisions. Similarly weak or inconsistent adherence to treatment guidelines contributing to low service quality in low-income settings has also been noted by other studies.^47–49 While our study revealed that most of the rural facilities had access to basic infrastructures, some structural differences may still remain not picked up by our survey. Comparing input and process indicators, we observed that lack of supplies hardly seemed to influence non-adherence of IMCI guidelines. Comparing eight low-income countries, Leslie et al also found limited correlation between structural aspects and the process of providing evidence-based maternal and child healthcare.⁵⁰

Several studies have reported on the effectiveness of IMCI guidelines^{5 6} and reasons of low adherence. Lack of IMCI-based training and shortage of equipment are commonly identified contributors to low adherence.⁵¹ Some authors also point to the lack of motivation to adhere to guidelines in combination with high workload.^{52 53} Besides inconsistencies in protocol adherence, an additional contribution to the low effectiveness of provided care might have been the fact that a large portion of observed U5YO consultations was actually conducted by health workers without adequate qualifications (ie, AIS or providers without IMCI training) in the absence of any supervision by a more qualified staff member. Officially, AIS are not authorised to independently provide any curative care in Burkina Faso²⁵ and usually do not receive any specific skill trainings, such as IMCI.⁵⁴ In addition, inadequate equipment and supplies might have also contributed to some of the deviations from protocol, for instance, many facilities had no malaria tests or otoscopes available, which might explain the less differentiated use of antimalaria drugs or the limited focus on ear-related symptoms.⁵¹

As with all studies on performance quality, our study faces some limitations regarding the assessment of the quality components included in our EC estimates. To determine the quality of curative CHS, we relied on both direct observations and vignettes.^{32 34} A common bias to direct observation is the so-called Hawthorne effect, which describes higher performance under observation compared with non-observed situations, and may cause overestimation of actual performance.⁵⁵ In contrast, clinical vignettes might underestimate actual clinical competence, as a testing format based on abstract case scenarios might be unfamiliar to many health workers and has limitations in reflecting the realities of actual case management. Still, both instruments are considered standard in the assessment of health worker performance.

Estimating effectiveness, we measured quality based on content of care focusing on both healthcare inputs (infrastructure, supplies, provider knowledge) and processes (aspects of actual or theoretical case management). While providing a comprehensive measure of effectiveness of care, a content of care approach may only approximate an individual’s health gain insofar, as it does not capture aspects such as patient adherence to treatment or individual health outcomes (recovery, complications, etc).¹² Nevertheless, the indicators included in our quality score are considered measures relevant for reducing child mortality and morbidity.^4–6

For the indicator on danger signs used in the MCCD dimension, we accepted positive performance already when at least two danger signs were reviewed. This was done in order to better facilitate score aggregation given the overall poor performance observed in respect to danger sign assessment. It needs to be noted that this approach actually overestimates providers’ overall performance. Similarly, the thresholds applied to categorising facility performance are relatively arbitrary even though we relied on the work of other authors.^{14 16} As the categorisation approach affects heavily whether a facility was grouped as high performing or low performing, we presented the two scenarios of EC to again allow for some room in our estimation.

Further, by defining true need we assumed every reported U5YO illness episode would actually require a medical care visit (including milder forms of illness). This rather conservative estimation might have overestimated the actual true need in our study population and thus likely underestimated both crude coverage and EC.

Another limitation of this study is that although we focus on U5YO and infants, we purposefully exclude early neonatal conditions directly related to birth. In addition, while our study focus was on primary-level healthcare facilities in rural areas, generalisability of our findings might be limited given that study regions and districts were purposely selected. Still, the relatively large facility sample available to us (around a third of primary-level healthcare facilities in the country) nevertheless provides a relatively broad overview on EC in Burkina Faso. With this study adding new evidence on the effectiveness of CHS coverage in low-income settings, the future focus should certainly include the EC of U5YO in more urban areas and the effectiveness of services provided by hospitals. Additional research exploring the determinants of EC (both demand side factors and supply side factors), will be necessary and helpful to decision makers to tailor health interventions more specifically to improve effective service coverage.

Conclusion

Comparing crude and effective service coverage of U5YO children in rural Burkina Faso resulted in two major findings. First, there are existing gaps between crude coverage and EC. Second, the effectiveness of services provided to U5YO children is extremely low, even when considering a less strict definition of service quality. While our quality assessment relied on content of care measured as guideline adherence, we also assessed the availability of essential equipment and supplies required to implement these protocols, as well as main provider-specific characteristics. The pattern observed in our study is that lack of supplies hardly seemed to influence non-adherence of IMCI guidelines. Non-adherence rather seems to be an issue specific to the individual provider or service staffing with quite a number of unqualified health workers actually providing clinical care to U5YO children. To improve effectiveness of U5YO service provision, both policy makers and health workers should review and adjust the implementation of evidence-based clinical protocols (eg, through trainings, performance evaluations, supervision and coaching) to the human and structural resources available at the CSPS level.