Abstract
Background
The Integrated eDiagnosis Approach (IeDA), centred on an electronic Clinical Decision Support System (eCDSS) developed in line with national Integrated Management of Childhood Illness (IMCI) guidelines, was implemented in primary health facilities of two regions of Burkina Faso. An evaluation was performed using a stepped-wedge cluster randomised design with the aim of determining whether the IeDA intervention increased Health Care Workers’ (HCW) adherence to the IMCI guidelines.
Methods
Ten randomly selected facilities per district were visited at each step by two trained nurses: One observed under-five consultations and the second conducted a repeat consultation. The primary outcomes were: overall adherence to clinical assessment tasks; overall correct classification ignoring the severity of the classifications; and overall correct prescription according to HCWs’ classifications. Statistical comparisons between trial arms were performed on cluster/step-level summaries.
Results
On average, 54 and 79% of clinical assessment tasks were observed to be completed by HCWs in the control and intervention districts respectively (cluster-level mean difference = 29.9%; P-value = 0.002). The proportion of children for whom the validation nurses and the HCWs recorded the same classifications (ignoring the severity) was 73 and 79% in the control and intervention districts respectively (cluster-level mean difference = 10.1%; P-value = 0.004). The proportion of children who received correct prescriptions in accordance with HCWs’ classifications were similar across arms, 78% in the control arm and 77% in the intervention arm (cluster-level mean difference = − 1.1%; P-value = 0.788).
Conclusion
The IeDA intervention improved substantially HCWs’ adherence to IMCI’s clinical assessment tasks, leading to some overall increase in correct classifications but to no overall improvement in correct prescriptions. The largest improvements tended to be observed for less common conditions. For more common conditions, HCWs in the control districts performed relatively well, thus limiting the scope to detect an overall impact.
Trial registration
ClinicalTrials.gov NCT02341469; First submitted August 272,014, posted January 19, 2015.
Supplementary Information
The online version contains supplementary material available at 10.1186/s12913-021-06317-3.
Keywords: Integrated Management of Childhood Illness, Electronic clinical decision support system, Health care workers’ adherence, Burkina Faso
Background
Currently, more than 75 low- and middle-income countries (LMIC) are implementing the Integrated Management of Childhood Illness (IMCI) strategy on a large scale. However, poor adherence of health care workers (HCWs) to guidelines has often been reported [1, 2], likely due to health system limitations, such as lack of training, coordination and supervision, or low availability of essential medicines and equipment [3–6]. In Burkina Faso, the IMCI strategy was introduced in 2003, but an evaluation conducted in 2011 reported a low coverage of training and poor performance in terms of adherence to guidelines [7].
Recent advances in Information and Communication Technologies (ICT) and the advent of electronic Clinical Decision Support System (eCDSS) could potentially transform health care services in LMICs, for instance by helping HCWs to correctly follow relatively complex charts. However, several reviews reveal the lack of evidence for a scalable and sustainable impact on health indicators [8–12]. In particular, the experience with using such technology to improve adherence to the IMCI guidelines is limited [13–17].
From 2014, Terre des hommes foundation (Tdh), in partnership with the Burkinabe Ministry of Health (MoH), implemented, in primary health facilities of two regions of Burkina Faso, the Integrated eDIagnosis Approach (IeDA), a complex intervention centred on an eCDSS developed in line with national IMCI guidelines, with the objective of improving HCWs’ adherence to the IMCI guidelines. Between 2014 and 2017, an evaluation was performed using a stepped-wedge cluster randomised design by an independent team from the London School of Hygiene and Tropical Medicine (LSHTM), United Kingdom, and Centre Muraz, Burkina Faso. The aim of the evaluation was to determine whether the IeDA intervention increased adherence to the IMCI guidelines and improved clinical assessment, classification, prescription, referral and counselling during under-five child consultations in primary health facilities.
Methods
Setting
In Burkina Faso, coverage of key effective interventions for preventing child deaths has steadily increased following the adoption of successive public health policies (e.g. free anenatal care, subsidies for child birth and emergency obstetric care, national distribution of insecticide treated nets, Artemisinin-based Combination Therapy (ACT) for treating uncomplicated malaria at facility and community level, expanded program for vaccination). Consequenttly, in 2015, the under-five mortality rate had declined by 56% compared to 1990, from an estimated 202 deaths per 1000 live births in 1990 to 89 deaths per 1000 live births in 2015 [18]. The government is the main health service provider and managed 83% of facilities within the country in 2014 [19]. The country is divided into 13 regions further subdivided into 63 health districts each with one district or regional hospital. In rural areas, primary health facilities, usually run by one or more nurses with the support of health assistants, are the most common point of care and provide a basic package of outpatient services. In 2014, there were 1824 primary health facilities, corresponding to about one facility per 10,000 inhabitants.
The evaluation took place in the Boucle du Mouhoun and Nord regions from September 2014 to November 2017. Of the 11 districts in these two regions, three districts were selected by the implementing agencies to pilot the first versions of the eCDSS in 2010 and were therefore excluded from the evaluation, which was restricted to the eight remaining districts (Fig. 1). In addition to IeDA, a performance-based financing (PBF) intervention was independently implemented in four trial districts (Nouna, Solenzo, Gourcy and Ouahigouya districts). From April 2016, free care for under-five children was also introduced by the MoH in all public facilities [20].
The IeDA intervention
The IeDA intervention comprised five components: 1. An eCDSS provided on tablets to primary health facilities for the management of under-five connsultations: Based on the information recorded by HCWs from the clinical assessment of the child (e.g. body temperature), the eCDSS displays the relevant charts on the screen to guide HCWs through the IMCI national protocol, from the classification (e.g. uncomplicated malaria), through prescription (e.g. first line antimalarial), referral and counselling. During the trial period, several versions were deployed following feedback from users and stakeholders; 2. A six-day training course provided to HCWs on IMCI guidelines and the use of the eCDSS. During the last year of the trial, learning modules with short videos were also available on the eCDSS to support continuous training; 3. A quality assurance coaching system involving team meetings two to four times a year through which health district authorities and HCWs discussed solutions to their local issues (e.g. organisation of care); 4. A supervision system including monthly visits to primary health facilities; 5. A health information system based on data collected through the eCDSS. During the last year of the trial, descriptive dashboards on under-five consultations were developed and shared with the health district authorities and HCWs.
Evaluation design
Since some components of the intervention could only be delivered at the district level, and rolling out the intervention in a phased manner was more practical for the implementing agencies, the evaluation used a stepped-wedge cluster randomised design, with health districts (“clusters”) receiving the intervention at different time points in a randomised order.
Nine steps, one every 4 months, were initially planned, with the first step used as baseline (Fig. 2a). However, funding and logistic issues resulted into delayed roll-out and only four out of eight districts with the intervention implemented. The baseline phase included the first two steps, and during each of the next four steps, from step 3 to step 6, a new district implemented the intervention (Fig. 2b). For the purposes of data collection, ten primary facilities with staff trained in IMCI were randomly selected in each district with stratification on the 2013 annual under-five consultations caseload [21]. Eight rounds of data collection were conducted in total (Fig. 2b).
Full implementation in a district was considered to have been achieved when the eCDSS was provided to all primary facilities and when all HCWs had been trained in its use and IMCI guidelines. In some control districts, data were collected after implementation started but before the full implementation was completed, resulting in some “contamination” of these control districts (Fig. 2b).
Randomisation and masking
Randomisation was restricted to ensure intervention and control clusters were balanced with respect to region and the PBF intervention. Details of the randomisation procedure used to allocate districts to receive the intervention have been published elsewhere [21]. Randomization was performed by JL, independently of Tdh. The nature of the intervention precluded formal masking of fieldworkers.
The allocation of the intervention to each district was gradually communicated by the research team to the implementing agencies and the list of surveyed facilities was not communicated to reduce the likelihood that more intensive support was provided to those facilities.
Sample size
The sample size was determined using the method described by Hussey and Hughes [22], assuming a design effect of 2 due to clustering within facilities and a between cluster coefficient of variation of 0.3. With a harmonic mean of ten children seen at each of the ten selected health facility of the eight districts per step (and therefore 100 children per district and 800 children per step), the trial would provide 90% power to detect an increase in any of the primary outcome from 25 to 33% %. With a harmonic mean of only four children seen per facility at each step, the trial would have 98% power to detect an increase from 25 to 40% [21].
Data collection
Data collection was conducted by two teams, each comprising two trained nurses. At each step, all ten selected primary facilities in each of the eight districts, were visited once for data collection. Data were collected for all consultations of children aged 2 months to 5 years old occurring during the research team’s visit to the facility. Each visit lasted 2 days or less if the required minimum sample size of children observed per facility was achieved. At each step, the newest intervention district was visited last to maximise the chances that HCWs had learnt how to use the new technology. Each visit was notified, by the data collection team, to the facility the day before the visit.
One independent trained nurse observed the consultation and recorded, using a structured and pre-tested observation form programmed into a tablet, the HCW’s clinical practices, illness classifications and prescriptions given to the child. Observations were passive, and the observer never intervened during the consultation. Validation data were collected by the second independent trained nurse, who conducted a repeat consultation with the child, using the eCDSS. These validation data were intended to provide a “gold standard” classification for each child. When there were discrepancies between the HCW and the validation nurse, the final management of the child was agreed by discussion between the two of them.
In addition, at each visit, a shortened version of the WHO Service Availability and Readiness Assessment (SARA) questionnaire [23] was completed to document the availability of essential medicines and equipment required by IMCI guidelines.
The four nurses recruited for data collection had previously been trained in IMCI by the MoH. The two nurses responsible for observation of consultations had at least 5 years of experience working in a health centre. The two validation nurses had at least 10 years of experience working in a health centre and were also IMCI trainers. In addition, all underwent 2 weeks of training, provided by the main investigators, on the study methods and tools prior to the trial, and benefited from two refresher trainings, provided by Tdh, on IMCI and the eCDSS during the trial.
Outcomes
The evaluation focussed on the adherence to IMCI charts designed for new consultations of children aged 2 months to 5 years old to assess, classify and treat danger signs, cough/difficult breathing, diarrhoea, fever and nutritional status.The evaluation did not consider IMCI charts designed for children who return after an intial consultation. We excluded charts related to HIV and ear problems due to their very low prevalence during the trial period (across all steps and according to the validation nurses, only 0.9% of children classified with HIV infection, and 2.7% of children classified with ear problems). We also excluded the charts related to vitamin A supplementation and vaccination as coverage was high in Burkina Faso. Upon the advice of the trial’s scientific advisory committee, for anaemia, only adherence to the clinical assessment task was evaluated due to the difficulty of assessing anaemia reliably when laboratory testing was locally unavailable.
Primary and secondary outcomes are defined in the Additional file 1. Briefly, the primary outcomes included: 1. overall adherence to clinical assessment tasks; 2. overall correct classification ignoring the severity of the classifications (upon the advice of the trial’s scientific advisory committee); and 3. overall correct prescription according to HCWs’ classifications. The secondary outcomes included: 1. adherence to assessment of danger signs; 2. correct identification of at least one danger sign; 3. overall correct classification accounting for the severity of the classifications; 4. overall correct prescription according to validation nurses’ classifications; 5 & 6. overall correct referral or hospitalisation according to HCWs’ assessment and to validation nurses’ assessment; and 7. overall correct treatment counselling.
Other reported outcomes are: sensitivity and specificity of the HCWs’ classifications; over-prescription of antibiotics and antimalarials; overall availability index of essential oral medicines and equipment (Additional file 2).
Analyses
Analyses were performed using Stata version 14. Analysis included all new consultations of children aged 2 months to 5 years old and excluded children who return after an intial consultation for a follow-up consultation. Primary analyses included “contaminated” control districts as control districts based on the intention-to-treat (ITT) principle.
Secondary analyses excluded these districts for the period when they were contaminated.
Descriptiive analyses were performed using individual-level data and point estimates and confidence intervals for all outcomes were computed accounting for the clustering of observations within districts and facilities using the svy family of commands in Stata.
Comparisons between trial arms and statistical tests to investigate evidence of an intervention effect were performed on cluster/step-level summaries as recommended by Hayes and Moulton [24] for trials with fewer than about 15 clusters per arm to account for the clustered nature of the data. A “vertical” stepped wedge analysis was performed with permutation test using the swpermute command in Stata [25]. This approach analyses each step as a parallel arm trial or, in other words, computes, for each step, one cluster summary per district and one effect estimate and then combines these step-level effect estimates into a weighted average (with the weights proportional to the harmonic mean of the number of clusters in each arm and step). This approach, recommended by Thomson et al. [26], preserves the randomisation and accounts for secular trends. “Horizontal” comparisons, i.e. comparison within a cluster over time (which are non-randomised), do not contribute to the analysis. Applied to our design, across the six steps and the eight clusters, 46 cluster/step summaries were computed (two cluster/step-level summaries were excluded from the analysis due to data lost in two districts at step 6 and 7 respectively) giving six effect estimates which were then combined into a weighted average for each of our outcome.
The above approach was used for all primary and secondaty outomes with the exception of correct identification of at least one danger sign and overall correct referral/hospitalisation. Given the very small number of children with danger signs or severe classifications warranting referral/hospitalisation who contributed to these two outcomes, Fisher’s exact test, performed on individual level data and ignoring clustering, was used to test for an intervention effect.
Statistical tests to investigate evidence of a difference between trial arms were only performed on the primary and secondary outcomes to reduce the problem of multiple testing. No formal adjustment was made for multiple testing. Because our ten endpoints are not all independent to each other, applying the Bonferronni correction would be overly conservative (as it assumes that all hypotheses being tested are independent of each other).
Results
After excluding 189 follow-up consultations, data were recorded for 2724 new consultations of children aged 2 months to 5 years old: 686 consultations at baseline, 1343 consultations in control districts and 695 consultations in intervention districts (Fig. 3, Additional file 4).
About half of consultations at baseline and in the control and intervention districts (46%, 317/686, 49%, 658/1343, and 48%, 335/695 respectively) were conducted in districts where the concurrent PBF intervention was implemented.
While the IMCI paper-form was used for 70% (479/686) and 68% (918/1343) of the consultations at baseline and in control clusters respectively, the eCDSS was used in nearly all consultations (97%, 674/694) in intervention clusters. The occasional use of the eCDSS at baseline (1%, 8/686) or in the control districts (9%, 120/1343) reflects instances of early roll-out of the eCDSS prior to training.
Gender and age distributions were similar at baseline and by trial arm (Table 1). Based on validation nurses’ assessment, the most common classification given to children was malaria (between 53 and 69% of children across baseline and trial arms) (Table 2). Other common classifications included: diarrhoea with no dehydration (about 27%) and pneumonia (between 16 and 27%). About 45% of children had one classification only and between 33 and 48% had two or more classifications (Table 3).
Table 1.
Baseline (N = 686) | Control arm (N = 1343) | Intervention arm (N = 695) | ||
---|---|---|---|---|
% | % | % | ||
Gender | Females | 45.3 | 44.4 | 45.9 |
Males | 54.7 | 55.6 | 54.1 | |
Total | 100.0 | 100.0 | 100.0 | |
Age (months) | 2–11 | 31.8 | 29.0 | 31.7 |
12–23 | 28.6 | 30.9 | 27.8 | |
24–35 | 18.4 | 17.9 | 18.9 | |
36–47 | 11.4 | 12.2 | 12.4 | |
48–60 | 9.9 | 10.0 | 9.4 | |
Total | 100.0 | 100.0 | 100.0 |
Table 2.
Baseline (N = 686) | Control arm (N = 1343) | Intervention arm (N = 695) | |
---|---|---|---|
Classification | % | % | % |
Severe pneumonia or very severe disease | 2.2 | 0.7 | 0.6 |
Pneumonia | 27.3 | 16.2 | 24.3 |
Severe dehydration | 0.7 | 0.1 | 0.0 |
Dehydration | 0.7 | 0.2 | 0.4 |
Diarrhoea with no dehydration | 25.7 | 25.5 | 27.2 |
Severe persistent diarrhoea | 0.0 | 0.0 | 0.0 |
Persistent diarrhoea | 0.0 | 0.0 | 0.0 |
Dysentery | 1.8 | 2.0 | 1.7 |
Severe malaria or severe febrile illness | 3.4 | 1.8 | 2.5 |
Malaria | 69.4 | 54.7 | 52.7 |
Severe and complicated measles | 0.0 | 0.0 | 0.0 |
Measles with eyes or mouth complications | 0.0 | 0.0 | 0.0 |
Measles | 0.0 | 0.0 | 0.0 |
Severe anaemia | 0.4 | 0.2 | 0.1 |
Anaemia | 12.5 | 6.6 | 4.5 |
Severe acute malnutrition | 4.6 | 4.9 | 4.0 |
Moderate acute malnutrition | 10.8 | 11.4 | 16.2 |
Table 3.
Number of classifications | Baseline (N = 686) | Control arm (N = 1343) | Intervention arm (N = 695) |
---|---|---|---|
% | % | % | |
0 | 10.5 | 21.3 | 17.3 |
1 | 41.3 | 45.5 | 44.6 |
2 | 32.7 | 24.7 | 28.4 |
3 | 11.8 | 6.9 | 8.5 |
4 | 3.2 | 1.5 | 1.2 |
5 | 0.6 | 0.2 | 0.1 |
Total | 100.0 | 100.0 | 100.0 |
Adherence to clinical assessment
Across the six IMCI charts, the average percentage of tasks completed by the HCWs was 48% at baseline, 54% in the control districts and 79% in the intervention districts with evidence for a difference between trial arms (cluster-level mean difference = 30%; P-value = 0.002) (Table 4). For all IMCI charts, HCWs in the intervention districts completed more of the recommended tasks compared to HCWs in the control districts (Table 5). In particular, more of the recommended tasks were completed for assessing danger signs: 95% versus 34% in the intervention and control districts respectively (cluster-level mean difference = 71%; P-value = 0.002) (Table 4).
Table 4.
Adherence to clinical assessment | Baseline | Control arm | Intervention arm | Cluster-level mean difference between arms | P-value* | |||||||||
N | % | 95%CI | N | % | 95%CI | N | % | 95%CI | ||||||
Overall adherence (13 to 33 tasks) | 686 | 48.0 | 44.3 | 51.6 | 1343 | 54.3 | 50.6 | 58.0 | 695 | 79.3 | 72.7 | 85.9 | 29.9 | 0.002 |
Adherence to danger signs’ assessment (3 tasks) | 686 | 18.4 | 12.0 | 24.9 | 1343 | 34.2 | 25.5 | 42.9 | 695 | 95.2 | 90.0 | 99.9 | 71.2 | 0.002 |
Identification of at least one danger sign (proportion of children correctly identified with at least one danger sign) | Baseline | Control arm | Intervention arm | Individual-level difference between arms | P-value** | |||||||||
Na | % | 95%CI | Na | % | 95%CI | Na | % | 95%CI | ||||||
24 | 66.7 | 47.2 | 81.7 | 25 | 56.0 | 30.8 | 78.4 | 16 | 75.0 | 50.5 | 89.8 | 19.0 | 0.322 | |
Overall correct classification (proportion of children correctly classified with x given classifications) | Baseline | Control arm | Intervention arm | Cluster-level mean difference between arms | P-value* | |||||||||
Nb | % | 95%CI | Nb | % | 95%CI | Nb | % | 95%CI | ||||||
Accounting for the severity of classifications | 609 | 70.6 | 63.7 | 76.7 | 1049 | 69.8 | 66.0 | 73.4 | 572 | 74.7 | 66.9 | 81.1 | 9.1 | 0.038 |
Ignoring the severity of the classifications | 609 | 75.0 | 68.0 | 81.0 | 1049 | 73.1 | 68.8 | 77.0 | 572 | 78.7 | 72.9 | 83.5 | 10.1 | 0.004 |
Overall correct prescription (proportion of children who received at least all the recommended prescriptions) | Baseline | Control arm | Intervention arm | Cluster-level mean difference between arms | P-value* | |||||||||
Nc | % | 95%CI | Nc | % | 95%CI | Nc | % | 95%CI | ||||||
According to the HCWs’ classifications | 614 | 75.7 | 68.0 | 82.1 | 1074 | 77.8 | 72.5 | 82.4 | 567 | 77.1 | 71.6 | 81.8 | −1.1 | 0.788 |
According to the validation nurses’ classifications | 610 | 65.3 | 59.8 | 70.4 | 1049 | 66.1 | 60.7 | 71.0 | 572 | 68.5 | 58.8 | 76.9 | 6.7 | 0.226 |
Overall correct referral/hospitalisation (proportion of children in need of referral/hospitalisation who were actually referred/hospitalised) | Baseline | Control arm | Intervention arm | Individual-level difference between arms | P-value** | |||||||||
Nd | % | 95%CI | Nd | % | 95%CI | Nd | % | 95%CI | ||||||
According to the HCWs’ classifications | 35 | 60.0 | 47.7 | 71.1 | 42 | 52.4 | 23.7 | 79.6 | 41 | 61.0 | 21.5 | 89.9 | 8.6 | 0.509 |
According to the validation nurses’ classifications | 29 | 55.2 | 36.0 | 72.9 | 32 | 53.1 | 36.5 | 69.1 | 22 | 68.2 | 47.8 | 83.4 | 15.1 | 0.398 |
Overall correct treatment counselling (proportion of caretakers who received information on home-based prescription) | Baseline | Control arm | Intervention arm | Cluster-level mean difference between arms | P-value* | |||||||||
Ne | % | 95%CI | Ne | % | 95%CI | Ne | % | 95%CI | ||||||
612 | 77.3 | 67.5 | 84.8 | 1143 | 91.5 | 88.8 | 93.6 | 576 | 87.9 | 77.9 | 93.7 | −4.1 | 0.355 |
* t test on cluster-level summaries & accounting for the stepped wedge design; ** Fisher’s exact test on individual-level data & ignoring clustering
aNumber of children identified, by the validation nurses, with a given danger sign
b Number of children classified, by the validation nurses, with x given classification
cNumber of children classified, by the HCWs or by the validation nurses, with x given classification
dNumber of children identified, by the HCWs or the validation nurses, with at least one danger sign or a classification requiring referral/hospitalisation
e Number of children who were prescribed, by the HCWs, x given treatment (regardless of the classification)
Table 5.
IMCI chart | Task: Questions to address to the mother or examinations to perform | Baseline | Control arm | Intervention arm | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
N | % | 95%CI | N | % | 95%CI | N | % | 95%CI | |||||
Danger signs b | Ask if the child is able to drink/breastfeed | 686 | 15.9 | 9.2 | 26.1 | 1343 | 28.7 | 20.4 | 38.8 | 695 | 94.5 | 75.1 | 99.0 |
Ask if the child vomits everything | 686 | 15.7 | 8.6 | 27.0 | 1342 | 40.8 | 33.0 | 49.0 | 695 | 95.8 | 81.2 | 99.2 | |
Ask about recent convulsions a | 686 | 23.6 | 15.0 | 35.1 | 1342 | 33.1 | 23.3 | 44.6 | 695 | 95.3 | 90.3 | 97.7 | |
Adherence index (3 tasks) | 686 | 18.4 | 12.0 | 24.9 | 1343 | 34.2 | 25.5 | 42.9 | 695 | 95.2 | 90.0 | 99.9 | |
Cough/difficult breathing | Ask about cough a | 686 | 94.8 | 90.4 | 97.2 | 1342 | 94.3 | 87.2 | 97.6 | 695 | 99.4 | 97.4 | 99.9 |
Ask about difficult breathing a | 686 | 2.8 | 1.6 | 4.8 | 1295 | 7.9 | 3.1 | 18.6 | 605 | 11.4 | 4.0 | 28.5 | |
Ask for duration (if cough/difficult breathing) | 188 | 93.1 | 85.4 | 96.9 | 450 | 82.2 | 74.4 | 88.1 | 324 | 96.9 | 87.5 | 99.3 | |
Count number of breaths per minute (if cough/difficult breathing) | 317 | 54.9 | 40.6 | 68.4 | 554 | 44.8 | 34.0 | 56.1 | 340 | 88.5 | 79.2 | 94.0 | |
Look for chest indrawing (if cough/difficult breathing) | 318 | 47.8 | 37.5 | 58.2 | 554 | 41.2 | 28.9 | 54.7 | 340 | 82.4 | 75.0 | 87.9 | |
Listen for stridor or “wheeze” breathing (if cough/difficult breathing) | 318 | 27.0 | 17.9 | 38.7 | 554 | 17.2 | 10.8 | 26.1 | 339 | 51.6 | 18.8 | 83.1 | |
Adherence index (2 to 6 tasks) | 686 | 48.3 | 44.4 | 52.2 | 1342 | 50.0 | 45.8 | 54.3 | 695 | 67.8 | 55.0 | 80.7 | |
Diarrhoea c | Ask about diarrhoea a | 686 | 94.8 | 87.4 | 97.9 | 1343 | 92.3 | 85.1 | 96.2 | 695 | 98.7 | 94.8 | 99.7 |
Ask for duration (if diarrhoea) | 152 | 92.1 | 86.2 | 95.6 | 340 | 87.1 | 80.0 | 91.9 | 178 | 96.6 | 90.5 | 98.9 | |
Ask for blood in the stool (if diarrhoea) a | 151 | 53.0 | 45.3 | 60.5 | 346 | 58.4 | 45.3 | 70.4 | 178 | 89.9 | 81.3 | 94.8 | |
Offer water to the child (if diarrhoea) | 151 | 9.9 | 3.1 | 27.5 | 347 | 6.6 | 3.6 | 11.8 | 178 | 41.6 | 21.7 | 64.6 | |
Pinch the skin of the abdomen (if diarrhoea) | 98 | 46.9 | 35.0 | 59.3 | 285 | 47.0 | 26.1 | 69.0 | 172 | 76.7 | 65.7 | 85.0 | |
Adherence index (1 to 5 tasks) | 686 | 86.2 | 81.3 | 91.2 | 1343 | 82.1 | 74.9 | 89.2 | 695 | 93.8 | 91.6 | 96.1 | |
Fever or history of fever d | Ask about current fever a | 686 | 88.2 | 72.7 | 95.4 | 1343 | 96.9 | 92.1 | 98.8 | 694 | 100.0 | – | – |
Ask about history of fever a | 686 | 23.5 | 14.0 | 36.6 | 1319 | 75.1 | 51.2 | 89.7 | 654 | 72.9 | 42.8 | 90.6 | |
Ask for duration (if fever) | 543 | 91.7 | 87.0 | 94.8 | 1027 | 88.4 | 84.5 | 91.4 | 575 | 96.7 | 95.0 | 97.8 | |
Ask if the child had fever every day (if fever > 7 days) | 8 | 25.0 | 2.8 | 79.5 | 10 | 40.0 | 13.6 | 73.9 | 4 | 75.0 | 17.4 | 97.7 | |
Ask if urine are dark or not abundant (if fever) | 544 | 10.9 | 6.8 | 16.9 | 1081 | 19.8 | 13.9 | 27.5 | 581 | 81.2 | 67.8 | 89.9 | |
Ask about abnormal bleeding (if fever) | 544 | 6.3 | 3.0 | 12.4 | 1081 | 14.7 | 7.5 | 26.8 | 581 | 75.7 | 59.0 | 87.1 | |
Ask about history of measles in the past 3 months (if fever) | 544 | 10.9 | 6.4 | 17.8 | 1081 | 17.7 | 13.2 | 23.2 | 581 | 65.8 | 29.7 | 89.7 | |
Take the temperature (if fever) | 544 | 99.8 | 98.4 | 99.9 | 1082 | 99.1 | 98.5 | 99.4 | 586 | 99.7 | 98.9 | 99.9 | |
Perform a RDT (if fever) | 544 | 74.3 | 53.8 | 87.7 | 1082 | 86.7 | 79.5 | 91.6 | 584 | 93.7 | 88.6 | 96.6 | |
Look for neck stiffness (if fever) | 544 | 16.2 | 7.6 | 31.3 | 1081 | 23.0 | 15.2 | 33.3 | 582 | 47.6 | 37.8 | 57.6 | |
Take the pulse (if fever) | 544 | 2.4 | 0.5 | 9.9 | 1075 | 8.2 | 2.7 | 22.3 | 582 | 1.6 | 0.4 | 5.6 | |
Look for cold hands or feet (if fever) | 544 | 1.3 | 0.4 | 4.5 | 1078 | 1.2 | 0.5 | 2.7 | 578 | 33.7 | 11.0 | 67.7 | |
Look for jaundice or redness in the eyes (if fever) | 544 | 82.5 | 70.8 | 90.2 | 1081 | 83.0 | 73.3 | 89.6 | 584 | 94.0 | 86.9 | 97.4 | |
Look for general rash (if fever) | 544 | 14.9 | 7.4 | 27.6 | 1082 | 14.1 | 9.3 | 20.6 | 583 | 34.3 | 21.0 | 50.6 | |
Adherence index (2 to 14 tasks) | 686 | 40.3 | 38.1 | 42.5 | 1343 | 52.5 | 47.8 | 57.2 | 694 | 71.5 | 65.6 | 77.4 | |
Anaemia | Look for palmar pallor | 686 | 59.9 | 44.2 | 73.8 | 1341 | 51.5 | 42.6 | 60.4 | 692 | 92.2 | 78.4 | 97.5 |
Nutritional status e | Measure height | 686 | 88.3 | 61.8 | 97.3 | 1343 | 90.4 | 79.1 | 95.9 | 695 | 99.6 | 98.9 | 99.8 |
Weigh the child | 686 | 98.5 | 94.3 | 99.6 | 1343 | 98.4 | 93.7 | 99.6 | 695 | 99.1 | 97.5 | 99.7 | |
Measure MUAC | 686 | 84.3 | 70.4 | 92.3 | 1342 | 86.1 | 82.2 | 89.2 | 692 | 88.4 | 83.5 | 92.1 | |
Look for feet oedema | 686 | 23.5 | 14.3 | 36.1 | 1342 | 35.1 | 22.8 | 49.8 | 692 | 86.6 | 55.0 | 97.1 | |
Offer RUTF (if age > 5 months & MUAC < 115 mm & no danger sign or severe classification) | 17 | 0.0 | – | – | 34 | 8.8 | 1.9 | 32.8 | 16 | 43.8 | 20.3 | 70.4 | |
Adherence index (4 to 5 tasks) | 686 | 73.2 | 64.5 | 81.9 | 1343 | 77.1 | 71.6 | 82.5 | 695 | 93.2 | 87.3 | 99.1 |
a Or reported by the caretakers spontaneously when the consultation started or when asked the reason for consulting; b Tasks recommended by the IMCI guidelines but not observable during consultations: Observe if the child convulses, observe if the child is lethargic/unconscious; c Tasks recommended by the IMCI guidelines but not observable during consultations: Observe if the child is lethargic/unconscious (if diarrhoea), observe if the child is restless/irritated (if diarrhoea), look for sunken eyes (if diarrhoea); d Tasks recommended by the IMCI guidelines but not observable during consultations: Look for running nose (if fever); e Tasks recommended by the IMCI guidelines but not observable during consultations: Look for severe and visible weight loss
Identification of danger signs
The proportion of children correctly identified, by the HCWs, with at least one danger sign was 67% (16/24) at baseline and 56% (14/25) in the control districts. It appeared to be somewhat higher (75%, 12/16) in the intervention districts but the small number of children with danger signs preclude firm conclusion (cluster-level mean difference = 19%; P-value = 0.322) (Table 4).
Classification
Ignoring the severity of the classifications, the proportion of children for whom the validation nurses and the HCWs recorded the same classifications was 75% (457/609) at baseline, 73% (767/1049) in the control districts and 79% (450/572) in the intervention districts with evidence for a difference between trial arms (cluster-level mean difference = 10%; P-value = 0.004) (Table 4). Accounting for the severity of the classifications slightly lowered the proportions of correct classifications (cluster-level mean difference = 9%; P-value = 0.038) (Table 4).
By IMCI chart, HCWs in the intervention districts correctly classified children having diarrhoea with no dehydration, dysentery and acute malnutrition (severe or moderate) more often than those in the control districts (Table 6). Although based on a small number of children, HCWs in intervention districts also appeared to correctly classify children with severe malaria or severe febrile illness more often than those in control districts.
Table 6.
Baseline | Control arm | Intervention arm | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Classification | Na | % | 95%CI | Na | % | 95%CI | Na | % | 95%CI | |||
Severe pneumonia or very severe disease | 14 | 14.3 | 2.5 | 51.7 | 10 | 20.0 | 1.3 | 82.6 | 4 | 25.0 | – | – |
Pneumonia | 185 | 75.7 | 66.6 | 82.9 | 218 | 71.6 | 65.6 | 76.9 | 169 | 76.3 | 46.8 | 92.2 |
Pneumonia ignoring severity | 199 | 75.9 | 67.0 | 83.0 | 228 | 73.3 | 65.8 | 79.5 | 173 | 78.6 | 50.5 | 93.0 |
Severe dehydration | 5 | 60.0 | 20.1 | 90.0 | 1 | 0.0 | – | – | 0 | – | – | – |
Dehydration | 5 | 80.0 | 17.1 | 98.7 | 2 | 50.0 | 0.6 | 99.4 | 3 | 66.7 | 1.1 | 99.7 |
Diarrhoea with no dehydration | 175 | 58.3 | 44.5 | 70.9 | 343 | 64.7 | 51.9 | 75.7 | 189 | 74.6 | 62.9 | 83.6 |
Severe persistent diarrhoea | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – |
Persistent diarrhoea | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – |
Diarrhoea ignoring severity | 185 | 61.1 | 48.1 | 72.7 | 346 | 65.9 | 52.6 | 77.1 | 192 | 76.6 | 64.8 | 85.3 |
Dysentery | 12 | 41.7 | 22.9 | 63.2 | 27 | 44.4 | 29.3 | 60.6 | 12 | 83.3 | 17.8 | 99.1 |
Severe malaria or severe febrile illness | 23 | 60.9 | 37.7 | 80.0 | 24 | 62.5 | 36.1 | 83.1 | 17 | 82.4 | 53.2 | 95.0 |
Malaria | 475 | 93.9 | 90.9 | 96.0 | 734 | 92.0 | 87.6 | 94.9 | 366 | 91.0 | 87.7 | 93.4 |
Malaria ignoring severity | 498 | 95.0 | 91.9 | 96.9 | 758 | 93.1 | 90.0 | 95.3 | 383 | 93.0 | 87.5 | 96.1 |
Severe acute malnutrition | 22 | 81.8 | 50.0 | 95.3 | 45 | 57.8 | 37.6 | 75.6 | 23 | 91.3 | 67.6 | 98.1 |
Moderate acute malnutrition | 56 | 46.4 | 29.9 | 63.8 | 120 | 41.7 | 31.6 | 52.5 | 95 | 62.1 | 52.2 | 71.1 |
Malnutrition ignoring severity | 78 | 68.0 | 52.8 | 80.0 | 165 | 55.2 | 45.5 | 64.4 | 118 | 75.4 | 63.1 | 84.6 |
a Number of children classified, by the validation nurses, with a given classification
HCWs in the intervention districts were also less likely to wrongly diagnose pneumonia as being present when it was not: 7% (38/521) versus 19% (209/1113) (Table 7). For other conditions, false positive diagnoses were rare (< 5%) in both arms.
Table 7.
Baseline | Control arm | Intervention arm | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Classification | Na | % | 95%CI | Na | % | 95%CI | Na | % | 95%CI | |||
Severe pneumonia or very severe disease | 640 | 99.8 | 98.6 | 99.9 | 1331 | 99.5 | 98.4 | 99.8 | 690 | 98.8 | 96.3 | 99.6 |
Pneumonia | 469 | 82.5 | 72.0 | 89.6 | 1123 | 81.2 | 72.5 | 87.6 | 525 | 93.1 | 78.4 | 98.1 |
Pneumonia ignoring severity | 455 | 83.7 | 73.4 | 90.6 | 1113 | 81.2 | 72.9 | 87.4 | 521 | 92.7 | 76.4 | 98.0 |
Severe dehydration | 675 | 100.0 | – | – | 1342 | 99.9 | 99.4 | 99.9 | 695 | 100.0 | – | – |
Dehydration | 681 | 99.7 | 98.3 | 99.9 | 1341 | 99.6 | 98.5 | 99.9 | 692 | 99.3 | 98.6 | 99.6 |
Diarrhoea with no dehydration | 505 | 97.0 | 94.9 | 98.3 | 1000 | 94.7 | 89.0 | 97.5 | 506 | 98.8 | 97.4 | 99.5 |
Severe persistent diarrhoea | 680 | 99.9 | 98.1 | 99.9 | 1343 | 100.0 | – | – | 695 | 100.0 | – | – |
Persistent diarrhoea | 680 | 99.9 | 98.1 | 99.9 | 1343 | 100.0 | – | – | 695 | 99.9 | 99.1 | 99.9 |
Diarrhoea ignoring severity | 495 | 97.0 | 94.8 | 98.2 | 997 | 94.5 | 88.8 | 97.4 | 503 | 98.4 | 95.6 | 99.4 |
Dysentery | 667 | 97.5 | 93.0 | 99.0 | 1316 | 97.5 | 95.0 | 98.8 | 683 | 99.6 | 98.9 | 99.8 |
Severe malaria or severe febrile illness | 660 | 98.8 | 97.7 | 99.4 | 1319 | 99.2 | 96.1 | 99.8 | 678 | 98.5 | 96.1 | 99.5 |
Malaria | 208 | 90.9 | 83.4 | 95.2 | 609 | 95.2 | 90.7 | 97.6 | 329 | 95.7 | 91.9 | 97.8 |
Malaria ignoring severity | 185 | 92.4 | 85.6 | 96.2 | 585 | 95.9 | 92.3 | 97.9 | 312 | 95.2 | 91.8 | 97.2 |
Severe acute malnutrition | 563 | 97.9 | 95.4 | 99.0 | 1100 | 98.9 | 98.4 | 99.3 | 589 | 98.5 | 96.7 | 99.3 |
Moderate acute malnutrition | 529 | 93.6 | 89.7 | 96.0 | 1025 | 96.8 | 94.0 | 98.3 | 517 | 96.5 | 94.3 | 97.9 |
Malnutrition ignoring severity | 507 | 92.7 | 89.2 | 95.2 | 980 | 96.9 | 94.0 | 98.5 | 494 | 96.4 | 94.1 | 97.8 |
a Number of children not classified, by the validation nurses, with a given classification
Prescription
Overall, the proportion of children who received all the recommended prescriptions in accordance with the HCWs’ classifications was 76% (465/614) at baseline, 78% (836/1074) in the control districts and 77% (437/567) in the intervention districts with no evidence for a difference between trial arms (cluster-level mean difference = − 1%; P-value = 0.788) (Table 4). According to the validation nurses’ classifications, these proportions were 65% (398/610) at baseline, 66% (693/1049) in the control districts and 69% (392/572) in the intervention districts (cluster-level mean difference = 7%; P-value = 0.226).
By IMCI chart, correct prescriptions for dysentery were much more common in the intervention districts than in the control districts, as were correct prescriptions for acute malnutrition (severe without complications or moderate) and severe malaria or severe febrile illness, although still infrequent (Tables 8 and 9). Correct prescriptions for diarrhoea with no dehydration were also higher in the intervention districts compared to the control districts (Table 9).
Table 8.
Baseline | Control arm | Intervention arm | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Classification | Nb | % | 95%CI | Nb | % | 95%CI | Nb | % | 95%CI | |||
Severe pneumonia or very severe disease | 3 | 33.3 | 2.2 | 91.7 | 9 | 44.4 | 9.7 | 85.7 | 9 | 55.6 | 25.5 | 82.0 |
Pneumonia | 222 | 93.2 | 89.7 | 95.6 | 367 | 94.8 | 86.7 | 98.1 | 165 | 95.2 | 92.8 | 96.8 |
All classifications related to pneumonia | 225 | 92.4 | 88.1 | 95.3 | 376 | 93.6 | 86.2 | 97.2 | 174 | 93.1 | 88.8 | 95.8 |
Severe dehydration with another severe classification | 3 | 33.3 | 2.2 | 91.7 | 1 | 0.0 | – | – | 0 | – | – | – |
Severe dehydration without another severe classification | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – |
Dehydration with other severe classification | 3 | 66.7 | 8.3 | 97.8 | 2 | 50.0 | 0.6 | 99.4 | 2 | 100.0 | – | – |
Dehydration without other severe classification | 3 | 100.0 | – | – | 5 | 80.0 | 7.1 | 99.5 | 5 | 100.0 | – | – |
Diarrhoea with no dehydration | 117 | 70.1 | 52.7 | 83.1 | 275 | 84.0 | 79.4 | 87.7 | 147 | 88.4 | 73.3 | 95.5 |
Severe persistent diarrhoea | 1 | 100.0 | – | – | 0 | – | – | – | 0 | – | – | – |
Persistent diarrhoea | 1 | 0.0 | – | – | 0 | – | – | – | 1 | 0.0 | – | – |
All classifications related to diarrhoea | 128 | 69.5 | 53.3 | 82.0 | 283 | 83.4 | 79.3 | 86.8 | 155 | 88.4 | 72.8 | 95.6 |
Dysentery | 22 | 0.0 | – | – | 45 | 11.1 | 2.3 | 39.4 | 13 | 69.2 | 34.7 | 90.5 |
Severe malaria or severe febrile illness | 22 | 9.1 | 1.6 | 38.6 | 26 | 7.7 | 1.3 | 34.6 | 24 | 33.3 | 7.9 | 74.5 |
Malaria | 465 | 98.9 | 97.4 | 99.6 | 704 | 98.9 | 97.3 | 99.5 | 347 | 98.3 | 96.1 | 99.2 |
All classifications related to malaria | 487 | 94.9 | 91.9 | 96.8 | 730 | 95.6 | 93.1 | 97.2 | 371 | 94.1 | 84.9 | 97.8 |
Severe acute malnutrition with complicationsa | 3 | 0.0 | – | – | 5 | 40.0 | 7.2 | 85.1 | 0 | – | – | – |
Severe acute malnutrition without complications | 25 | 32.0 | 17.3 | 51.5 | 35 | 17.1 | 5.0 | 45.0 | 30 | 40.0 | 30.7 | 50.1 |
Moderate acute malnutrition | 64 | 12.5 | 7.0 | 21.2 | 84 | 1.2 | 0.1 | 11.1 | 82 | 8.5 | 5.1 | 14.0 |
All classifications related to malnutrition | 95 | 16.8 | 10.3 | 26.3 | 124 | 7.3 | 4.2 | 12.3 | 112 | 17.0 | 10.1 | 27.0 |
a Any danger sign or other severe classification
b Number of children classified, by the HCWs, with a given classification
Table 9.
Baseline | Control arm | Intervention arm | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Classification | Nb | % | 95%CI | Nb | % | 95%CI | Nb | % | 95%CI | |||
Severe pneumonia or very severe disease | 15 | 6.7 | 0.7 | 43.6 | 10 | 0.0 | – | – | 4 | 0.0 | – | – |
Pneumonia | 187 | 81.8 | 74.1 | 87.6 | 218 | 78.9 | 70.8 | 85.2 | 169 | 81.1 | 61.5 | 92.0 |
All classifications related to pneumonia | 202 | 76.2 | 69.1 | 82.2 | 228 | 75.4 | 65.3 | 83.4 | 173 | 79.2 | 55.2 | 92.1 |
Severe dehydration with another severe classification | 5 | 20.0 | 2.7 | 69.1 | 1 | 0.0 | – | – | 0 | – | – | – |
Severe dehydration without another severe classification | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – |
Dehydration with other severe classification | 1 | 100.0 | – | – | 2 | 50.0 | – | – | 0 | – | – | – |
Dehydration without other severe classification | 4 | 50.0 | 4.3 | 95.7 | 0 | – | – | – | 3 | 66.7 | – | – |
Diarrhoea with no dehydration | 176 | 50.0 | 40.5 | 59.5 | 343 | 65.6 | 57.5 | 72.9 | 189 | 76.7 | 52.9 | 90.6 |
Severe persistent diarrhoea | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – |
Persistent diarrhoea | 0 | – | – | – | 0 | – | – | – | 0 | – | – | – |
All classifications related to diarrhoea | 186 | 49.5 | 39.8 | 59.2 | 346 | 65.3 | 57.5 | 72.4 | 192 | 76.6 | 52.1 | 90.8 |
Dysentery | 12 | 8.3 | 0.7 | 54.0 | 27 | 11.1 | 2.1 | 42.4 | 12 | 75.0 | 29.2 | 95.6 |
Severe malaria or severe febrile illness | 23 | 8.7 | 1.4 | 38.7 | 24 | 0.0 | – | – | 17 | 29.4 | 4.5 | 78.7 |
Malaria | 476 | 94.5 | 91.7 | 96.4 | 734 | 94.1 | 90.3 | 96.5 | 366 | 95.4 | 92.1 | 97.3 |
All classifications related to malaria | 499 | 90.6 | 87.1 | 93.2 | 758 | 91.2 | 87.0 | 94.1 | 383 | 92.4 | 82.4 | 97.0 |
Severe acute malnutrition with complicationsa | 3 | 0.0 | – | – | 3 | 33.3 | 0.1 | 99.9 | 0 | – | – | – |
Severe acute malnutrition without complications | 19 | 15.8 | 5.5 | 37.8 | 42 | 14.3 | 4.1 | 39.6 | 23 | 43.5 | 33.0 | 54.6 |
Moderate acute malnutrition | 57 | 8.8 | 4.6 | 16.2 | 121 | 2.5 | 0.5 | 11.2 | 95 | 8.4 | 5.4 | 12.9 |
All classifications related to malnutrition | 79 | 10.1 | 5.3 | 18.6 | 166 | 6.0 | 3.9 | 9.2 | 118 | 15.3 | 11.1 | 20.6 |
a Any danger sign or other severe classification
b Number of children classified, by the validation nurses, with a given classification
Over-prescription
According to the HCWs’ classifications, the proportion of children who were not in need of an antibiotic but who were actually prescribed one was 11% (77/681) at baseline, 14% (187/1341) in the control districts and 8% (56/694) in the intervention districts (Table 10). According to validation nurses’ classifications, these proportions were 18% (123/668) at baseline, 23% (289/1252) in the control districts and 10% (69/676) in the intervention districts (Table 11). With respect to antimalarials, a Rapid Diagnostic Test (RDT) was performed for about 90% of febrile children in both arms (Table 5) and over-prescription was low and similar at baseline and between trial arms: around 2 to 4%.
Table 10.
Baseline (N = 686) | Control arm (N = 1343) | Intervention arm (N = 695) | |||||||
---|---|---|---|---|---|---|---|---|---|
Medicines | % | 95%CI | % | 95%CI | % | 95%CI | |||
Ampicillin injectable | 0.0 | – | – | 0.1 | – | – | 0.3 | 0.0 | 1.8 |
Gentamycin injectable | 0.0 | – | – | 0.0 | – | – | 0.1 | – | – |
Cotrimoxazole | 6.1 | 4.0 | 9.3 | 6.7 | 4.5 | 9.8 | 1.9 | 0.8 | 4.3 |
Amoxicillin | 4.0 | 2.2 | 7.1 | 5.8 | 3.3 | 10.0 | 5.0 | 3.3 | 7.6 |
Ciprofloxacin | 0.3 | – | – | 0.0 | – | – | 0.3 | – | – |
Metronidazole | 0.9 | 0.2 | 3.8 | 2.0 | 1.4 | 2.8 | 0.6 | 0.1 | 2.2 |
All antibiotics | 11.3 | 8.3 | 15.3 | 13.9 | 9.2 | 20.5 | 8.1 | 5.2 | 12.4 |
Artesunate or Artemether injectable | 0.4 | 0.1 | 1.5 | 0.1 | – | – | 0.1 | – | – |
Quinine injectable | 0.0 | – | – | 0.0 | – | – | 0.0 | – | – |
Artemisinin-based Combinnation Therapy (ACT) | 1.3 | 0.5 | 3.8 | 1.7 | 0.7 | 4.0 | 2.6 | 1.0 | 6.8 |
All antimalarials | 1.8 | 0.8 | 3.8 | 1.8 | 0.8 | 4.0 | 2.7 | 1.0 | 7.2 |
ACT Artesunate + Amodiaquine or Artemether/ Lumefantrine
Table 11.
Baseline (N = 686) | Control arm (N = 1343) | Intervention arm (N = 695) | |||||||
---|---|---|---|---|---|---|---|---|---|
Medicines | % | 95%CI | % | 95%CI | % | 95%CI | |||
Ampicillin injectable | 0.6 | 0.2 | 1.4 | 0.3 | 0.1 | 1.5 | 1.0 | 0.3 | 2.9 |
Gentamycin injectable | 0.0 | – | – | 0.3 | 0.1 | 1.6 | 0.9 | 0.3 | 2.2 |
Cotrimoxazole | 10.4 | 6.0 | 17.3 | 8.6 | 5.4 | 13.5 | 1.3 | 0.4 | 3.7 |
Amoxicillin | 5.3 | 3.1 | 8.9 | 10.7 | 5.7 | 19.2 | 6.5 | 3.2 | 12.5 |
Ciprofloxacin | 0.2 | – | 1.5 | 0.2 | – | 0.7 | 0.3 | 0.1 | 0.6 |
Metronidazole | 1.9 | 0.7 | 4.8 | 3.0 | 2.0 | 4.4 | 1.0 | 0.3 | 2.9 |
All antibiotics | 18.4 | 11.8 | 27.5 | 23.1 | 16.3 | 31.7 | 10.2 | 4.7 | 20.7 |
Artesunate or Artemether injectable | 0.7 | 0.3 | 2.0 | 0.6 | 0.1 | 4.1 | 0.7 | 0.3 | 1.7 |
Quinine injectable | 0.4 | 0.1 | 1.5 | 0.2 | – | 1.4 | 0.7 | 0.3 | 1.7 |
Artemisinin-based Combinnation Therapy (ACT) | 2.8 | 2.0 | 3.8 | 2.1 | 0.8 | 5.6 | 1.6 | 0.5 | 4.5 |
All antimalarials | 3.9 | 2.6 | 6.0 | 2.8 | 1.2 | 6.3 | 3.0 | 1.2 | 7.4 |
ACT Artesunate + Amodiaquine or Artemether/ Lumefantrine
Referral/ hospitalisation
Overall, the proportion of children in need of referral or hospitalisation according to the HCWs’ assessment who were actually referred or hospitalised by the HCWs was 60% (21/35) at baseline, 52% (22/42) in the control districts and 61% (25/41) in the intervention districts with no evidence for a difference between trial arms (individual-level mean difference = 9%; P-value = 0.509) (Table 4). According to the validation nurses’ assessment, these proportions were 55% (16/29) at baseline, 53% (17/32) in the control districts and 68% (15/22) in the intervention districts (individual-level mean difference = 15%; P-value = 0.398).
Treatment counselling
The proportion of caretakers to whom the HCWs mentioned both the number of doses a day and the number of days for all the relevant oral medicines prescribed for treating the child at home was 77% (473/612) at baseline, 92% (1046/1143) in the control districts and 88% (506/576) in the intervention districts with no evidence for a difference between trial arms (cluster-level mean difference = − 4.1%; P-value = 0.355) (Table 4). For all oral medicines, both the number of doses per day and the number of days were mentioned by the HCWs to a high proportion of caretakers at baseline and in both trial arms (Table 12).
Table 12.
Treatment | Baseline | Control arm | Intervention arm | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Na | % | 95%CI | Na | % | 95%CI | Na | % | 95%CI | ||||
Oral antibiotics for pneumonia b | 290 | 81.7 | 70.3 | 89.4 | 538 | 93.9 | 90.9 | 95.9 | 228 | 91.7 | 88.8 | 93.8 |
Artemisinin-based Combinnation Therapy (ACT) c | 470 | 93.6 | 89.1 | 96.3 | 721 | 96.4 | 94.1 | 97.8 | 360 | 96.7 | 94.5 | 98.0 |
ORS | 99 | 46.5 | 25.1 | 69.2 | 264 | 86.7 | 77.4 | 92.6 | 148 | 84.5 | 59.6 | 95.2 |
Zinc | 99 | 83.8 | 74.9 | 90.0 | 268 | 94.0 | 91.7 | 95.7 | 153 | 87.6 | 63.9 | 96.6 |
Oral anti-infectious for dysentery d | 19 | 57.9 | 26.5 | 84.0 | 54 | 83.3 | 69.5 | 91.7 | 18 | 94.4 | 70.0 | 99.2 |
Deworming treatments e | 73 | 80.8 | 59.4 | 92.4 | 111 | 94.6 | 77.2 | 98.9 | 68 | 89.7 | 73.6 | 96.5 |
Iron/ folic acid | 42 | 47.6 | 19.5 | 77.3 | 40 | 92.5 | 69.0 | 98.6 | 72 | 83.3 | 58.8 | 94.6 |
Plumpy nut or equivalent | 28 | 32.1 | 17.6 | 51.3 | 16 | 75.0 | 41.0 | 92.8 | 27 | 63.0 | 55.8 | 69.6 |
aNumber of children who were prescribed, by the HCWs, a given treatment (regardless of the classification)
b Amoxicillin, Cotrimoxazole; c Artesunate + Amodiaquine, Artemether/ Lumefantrine; d Ciprofloxacin, Metronidazole; e Albendazole, Mebendazole
Availability of essential oral medicines and equipment
The average proportion of essential oral medicines that were observed to be available at the health facilities was high: 98% at baseline, 94% in the control districts and 89% in the intervention districts (Table 13). However, deworming treatments, amoxicillin, ORS and multivitamins were less frequently available in the intervention districts compared to the control districts.
Table 13.
Baseline (N = 158) | Control arm (N = 292) | Intervention arm (N = 168) | ||||
---|---|---|---|---|---|---|
N | % | N | % | N | % | |
Oral medicine (availability observed with at least one unexpired) | ||||||
Albendazole | 158 | 94.3 | 290 | 93.5 | 166 | 79.5 |
Amoxicillin | 158 | 97.5 | 291 | 89.7 | 166 | 73.5 |
Artesunate + Amodiaquine or Artemether/ Lumefantrine (ACT) | 158 | 99.4 | 291 | 99.3 | 166 | 97.6 |
Ciprofloxacin | 158 | 99.4 | 291 | 97.3 | 166 | 94.6 |
Cotrimoxazole | 158 | 99.4 | 291 | 99.3 | 166 | 98.2 |
Iron and folic acid | 157 | 98.7 | 291 | 98.6 | 165 | 94.6 |
Mebendazole | 158 | 98.1 | 291 | 97.6 | 166 | 89.2 |
Metronidazole | 158 | 100.0 | 290 | 99.7 | 166 | 97.6 |
Multivitamins | 158 | 95.6 | 290 | 75.9 | 166 | 68.1 |
Oral Rehydration Salt (ORS) | 158 | 94.3 | 291 | 93.1 | 166 | 85.5 |
Ready to use therapeutic food (RUTF) | 158 | 96.2 | 289 | 92.0 | 166 | 91.0 |
Zinc | 158 | 95.6 | 288 | 93.1 | 165 | 87.3 |
Vitamin A | 158 | 100.0 | 290 | 98.6 | 164 | 97.0 |
Availability index of essential oral medicines (13 items) | 158 | 97.6 | 291 | 94.4 | 166 | 88.7 |
Equipment (availability observed or reported) | ||||||
Source of electricity | 158 | 91.1 | 292 | 98.6 | 168 | 100.0 |
Electricity without any power cut in the last 7 days | 155 | 41.9 | 198 | 33.3 | 254 | 66.7 |
Baby weighing scale (graduation 100 g) | 157 | 100.0 | 291 | 99.3 | 168 | 100.0 |
Children weighing scale (graduation 250 g) | 157 | 100.0 | 291 | 99.3 | 168 | 100.0 |
Measuring rod | 157 | 100.0 | 291 | 99.3 | 168 | 100.0 |
Mid-upper arm circumference tape | 157 | 100.0 | 290 | 99.7 | 168 | 100.0 |
Thermometer | 157 | 100.0 | 256 | 99.6 | 166 | 99.4 |
Rapid Diagnostic Test | 157 | 89.8 | 291 | 94.9 | 167 | 98.2 |
Source of clean water | 158 | 91.8 | 292 | 96.2 | 163 | 97.0 |
Spoons, cups and jugs to mix and administer ORS | 158 | 21.5 | 289 | 32.9 | 168 | 85.1 |
Kit for intravenous injection | 157 | 94.3 | 292 | 64.7 | 166 | 31.3 |
Single-use syringes with disposable needles | 157 | 100.0 | 292 | 99.0 | 167 | 99.4 |
Refrigerator | 152 | 99.3 | 287 | 99.3 | 168 | 99.4 |
Availability index of essential equipment (13 items) | 158 | 86.9 | 292 | 87.1 | 161 | 90.6 |
With respect to essential equipment, availability at the health facilities was high: 87% at baseline, 87% in the control districts and 91% in the intervention districts. Better availability of electricity and equipment to administer ORS was observed in the intervention districts compared to the control districts.
Explanatory analyses
Comparison of HCWs’ performance with and without use of IMCI paper-forms in the control districts
In order to assess whether the frequent use of IMCI paper-based form in the control districts had an effect on HCWs performance, primary and secondary outcomes in the control districts were compared between HCWs who were observed to use an IMCI paper-form and those who did not.
Surprisingly, HCWs who did not use an IMCI paper-form in the control districts seem to have better assessed danger signs than those who used a form: on average they performed 45% versus 22% of the recommended tasks respectively (Additional file 5). For all other outcomes, HCWs’ performance was similar between the two groups.
Agreement between HCWs and validation nurses’ clinical assessment
The square root of the mean square errors (RMSE) for the differences in child’s weight, height and temperature measurements between HCWs and validation nurses indicate differences of a small magnitude (< 1 kg, < 3 cm or < 1 °C) at baseline and in the trial arms (Additional file 6a). Higher RMSE were observed between HCWs and validation nurses’ measurements of mid-upper arm circumference (MUAC) (around 5 mm) and respiratory count (around 9 counts). All differences were fairly balanced between trial arms.
With respect to RDT results and caretakers’ answers about children’s key symptoms, actual agreement between HCWs and validation nurses were high (> 90%) at baseline and in the trial arms (Additional file 6b). The Kappa coefficients indicate that 90% or more of RDT results were in agreement beyond that expected by chance. The Kappa coefficients for caretakers’ answers range from 0.60 to 0.88.
Secondary analyses
Excluding “contaminated” control districts for the period when they were contaminated removed a total of 173 consultations from the analysis and made little or no difference to the results (Additional file 7).
Discussion
The IeDA intervention improved substantially HCW’s adherence to IMCI’s clinical assessment tasks (30% point increase on average across the intervention districts compared to the control districts), including the assessment of danger signs, which led to some overall increase in the proportion of children being correctly classified (around 10% point increase on average across the intervention districts compared to the control districts) but to no improvement in overall proportion of children receiving correct prescriptions. The intervention, however, appeared to have reduced over-prescription of antibiotics by 6 to 13% points.
Achieving correct classification depends, at least in part, on the clinical skills of the HCWs, which may be more difficult to improve than task adherence itself and may have limited the effect of the intervention on correct classification. Recent more advanced clinical charts, also built on electronic tools, such as electronic point-of-care tests (ePOCT) integrating malaria RDT, haemoglobin, pulse oximetry in all febrile patients and other tests (e.g. glucometer, C-reactive protein) in subgroups of them, have led to major improvements in febrile disease classification and a considerable reduction of antibiotic prescription [27].
In addition, using the eIMCI in Burkina Faso, improvements in classifications and prescriptions tended to be observed for less common conditions, such as dysentery and malnutrition, for which HCWs in the control districts performed relatively poorly. The data were also consistent with an improvement in danger sign identification, correct referrals/hospitalisations and management of severe malaria or severe febrile illness, although small numbers of such children preclude firm conclusions. For other, more common, conditions (e.g. malaria or pneumonia), HCWs in the control districts performed relatively well in classifying and prescribing the correct medicines, thus limiting the scope to detect an overall impact.
There were some notable differences between findings at baseline and in the control arm with respect to prevalence of pneumonia (27 and 16% respectively), malaria (69 and 55% respectively) and anaemia (13 and 7% respectively). At baseline and in the control arm, 33 and 18% of observations respectively occurred from January to March, during the peak of the pneumonia season. Observations during the malaria season (July to November) were less frequent at baseline (49%) compared to the control arm (61%). However, the higher prevalence at baseline is consistent with the higher proportion of positive RDT: 82% of RDTs were positive at baseline compared to 66% during the control steps. These results may reflect a more intense malaria season during the baseline steps. This could also explain the difference in anaemia prevalence, which is associated with malaria.
Our findings are broadly consistent with the limited evidence available on the effectiveness of eCDSS for improving adherence to IMCI (eIMCI). In 18 primary facilities in four districts of Tanzania, only 21% of children had all ten critical IMCI tasks assessed under paper-based IMCI compared to 71% under eIMCI (p < 0.001) [14]. In two basic health centres in the Kabul province of Afghanistan, only 24% of children underwent a physical examination in line with IMCI at baseline compared to 84% after 1 year of implementation (p < 0.05) [17]. Comparison of HCWs classifications with classifications given by an independent nurse in Tanzania showed that the electronic protocol improved overall correct classification: 83% under paper-based IMCI compared to 91% under eIMCI (p < 0.001) [14]. In Afghanistan, only 35% of children received a treatment in line with HCWs’ classifications at baseline compared to 99% after 1 year of implementation [17]. Reduction in over-prescriptions of antibiotic have also been reported using eIMCI in Afghanistan [17] and Tanzania [15].
In Burkina Faso, interviews with HCWs indicated that IeDA was well accepted, in particular with respect to the usefulness of the eCDSS in guiding through the clinical assessment (Blanchet K et al.: Realist evaluation of the Integrated electronic Diagnostic Approach (IeDA) for the management of childhood illness at primary health facilities in Burkina Faso, submitted). In Ghana, South Africa and Tanzania, HCWs reported similar opinions [13, 16]. Nevertheless, our realistic evaluation in Burkina Faso also revealed contextual factors that may have limited the effect of the IeDA intervention. First, staff turnover was reported to be common by district managers, in particular in remote rural facilities where most HCWs do not want to spend more than a few years. A visit in July 2017 in all intervention facilities revealed that around a third of HCWs (36%) had been changed within the last 12 months and that a relatively large proportion (36%) of HCWs had not benefited from the eIMCI training (Blanchet K et al.: Realist evaluation of the Integrated electronic Diagnostic Approach (IeDA) for the management of childhood illness at primary health facilities in Burkina Faso, submitted). Second, while supervision and audit with feedback can be effective in improving performance [28–30], monthly supervision visits planned under the IeDA intervention in Burkina Faso faced challenges. The district management teams reported limited budget, access to vehicles and time to dedicate to these visits (Blanchet K et al.: Realist evaluation of the Integrated electronic Diagnostic Approach (IeDA) for the management of childhood illness at primary health facilities in Burkina Faso, submitted).
In addition to incomplete coverage of the IeDA intervention, while pressure from children’s caretakers, sometimes reported during interviews with HCWs (Blanchet K et al.: Realist evaluation of the Integrated electronic Diagnostic Approach (IeDA) for the management of childhood illness at primary health facilities in Burkina Faso, submitted), may have limited the reduction in over-prescription of antibiotics, the relatively lower availability of some essential medicines, such as amoxicillin and ORS, in the intervention facilities compared to the control facilities may have limited improvement in correct prescriptions for pneumonia, severe acute malnutrition without complications and diarrhoea. Multiple conditions may also have influenced the medicines prescribed. Across baseline and trial arms, about a third or more of children were diagnosed with two or more classifications. In Tanzania, a large know-do gap was observed, and a lack of knowledge was not the only constraint identified for improved performance. HCWs’ weak belief in the importance of following guidelines and confidence in their own experience, lack of intrinsic motivation, and physical or cognitive “overload” were also reported, with poor remuneration contributing to several of these factors [31].
Limitations
Some limitations of our evaluation should be acknowledged. First, the “gold standard” classifications were provided by a repeat consultation after the initial consultation and it is possible that the clinical status of some children (e.g. respiratory rate, temperature, current convulsions) may have changed in the interval between the two. Therefore, we should not expect full agreement between HCWs and validation nurses. Our “gold standard” is certainly less than perfect, and this would tend to reduce the apparent magnitude of any improvement in classifications.
Second, it is likely that the behaviour of HCWs was impacted by the fact that they were observed [32]. The high proportion of HCWs observed using IMCI paper-forms in the control districts (68% overall) compared to routine practice (less than 8% of under-five consultations in 2012 [33]) suggests that HCWs in this arm were motivated to perform better than usual. Even if HCWs in the control districts who used IMCI paper-forms did not seem to have performed better compared to those who did not use IMCI paper-forms, repeated observations might explain improvements in some indicators from baseline to control steps, for instance adherence to assessment of danger signs (18% at baseline compared to 34% during control steps). Nevertheless, the behaviour of HCWs in the intervention districts may also have been affected by the presence of observers. Therefore, our findings may over-estimate how well HCWs perform in the absence of an observer, but it is difficult to assert whether or in which direction this may have affected the comparison of intervention and control districts.
Third, the initial evaluation design was not followed. In particular, rolling out the intervention to all districts as planned would have led to more data in the intervention arm, which could have strengthen our findings. In addition, the evaluation design could not address the multi-faceted nature of the intervention and evolving version of the eCDSS. It is therefore not possible to distinguish which component of the intervention led to observed improvements or whether improvements were the result of the combination of components.
Lastly, with respect to statistical analyses, multiple comparisons between arms were performed and can increase the overall error in hypothesis testing, so that P-values should be interpreted with caution. The small number of clusters per trial arm precluded using random effects models on individual level data, thus limiting our ability to control for individual child-level factors.
Conclusion
To conclude, the IeDA intervention was well accepted and improved substantially HCW’s adherence to IMCI clinical assessment which led to some improvements in overall correct classifications but little or no improvement in overall correct prescriptions. Nevertheless, substantial improvements were observed in correct classifications and prescriptions for dysentery and malnutrition. To some degree, we also observed an improvement in danger sign identification, correct referrals/hospitalisations and management of severe malaria, although small numbers prevent firm conclusions. For the most common conditions, HCWs in the control districts, who may have been influenced by a Hawthorne effect, performed relatively well, limiting the scope to detect an overall impact.
HCWs’ practices are complex behaviours that have many potential contextual and intrinsic influences. Lower availability of some essential medicines in the intervention districts was observed and our realistic evaluation concurrently reported staff turnover and incomplete coverage of training and supervision which may have limited the effect of the IeDA intervention on correct classification and prescription. Task adherence may be easier to achieve than correct classifications which require clinical skills. In the context of national scaling up, disparities between regions exist in terms of structures, staff and resources. Nevertheless, complete coverage of the eIMCI training could be achieved by its integration into the initial nursing curriculum. Supervision will inevitably require resources but also management capacity to deal with relationships, organisation culture and HCWs’ professional norms, experiences and motivation (Blanchet K et al.: Realist evaluation of the Integrated electronic Diagnostic Approach (IeDA) for the management of childhood illness at primary health facilities in Burkina Faso, submitted).
Supplementary Information
Acknowledgments
We gratefully acknowledge all the fieldworkers for their work in the field and the study population for their participation. We also thank the Independent Scientific Advisory Committee (ISAC) members (P Alain Labrique, Pr Antoine Gueissbhuler, Pr Valerie D’Acremont) for their thoughtful advice over the course of the study, and Terre des hommes foundation for their collaboration.
Abbreviations
- ACT
Artemisinin-based Combination Therapy
- eCDSS
electronic Clinical Decision Support System
- ePOCT
electronic point-of-care tests
- HCW
Health Care Worker
- IeDA
integrated eDiagnosis Approach
- IMCI
Integrated Management of Childhood illness
- LMIC
Low- and Middle-Income Countries
- LSHTM
London School of Hygiene and Tropical Medicine
- MoH
Ministry of Health
- MUAC
Mid-Upper Arm Circumference
- PBF
performance-Based Financing
- RDT
Rapid Diagnostic Test
- RMSE
square root of the mean square errors
- Tdh
Terre des hommes foundation
Authors’ contributions
KB, JJL and SC conceived the project. SoS designed the data collection instruments with inputs from other authors. AS and SeS implemented and supervised the fieldwork. SeS was responsible of data management. JJL and SC developed the analysis strategy, with inputs from SoS and KB. SoS analysed the data and wrote the first draft of the manuscript. All authors reviewed, made inputs to and approved the final paper. KB and SC are the overall guarantors and SoS is the corresponding author.
Funding
The trial was funded by the Bill and Melinda Gates foundation (Grant No. OPP1084359) and the Swiss Agency for Development and Cooperation. The funders of the study had no role in study design, in the collection, analysis, and interpretation of data, in the writing of the report, and in the decision to submit the paper for publication. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Declarations
Ethics approval and consent to participate
Ethical approval was granted by the National Health Ethics Committee of the MoH of Burkina Faso (Reference 2014–4-026), and the LSHTM (Reference 7261). Written informed consent was obtained from the HCW and the parent/guardian of all children aged under-5 prior to the observation of the consultation and the repeat consultation. The trial was registered at ClinicalTrials.gov (NCT02341469), first submitted in August 27 2014 and posted in January 19 2015.
Consent for publication
Not applicable.
Competing interests
The authors declare no conflict of interest.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Kruger C, Heinzel-Gutenbrunner M, Ali M. Adherence to the integrated management of childhood illness guidelines in Namibia, Kenya, Tanzania and Uganda: evidence from the national service provision assessment surveys. BMC Health Serc Res. 2017;17:822. doi: 10.1186/s12913-017-2781-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Kruk ME, Gage AD, Arsenault C, Jordan K, Leslie HH, Roder-DeWan S, Adeyi O, Barker P, Daelmans B, Doubova SV, English M, Elorrio EG, Guanais F, Gureje O, Hirschhorn LR, Jiang L, Kelley E, Lemango ET, Liljestrand J, Malata A, Marchant T, Matsoso MP, Meara JG, Mohanan M, Ndiaye Y, Norheim OF, Reddy KS, Rowe AK, Salomon JA, Thapa G, Twum-Danso NAY, Pate M. High-quality health systems in the sustainable development goals era: time for a revolution. Lancet Glob Health. 2018;6(11):Pe1196–Pe1252. doi: 10.1016/S2214-109X(18)30386-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Bryce J, Victora CG, Habicht JP, Black RE, Scherpbier RW. Programmatic pathways to child survival: results of a multi-country evaluation of integrated Management of Childhood Illness. Health Policy Plan. 2005;20(Suppl 1):i5–17. doi: 10.1093/heapol/czi055. [DOI] [PubMed] [Google Scholar]
- 4.Pariyo GW, Gouws E, Bryce J, Burnham G. Improving facility-based care for sick children in Uganda: training is not enough. Health Policy Plan. 2005;20(Suppl 1):i58–i68. doi: 10.1093/heapol/czi051. [DOI] [PubMed] [Google Scholar]
- 5.Huicho L, Davila M, Campos M, et al. Scaling up integrated management of childhood illness to the national level: achievements and challenges in Peru. Health Policy Plan. 2005;20(1):14–24. doi: 10.1093/heapol/czi002. [DOI] [PubMed] [Google Scholar]
- 6.Mushi HP, Mullei K, Macha J, Wafula F, Borghi J, Goodman C, Gilson L. The challenges of achieving high training coverage for IMCI: case studies from Kenya and Tanzania. Health Policy Plan. 2011;26(5):395–404. doi: 10.1093/heapol/czq068. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Kouanda S, Baguiya A. Evaluation de la qualité des soins prodigués aux enfants de moins de cinq ans dans les formations sanitaires des régions du Nord et de Centre-Nord du Burkina Faso. Ministère de la Santé; 2012.
- 8.Gurman TA, Rubin SE, Roess AA. Effectiveness of mHealth behavior change communication interventions in developing countries: a systematic review of the literature. J Health Commun. 2012;17(Suppl 1):82–104. doi: 10.1080/10810730.2011.649160. [DOI] [PubMed] [Google Scholar]
- 9.Kallander K, Tibenderana JK, Akpogheneta OJ, et al. Mobile health (mHealth) approaches and lessons for increased performance and retention of community health workers in low- and middle-income countries: a review. J Med Internet Res. 2013;15(1):e17. doi: 10.2196/jmir.2130. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Aranda-Jan CB, Mohutsiwa-Dibe N, Loukanova S. Systematic review on what works, what does not work and why of implementation of mobile health (mHealth) projects in Africa. BMC Public Health. 2014;14(1):188. doi: 10.1186/1471-2458-14-188. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Hall CS, Fottrell E, Wilkinson S, Byass P. Assessing the impact of mHealth interventions in low- and middle-income countries – what has been shown to work? Glob Health Action. 2014;7(1):25606. doi: 10.3402/gha.v7.25606. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Chib A, van Velthoven MH, Car J. mHealth adoption in low-resource environments: a review of the use of mobile healthcare in developing countries. J Health Commun. 2015;20(1):4–34. doi: 10.1080/10810730.2013.864735. [DOI] [PubMed] [Google Scholar]
- 13.Mitchell M, Getchell M, Nkaka M, Msellemu D, van Esch J, Hedt-Gauthier B. Perceived improvement in integrated management of childhood illness implementation through use of mobile technology: qualitative evidence from a pilot study in Tanzania. J Health Commun. 2012;17(Suppl 1):118–127. doi: 10.1080/10810730.2011.649105. [DOI] [PubMed] [Google Scholar]
- 14.Mitchell M, Hedt-Gauthier BL, Msellemu D, Nkaka M, Lesh N. Using electronic technology to improve clinical care – results from a before-after cluster trial to evaluate assessment and classifications of sick children according to integrated Management of Childhood Illness (IMCI) protocol in Tanzania. BMC Med Inform Decis Mak. 2013;13(1):95. doi: 10.1186/1472-6947-13-95. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Shao AF, Rambaud-Althaus C, Samaka J, Faustine AF, Perri-Moore S, Swai N, Kahama-Maro J, Mitchell M, Genton B, D’Acremont V. New algorithm for managing childhood illness using Mobile technology (ALMANACH): a controlled non-inferiority study on clinical outcome and antibiotic use in Tanzania. PLoS One. 2015;10(7):e0132316. doi: 10.1371/journal.pone.0132316. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Ginsburg AS, Tawiah Agyemang C, Ambler G, et al. mPneumonia, an innovation for diagnosing and treating childhood pneumonia in low-resource settings: a feasibility, Usability and Acceptability Study in Ghana. PLoS One. 2016;11:e0165201. doi: 10.1371/journal.pone.0165201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Bernasconi A, Crabbé F, Raab M, et al. Can the use of digital algorithms improve quality of care? An example from Afghanistan. PLoS One. 2018;13:e0207233. doi: 10.1371/journal.pone.0207233. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.You D, Hug L, Ejdemyr S, Idele P, Hogan D, Mathers C, Gerland P, New JR, Alkema L. Global, regional, and national levels and trends in under-5 mortality between 1990 and 2015, with scenario-based projections to 2030: a systematic analysis by the UN inter-agency Group for Child Mortality Estimation. Lancet. 2015;386(10010):2275–2286. doi: 10.1016/S0140-6736(15)00120-8. [DOI] [PubMed] [Google Scholar]
- 19.Ministère de la Santé, Direction Générale des Études et des Statistiques Sectorielles. Enquête nationale sur les prestations des services de santé et la qualité des données sanitaires (EN-PSQD/SARA II). 2014, Ouagadougou, Burkina Faso. Available from: http://www.who.int/healthinfo/systems/SARA_Burkina_Faso_2012_fullreport.pdf
- 20.Ridde V, Yaméogo P. How Burkina Faso used evidence in deciding to lauch its policy of free healthcare for children under five and women in 2016. Palgrave Communications. 2018;4(1):119. doi: 10.1057/s41599-018-0173-x. [DOI] [Google Scholar]
- 21.Blanchet K, Lewis JJ, Pozo-Martin F, et al. A mixed methods protocol to evaluate the effect and cost-effectiveness of an integrated electronic diagnosis approach (IeDA) for the management of childhood illnesses at primary health facilities in Burkina Faso. Implement Sci. 2016;11:111. doi: 10.1186/s13012-016-0476-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28(2):182–191. doi: 10.1016/j.cct.2006.05.007. [DOI] [PubMed] [Google Scholar]
- 23.World Health Organisation (WHO). Service Availability and Readiness Assessment (SARA): an annual monitoring system for service delivery - Reference Manual. Health Statistics and Information Systems, WHO: Geneva. 2013. Available from: https://www.who.int/healthinfo/systems/SARA_Reference_Manual_Full.pdf
- 24.Hayes RJ, Moulton LH. Cluster randomised trials. Boca Raton (FL): Chapman and Hall; 2009. [Google Scholar]
- 25.Thompson JA, Davey C, Hayes RJ, Hargreaves J, Fielding K. Swpermute: permutation tests for stepped-wedge cluster randomised trials. Stata J. 2019;19(4):803–819. doi: 10.1177/1536867X19893624. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Thompson JA, Davey C, Fielding K, Hargreaves JR, Hayes RJ. Robust analysis of stepped wedge trials using cluster-level summaries within periods. Stat Med. 2018;37(16):2487–2500. doi: 10.1002/sim.7668. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Keitel K, Kagoro F, Samaka J, Masimba J, Said Z, Temba H, Mlaganile T, Sangu W, Rambaud-Althaus C, Gervaix A, Genton B, D’Acremont V. A novel electronic algorithm using host biomarker point-of-care tests for the management of febrile illnesses in Tanzanian children (e-POCT): a randomised, controlled non-inferiority trial. PLoS Med. 2017;14(10):e1002411. doi: 10.1371/journal.pmed.1002411. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Rowe AK, de Savigny D, Lanata CF, Victora CG. How can we achieve and maintain high-quality performance of health workers in low-resource settings? Lancet. 2005;366(9490):1026–1035. doi: 10.1016/S0140-6736(05)67028-6. [DOI] [PubMed] [Google Scholar]
- 29.Rowe AK, Onikpo F, Lama M, Osterholt DM, Rowe SY, Deming MS. A multifaceted intervention to improve health worker adherence to integrated management of childhood illness guidelines in Benin. Am J Public Health. 2009;99(5):837–846. doi: 10.2105/AJPH.2008.134411. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Magge H, Anatole M, Cyamatare FR, Mezzacappa C, Nkikabahizi F, Niyonzima S, Drobac PC, Ngabo F, Hirschhorn LR. Mentoring and quality improvement strengthen integrated management of childhood illness implementation in rural Rwanda. Arch Dis Child. 2015;100(6):565–570. doi: 10.1136/archdischild-2013-305863. [DOI] [PubMed] [Google Scholar]
- 31.Lange S, Mwisongo A, Maestad O. Why don’t clinicians adhere more consistently to guidelines for the integrated Management of Childhood Illness (IMCI)? Soc Sci Med. 2014;104:56–63. doi: 10.1016/j.socscimed.2013.12.020. [DOI] [PubMed] [Google Scholar]
- 32.Leonard K, Masatu MC. Outpatient process quality evaluation and the Hawthorne effect. Soc Sci Med. 2006;63(9):2330–2340. doi: 10.1016/j.socscimed.2006.06.003. [DOI] [PubMed] [Google Scholar]
- 33.Ministère de la Santé, Direction. Générale de la Santé de la Famille, Direction de la Santé de la Mère et de l’Enfant. Évaluation de la mise en œuvre du plan de couverture PCIME 2005–2010 du Burkina Faso. Ouagadougou; 2013.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.