Table 1.

Comparison between the methodologies of program supervision and the RADAR evaluation.

Program supervision	RADAR Evaluation
Evaluators
130 PHC clinicians trained to supervise CHWs in their catchment areas. On average, two clinicians consistently supervised three to four CHWs.	20 trained clinicians used as the gold standard and 10 supervisors.
Number of CHWs evaluated
Cycle 3 was used for the RADAR comparison, whereby 440 CHWs out of the 441 in the program were supervised. CHWs were supervised between five to seven times during the program. Two districts adopted a targeted approach focusing on lower performing CHWs in Cycle 4 (Kolokani and Sikasso) and Cycle 5 (Sikasso).See Supplement Table 2.	Of CHWs working in 441 functional iCCM sites, 300 were selected using a stratified random sample proportional to the number of CHWs in each district. From the stratified sampling scheme, the CHW distribution was per the number of CHW’s working in the district. The sample size provided an estimate of 50% prevalence with 6% precision and a type I error of 0.05, with 12% refusal to participate. Due to access and security constraints, only 237 CHWs were evaluated.
Consent
Consent was obtained from the sick child’s companion in case management observations and in simulations. If the companion was under 18 years of age and was not a parent, the supervisor and the CHW asked for a related adult to give consent. In the absence of that, another sick child was sought.	Consent was obtained from the CHW and the sick child’s companion before conducting the evaluation, including from mothers between 15 and 17 years of age.
Evaluation of quality of care
A clinician observed a CHW examining a sick child brought by a companion and recorded the observations in accordance with the MoH protocol (assessment of symptoms and danger signs, classification, treatment, or referral).CHW accuracy in measuring the respiratory rate was determined by the supervisor and the CHW simultaneously counting the child’s breaths using a timer and the CHW first revealing their count to the supervisor. The count by the CHW was considered correct if it was within 2 breaths of the count of the supervisor’s.Evaluated correct usage of the rapid diagnostic kit to classify malaria.CHW treatment was considered correct if both the prescription (identification of medicine) and dosage (in quantity and frequency) were correct.In the absence of a sick child, one with mild symptoms or whom the CHW had recently seen was sought. In the absence of that, a customized simulation to improve insufficiencies in CHW’s performance was employed using a healthy child, recruited with her/his caregiver.Five Care of Sick Children forms from the CHW the previous month were randomly selected and reviewed.	First, a data collector observed a CHW examining a sick child who met inclusion criteria (child between 2 and 59 months, symptoms relevant to iCCM, first consultation by CHW for the episode, mother or companion aged at least 18 years and those aged 15 to 17 years who were married or had at least one child) and recorded the CHW’s actions based on the MoH protocol (assessment of symptoms and danger signs, classification, treatment, or referral).Then, the data collector held an exit interview with the child’s companion to ascertain how well they had had comprehended the instructions. After the exit interview, a clinician who had not done the observation conducted a re-exam to avoid bias.In the absence of two sick children spontaneously presenting for care, study teams and CHWs were trained to go to the village and find sick children in the community.The CHW counted the child’s respiratory rate during the exam and the clinician counted it during the re-examination. Counting was considered correct when the CHW’s measurement was within ±5 counts of the clinician’s.Treatment by the CHW was considered correct if both the prescription (identification of medicine and dosage for quantity and frequency) was correct.
Number of children observed receiving care from a CHW
Target of one case per CHW.	Target of two cases per CHW.
Data collectionUsed MoH supervision form to assess the QoC for sick children (classification and treatment).Data collected in paper forms and kept at the PHC to which the CHW is attached. The District Counsellor ensured complete data collection during supervision and entered data later in MS Excel. Data from all districts was compiled by the program’s Monitoring and Evaluation (M&E) Officer of the Malian Red Cross.	Used RADAR data collection forms to assess steps of correct classification and treatment.Observation data collected in paper forms and then entered into an electronic database for analysis; other data collected directly on tablets using ODK Collect Data quality assurance and data cleaning included checking for any doubles with the same site and CHW name, labelling, and categorizing variables, using a reproducible ‘Do’ file in Stata, looking for missing values, or things that were either much too high or much too low, and verifying 100% of paper forms with the data entered.
Data analysisDescriptive analyses were done using MS Excel by the M&E Officer.Enabled findings on correct classification and treatment by district, but not by illness.Enabled examining association between correct classification and treatment with CHW sex only.	Enabled findings on correct classification and treatment, by illness, and by district. Enabled examining association between correct classification and treatment and factors such as CHW sex, age, level of education, level of experience, and age of child.