Management of Childhood Illness at Health Facilities in Benin: Problems and Their Causes

Abstract

Objectives. To prepare for the implementation of Integrated Management of Childhood Illness (IMCI) in Benin, we studied the management of ill children younger than 5 years at outpatient health facilities.

Methods. We observed a representative sample of consultations; after each consultation, we interviewed caregivers and reexamined children. Health workers' performance was evaluated against IMCI guidelines. To identify determinants of performance, statistical modeling was performed and 6 focus groups with health workers were conducted to solicit their opinions.

Results. Altogether, 584 children were enrolled and 101 health workers were observed; 130 health workers participated in focus group discussions. Many serious deficiencies were found: incomplete assessment of children's signs and symptoms, incorrect diagnosis and treatment of potentially life-threatening illnesses, inappropriate prescription of dangerous sedatives, missed opportunities to vaccinate, and failure to refer severely ill children for hospitalization. Quantitative and qualitative analyses showed various health facility–, health worker–, caregiver-, and child-related factors as possible determinants of health worker performance.

Conclusions. Action is urgently needed. Our results suggest that to improve health care delivery, interventions should target both the health system and the community level.

Each year in developing countries, more than 12 million children younger than 5 years die. About 70% of these deaths are a result of 5 conditions: pneumonia, diarrhea, malaria, measles, and malnutrition.1 To reduce child mortality from these conditions, the World Health Organization developed the Integrated Management of Childhood Illness (IMCI) strategy.1 At the health facility level, IMCI defines a minimum standard of care for ill children aged 1 week to 59 months. These guidelines describe steps for assessing, classifying, treating, and referring ill children; counseling the child's caregiver; and vaccinating underimmunized children. Diagnoses (or “classifications”) are based on signs and symptoms (e.g., fever = malaria; cough with fast breathing = pneumonia), and the only equipment required is a scale and a watch.

In March 1999, the Benin Ministry of Public Health formally adopted the IMCI strategy and selected Ouémé Département in southeastern Benin as a pilot site; the population is about 1 million and mostly rural, and the mortality rate for children younger than 5 years is 158 per 1000 live births.2 To prepare for the implementation and evaluation of IMCI, we studied the management of ill children seen as outpatients at health facilities in Ouémé Département. The objectives were to identify areas needing emphasis during IMCI training and to identify institutional factors, such as supervision, drug supplies, and time management, that may have affected IMCI implementation.

At first analysis of the results, we found serious deficiencies in the care children were receiving. In addition, the results suggested that public health facilities were underused and that caregivers may not be complying with health workers' recommendations to refer seriously ill children to a hospital. To understand the causes of these deficiencies, we used 2 complementary strategies: statistical modeling to identify predictors of health workers' performance, and focus group discussions with health workers to solicit their opinions on why the problems exist.

METHODS

Study Design

The design was a cluster survey that included all health facilities in Ouémé Département in which IMCI would be implemented (all 78 government health facilities and 9 of the 11 licensed nongovernment health facilities). The observational unit was the encounter between a health worker, an ill child, and the child's caregiver. Primary sampling units were defined as all ill child consultations occurring at a health facility on a single working day (Monday to Friday) during the study period. A primary sampling unit for each health facility was selected by systematic sampling.3 All children aged 1 week to 59 months who were brought to the facility for consultation for any illness were eligible for inclusion. This sampling plan yielded an equal probability sample of ill child consultations.

The study was conducted during the 2 rainy seasons (approximately May to July and September to October), the time of peak malaria transmission. Health facilities were divided into 2 groups according to the accessibility by road during the May to July rainy season. We visited the 75 accessible facilities from July 28 to September 8, 1999, and the 12 inaccessible facilities from October 11 to October 29, 1999.

Data Collection

Three survey teams received training until the agreement of practice results of surveyors and study investigators was greater than 90% (1 week). We did not give advance notice of our visit to health facility staff. Survey teams arrived at health facilities before the official opening time (8:00 am) and stayed until the official closing time (6:30 pm). We collected data with 5 methods: (1) observation of consultations with a checklist; (2) standardized exit interviews with caregivers (usually the mother); (3) repeat examination of children by an IMCI-trained study clinician using IMCI guidelines, out of the health worker's view; (4) assessment of health facility supplies with a checklist; and (5) standardized interviews with health workers. After the repeat examination, the survey clinician gave appropriate medications free of charge to any child who had been given inadequate treatment. To prevent undermining caregiver confidence, we asked health workers to tell caregivers that their child would be reexamined with new guidelines at the end of their visit and that additional treatment might be given.

Analysis

Data from paper questionnaires were double-entered into an electronic database and verified with Epi Info.4 We applied the Benin adaptation of IMCI guidelines5 to the assessment findings from the survey clinician's repeat examination to generate “gold standard” classifications and used these classifications to determine which assessment tasks and treatments were needed for each child. Indicators of health care quality were calculated with SAS.6 We estimated the precision of proportions (95% confidence intervals) with SUDAAN,7 which accounts for the cluster sampling design. Confidence intervals were estimated for all proportions presented in this report and are available from the authors.

To identify predictors of case management quality, we used logistic regression modeling with the SAS GENMOD procedure, which adjusts for the correlation among children managed by the same health worker. For each of 7 outcomes, we estimated odds ratios and P values for all variables of interest (see Table 4 ▶ later in this article) with a series of univariate models; we entered variables with P < .15 into a model and used a backward elimination strategy to eliminate variables with P > .05. In general, the outcomes reflected quality as defined by both IMCI guidelines and guidelines that existed in Benin before IMCI was adopted. One key outcome, malaria treatment, was omitted because we intend to analyze it differently and submit that analysis for publication separately.

TABLE 4—

Statistically Significant (P < .05) Predictors of Health Workers' Performance in the Management of Childhood Illness at Outpatient Health Facilities in Ouémé Département, Benin

Outcomes and Predictorsa (No. of Children in Analysis)	No. (%) of Children With the Outcome	% of Children With the Outcome for Each Level of the Predictor	Adjusted OR (95% CI)	P
Outcome 1. Health worker pinches skin to assess dehydration for children with diarrhea (n = 52)	7 (13.5)
In-service diarrhea training (trained health workers vs untrained health workers)		41.7 vs 5.0	12.4 (1.8, 83.9)	.01
Outcome 2. ORS treatment for children with diarrhea (n = 52)	20 (38.5)
Chief complaint of diarrhea (yes vs no)		73.3 vs 24.3	9.9 (2.7, 36.3)	.0006
Child's sex (male vs female)		42.9 vs 33.3	5.3 (1.2, 23.7)	.03
Health worker type (nursing aide vs nurse vs physician)b		57.1 vs 37.5 vs 20.0	3.5 (1.2, 9.9)	.02
Outcome 3. Iron treatment for children with IMCI-defined nonsevere anemia who did not need urgent referral (n = 100)	48 (48.0)
Health worker diagnosed anemia (diagnosed vs not diagnosed)		81.0 vs 39.2	7.9 (1.9, 33.8)	.005
Child's temperature (≥38.5°C vs <38.5°C)		26.7 vs 57.1	0.24 (0.1, 0.6)	.004
Outcome 4. Effective oral antibiotic treatment for children with IMCI-defined nonsevere pneumoniac (n = 117)	66 (56.4)
Health worker had knowledge of pneumonia treatment (yes vs no)d		65.1 vs 32.3	5.1 (2.0, 12.9)	.0007
Child's age (per year increase, ranging from 0 to 4 years)b		…	0.7 (0.5, 0.9)	.02
Consultation length (≥15 min vs 10–14 min vs <10 min)b		75.0 vs 55.3 vs 33.3	3.2 (1.8, 5.6)	<.0001
Outcome 5. Referral for hospitalization for children with an IMCI-defined need for urgent referral (n = 70)	23 (32.9)
Health facility had an inpatient service (yes vs no)		51.6 vs 17.9	15.6 (3.0, 81.0)	.001
Child was lethargic or unconscious (yes vs no)		75.0 vs 24.1	5.2 (1.1, 25.4)	.04
Case load (per additional patient, ranging from 2 to 34 patients)b		…	0.9 (0.8, 0.96)	.005
Health worker's age (per year increase, ranging from 25 to 52 years)b		…	0.9 (0.8, 0.97)	.01
Outcome 6. DPT vaccination for children needing DPT who did not need urgent referral (n = 53)	13 (24.5)
Child's age (<1 y vs ≥1 y)e		35.1 vs 0	Undefined	.005
Outcome 7. Unnecessary injections (health worker prescribed at least 1 injection not recommended by IMCI guidelines) (n = 397)	120 (30.2)
Child's temperature (≥38.5°C vs <38.5°C)		53.8 vs 23.2	2.9 (1.7, 5.0)	<.0001
Health worker's diagnosis was malaria (yes vs no)		35.8 vs 21.4	1.8 (1.1, 3.0)	.01

Note. OR = odds ratio; CI = confidence interval; ORS = oral rehydration solution; IMCI = Integrated Management of Childhood Illness; DPT = diphtheria-pertussis-tetanus.

^aVariables examined are child's age, sex, chief complaints, temperature, and treatment with an antimalarial before being brought to the health facility; consultation length; health worker's age, sex, preservice and in-service training, knowledge, diagnostic accuracy, case load, and supervision in the past 6 months; health facility type (small government health facility, large government health facility, or nongovernment health facility); and whether the health facility had an inpatient service, a resident supervisor, wall charts in the consultation room, and medicines in stock. Most, but not all, variables in this list were analyzed as potential predictors for the 7 outcomes examined.

^bVariable analyzed as a continuous variable; the odds ratio and P value reflect a test for trend.

^cEffective oral antibiotics are amoxicillin, ampicillin, cotrimoxazole, and erythromycin.

^dKnowledge of pneumonia treatment means that when the health worker was asked for the treatment of a child who had fever, cough, and fast breathing in a hypothetical case scenario, the health worker's response included amoxicillin, ampicillin, cotrimoxazole, or erythromycin.

^eBecause a proportion equaled zero, the variable was analyzed with 2-sided Fisher exact test.

Definitions

For several assessment tasks, the definition of correct performance was whether the health worker “determined” if a child had a particular sign or symptom. “Determined” means that the health worker was exposed to the information because the health worker specifically asked for the information or because the information was offered spontaneously by the caregiver or because the child obviously had or did not have the sign (e.g., a child who was awake and alert was obviously not lethargic). This definition prevented a health worker's performance from being judged as incorrect when the health worker did not ask for information that was already available.

To evaluate treatments, we used 2 definitions of correct treatment: “recommended,” defined by the Benin adaptation of IMCI guidelines, and “adequate,” judged by the investigators to be about as effective as the recommended treatments (see Table 3 ▶ later in this article). IMCI danger signs included convulsions (during the consultation or in the past 3 days), the inability to drink or breastfeed, “vomiting everything,” and lethargy or unconsciousness. IMCI main symptoms included cough or difficult breathing, diarrhea, fever, and ear problems.

TABLE 3—

Treatments^a Administered to or Prescribed for Children Aged 2 to 59 Months With Moderate or Severe IMCI Classifications Who Were Brought to an Outpatient Health Facility for an Initial Consultation in Ouémé Département, Benin

IMCI Classification (Treatment)	No. With Classification	No. (%) Treated	95% CI
Severe pneumonia or very severe disease
Recommended treatment (referral and injectable ampicillin and immediate diazepam treatment for convulsions)	24	0 (0)	…
“Adequate” treatment (referral and an effective antibiotic and immediate treatment for convulsions with diazepam or phenobarbital)b	24	1 (4.2)	0, 11.9
Pneumonia (nonsevere)
Recommended treatment (cotrimoxazole)	117	33 (28.2)	18.9, 37.6
“Adequate” treatment (an effective antibiotic)b	117	67 (57.3)	47.6, 67.0
Diarrhea with moderate or no dehydration
Recommended treatment (oral rehydration solution)c	52	20 (38.5)	23.4, 53.5
Very severe febrile disease
Recommended treatment (referral and injectable ampicillin and injectable quinine and treatment for low blood sugar and paracetamol for fever ≥39.0°C and immediate diazepam treatment for convulsions)	36	0 (0)	…
“Adequate” treatment (referral and an effective antibiotic and an effective antimalarial and paracetamol for fever ≥39.0°C and immediate treatment for convulsions with diazepam or phenobarbital)b	36	2 (5.6)	0, 13.1
Malaria (nonsevere)
Recommended treatment (chloroquine and paracetamol for fever ≥39.0°C)	301	179 (59.5)	48.7, 70.3
“Adequate” treatment (an effective antimalarial and paracetamol for fever ≥39.0°C)b	301	253 (84.1)	79.5, 88.6
Acute ear infection
Recommended treatment (cotrimoxazole and paracetamol)	23	0 (0)	…
“Adequate” treatment (an effective antibiotic and paracetamol)b	23	2 (8.7)	0, 20.6
Severe anemia
Recommended treatment (urgent referral)c	29	19 (65.5)	44.2, 86.9
Anemia (nonsevere, among children not needing urgent referral)
Recommended treatment (iron and folate and chloroquine and mebendazole if aged ≥24 months and last dose was >6 months ago)a	100	15 (15.0)	6.4, 23.6
“Adequate” treatment (iron and an effective antimalarial and mebendazole or albendazole if aged ≥24 months and last dose was >6 months ago)b	100	45 (45.0)	34.0, 56.0

Note. IMCI = Integrated Management of Childhood Illness; CI = confidence interval; ellipses indicate not available (CIs were not estimated if a proportion = 0).

^aOnly drug selection is considered; other aspects (e.g., dosages) are not considered. Recommendations are based on the Benin adaptation of IMCI standards except for nonsevere anemia and choice of antipyretic, which are based on generic World Health Organization standards. “Adequate” treatments were treatments judged by the investigators to be about as effective as the recommended treatments. All drugs are oral unless otherwise stated.

^bEffective antibiotics are ampicillin, cotrimoxazole, amoxicillin, erythromycin, and penicillin G. Effective antimalarials are quinine, chloroquine, sulfadoxine-pyrimethamine, halofantrine, and cotrimoxazole. Oral medications are considered effective only if the child is awake, able to drink, not “vomiting everything,” and not having seizures.

^cFailure to provide the recommended treatment was considered inadequate; therefore, no “adequate” treatment definition was proposed.

Focus Group Discussions

To disseminate results of the health facility study, we convened 6 “feedback meetings” for health workers in February and March 2000; 2 meetings were held in Porto Novo, the capital, and 1 was held in each of the 4 remaining health zones in Ouémé Département. We sent invitations to health workers at all 87 facilities that were included in the 1999 health facility study, and all health workers who came to a feedback meeting agreed to participate in a focus group discussion.

After presenting the health facility study results, an investigator (F. O. or M. L.) conducted a focus group discussion. The discussions covered 14 topics—10 on deficiencies in case management (see problem list in Figure 1 ▶) and the other 4 on underuse of health facilities, caregiver noncompliance with health workers' recommendations to take severely ill children to a hospital, recognition of good performance as a source of motivation, and barriers to IMCI implementation. We used a standard list of questions to lead a semistructured discussion to solicit health workers' opinions on why aspects of the management of childhood illness were inadequate and what could be done to improve case management at health facilities.

FIGURE 1—

Causes for deficiencies in the care delivered to ill children at public health facilities (HFs) according to health worker (HW) opinions: Ouémé Département, Benin.

All focus group discussions (in French) were tape recorded, transcribed, and translated into English. We also collected the names of health workers and their health facilities to determine the extent to which the views of focus group participants represented those of health workers observed in the health facility study. However, to encourage health workers to be frank and honest and unafraid of being penalized for their views, we did not attach names to specific comments. We determined the sex of participants from their names.

We reviewed responses to the discussion questions and grouped nearly identical responses into “opinions” and then reviewed opinions for the presence of causes and causal relations to develop a series of causal chains.8 We defined a root cause as a cause at the end of a causal chain furthest from the problem being studied (i.e., a cause that had no identifiable cause, according to the opinions gathered in the study) and an intermediate cause as any cause in the chain between the root cause and the problem. To simplify the presentation, we grouped case management deficiencies into a single problem (Figure 1 ▶). This simplification reflects the close relation and often common root causes of case management deficiencies.

RESULTS

Enrollment and Study Participants

We visited all 87 health facilities in the sampling frame, although 2 were closed because of severe flooding. Seventy-eight health facilities surveyed were public; the remainder included private facilities, public–private cooperatives, and health facilities operated by religious groups. During visits to the 85 open health facilities, we enrolled 109 health workers and 584 children and their caregivers (100% participation). Not every facility had child consultations on the day of the visit: 101 health workers performed 584 child consultations at 79 health facilities.

Most health facilities (68 of 87, or 78.2%) were open during the official hours of operation. Among the 85 open health facilities, 51 (60.0%) had running water, 38 (44.7%) had electricity, and 37 (43.5%) had a scale for weighing infants and a scale for older children. All IMCI-recommended drugs except vitamin A were in stock at most facilities. Only 60 (70.6%) facilities had BCG, polio, diphtheria-pertussis-tetanus, and measles vaccines in stock, and only 2 (2.4%) facilities had yellow fever vaccine in stock.

Of the 109 health workers interviewed, 74 (67.9%) were men; 78 (71.6%) had a nursing degree, 6 (5.5%) were physicians, and 25 (22.9%) had no formal medical training (i.e., “nursing aides”). The mean age was 39 years (range = 25–53), and the average amount of time spent working as a health worker was 13 years. The proportions of health workers who had received in-service training were 59.6% for vaccinations, 34.9% for diarrhea, 28.4% for malaria, 8.3% for nutrition, and 0% for acute respiratory infections. The duration of in-service training courses was typically 1 to 5 days, and about one third included clinical practice. In the 6 months preceding the survey, 64 (58.7%) health workers received a supervisory visit, 31 (28.4%) received a supervisory visit in which the health worker was observed during a child consultation, and 14 (12.8%) reported, when asked about their last visit, that it included observation during a child consultation and feedback on clinical performance.

Health workers managed a median of 4 children per day (range = 0–23). They also had a median of 2 other duties (range = 0–7), such as caring for older children and adults, administrative work, and education activities. When consultations with patients of all ages were considered, the median caseload was 9 patients per day (range = 0–34).

Among the 584 children enrolled, 317 (54.3%) were male, the median age was 16 months (range = 0–59), and 431 (73.8%) represented initial consultations at the health facility for the illness. Most (88.5%, or 516 of 583; 1 value missing) caregivers were the child's mother.

The most common gold standard IMCI classifications for the 550 children aged 2 to 59 months were malaria (410 of 547, 3 values missing, or 75.0%), nonsevere anemia (168, or 30.5%), and pneumonia (144, or 26.2%); diarrhea with dehydration, dysentery, measles, and ear problems were uncommon. The distribution of IMCI classifications among children brought for an initial consultation was similar to that in children brought for a follow-up visit. Among the 34 children aged 1 week to 2 months, the most common classifications were feeding problem or low weight (23, or 67.6%) and possible serious bacterial infection (7, or 20.6%). Altogether, children had a median of 2 classifications each, 85 (14.6%) had at least 1 severe classification, and 436 (74.7%) had at least 1 moderate classification (but no severe classification). In all, 87 (14.9%) children needed urgent referral, and 16 (2.7%) needed nonurgent referral. Among the 431 children brought for an initial consultation, 42.4% (182 of 429, 2 values missing) were brought within 2 days of the illness's onset. Before being brought to the health facility, 228 (52.9%) children had received an antipyretic, 201 (46.6%) had received an antimalarial, and 28 (6.5%) had received an antibiotic.

Characteristics of Patient Consultations

Most (75.5%, or 440 of 583; 1 value missing) caregivers arrived at health facilities before 12:00 pm. The median waiting time was 20 minutes (range = 0–250 minutes), although 20.9% (122 of 583, 1 value missing) of the caregivers waited 1 hour or more. The median consultation length was 10 minutes; however, consultations were twice as long for initial consultations (median = 12 minutes; range = 2–127) as for follow-up consultations (median = 6 minutes; range = 1–74).

We analyzed case management quality primarily on the 397 children aged 2 to 59 months who were brought for an initial consultation and whose consultation was observed by a surveyor. Health workers rarely (13 of 397, or 3.3%) washed their hands before consultations. Health workers determined whether children had IMCI danger signs in fewer than 9% of the consultations (except for determining whether a child was awake and alert) (Table 1 ▶). IMCI main symptoms were more often determined, but the proportion of children assessed varied widely from 9.8% (ear problem) to 91.4% (fever). A median of 1 of 4 danger signs and 2 of 4 main symptoms were assessed per child.

TABLE 1—

Assessment Tasks Performed on Ill Children Aged 2 to 59 Months Who Were Brought to an Outpatient Health Facility for an Initial Consultation in Ouémé Département, Benin

Task	No.	%	95% CI
For all children (N = 397)
Health worker “determined”a if the child
Had convulsions	33	8.3	5.0, 11.6
Was lethargic or unconscious	389	98.0	96.7, 99.3
Was unable to drink or breastfeed	21	5.3	2.9, 7.7
Vomited everything	14	3.5	1.3, 5.8
Health worker “determined”a if the child had
Cough or difficult breathing	265	66.8	59.8, 73.7
Diarrhea	149	37.5	29.7, 45.3
Fever (by history and temperature measurement)	363	91.4	87.9, 95.0
An ear problem	39	9.8	5.7, 13.9
Health worker assessed nutritional status by
Weighing the child	155	39.0	27.6, 50.5
Checking for edema of both feet	3	0.8	0, 1.6
Checking for palmar pallor	17	4.3	1.8, 6.8
For children with cough or difficult breathing (n = 206)
Health worker “determined”a duration of symptom	54	26.2	19.8, 32.7
Health worker checked 60-second respiratory rate	0	0	…
For children with diarrhea (n = 52)
Health worker “determined”a duration of diarrhea	17	32.7	19.2, 46.2
Health worker asked if there was blood in the stool	11	21.2	7.9, 34.4
Health worker offered the child fluid to assess thirst	0	0	…
Health worker pinched the skin of the abdomen	7	13.5	2.4, 24.5
For children with fever (n = 340)
Health worker “determined”a duration of the fever	224	65.9	60.5, 71.3
Health worker checked for neck stiffness	42	12.4	5.8, 18.9

Note. CI = confidence interval; ellipses indicate not available (CIs were not estimated if a proportion = 0).

^a”Determined” means that the health worker was exposed to the information (i.e., either the health worker asked for the information or the caregiver spontaneously offered the information or the surveyor observing the consultation could obviously tell the child had the symptom of interest).

To examine whether the determination of main symptoms, as defined above, was a result of health workers actively asking for information or caregivers spontaneously offering information, we analyzed the subset of consultations in which the information was not readily available (i.e., the health worker had to ask). For cough or difficult breathing, health workers asked about the symptom in 184 (58.2%) of the 316 consultations in which the caregiver did not spontaneously identify the symptom. The same analysis for other main symptoms gave the following proportions: 33.3% (124 of 372) for diarrhea, 53.2% (33 of 62) for fever, and 7.0% (27 of 385) for an ear problem. The importance of asking about symptoms when caregivers did not offer the information spontaneously can be shown by examining the prevalence of IMCI classifications among children whose caregivers did not spontaneously identify main symptoms. For example, of the 316 children whose caregiver did not spontaneously report cough or difficult breathing, 75 (23.7%) had pneumonia and another 20 (6.3%) had severe pneumonia. Similarly, of the 62 children whose caregiver did not spontaneously report fever (and who did not have a recorded temperature of 37.5°C or greater), 32.8% (20 of 61, 1 value missing) had malaria and another 4.9% (3 of 61, 1 value missing) had very severe febrile disease.

Health workers usually did not make symptom-specific assessments; indeed, in the 206 children with cough or difficult breathing, they never measured a 60-second respiratory rate, and they determined duration in only 54 (26.2%) consultations (Table 1 ▶). Similarly, assessment of malnutrition usually was incomplete; for example, only 155 (39.0%) children were weighed. Among the 304 children seen at a health facility with a scale, 136 (44.7%) were weighed.

In general, health workers' diagnoses did not match gold standard IMCI classifications (Table 2 ▶). For example, health workers identified only 201 of the 301 children with a gold standard IMCI classification of malaria (sensitivity = 201 of 301, or 66.8%). Except for malaria, sensitivity estimates were all less than 21%.

TABLE 2—

Sensitivity of Health Worker Diagnoses for Detecting “Gold Standard” IMCI Classifications for Ill Children Aged 2 to 59 Months Who Were Brought to an Outpatient Health Facility for an Initial Consultation in Ouémé Département, Benin

IMCI Classification	No. of Children Correctly Diagnosed/No. of Children With Gold Standard Classification	Sensitivitya	95% CI
Classifications for cough or difficult breathing
Severe pneumonia or very severe disease	0/24	0	…
Pneumonia (nonsevere)	7/117	6.0	0.7, 11.2
No pneumonia: cough or cold	8/66	12.1	3.1, 21.2
Classifications for diarrhea
Diarrhea with no dehydration	0/45	0	…
Classifications for fever
Very severe febrile disease	0/36	0	…
Malaria (nonsevere)	201/301	66.8	59.6, 73.9
Classifications for ear problem
Acute ear infection	2/23	8.7	0, 20.6
Classifications for malnutrition or anemia
Severe malnutrition or severe anemia	6/37	16.2	5.4, 27.1
Anemia or very low weight	22/114	19.3	10.4, 28.2
Classifications for anemia
Severe anemia	6/29	20.7	6.3, 35.1
Mild anemia	22/112	19.6	10.7, 28.6

Note. IMCI = Integrated Management of Childhood Illness; CI = confidence interval; ellipses indicate not available (CIs were not estimated if a proportion = 0).

^aSensitivity is the percentage of children with a gold standard IMCI classification who receive correct diagnoses by the health worker under observation.

Health workers often did not refer children for hospitalization when indicated by IMCI guidelines. Of the 70 children who were brought for an initial consultation and needed urgent referral, only 23 (32.9%) were referred. In addition, caregivers may not always comply with the health worker's recommendations to take their child to a hospital; of the 39 caregivers who said that the health worker had referred their child (including some children who did not need referral by IMCI guidelines), 8 (20.5%) indicated that they would not take their child to the referral center the same day.

Health workers often did not treat children according to IMCI guidelines, but a deviation from IMCI guidelines did not necessarily mean that the treatment was ineffective (Table 3 ▶). For moderate IMCI classifications, the proportion of children for whom the selection of medicines agreed with IMCI recommendations varied from 0% (acute ear infection) to 59.5% (malaria), whereas adequate treatment varied from 8.7% (acute ear infection) to 84.1% (malaria). For severe classifications, the proportion of children for whom the selection of treatments (including referral) agreed with IMCI recommendations was 0% for severe pneumonia and very severe febrile disease and 65.5% for severe anemia; adequate treatment rates were 4.2% for severe pneumonia and 5.6% for very severe febrile disease.

Other important aspects of treatment include dosage, duration of treatment, and whether the caregiver actually obtains the medicine and knows how to administer it at home. For the 301 children with malaria, health workers selected an adequate antimalarial for 91.4% of the children and selected the recommended antimalarial drug for 64.8% of the children, but they selected the recommended drug and prescribed the correct dosage and duration for only 26.3% of the children. Only 16.0% of the caregivers left the health facility with the recommended antimalarial and the knowledge of how to administer it.

Health workers often missed opportunities to vaccinate. Of the 168 children aged 2 to 59 months with an immunization record, a valid birthdate, and no need for urgent referral, none of the 9 children needing BCG vaccine received it, 13 of 52 (25.0%) received oral polio vaccine, 13 of 53 (24.5%) received diphtheria-pertussis-tetanus vaccine, and 2 of 25 (8.0%) received measles vaccine.

Health workers often treated children unnecessarily and sometimes with dangerous drugs. At initial consultations, they administered or prescribed an unnecessary injection to 120 (30.2%) of the 397 children. Of the 51 children with neither fever nor anemia, 20 (39.2%) received an unnecessary antimalarial. A particular concern was the inappropriate use of the sedatives diazepam and phenobarbital, which may cause respiratory depression and death in children with a respiratory illness such as pneumonia; about 15% of the children with pneumonia or severe pneumonia received unnecessary treatment with a sedative.

A large number of medicines were prescribed for some children. On average, health workers prescribed 4.2 medicines per child (range = 0–10); nevertheless, caregivers actually obtained an average of only 2.6 medicines per child (range = 0–9).

Health workers rarely performed counseling tasks recommended by IMCI. For example, they told the child's diagnosis to caregivers in 11.1% of the consultations, verified comprehension of instructions in 6.0% of the consultations, and asked caregivers if they had questions in only 0.8% of the consultations. In addition, health workers praised caregivers in only 2.0% of the consultations. For children not needing urgent referral, in 4.2% of consultations health workers counseled caregivers to administer more fluids; in 2.7% of consultations, to continue feeding or breastfeeding; and in 5.7% of consultations, to return immediately if the child became sicker. Health workers never counseled caregivers to return immediately if the child was unable to drink. Instructions on administering medicines were given in 49.6% of the 246 consultations in which the caregiver had obtained at least 1 medicine, and advice on using a mosquito bed net was given in 18.0% of the consultations.

Health workers did not always explain effectively how to administer medications. Exit interviews with 148 caregivers whose child had malaria and who had obtained oral chloroquine with clear written instructions showed that only 68 (45.9%) caregivers recalled instructions on dosage and treatment duration that perfectly matched the instructions of the health worker, 66 (44.6%) of the caregivers recalled instructions that partially matched those of the health worker, and 14 (9.5%) of the caregivers could not correctly recall any of the instructions.

Predictors of Case Management Quality

Statistically significant (P < .05) predictors of 7 indicators of case management quality are shown in Table 4 ▶. The number of children in each analysis was relatively small, ranging from 52 to 397. Factors associated with at least 1 indicator of correct performance (e.g., treat anemia with iron) included chief complaint of diarrhea, male sex of the child, younger age of the child, the child's being lethargic or unconscious, being treated by a nursing aide, in-service diarrhea training, correct health worker diagnosis and knowledge, longer consultation time, and having an inpatient service at the health facility. Factors associated with incorrect performance (i.e., not referring a child for hospitalization; giving an unnecessary injection) included having a high temperature, higher caseload, older age of the health worker, and a health worker diagnosis of malaria. Factors not associated with any of the 7 outcomes were health worker's sex; supervision; in-service training in malaria, nutrition, or vaccinations; health facility type; and presence of a resident supervisor at the facility.

Focus Group Discussions

Altogether, 130 health workers attended a feedback meeting and participated in a focus group discussion; 99 (76.2%) were nurses, 14 (10.8%) were physicians, 13 (10.0%) were nursing aides, and 4 (3.1%) had other basic training; 83 (63.8%) had male given names, 43 (33.1%) had female given names, and 4 (3.1%) had names that were ambiguous. Fifty-eight (53.2%) of the 109 health workers enrolled in the health facility study participated in a focus group discussion, and 69 (79.3%) of the 87 health facilities were represented in the focus groups. Each focus group had 15 to 33 participants, and all focus groups included both men and women. Facilitators reported that health workers appeared to be at ease during the focus group discussions and that participation was good.

We collected 677 responses representing 394 unique opinions; 50 (12.7%) opinions were excluded from the causal chain analysis because they were too vague to be interpreted, did not describe a cause of a problem, or indicated an obvious misunderstanding of the issue. Causal chain analysis identified 17 root causes for inadequate case management (Figure 1 ▶), underuse of public health facilities (Figure 2 ▶), and caregiver's noncompliance with referral recommendations (Figure 2 ▶).

FIGURE 2—

Causes for the low use of public health facilities (HFs) and poor caregiver compliance with health worker (HW) recommendations to refer seriously ill children to a hospital, according to HW opinions: Ouémé Département, Benin.

DISCUSSION

We found many serious deficiencies in the management of ill children, although care was not uniformly poor. Assessment of children's clinical signs and symptoms was incomplete, diagnosis and treatment of potentially life-threatening illnesses were incorrect, opportunities to vaccinate were missed, and severely ill children were not referred for hospitalization. Even when health workers did prescribe the correct medicine, caregivers rarely left health facilities with both the medicine in hand and the knowledge of how to give it. In addition to these errors of omission, health workers unnecessarily prescribed dangerous sedatives to 1 of every 7 children.

Although health workers often did not treat children according to IMCI guidelines, some children not receiving the recommended treatment still received adequate treatment. This suggests that the categorization of case management quality as either correct or incorrect may blur an important distinction between “incorrectly” managed children who were treated adequately enough to avert death and “incorrectly” managed children who were treated so poorly that the consultation did not reduce the risk of dying.

Limitations of the Quantitative Study

The health facility study had several important limitations. First, the presence of the study team may have affected health workers' performance, either by making them nervous or by motivating them to perform better than usual. Surveyors may have had other effects as well. For example, the average number of children per health facility seen on the day of the study visit was higher than the average of the 5 preceding weekdays (6.7 vs 4.7). Perhaps the presence of the team attracted more patients than usual. The resulting increased workload may have caused performance to be worse than usual.

Second, the clinical status of the child (e.g., the child's temperature or respiratory rate) may have changed from the time of the health worker's consultation to that of the study clinician's examination. Such changes could cause health workers' diagnoses and treatments to appear incorrect more often than is truly the case. Third, data on treatments were not always complete because health worker prescriptions often were incomplete. Finally, small numbers of children probably limited our ability to identify statistical associations.

Despite these limitations, most of the deficiencies in case management identified by this study are probably real. None of the health workers had received IMCI training, so it is not surprising that certain aspects of case management would appear incorrect. However, the IMCI strategy is generally consistent with disease-specific guidelines that have existed for years. Not all deficiencies can be ascribed to the fact that IMCI has not yet been implemented.

Why Do Children Receive Poor Care?

Considering that most health facilities were well stocked with medicines, caseloads were relatively light, many health workers had received years of basic training, and at least some had received additional in-service training and supervision, why was the quality of care often poor? Statistical modeling found that preservice training, in-service diarrhea training, health workers' age, correct health worker diagnosis and knowledge, longer consultation times, lower caseloads, and having an inpatient service at the health facility were associated with correct performance. However, most factors were associated with only 1 indicator of correct performance. For example, in-service training was associated with improved performance of an assessment task but not with treatment or vaccination.

Child factors had particularly interesting relations with health worker performance: children with higher temperatures were more likely to be prescribed unnecessary injections and less likely to have iron prescribed for anemia; younger children were more likely to be vaccinated and to have pneumonia treated with an antibiotic; being male and a chief complaint of diarrhea were associated with oral rehydration treatment; and the presence of lethargy or unconsciousness was associated with being referred for hospitalization. Moreover, factors such as supervision and drug supply were not associated with any of the performance indicators. These results remind us that health workers' performance is a complex set of behaviors, with many possible determinants. Statistical modeling, however, is inherently limited by the data fed into the models. To address this limitation, we can draw on results from the focus group discussions.

Health Workers' Opinions

Many health workers' explanations were predictable. For example, inadequate training, supervision, and motivation and a lack of equipment and supplies could be expected to be reasons for deficiencies in case management quality. Identification of such factors (or barriers), however, is still useful for 3 reasons. First, it is reassuring that common assumptions are supported by data. Second, the detailed nature of the opinions and causal chains can be used to craft specific strategies for overcoming barriers. Third, strategies based on locally obtained data may be more readily accepted and more likely to overcome the barriers.

Some of the explanatory factors might have been more difficult to predict. For example, poor communication may in part be a result of health workers' discomfort with spending too much time speaking with someone else's wife. If such cultural barriers are not recognized and addressed, the potential for strategies such as IMCI (which requires considerable communication between health workers and caregivers) to improve case management might not be realized fully.

This last example, in which poor performance was explained by a cultural factor, also illustrates the more general finding that caregivers and “the community” may have an important influence on the quality of care in health facilities. In addition to cultural factors, health workers frequently mentioned caregiver beliefs (e.g., vaccines should not be given to an ill child) and attitudes (e.g., caregivers demand sedatives to calm their child) as reasons for deficiencies in case management. This finding underscores the role of factors influencing health workers that are outside the direct control of the health system and suggests that community interventions may be needed to improve the quality of care at health facilities. In particular, interventions could help caregivers understand how the health system operates, so that they know what to expect. For example, caregivers might need to be told that with IMCI, consultation and waiting times may increase because health workers will be evaluating their children more thoroughly. In addition, interventions could be designed to bring health workers and community leaders together to help bridge cultural gaps between caregivers and the health system.

Results from the focus groups agreed with findings of other studies. Cabana et al.9 reviewed 125 studies of physician opinions to explain why physicians in industrialized countries do not comply with clinical practice guidelines. Their findings, like ours, indicated a web of interconnected factors, including lack of familiarity with guidelines, disagreement with guidelines, lack of motivation, difficulty in changing old habits, and patient-, guideline-, and environmental-related factors. A study from Nepal10 found that health workers believed that poor health worker management, a lack of incentives and motivation, and poor working conditions led to poor quality of care. A survey from the Central African Republic11 reported that health workers thought that the barriers to providing correct treatment were a lack of training, medications, equipment, personnel, and feedback. Studies from Peru12 and Ghana13 illustrated the influence of caregivers on health worker prescribing practices. Health workers not only respond to caregiver demands for inappropriate treatments but also seem to be affected by their perceptions of what caregivers want—even when caregivers do not actually make demands.

Limitations of Focus Group Results

Results from the focus groups had several limitations. First, the health workers enrolled were a convenience sample, and their views may not represent those of all health workers in the Département. Second, health workers' opinions may not reflect reality. For example, when describing caregiver beliefs, health workers were reporting only their perceptions of the beliefs held by caregivers. Similarly, just because health workers state that a particular factor leads to poor performance does not mean the factor is truly related to performance. For example, in the Central African Republic, a lack of training was the barrier to providing correct treatment most often reported by health workers. However, the quality of care received by children with fever was not related to whether health workers had received in-service training on fever management.11

Third, health worker opinions may have been influenced by the setting of the focus group discussions. Participants knew that the meetings were sponsored by the Ministry of Public Health and other participants may have been coworkers or supervisors; thus, health workers may have expressed opinions that they thought would benefit them or avoided expressing opinions that they thought would harm them. Fourth, the study did not provide information on the relative importance of the root causes. Fifth, the focus group discussions were conducted in French and then translated to English and analyzed; translation may have altered some of the opinions. Finally, the analysis of opinions may have been unintentionally influenced by beliefs and interpretations of the investigators. This is particularly true for the causal chain analysis, because each chain was constructed by piecing together opinions on a variety of topics. Thus, causal chains may reflect opinions that were misinterpreted or taken out of context.

Although we have no proof that health workers' explanations were correct, they provide insights into how health workers perceive the health system in which they work and the beliefs and attitudes of the caregivers of the children they treat. Thus, the results can be used to develop recommendations for overcoming potential barriers to strategies such as IMCI for improving the quality of care. In particular, if health workers' opinions were correct, the study suggests that improving the management of childhood illness at health facilities requires changes in both the health system and the communities. The implication, therefore, is that the community component of IMCI should include interventions to support case management at health facilities. Finally, if studies such as this one were conducted in a variety of settings, comparisons of root causes could help identify determinants of health worker performance that transcend specific cultural settings and contribute to global strategies for improving performance.

Future Directions

What can be done to improve or maintain health worker performance? Studies from developing countries suggested that training alone, the most common intervention for improving performance, has little lasting effect.11–14 Other studies15–18 suggested that supervision, incentives, accreditation, and the use of job aids may be helpful; however, our understanding about which interventions are truly effective (and affordable) is quite limited.

Some evidence from industrialized countries suggests that multiple interventions are more likely to improve physician performance than are single interventions.19 Thus, a reasonable strategy might be to strengthen several health worker supports simultaneously: supervision, job aids, and incentives. This strategy forms the basis of an intervention trial being planned in Ouémé Département to support health workers after IMCI training.

Despite its problems, Ouémé Département's health system has a good infrastructure and health workers and managers who seem eager to embrace the IMCI strategy and improve the quality of care. If IMCI is implemented with special attention to supporting health workers after training and working with communities to improve caregivers' understanding and acceptance of IMCI goals, there should be optimism about public health improvements.