Validation of an Integrated Management of Childhood Illness algorithm for managing common skin conditions in Fiji

Abstract

Objective

To assess the sensitivity of an Integrated Management of Childhood Illness (IMCI) algorithm to detect common skin conditions in children in Fiji.

Methods

We collected data from the assessments of children aged between 2 months and 5 years who presented to one of two health clinics. Every child was assessed by a nurse trained in the use of the IMCI algorithm and also an expert paediatrician. We used a kappa statistic to measure agreement between the nurse/algorithm assessment method and the paediatrician’s diagnosis.

Findings

High sensitivity for identifying skin problems (sensitivity: 98.7%; 95% confidence interval, CI: 95.5–99.9) was found for the algorithm applied by IMCI-trained nurses, who were able to identify the one child with a severe skin infection and all three children with periorbital cellulitis. Sensitivity was high for the classification of abscess/cellulitis (sensitivity: 95%; 95% CI: 75.1–99.9) and infected scabies (sensitivity: 89.1%; 95% CI: 77.8–95.9), but lower for identification of impetigo, fungal infection and, in particular, non-infected scabies.

Conclusion

The IMCI skin algorithm is a robust tool that should be incorporated into the IMCI after some modifications relating to scabies and impetigo. Its use by primary health-care workers will reduce the burden of skin diseases in children in Fiji through improved case identification and management. The algorithm should be considered in other countries where skin diseases in children are a priority, particularly in the Pacific region.

Introduction

The Integrated Management of Childhood Illness (IMCI) is an evidence-based, cost-effective and comprehensive child health strategy developed by WHO and the United Nations Children’s Fund (UNICEF) for middle- and low-income countries.¹ The strategy aims to reduce child morbidity and mortality and to enhance child growth and development in these countries. IMCI has been shown to improve the quality of care and to increase the utilization of health facilities.² The core of the IMCI strategy is the application of clinical guidelines for the integrated case management of sick children aged 5 years and under at first-level health facilities. The IMCI guidelines have been individually adapted for conditions in more than 100 countries since they were launched in 1996.¹

The WHO/UNICEF Regional Child Survival Strategy for the Western Pacific highlights the importance of IMCI in delivering essential child survival interventions.³ The Fiji Ministry of Health adopted the IMCI strategy as a pilot project in three subdivisions of the Central Division of Fiji (Suva, Rewa and Serua Namosi) in 2003. Following a review of the pilot project in 2004, the IMCI strategy was implemented across the country. The review in 2004 also showed that health workers with IMCI training consistently identified skin conditions as a problem that the existing algorithm did not address.

Skin conditions, in particular impetigo and scabies, are common in Fiji. One study on Taveuni Island, a northern island, found that of 258 children aged 5 to 15 years, 84 (33%) had evidence of scabies infestation.⁴ In a study of 3462 primary-school children in the Central Division of Fiji in 2006, 1259 children (36%) had impetigo and 640 (18%) had scabies, with prevalence as high as 80% for impetigo and 40% for scabies in some areas (A Steer, unpublished data).

Scabies is often not identified as a health problem by parents; the condition is so common that consultation is not sought except when skin lesions worsen. In a recent cross-sectional survey of 773 infants and their parents attending maternal and child health-care clinics, 162 (21%) parents reported that their child had infected scabies, but the actual number of cases of infected scabies diagnosed by a medical practitioner at the same visit was 509 (66%), (F Russell, personal communication).

Scabies and impetigo in tropical and subtropical countries are more than just nuisance problems in children, and skin diseases are among the most common reasons for attending child health clinics worldwide.⁵ Scabies is a neglected disease⁶ and lesions are often secondarily infected by bacteria, most commonly Staphylococcus aureus and Streptococcus pyogenes. In studies of invasive S. aureus and S. pyogenes infections in Australian aborigines, impetigo is associated with up to 37% of cases.^7,8 Such findings are also likely to apply to Fiji, where invasive infections with S. pyogenes have a higher incidence than in developed countries and carry a high case-fatality rate.⁹ Impetigo caused by S. pyogenes can also lead to acute post-streptococcal glomerulonephritis and may play a role in the pathogenesis of acute rheumatic fever.¹⁰

An algorithm for the identification and management of skin conditions was developed by the Fiji IMCI pilot review panel that included WHO consultants, Fiji Ministry of Health staff, nursing representatives and selected Fijian paediatricians. We are not aware of an IMCI algorithm for the treatment of common childhood skin conditions in any other setting. The IMCI skin algorithm focuses on the most common skin conditions among children in Fiji, including impetigo, scabies, infected scabies and tinea (ringworm) infection (Box 1, Table 1). The algorithm also guides primary health-care workers in the identification of serious skin conditions that require prompt referral, such as extensive cellulitis and orbital and periorbital cellulitis.

Box 1. Child skin assessment algorithm applied by IMCI-trained nurses in two clinics in Fiji.

1. Ask the mother: “Does the child have a skin problem?”

2. Look for a skin problem.

If the answer to question 1 is “yes” or if a skin problem is identified, then ask:

• “Does the child have skin itchiness?”

• “Does the child have pain from the skin problem?”

Then, look and feel more closely:

• Look for extensive warm redness and swelling.

• Look for localized warm, tender swelling or redness.

• Look for swelling or redness around the eyes.

• Look for discrete lesions with pus or crusts.

• Look for papules on the hands, knees, elbows, feet, trunk.

• Look for round to oval scaly patches.

Table 1. IMCI algorithm used by nurses in two clinics in Fiji to classify common childhood skin conditions.

Sign	Diagnosis	Action
• Any general danger sign	Very severe skin infection	• Give first dose of appropriate antibiotic
• Extensive warm redness or swelling		• Refer URGENTLY to hospital
Swelling or redness around eyes	Periorbital or orbital cellulitis	• Give first dose of appropriate antibiotic
		• Refer URGENTLY to hospital
Localized warm tender swelling and redness	Abscess or cellulitis	• Give first dose of appropriate antibiotic
		• Refer to hospital
Discrete sores/lesions with pus or crusts	Impetigo	• Give appropriate oral antibiotic for 7 days
		• Follow up in 5 days
Itchiness AND papules AND lesions with pus or crusts	Infected scabies	• Give appropriate oral antibiotic for 7 days
		• Give appropriate topical skin cream
		• Treat the whole family with the cream
		• Follow up in 5 days and 2 weeks
Itchiness AND papules	Non-infected scabies	• Give appropriate topical skin cream
		• Treat the whole family with the cream
		• Follow up in 2 weeks
Round to oval flat scaly patches, often itchy	Fungal infection	• Give appropriate topical antifungal for 2 weeks
		• Follow up in 2 weeks
If there are not enough signs to classify in any of the above boxes OR if other signs present are not found in the above boxes	Other skin conditions	• Refer to the doctor or skin clinic

IMCI, Integrated Management of Childhood Illness.

Ideally, the algorithm should have very high sensitivity for identifying the presence of any skin condition, a sensitivity and specificity above 70–80% for classifying common skin conditions, and very high sensitivity for classifying severe skin infections.

To validate this algorithm we used an approach similar to that used in the original evaluation of the IMCI core clinical guidelines.^1,11–13 In these original studies, health-care workers’ decisions based on the IMCI case-management algorithm were compared with those of an expert paediatrician with or without laboratory support. In practice, the diagnosis of common skin conditions of childhood in tropical countries is clinical, and laboratory and microbiological testing is rarely indicated. Therefore, we designed a study to compare the assessment and clinical classification of common skin conditions by primary health-care workers trained in the use of a skin disease algorithm with those of two expert paediatricians.

Methods

Setting

Fiji is an independent republic of some 300 islands in the western Pacific and is north of the Tropic of Capricorn. In 2007 it had 827 900 inhabitants, mainly native Fijians (approximately 57%) and Indo-Fijians (about 37%)¹⁴ It is ranked 90th out of 177 countries on the United Nations Development Programme Human Development Index and has a gross domestic product per capita of 6066 United States dollars. Its infant mortality rate is 16 deaths per 1000 live births.¹⁵

This study took place in two clinics: the paediatric emergency department of the Colonial War Memorial Hospital (CWMH) in the capital Suva, and the Nausori Health Centre (NHC), a busy periurban health centre approximately 23 km north-east of Suva. These clinics were chosen for their convenient access and because they had already participated in the initial pilot project.

Training

Instruction on the use of the skin algorithm was provided to 10 nurses with IMCI training during an 8-hour workshop taught in 1 day by the two paediatricians. The workshop agenda followed a standard IMCI format and included background information on skin condition diagnoses. Nurses were trained in the use of the algorithm to identify, assess and classify skin conditions, to identify proper treatment, and to provide counselling to the mother. During the workshop, a written guide and a photograph booklet were used, individual and group exercises and role plays were conducted, and supervised practice assessment of clinical cases in the paediatric emergency department was performed.

Validation of the algorithm

Five of the nurses were from the CWMH and five were from the NHC. All nurses were experienced in child health, public health and IMCI. We chose nurses already trained in IMCI because an understanding of the IMCI system is needed to use the skin algorithm and because the goal is to eventually instruct all IMCI-trained nurses in Fiji in the use of the algorithm.

The two paediatricians chosen were very experienced in the diagnosis of common skin conditions in Fiji. Both attended the two clinics during the study, but at different times. Thus, the inter-rater reliability between the two paediatricians could not be measured. Similarly, inter-rater reliability between the nurses was not assessed.

Children aged 2 months to 5 years presenting consecutively to the clinics for any reason were seen initially by a nurse, who assessed them using the existing IMCI guidelines with the skin algorithm inserted after the “ear problem” section and before the “assess nutritional status” section.

The skin algorithm was used irrespective of the child’s presenting complaint; this practice is standard, as health workers complete all parts of the IMCI assessment guidelines for every child.¹ The nurse asked the parent if their child had a skin problem and then proceeded to examine the child’s skin from head to toe. It is standard procedure for the child to be properly exposed for most aspects of the IMCI guidelines, so a full skin examination does not cause the child further embarrassment.¹

Following the examination, nurses completed a simple data collection form that corresponded to the assessment and classification portions of the skin algorithm. A paediatrician who was blinded to the nurse’s assessment results then examined the same child and completed an identical data collection form. All children were examined by one IMCI-trained nurse and one paediatrician. The data collection form included pre-coded information about the presenting complaint.

Statistics

The primary endpoint was sensitivity of the algorithm employed by IMCI-trained nurses compared with the paediatricians’ diagnosis of any skin condition. We calculated that a sample size of 250 children was required to detect a sensitivity of 80% with a confidence range of ±10%. Data were entered into the EpiData, version 3.1, (EpiData Association, Odense M, Denmark) data entry platform and analysed using STATA, version 9.0, (StataCorp. LP, College Station, TX, United States of America). Sensitivity and specificity, with binomial 95% confidence intervals (CIs), were calculated for all endpoints.

We used a kappa statistic to measure agreement between the IMCI-trained nurses and expert paediatricians for all endpoints. It is generally accepted that kappa values greater than 0.75 indicate excellent agreement, those from 0.4 to 0.75 indicate fair to good agreement, and those less than 0.4 indicate moderate or poor agreement.¹⁶

Ethics approval

Ethics approval for the study was obtained from the Fiji National Research Ethics Committee and the Fiji National Health Research Committee.

Results

Participants’ characteristics

In September 2007, 250 children were enrolled over an 8-day period. We excluded 2 children because one was younger than 2 months and the other older than 5 years. There were 152 boys (61.3%) and 96 girls (38.7%). Of the 248 participants, 159 were native Fijian (64.1%), 70 were Indo-Fijian (28.2%) and 19 were of other ethnicities (7.7%). Mean age at enrolment was 1 year and 11 months; 139 (56.0%) children were enrolled at CWMH and 109 (44.0%) at NHC.

One paediatrician assessed 132 children (53.2%), while the other saw 116 children (46.8%). The number of children seen by the nurses was not equally divided among all 10 nurses; one nurse saw 9 children (4%), six nurses saw between 14 and 20 children (42% in total) and three nurses saw 39 children or more (54% in total), with a median of 25 children and a range of 9 to 55 children.

Clinical features

About half of the children visited the clinic for respiratory problems (119 children, or 50.0%). The next most common presenting complaint was a skin problem (67 cases, or 27%), and fever and gastrointestinal problems followed (24 cases, or 9.7%).

A skin problem was diagnosed by the paediatricians in 157 patients (63.3%) and by the nurses in 163 cases (65.7%) (Table 2). Infected and non-infected scabies and impetigo were the most common diagnoses made by a paediatrician. Four children had serious skin problems: 1 had a severe skin infection and 3 had periorbital cellulitis. The most common diagnosis in the group of children identified with other skin diagnoses was napkin dermatitis (10 children), followed by molluscum contagiosum (4 children). Of note was our finding that parents did not identify a skin condition as the primary presenting problem in their child, despite the fact that such a condition was detected by a paediatrician in 90 cases (36.3% of all children and 57.3% of children with a skin problem).

Table 2. Overall sensitivity and specificity and agreement with paediatrician diagnosis for IMCI algorithm used by nurses to assess and classify childhood skin problems in two clinics in Fiji.

Variable	No. of cases identified by paediatricians (%)	No. of cases identified by nurses		Sensitivity, in % (95% CI)	Specificity, in % (95% CI)	Agreement, in %		Kappa statistic
Detection of skin problem	157 (63.3)	163		98.7 (95.5–99.9)	91.2 (83.4–96.1)	96		0.91
Itch	97 (39.1)	88		87.1 (78.6–93.2)	88.7 (78.1–95.3)	87.7		0.75
Skin pain	41 (16.5)	45		80.5 (65.1–91.2)	89.5 (82.3–94.4)	87.1		0.68
Extensive redness or swelling	3 (1.2)	3		33.3 (0.8–90.6)	98.7 (95.3–99.8)	97.4		0.32
Eye swelling or redness	5 (2)	5		60 (14.7–94.7)	98.7 (95.2–99.8)	90.3		0.65
Localized swelling or redness	21 (8.5)	30		85.1 (63.7–97)	91 (84.9–95.3)	97.4		0.59
Lesion with crusts or pus	92 (37.1)	94		91.3 (83.6–96.2)	84.1 (72.7–92.1)	88.4		0.76
Papules	90 (36.3)	75		76.7 (66.6–84.9)	90.8 (81–96.5)	82.6		0.65
Patches	12 (4.8)	10		58.3 (27.7–84.8)	97.9 (94–99.6)	94.8		0.61
Severe skin infection	1 (0.4)	1		100 (2.5–100)	100 (97.6–100)	100		1
Periorbital cellulitis	3 (1.2)	4		75 (29.2–100)	99.3 (96.4–100)	99.4		0.85
Abscess or cellulitis	20 (8.1)	24		95 (75.1–99.9)	96.3 (91.6–98.8)	96.1		0.84
Impetigo	50 (20.2)	52		70 (55.4–82.1)	83.8 (75.4–90.3)	79.4		0.53
Infected scabies	55 (22.2)	61		89.1 (77.8–95.9)	88 (78–93.6)	88.4		0.75
Scabies	24 (9.7)	22		58.3 (36.6–77.9)	93.9 (88.3–97.3)	88.4		0.54
Fungal infection	13 (5.2)	11		69.2 (38.6–90.9)	98.6 (95–99.8)	96.1		0.73
Other	28 (11.3)	36		75 (55.1–93.2)	88.2 (81.3–93.2)	85.8		0.57

CI, confidence interval; IMCI, Integrated Management of Childhood Illness.

Sensitivity of algorithm applied by nurses

When compared with paediatricians’ diagnoses, the IMCI algorithm applied by IMCI-trained nurses showed a sensitivity of 98.7% (95% CI: 95.5–99.9) for detecting a skin problem; the kappa statistic was 0.91, indicative of excellent agreement (Table 2). This sensitivity changed little in patients whose primary presenting complaint was not a skin problem (sensitivity: 97.8%; 95% CI: 92.3–99.7).

When the nurses’ and the paediatricians’ assessments of skin problems were compared, sensitivity of the IMCI algorithm used by the nurses was > 75% for all aspects of assessment except extensive redness (33.3%), eye swelling (60%) and patches (58.3%). Extensive redness and eye swelling are important assessment features for the classification of severe skin infection and periorbital cellulitis. Importantly, the 1 child with a severe skin infection and all 3 children with periorbital cellulitis were classified correctly by the nurses.

When the nurses’ classification of skin problems was compared with the paediatricians’ diagnoses, the sensitivity of the IMCI-trained nurses’ algorithm was 75% or over for five of eight classifications, the specificity for all aspects of the classification was > 88%, and the kappa statistic was ≥ 0.73 for five of eight classification categories. The sensitivity for classifying impetigo was only 70%, but in some cases the nurses identified infected scabies, whereas the paediatrician assigned two discrete classifications of scabies and impetigo. After including infected scabies along with impetigo, the sensitivity rose to 92% (95% CI: 80.8–97.8). Sensitivity for classifying scabies alone was poor (58.3%); after including infected scabies along with scabies, sensitivity rose to 75% (95% CI: 53.3–90.2).

More than one feature of skin disease was detected on assessment in 73 children, and more than one classification was found in 35. The sensitivity and specificity of the nurses’ algorithm when compared with paediatricians’ diagnoses were 91.4% (95% CI: 85.5–95.5) and 96.3% (95% CI: 90.8–99), respectively, for the identification of two or more features in the examination portion of the assessment. The sensitivity and specificity in identifying two or more classifications were 96.8% (95% CI: 92.7–99) and 90.2% (95% CI: 82.2–95.4), respectively.

Variability among nurses

Because the number of children seen by each nurse differed, we calculated sensitivity, specificity and agreement for all categories for all nurses. Table 3 shows data for identification of a skin problem and for classification of impetigo. Overall, the uneven distribution of nurse examiners did not seem to be a major source of bias, although this effect was difficult to assess for the less common classifications because of the small numbers involved.

Table 3. Sensitivity, specificity and agreement with paediatrician diagnosis for IMCI algorithm applied by nurses in two clinics in Fiji to identify skin problems and classify impetigo among children.

Nurse	No. children seen		Identification of a skin problem									Impetigo
			Sensitivity, in %		Specificity, in %		Agreement, in %		Kappa			Sensitivity, in %		Specificity, in %		Agreement, in %		Kappa
A	39		95.8		86.7		92.3		0.84			66.7		90.0		87		0.5
B	9		100		75		88.9		0.77			75		100		80		0.55
C	18		100		66.7		94.4		0.77			66.7		88.9		80		0.57
D	18		100		100		100		1			40		100		75		0.44
E	19		100		100		100		1			66.7		100		91		0.74
F	15		100		100		100		1			100		100		100		1
G	18		100		100		100		1			75		85.7		66		0.61
H	55		97.4		75		90.9		0.77			75.0		61.5		82		0.32
I	17		100		100		100		1			50		75		83		0.25
J	40		100		100		100		1			80		88.2		86		0.6
Total	248		98.7		91.2		96.0		0.91			70.0		83.8		79		0.53
Mean	24.8		99.3		90.3		96.7		0.9			69.5		88.9		83		0.56

IMCI, Integrated Management of Childhood Illness.

Discussion

Almost two-thirds (63.6%) of children presenting to the IMCI clinic for any complaint had a skin problem diagnosed by a paediatrician. However, a skin problem was the primary presenting complaint among fewer than half of these children with a skin problem (42.7%). This concurs with unpublished and published findings by other researchers in Fiji⁴ and suggests that many children with skin conditions in Fiji do not present for medical care.

The most common skin problems in the children in our study were infected scabies (22.2%), impetigo (20.2%), non-infected scabies (9.7%) and abscess or localized cellulitis (8.1%). Other skin problems not included in the algorithm accounted for 11.3% of all skin problems.

Overall, the IMCI skin algorithm was found to be a robust tool when used by IMCI-trained nurses. High sensitivity and strong agreement with paediatricians’ diagnoses were found when nurses applied the IMCI skin algorithm, even in the group of children whose primary presenting complaint was not a skin problem. Detecting skin conditions in this group is important because a large proportion of children have a skin problem as an incidental finding. Assessment by the IMCI-trained nurses showed high sensitivity for classifying children with the most serious diagnoses (severe skin infection, periorbital cellulitis, abscess and cellulitis), as well as children with infected scabies, which was the most common diagnosis made by the paediatricians.

The major flaw in the algorithm appeared to be its low sensitivity for classifying non-infected scabies and its borderline sensitivity for classifying impetigo and fungal infections. The lower sensitivity for classifying non-infected scabies was probably due in part to lower sensitivity in identifying papules, which can be subtle. More clinical practice during the training may help improve recognition. However, the IMCI-trained nurses tended to classify children with discrete lesions of scabies and impetigo as having infected scabies, rather than as having impetigo and scabies separately. This error could be rectified by removing the classification of infected scabies from the algorithm. In this way, a child having impetigo alone would be classified as having impetigo; a child having non-infected scabies alone would be classified as having scabies; and a child having both impetigo and scabies would be classified as having both and be treated for each, whether or not there was secondary bacterial infection of the scabies lesions or the scabies and impetigo lesions were discrete. This classification system would achieve the aims of the algorithm and also be simpler.

The lower sensitivity of the IMCI algorithm applied by nurses for classifying fungal infections reflected lower sensitivity in classifying scaly patches. This problem could be remedied by emphasizing the issue in the training workshop, particularly by providing more clinical exposure.

The design of our study was similar to that of studies that validated the original IMCI clinical algorithms; that is, we compared the classification made by primary health-care workers trained in the use of an algorithm with diagnoses made by expert paediatricians. We consider this method appropriate in view of the clinical nature of the diagnosis of common childhood skin conditions in tropical countries. Unfortunately, for logistical reasons we were unable to measure inter-rater reliability between the two paediatricians and among the 10 nurses, although variability in the number of children seen by the nurses did not appear to be a major source of bias.

This study was not designed to assess treatment or follow-up. We have assumed that if a correct classification were made, appropriate treatment would be prescribed as per the algorithm. Given the adherence to the algorithm found in this study (data not shown), appropriate treatment would likely follow the corresponding classification. Our study was not sufficiently powered to evaluate sensitivity for the less frequent classifications, such as severe skin infection and periorbital cellulitis. We therefore advise caution in interpreting the good performance of the algorithm for these conditions.

The IMCI skin algorithm will be useful in Fiji to ensure that children presenting for medical care, whether for skin conditions or not, are accurately diagnosed and treated by IMCI-trained nurses. However, there are probably many children with skin diseases who require treatment but do not present for care. Therefore, for the greater public health aim of controlling skin diseases in children in Fiji, broader public health interventions, including health education, health promotion and possibly community-based treatment and prevention programmes, require consideration.

The skin algorithm is not being used in countries other than Fiji but could be considered for inclusion in IMCI programmes in other countries. A decision on including a skin algorithm should be proportionate to the level of priority of skin diseases for the country in question, and this prioritization should, in turn, be based on competing causes of mortality and morbidity as well as on the burden of skin disease.

The skin algorithm may not be appropriate for countries where diseases causing high mortality are still a priority, but it may be appropriate in countries such as Fiji that have been able to reduce the prevalence of such diseases and where skin diseases are common. In such countries, an IMCI skin algorithm may be able to significantly reduce the burden of common tropical childhood skin diseases as well as mortality rates through the prevention of invasive bacterial infections, post-infectious renal disease and rheumatic heart disease. Because the skin algorithm has been designed for easy incorporation into the existing IMCI algorithm, and because the treatment options recommended are all inexpensive and widely available, the algorithm could easily be adapted for use in countries other than Fiji where IMCI exists and where the identification and treatment of skin diseases are a priority. ■