A Comparative Study of the Management Decisions by IMNCI Algorithm and by Pediatricians of a Teaching Hospital for the Children Between 2 Months to 5 Years

Abstract

Background:

Integrated management of neonatal and childhood illness (IMNCI) is already operational in many states of India, but there are very few studies in Indian scenario comparing its validity and reliability with the pediatricians’ decisions.

Objectives:

1) To compare the IMNCI decision with the decision of pediatricians; 2) to assess the significance of multiple presenting symptoms in the IMNCI algorithm.

Materials and Methods:

The study was conducted among the sick children between 2 months to 5 years presented in pediatric department from January to March 2009. The IMNCI decision was compared with pediatrician's decisions by percent agreement, Kappa and weighted Kappa with the aids of SPSS version 10.

Results:

The overall diagnostic agreement between IMNCI algorithm and pediatrician's decisions was 36.64%, (Kappa 0.16 and weighted Kappa 0.29) with 51.15% over diagnosis and 12.21% under diagnosis. The importance given by IMNCI algorithm in cases of multiple presenting symptoms was also reflected as it was evident that 37.50% children presented with three symptoms were categorized as red, whereas it was 28.57% and 11.67% for those presented with two and one symptom, respectively, (P < 0.0001). Pediatricians also gave importance for presence of multiple symptoms by considering 50% as admissible in the group presented with three symptoms, 30.16% in the group presented with two symptoms, and 16.67% in the group presented with only one symptom. The association was also statistically significant (P = 0.018).

Conclusion:

Diagnostic discordance is seen mainly due to over diagnosis of all fever cases as malaria. Importance of presence of comorbidities was also reflected.

Introduction

Annually, over 10 million children in low and middle income countries including India die before they reach their fifth birthday and 7 out of 10 of these deaths are due to some common treatable or preventable conditions such as diarrh eal dehydration, acute respiratory infections, measles, malaria, and malnutrition.(1) Delay in seeking treatment and lack of quality care in health facilities are important cause of death in such conditions.(2) Moreover, according to the World Bank Report 1993, in situations where laboratory support and clinical resources are limited, a syndromic approach with comprehensive health care strategies is more realistic and cost effective. It also has the potential to make the greatest impact on the global burden of disease.(3) Considering this situation, during the year 1992, WHO, in collaboration with UNICEF and some other agencies, institutions and individuals, responded to the challenge by adopting a strategy known as Integrated Management of Childhood Illness (IMCI).(4)

The Government of India along with other experts also recognized the need for a strategy like IMCI. Moreover, as in Indian scenario, neonatal mortality constitutes a substantial (64%) proportion of “under 5 mortality,” all neonates starting from the day of birth were included in the strategy. Thus for Indian version, IMCI was adapted as Integrated Management of Neonatal and Childhood Illness (IMNCI).(5)

IMNCI was already operational in many of the states of India, but there are not much study testing its validity and reliability regarding the decisions arrived at by using the algorithm. However, a study was done by Prof. HPS Sachdev et al. at Maulana Azad Medical College, during 1999–2000 testing the validity of IMCI algorithm, with the age group 2 months to 5 years.(6) But there is hardly any evidence of published study in West Bengal though IMNCI was already implemented in Purulia district from the year 2005. In the present study, validity and reliability of IMNCI algorithm were assessed in comparison with the decision of senior pediatricians in Medical College, Kolkata. The significance of the multiple presenting symptoms was also assessed.

Materials and Methods

The study was an observational, cross-sectional one done in the Department of Pediatrics, Medical College, Kolkata. The study was conducted among the sick children in the age group 2 months to 5 years, presented with a fresh episode of illness. Residence at Kolkata Municipal Corporation (High malaria zone) was also a criterion for inclusion in the study. From the previous records of 3 years, it was found that number of children in that age group fulfilling the above-mentioned criteria was around 525 and 25% of the annual attendance was considered feasible to be taken as study subject. Thus, the number of the study subjects was 131. The study was continued till the required number of children was available, which was around 3 months. All the children fulfilling the criteria were included (Census). No sampling was done. Mother or other caregivers were interviewed with a predesigned, pretested; semistructured schedule and the children were examined according to the IMNCI algorithm using the Physician's chart booklet for age group 2 months to 5 years. They were classified and categorized according to the severity using the color code, such as red (urgent referral), yellow (treatment in the same health facility), or green (treatment at home). The color code was recorded.

After this assessment those children were sent to pediatricians and as in any tertiary care hospital, all of them were examined by the pediatricians. Pediatricians recorded the presenting symptoms, clinical features, provisional diagnosis, and their decision for management in the OPD tickets, from where the presenting symptoms and their decision for management were recorded. The decisions of the pediatricians were compared with the decision by the IMNCI algorithm. IMNCI classified the symptoms according to color code such as red, yellow, or green. When there were multiple classifications with varying severity, the color code of the most severe classification was considered as the appropriate one for the particular study subject and recording was done accordingly.

In the present study the decisions of the pediatricians were also categorized in three ways as 1) admission and treatment, 2) treatment with advice for follow-up or investigation, and 3) only home treatment without any advice for follow-up or investigation.

The decisions of IMNCI algorithm were compared with the decisions of the pediatricians regarding management. In the present study, the red category was considered equivalent with admission and treatment, yellow category with treatment with advice for follow-up or investigation, and green category was considered comparable with the pediatrician's decision, only home treatment. This comparison resulted in three different ways such as over diagnosis, under diagnosis, and percent agreement.

Over diagnosis was considered when IMNCI classification was red but pediatricians did not decide to admit the cases. They either advised for follow-up or investigation or for only home treatment. It was also considered when IMNCI classification was yellow but pediatricians did not provide any advice for follow-up or investigation. They advised for only home treatment.

Under diagnosis was considered when pediatricians decided to admit the cases but IMNCI coded them as yellow or green. It was also considered when IMNCI classification was green, but pediatricians advised for follow-up or investigation. In the other conditions, where both the decisions were comparable percent agreement was considered [Figure 1].

Figure 1.

Figure showing the comparison of IMNCI classification pediatricians’ decision

In the above example the cases present in the cells a, e and i were considered to show agreement. The cases in the cells b, where IMNCI decisions were urgent referral (color code red), but pediatricians considered investigation, treatment and follow up, c, where IMNCI decisions were urgent referral (color code red), but pediatricians considered home treatment only and f, where IMNCI decisions were treatment in the same health facility (color code yellow) but pediatricians considered home treatment only were considered to show over diagnosis. The cases in the cells d, where IMNCI decisions were treatment in the same health facility (color code yellow) but pediatricians considered admission and treatment, g, where IMNCI decisions were treatment at home (color code green) but pediatricians considered admission and treatment and h, where IMNCI decisions were treatment at home (color code green but pediatricians considered investigation, treatment and follow up were considered to show under diagnosis.

To exclude the agreement, which was merely due to chance along with percent agreement, Kappa and Weighted Kappa(7,8) values were also determined. For analysis of data, the help of SPSS version 10 windows compatible was taken.

Kappa test excludes the extent of the agreement, which was due to chance. Kappa is a ratio, where the numerator is the improvement of observed agreement over chance agreement and the denominator is the maximum possible improvement over chance agreement. The result of Kappa has some arbitrary divisions, which are as follows:(7,8)

• Under 0.2 is negligible agreement

• 0.2 to 0.4 is minimal agreement

• 0.4 to 0.6 is fair agreement

• 0.6 to 0.8 is good agreement

• And over 0.8 is excellent agreement

Weighted Kappa was considered more appropriate test for comparison of IMNCI and pediatricians’ decisions, as those categories are not merely nominal. Rather the categories are considered ordinal as the disagreement between green category of IMNCI algorithm and pediatrician's decision for admission was more than disagreement between yellow category and pediatrician's decision for admission.(7,8) In weighted Kappa, each cell in a row is weighted in accordance with how near it is to the cell in that row that includes the concordant items.

This weightage was applied to each cell value to obtain a new weighted cell value. When the observed and expected agreement was calculated from those weighted cell value, subsequent Kappa ratio was the weighted Kappa ratio. The result of weighted Kappa also has same arbitrary divisions as simple Kappa.(7,8)

Results and Discussion

The number of study subjects was 131 and among them 34.98% were in the age group 2 months to 12 months, 29.55% were between 13 months and 24 months and 35.47% were in the age group 25 months to 60 months. 57.64% of the study subjects were male and 42.36% young infants were female. The socioeconomic status of the study subjects has been assessed by applying the scale suggested by B.G. Prasad. It was evident that 38.92% were in grade IV and 35.96%, 11.82%, 11.80%, and 1.48% were in grade III, II, V, and I, respectively.

In the present study, the children residing at Kolkata Municipal Corporation were only included; therefore, all the children included in the study were of high malaria zone and accordingly all fever cases were classified as malaria.

All the study subjects in the age group were assessed according to the IMNCI algorithm, classified and categorized using the color code.

As per IMNCI algorithm 21.37% children were categorized as red, 61.83% as yellow, and 16.80% as green [Table 1]. Pediatricians advised admission for 25.19%, follow-up or investigation for 14.50%, and only home management for 60.31% [Table 1].

Table 1.

Distribution of study subjects according to comparison between decision of IMNCI algorithm and pediatrician's decisions (n = 131)

82.15% of the cases of red category, 11.11% of yellow category, and 4.55% cases in green category were considered admissible by pediatricians [Table 1]. When both the decisions were compared it was shown that overall agreement was 36.64% with simple kappa value 0.16 (negligible agreement)(7) and weighted Kappa value 0.29 (minimal agreement)(8) [Table 1]. Among the disagreements 51.15% was due to over diagnosis and 12.21% was due to under diagnosis. In the study, done by Sachdev et al.,(6) in the high malaria zone, there was total agreement of 54.2%, with 29.6% over diagnosis and 24.6% under diagnosis. In that study, pediatricians took the decision after all relevant investigation, like blood slide examination for malaria, whereas in the present study the decisions taken by them as per the provisional diagnosis based on only clinical examination; and this might be a cause of difference in the result.

Under diagnosis was present in 16 cases among which there were nine cases, classified as yellow, where pediatricians advised admission. In this group, pediatricians provisionally diagnosed two cases as diarrhea with severe dehydration, one case each as pneumonia, wheezy bronchitis, spastic diplegia, malaria, infective endocarditis, measles, and extrahepatic portal hypertension. Pediatricians also admitted one case, with provisional diagnosis of insect bite but it was classified by the algorithm in the green category. In six other conditions of under diagnosis, pediatrician's decision was investigation, treatment, and follow-up, but IMNCI algorithm classified those in the green category. In this category, pediatricians provisionally diagnosed two cases as breast-fed stool, one case as diarrhea with some dehydration, and one case as diarrhea without dehydration, and one case each as Koch's lymphadenitis and urinary tract infection.

Over diagnosis were present in 67 cases, among which 62 were categorized as yellow and it was reflected that diagnosis of any fever case as malaria is an important cause of over diagnosis. As among these 62 cases there were 42 cases, classified by the algorithm as malaria and categorized as yellow. But in all those cases pediatricians advised only home treatment with diagnoses of upper respiratory tract infection for 37 and pneumonia for 5 cases. There were other 25 cases, categorized by the algorithm as yellow, where pediatricians advised only home treatment. In this group, there were seven cases diagnosed by pediatricians as diarrhea with no dehydration, four cases diagnosed as wheezy bronchitis, three cases as worm infestation, two cases as protein energy malnutrition, and one case each were diagnosed as boil and congenital cyanotic heart disease. In two other cases, pediatricians’ diagnoses were healthy baby but due to presence of pallor of the palmer crease, they were classified as anemia and included in the yellow category. In other group of over diagnosis, there were three cases where IMNCI classification was red and pediatricians advised investigation, treatment, and follow-up. In this group, pediatricians diagnosed provisionally two cases as febrile convulsion, which was classified by the algorithm as very severe febrile disease and one case as upper respiratory tract infection, where severe malnutrition was the cause for inclusion in the red category. In another two conditions, pediatricians diagnosed one case as upper respiratory tract infection and another as pneumonia and advised home treatment, but IMNCI coded those in red category also due to presence of severe malnutrition.

Along with the decisions for management, the presenting symptoms as recorded by the pediatricians in the OPD or emergency tickets were assessed. There were multiple presenting symptoms for each study subjects [Table 2]. 45.80% of the study subjects presented with single presenting symptom, 48.09% presented with two presenting symptoms, and 6.11% presented with three presenting symptoms. The mean number of presenting symptoms was 1.60 ± 0.60 [Table 2]. In the study done by Sachdev et al.,(6) the mean number of morbidities in high malaria zone was 2.2 ± 1.1; therefore, in both the studies children are shown to be presented with multiple presenting symptoms.

Table 2.

Distribution of study subjects according to the number of presenting symptoms (n = 131)

The common presenting symptoms were cough and cold (57.25%), fever (39.69%), diarrhea (14.50%), vomiting (9.92%), convulsion (9.92%), respiratory distress (6.11%), worm infection (3.05%), etc. [Table 3]. Similarly in the study done by Weber et al.,(9) the common presenting symptoms in 2 months to 5 years age group were fever (66%), cough (36%), diarrhea (29%), chest pain (28%), vomiting (24%), skin problems (15%), abdominal pain (13%), difficulties in feeding (8%), headache (5%), and eye problems (5%).

Table 3.

Distribution of study subjects according to presenting symptoms (Multiple response) (n = 131)

According to Perkins et al.,(10) in a study at Kenya, fever, cough, and diarrhea were the main presenting symptoms.

IMNCI is already proved to give due importance to children with multiple conditions of ill health.(3) It was also reflected in the present study. 37.50% was categorized as red in the group presented with three symptoms, 28.57% in the group presented with two symptoms, and 11.67% in the group presented with only one symptom. The association was statistically significant (P < 0.0001) [Table 4].

Table 4.

Distribution of study subjects according to number of presenting symptoms with comparison between decision of IMNCI algorithm and pediatrician's decision (n = 131)

Also when pediatrician's decisions were considered for the same study subjects, it was found that presence of multiple symptoms was given importance. It was found that 50% was considered for admission in the group presented with three symptoms, 30.16% in the group presented with two symptoms, and 16.67% in the group presented with only one symptom. The association was also statistically significant (P = 0.018) [Table 4].

Therefore, presence of multiple presenting symptoms was given due importance both in the algorithm and by the pediatricians.

Conclusion

Therefore, in conclusion, it could be mentioned that IMNCI algorithm had shown 36.64% diagnostic agreement with the decisions of pediatricians and among the disagreements, 51.15% was due to over diagnosis and 12.21% was due to under diagnosis. Classification of any fever as malaria in high malaria zone remained a broad area for diagnostic discordance. As rapid diagnostic kit is available in primary health care settings, it can be utilized for the fever cases to avoid consideration of all fever cases as malaria. However, many other studies are required in other settings of West Bengal. For severe and serious conditions, over diagnosis was considered desirable, as it will help to identify more cases from the primary care setting for referral to the higher health facility. Presence of co morbidities was also given due importance in the IMNCI algorithm as well as by the pediatricians. This study was done in the setting of a tertiary care hospital and further study is required, particularly in primary health care setting.