Comparison of Routine Health Management Information System Versus Enhanced Inpatient Malaria Surveillance for Estimating the Burden of Malaria Among Children Admitted to Four Hospitals in Uganda

Abstract

The primary source of malaria surveillance data in Uganda is the Health Management Information System (HMIS), which does not require laboratory confirmation of reported malaria cases. To improve data quality, an enhanced inpatient malaria surveillance system (EIMSS) was implemented with emphasis on malaria testing of all children admitted in select hospitals. Data were compared between the HMIS and the EIMSS at four hospitals over a period of 12 months. After the implementation of the EIMSS, over 96% of admitted children under 5 years of age underwent laboratory testing for malaria. The HMIS significantly overreported the proportion of children under 5 years of age admitted with malaria (average absolute difference = 19%, range = 8–27% across the four hospitals) compared with the EIMSS. To improve the quality of the HMIS data for malaria surveillance, the National Malaria Control Program should, in addition to increasing malaria testing rates, focus on linking laboratory test results to reported malaria cases.

Uganda has made significant efforts to reduce the burden of malaria through scaling up of effective tools for malaria control, notably long-lasting insecticide-treated nets, indoor residual spraying of insecticides, treatment with artemisinin-based combination therapies, and intermittent preventive therapy in pregnancy.¹ Despite progress in these efforts, there is much work to be done to achieve the objective of a > 75% decrease in the malaria admission rate by 2015.^2,3

Demand for high-quality malaria surveillance data to monitor progress and evaluate the effectiveness of interventions is greater than ever before.⁴ However, malaria surveillance is often limited by non-specific case definitions and imperfect data collection tools and indicators.^3,5 The Ugandan National Malaria Control Program (NMCP) malaria surveillance relies on Health Management Information System (HMIS) data from monthly health facility summary forms. However, the usefulness of the HMIS data is limited by incomplete or delayed reporting, poor data quality, and lack of malaria-specific data based on laboratory confirmation.^6,7

To improve malaria surveillance in Uganda, the NMCP working with the Uganda Malaria Surveillance Program (UMSP) with support from the US President's Malaria Initiative implemented the enhanced inpatient malaria surveillance system (EIMSS) at six public hospitals.⁸ This program collects high-quality malaria data by emphasizing universal malaria laboratory testing for all hospitalized children and collecting and reporting laboratory-confirmed malaria cases using a computerized patient database. The program is limited to collection of data in the children's wards of participating hospitals. We evaluated the accuracy of the HMIS malaria data by using the EIMSS data as the gold standard.

The EIMSS was set up from April of 2010 to June of 2011 in four government-run hospitals: Tororo, Apac, Mubende, and Kambuga (Figure 1). Tororo, Apac, and Kambuga are general hospitals providing basic inpatient health services. Mubende was recently elevated to a regional referral hospital. Tororo District Hospital and Mubende Regional Hospital are in areas of high malaria endemicity (parasite rates in children 2–9 years old > 50%). Apac District Hospital is in an area that was previously highly endemic but has become moderately endemic (parasite rate = 30–50%) after the introduction of indoor residual spraying of insecticide. Kambuga District Hospital is in a highland area with relatively low endemicity (parasite rate < 20%).

Figure 1.

Study sites. Parasite rate is based on the estimated proportion of children 2–9 years of age with a positive blood smear based on cross-sectional surveys.

The EIMSS included (1) the use of a standardized structured medical record form (MRF) for the documentation of patient details (presenting symptoms and signs, laboratory test results, treatment administered, diagnosis, and final outcome on discharge); (2) laboratory testing for parasitemia using microscopy for all admitted children, regardless of presentation; and (3) training and routine supervision of hospital staff as well as supply of essential materials, like MRFs. Individual patient information captured on MRFs was entered into an Epi Info database (Centers for Disease Control and Prevention, Atlanta, GA). To increase malaria testing rates at the sites, the UMSP provided direct support to the laboratories in the form of training of microscopists and supplying essential materials and reagents required for malaria microscopy.

Data were transmitted approximately 2 weeks after the end of each month to a central location for cleaning, analysis, and reporting. The reported results included total number of admissions, number tested for malaria, and number of laboratory-confirmed malaria cases. For the HMIS, data were obtained from the HMIS monthly summary forms and included total admissions and numbers of malaria cases (final diagnosis of clinical malaria). Comparison between the EIMSS and the HMIS was limited to the children under 5 years old age group, because the EIMSS is limited to the children's ward.

The indicator of interest for the comparison between the HMIS and the EIMSS was the proportion of children admitted with malaria. For the HMIS, this indicator was calculated using the number of children given a clinical diagnosis of malaria as the numerator and the total number of admissions as the denominator. For the EIMSS, the number of children with laboratory-confirmed malaria was the numerator, and the number of children tested for malaria was the denominator. Data from both systems were analyzed for 12 months after the implementation of the EIMSS. For comparison purposes, the HMIS data from the 12-month period before the implementation of the EIMSS were also reviewed. Results are presented for only four of six sites where the EIMSS was introduced because of problems with the HMIS forms in two hospitals: the forms were destroyed at one site (Kabale) and not filled out at another site (Jinja) because of a staffing shortage. The proportions of malaria cases admitted during a 12-month period after the start of the program at each site were compared using the χ² test. Any P value < 0.05 was considered statistically significant.

During the 12-month period after the implementation of the EIMSS, the total numbers of admissions reported by the HMIS and the EIMSS were similar in three hospitals (Apac, Mubende, and Kambuga), but they were different at Tororo Hospital (HMIS: 4,385 versus EIMSS: 5,571) (Table 1). The proportion of admitted children who were laboratory-tested ranged from 94% to 98% at the four hospitals after the EIMSS implementation (Table 1). Through training, supply of essential reagents, and continuous supervision, the UMSP was able to maintain high testing rates at all four hospitals. This achievement suggests that near-universal testing for malaria in the inpatient setting is feasible beyond the sentinel sites.

Table 1.

Routine and enhanced surveillance reporting of pediatric malaria in four district hospitals in Uganda

Hospital	Start date	Routine HMIS surveillance data
		12-Month period before start date		12-Month period after start date		Enhanced surveillance data (12-month period after start date)
		Total admissions	Number of malaria cases* (% total admissions)	Total admissions	Number of malaria cases* (% total admissions)	Total admissions	Number tested (% total admissions)	Number of confirmed malaria cases† (% number tested)
Tororo Hospital	April of 2010	5,526	5,214 (94)	4,385	3,747 (85)	5,657	5,571 (98)	3,459 (62)
Kambuga Hospital	January of 2011	2,106	1,767 (84)	1,538	793 (52)	1,559	1,530 (98)	384 (25)
Mubende Hospital	April of 2011	2,215	1,444 (65)	2,930	1,581 (54)	2,890	2,717 (94)	1,020 (38)
Apac Hospital	June of 2011	2,115	1,429 (68)	1,925	904 (47)	2,110	1,927 (98)	754 (39)

Only data for children less than 5 years of age are included.

^*Based on a clinical diagnosis of malaria.

^†Laboratory confirmed (microscopy).

The HMIS reported significantly greater numbers of malaria cases and therefore, higher proportions of patients with malaria diagnosis compared with the EIMSS at all four hospitals. This is a reflection of the case definitions used by the two systems—clinical versus laboratory-confirmed (Table 1): Tororo (85% versus 62%, P < 0.001), Kambuga (52% versus 25%, P < 0.001), Mubende (54% versus 38%, P < 0.001), and Apac (47% versus 39%, P < 0.001). The differences between the two systems were relatively consistent month to month at two of the sites (Tororo and Kambuga) but inconsistent at the two other sites (Apac and Mubende) (Figure 2).

Figure 2.

Trends in malaria admission rates 12 months before and 12 months after the start of an enhanced malaria surveillance program at four public hospitals in Uganda. Vertical bars represent the month when the enhanced surveillance program started collecting data.

Interestingly, the HMIS reported a statistically significant reduction in the proportion of children under 5 years of age diagnosed with malaria at all four hospitals after the implementation of the EIMSS: Tororo (94% versus 85%, P < 0.001), Kambuga (84% versus 52%, P < 0.001), Mubende (65% versus 54%, P < 0.001), and Apac (68% versus 47%, P < 0.001) (Figure 2 and Table 1). Although these differences could have been caused by real changes in the burden of malaria, these decreases were more likely because of the greater use of laboratory test results when making malaria diagnoses after the implementation of the EIMSS.

There are several limitations to this study that could influence the interpretation of the results. First, the study was conducted in hospitals serving as malaria sentinel sites, where the quality of the HMIS could have been influenced by the ongoing sentinel surveillance work. Second, we do not know whether the quality of the HMIS at two sites not included in the analysis was substantially different from the quality at the other sites. Third, because the EIMSS is limited to children under 5 years of age, we do not know about the performance and the quality of data reporting for older age groups.

Despite these limitations, we show that the proportions of children with malaria reported by the HMIS were statistically significantly greater than those reported by the EIMSS. Most of these differences are likely caused by the differences in case definitions used by the HMIS and the EIMSS.^3,9 Even when malaria laboratory tests are performed, the HMIS is not designed to link test results with diagnosis, which is a fundamental limitation of current HMIS malaria data. In addition, the reliance of the HMIS on paper forms and manual tabulation could contribute to observed differences.¹⁰

Malaria diagnosis based on clinical observation without laboratory confirmation is a common practice in Uganda for many reasons. First, until recently, the World Health Organization recommended presumptive treatment of suspect malaria in endemic countries.¹¹ Second, clinicians lack skills and capacity to confirm alternate diagnosis. Third, health workers continue to believe in the benefits of presumptive diagnosis, even in the setting of negative test results.^12,13 The practice of diagnosing malaria without laboratory confirmation coupled with the non-specific nature of this diagnosis have resulted in the HMIS reporting cases that were not actually malaria, which may consistently overreport the true burden of malaria.^14,15

As a result, evaluating the impact of interventions using the HMIS data is likely to be problematic.^16–18 For example, as malaria diagnostics are scaled up in Uganda, decreases in the number of malaria cases reported by the HMIS may be misinterpreted as a reduction in the burden of malaria rather than simply improvements in the accuracy of the diagnosis of malaria.¹⁹ Our data suggest that, for the HMIS data to be useful in monitoring malaria trends, there is urgent need to increase the proportion of malaria diagnoses reported based on laboratory confirmation and link the laboratory results to reported malaria cases.²⁰

Disclaimer: The opinions expressed herein are those of the authors and do not necessarily reflect the views of the US Agency for International Development.