Abstract
To improve healthcare quality for hospitalized patients with malaria in Benin, a feasible and valid evaluation method is needed. Because observation of inpatients is challenging, chart abstraction is an attractive option. However, the quality of inpatient charts is unknown. We employed three methods in five hospitals to assess 11 signs of malaria and severe disease: 1) chart abstraction (probability sample of inpatients), 2) chart abstraction compared to interviews of inpatients and health workers (HWs), and 3) abstraction from charts of recently discharged inpatients compared to interviews with HWs. Method 1 showed that of 473 malaria signs (from 43 charts), 178 (38%, 95% confidence interval 24–51%) were documented. Method 2 showed that 96% (45 of 47) of documented signs were valid. Method 3 suggests that 65% (36 of 55) of non-documented signs were assessed (but not documented) by HWs. Chart abstraction was feasible and documented data were valid, but results should be interpreted cautiously in consideration of low levels of documentation.
Introduction
Malaria is a leading cause of morbidity and mortality in Benin,1 and interventions promoting correct case management of severe cases can prevent malaria deaths.2 Health facility surveys, in both outpatient and inpatient settings, can be used to assess the quality of care that health workers (HWs) provide to patients with malaria, and survey results can identify specific problems that can be addressed by health facility directors, supervisors, and program managers to improve care. Key aspects of the quality of malaria care include the correct assessment, diagnosis, treatment (including drug choice, timing, and dosage), and counseling. Methods to assess outpatient quality of care in resource-limited settings include direct observation of consultations by either conspicuous surveyors3–7 or simulated clients8; HW interviews, which might include clinical vignettes5; patient or guardian interviews9,10; and chart abstractions.6,10
The nature of the inpatient experience, however, makes it more difficult to assess healthcare quality. For example, it is not logistically feasible to observe or simulate an entire inpatient hospital admission, which could last days or weeks. Analyzing data abstracted from patients' medical records is a common method for determining the quality of care in many high-income countries, and medical records are potentially the most accessible data source for an inpatient assessment. However, we were concerned that medical charts of discharged patients in Benin might i) be difficult to find because of disorganized record keeping, ii) be difficult to read because of illegible handwriting and physical deterioration of the charts, iii) have documentation gaps that make it challenging to reconstruct what happened during the hospitalization, and iv) contain inaccuracies, even for information that was documented. Indeed there are a limited number of assessments published on the quality of care of hospitalized patients with severe febrile illness, including malaria.7,10–17
To evaluate efforts to improve malaria case management in Benin, surveys of outpatients and inpatients are being planned or are currently underway. We sought to identify an affordable hospital survey methodology that would provide reasonably valid data on inpatient care. Thus, we first conducted an evaluation of methods to assess the quality of inpatient care. In consideration of the potential limitations of chart abstraction outlined previously, the objectives of this assessment were to determine the i) feasibility of sampling and abstracting data from inpatient medical charts in Benin, ii) proportion of malaria signs that were documented in charts, and iii) validity of data documented in charts.
Methods
Design and data collection.
We conducted assessments using three methods: first, chart abstraction from a probability sample of inpatients to reconstruct events during a hospitalization for the evaluation of healthcare quality; second, chart abstraction plus interviews of inpatients and their HWs to fill in gaps in the chart; and third, abstraction from charts of recently discharged inpatients plus interviews with their HWs to provide a reason why documentation did not occur (e.g., the HW assessed the sign but did not document it, or the HW did not assess the sign). As the process of collecting data by multiple methods was labor intensive, we chose a small, manageable sample of five hospitals in three departments of southern Benin that were purposively selected based on geographical location, size, and type; the selected hospitals included three regional hospitals (Hôpital de Zone), one central departmental hospital (Center Hospitalier Départemental), and one national university hospital (Center National Hospitalier Universitaire). Care at these hospitals is typically delivered by nurses and physicians, some of whom are specialists. Field work for this assessment was conducted from July 5 to 20, 2010, corresponding to the end of Benin's principal rainy season. There were no known epidemics of severe seasonal-illness occurring in Southern Benin during this time; the meningitis attack rate in this region was low.18
In June 2010, staff of the selected hospitals was informed that an assessment team would make an unannounced visit in the next month. Each hospital was assessed over 2 days. Adult, pediatric, and medical maternity wards were eligible. Surgical and delivery wards were excluded, as these were considered unlikely places to find patients with severe malaria. Data were collected by one team, with one supervisor and two survey nurses. Survey nurses were trained over 5 days in chart abstraction and the interview-based methods. Before the team departed from each hospital, the supervisor reviewed all forms and, in cooperation with the survey nurses, ensured that they were corrected for omissions, clarity, and accuracy. After completing the work, the supervisor provided constructive feedback on improving chart documentation to the hospital director. Chart abstraction and interviews for all methodologies were conducted using standardized data collection instruments (designed for this evaluation) to record information on patient history, laboratory tests, diagnoses, and medications administered. Data were double-entered into EpiInfo 3.5.1 (Centers for Disease Control and Prevention; Atlanta, GA) databases, and the analysis was performed with SAS 9.3 (SAS Institute Inc., Cary, NC).
The assessment protocol was approved by the Centers for Disease Control and Prevention (CDC) internal review board and the Benin ethical review committee. Informed consent was obtained from all adult participants and from the parents or guardians of minors.
Definitions.
The following terms were defined using the Benin national malaria treatment guidelines.19 Signs of illness severity included: impaired consciousness, inability to eat or drink, uncontrollable vomiting, seizures, jaundice, pallor of mucous membranes or anemia, dark-colored urine, difficulty breathing, hypoglycemia, decreased urine output, shock, pulmonary edema, or abnormal bleeding. Severe malaria (diagnostic criteria for hospitals) was any patient with fever and parasitological confirmation of Plasmodium falciparum, plus one or more signs of illness severity.
We defined the following assessment-specific terms. Medical charts were documents describing a patient's clinical encounter that occurred during the eligible hospitalization (e.g., forms or paper on which clinical notes, diagnoses, prescriptions, and administered medications were written; laboratory registers; and printed output from laboratory tests or imaging studies). Documented meant a clearly written description in the chart explaining the presence (documented present) or absence (documented absent) of a clinical feature or test result. Suspected malaria was defined as any of the following documented in the chart or observed in the admitted patient: history of fever, measured fever (≥ 37.5°C), pallor of mucous membranes, anemia by hemoglobin (Hb) measurement (Hb < 11 g/dL), a malaria test ordered, an antimalarial prescribed, or a diagnosis of malaria. Confirmed malaria was a suspected-malaria case with a documented positive test result from a malaria blood smear or rapid diagnostic test. Malaria signs were the subset of malaria-associated symptoms and signs of severity that can be recalled from a patient history or caregiver's physical examination and do not involve laboratory testing. These include: history of fever, measured temperature, pallor of mucous membranes, altered consciousness, convulsions, difficulty breathing, jaundice, inability to eat/drink (includes uncontrollable vomiting), dark-colored urine, decreased urine output, and abnormal bleeding. Normal was defined as signs, symptoms, or measurements found within healthy or expected parameters. Abnormal described signs, symptoms, or measurements that are outside of healthy or expected parameters.
Method 1: Probability sampling and abstraction from charts of discharged (or deceased) patients.
Before the field-work, the assessment team requested that the selected hospitals collect and organize all inpatient records from June 2010 (a payment of about $20 USD was given for this work). A probability sample of inpatients was selected at each hospital. We estimated that the survey team could sample and abstract 10 charts during the hospital evaluation; thus 10 charts per hospital were set as the target for sampling. The gold standard source for identifying admissions was the inpatient ward registers, in which all admissions were sequentially recorded by hospital staff. In any given hospital, all eligible patients/charts registered during June 2010 had the same chance of being enrolled. However, selection probabilities were different in each hospital. The steps to probability sampling were i) identifying eligible wards; ii) collecting ward inpatient registers; iii) identifying and chronologically organizing the inpatient medical charts for June 2010; iv) estimating the expected number of missing records by comparing the collected charts to the number of admissions according to the ward registers; v) calculating the number of charts to sample (the total number of target charts increased by the estimated percentage of missing charts); vi) determining the sampling fraction and choosing a random start number between 1 and the skip interval (the nth chart systematically sampled); vii) selecting charts using the ward registers; viii) collecting the selected charts (no substitution was made for missing charts); and ix) abstracting all sampled charts, regardless of whether the target number was reached or exceeded. If malaria test results were missing from the chart, the assessment team then reviewed laboratory registers in an attempt to retrieve test results. The statistical analysis involved simple counts and percentages. The 95% confidence intervals (CIs) were estimated for the percentage of malaria signs that were documented using the SAS SURVEYFREQ procedure, which accounts for the clustering of patients within hospitals.
Method 2: Chart abstraction plus interviews with currently admitted patients and their HWs.
In each hospital, the assessment team attempted to interview one currently admitted patient (or his/her parent or guardian) from each of two age groups (< 5 years and ≥ 5 years of age), and a HW who directly treated those patients. Inclusion criteria were: being admitted in the past 48 hours with a febrile illness, and the availability of both the patient's medical chart and the attending HW. The inpatients in an eligible ward were systematically assessed according to their proximity to the main ward door to identify the first patient who satisfied the inclusion criteria. Once a patient was selected and informed consent was obtained, we did the following: i) interviewed the parent or guardian as the reference source on the patient history for all malaria signs, ii) abstracted the chart, iii) determined concordance between the patient interview and chart abstraction, and iv) interviewed the HW to clarify missing or discordant information between the patient report and the medical chart.
If malaria test results were missing from the chart, the assessment team reviewed laboratory registers. During the course of the patient interview and chart review, the assessment team evaluated if the patient's care needed modification (i.e., if a previously unrecognized sign of disease severity warranted more aggressive treatment). If treatment modification required an essential medication that was out of stock, the team then provided it free of charge.
Method 3: Chart abstraction plus HW interview for recently discharged patients.
The target for each hospital was to abstract the chart and interview the attending HW for one recently discharged patient in each of two age groups (< 5 years and ≥ 5 years of age). Inclusion criteria were being admitted to an eligible ward with a febrile illness, being discharged within the previous 72 hours, and the availability of both the medical chart and the attending HW. Once a chart was identified, we did the following: i) abstracted the chart, ii) determined if a sign was not documented or if information was unclear, and iii) interviewed the HW on all missing or unclear data elements. If malaria test results were missing from the chart, the assessment team reviewed laboratory registers. The purpose of the method 3 HW interview was to determine if the non-documented malaria sign was assessed but not recorded, or not assessed. An additional advantage of method 3 is that it allowed us to evaluate completed hospitalizations, thus avoiding bias from the Hawthorne effect.20
Table 1 provides an overview of the three methodological approaches used in this evaluation.
Table 1.
Methodological approach
| Method 1 | Method 2 | Method 3 | |
|---|---|---|---|
| Purpose: | |||
| Existing, standard method | X | ||
| Fill in chart gaps from patient and health worker (HW) interviews, and assess chart validity | X | ||
| Fill in chart gaps from HW interviews, assess the entire hospitalization, avoid bias from the Hawthorne effect | X | ||
| Approach: | |||
| Probability sampling | X | ||
| Chart abstraction | X | X | X |
| Patients were not interviewed (charts were abstracted after discharge) | X | X | |
| Patients were interviewed about all malaria signs | X | ||
| HWs were interviewed about discordances with patient interview and chart | X | ||
| HWs were interviewed about non-documented malaria signs in the chart (classified as assessed but not documented or not assessed) | X | ||
Results
Probability sampling and abstraction from charts of discharged (or deceased) inpatients (method 1).
In June 2010, in the five selected hospitals, 1,383 inpatients were admitted to eligible wards. The target number of charts to abstract was 50 (10 for each of 5 hospitals). During the sampling procedures, the survey team estimated that the overall percent of missing charts was 16%; to account for these missing charts a total of 60 records were sampled from ward registers. Among the 60 sampled, 45 charts (75%) were retrieved and abstracted; 28 (62%) were from pediatrics wards, 11 (24%) from medicine wards, and 6 (13%) from maternity wards. Among the 45 patients whose charts we abstracted, 43 had documented signs or symptoms that fit criteria for suspected malaria. Of the 43 patients with suspected malaria, 24 (56%) were < 5 years of age.
Documentation of malaria signs and classification of inpatients by disease severity and malaria test results (method 1).
Altogether, 473 malaria signs were assessed (43 charts of patients with suspected malaria × 11 signs per chart). Only 178 (38%, 95% confidence interval [CI] 24–51) malaria signs were documented in patient charts (Table 2). Among all 11 malaria signs, only three were documented in > 80% of charts: history of fever, measured temperature, pallor of mucus membranes. Of the 43 charts, 29 (67%) documented a measured temperature of ≥ 37.5°C. Despite poor documentation of individual malaria signs, chart abstraction detected the presence of at least one sign of severity, which could be recalled in a patient history or physical exam, among 32 of 43 (74%) suspected malaria charts.
Table 2.
Documentation of malaria signs (method 1)
| Malaria sign | Presence documented n (%) | Absence documented n (%) | Total documented (presence + absence) n (%, 95% CI*) | Not documented/not clear n (%) |
|---|---|---|---|---|
| History of fever | 35 (81) | 1 (2) | 36 (84, 70–97) | 7 (16) |
| Measured temperature | 40 (93) | 0 (0) | 40 (93, 74–100) | 3 (7) |
| Pallor of mucous membranes | 29 (67) | 9 (21) | 38 (88, 65–100) | 5 (12) |
| Altered consciousness | 7 (16) | 3 (7) | 10 (23, 0–53) | 33 (77) |
| Convulsions | 3 (7) | 3 (7) | 6 (14, 0–39) | 37 (86) |
| Difficulty breathing | 12 (28) | 9 (21) | 21 (49, 14–83) | 22 (51) |
| Jaundice | 1 (2) | 11 (26) | 12 (28, 0–68) | 31 (72) |
| Inability to eat/drink | 4 (9) | 2 (5) | 6 (14, 0–31) | 37 (86) |
| Dark colored urine | 5 (12) | 2 (5) | 7 (16, 7–26) | 36 (84) |
| Diminished urine output | 0 (0) | 1 (2) | 1 (2, 0–9) | 42 (98) |
| Abnormal bleeding | 0 (0) | 1 (2) | 1 (2, 0–9) | 42 (98) |
| Total malaria signs, N = 473 | 136 (29) | 42 (9) | 178 (38, 24–51) | 295 (62) |
CI = confidence interval.
Malaria diagnostic tests were ordered for 35 (81%) of the 43 suspected malaria inpatients. Of these 35 tested patients, 27 (77%) had a result documented in the chart or laboratory register. Of those with documented results, 16 (59%) were positive for malaria.
In summary, among all 45 patients with abstracted charts: 2 (4%) did not have suspected malaria; 4 (9%) had what appeared to be uncomplicated malaria (positive malaria test without a sign of severity documented as present, although documentation could be incomplete); 12 (27%) had severe malaria (positive malaria test plus at least one sign of severity documented as present); 9 (20%) had a severe non-malarial illness (negative malaria test plus at least one sign of severity documented as present); 2 (4%) had what appeared to be a non-severe non-malarial illness (negative malaria test without a sign of severity documented as present, although documentation could be incomplete); 5 (11%) had what appeared to be suspected malaria without severe signs (although documentation could be incomplete) and no test result; and 11 (24%) had a severe illness suspected to be malaria without a test result.
Supplemental Table 1 provides information on the variation among records at the facility-level, specifically related to the documentation of malaria signs and the confirmation of malaria cases.
Feasibility of chart abstraction.
We found that obtaining a probability sample of charts for abstraction was feasible. However, an underestimation of missing charts occurred in hospitals with poorly organized medical records, resulting in an overall discrepancy between the 16% estimated missing rate and the 25% actual missing rate. The reasons for the inaccuracy of the missing chart estimation were that sometimes multiple pieces from one chart appeared to be the medical record of more than one patient, or charts from other wards were occasionally mixed together.
At each hospital, it took ∼0.5–0.75 days for one supervisor and two survey nurses to collect, organize, and sample the previous month's inpatient charts. It took an additional day for a two-person team to abstract about 10 inpatient charts. Altogether, a study team of one supervisor and two survey nurses collected, sampled, and abstracted 10 charts during a two-day facility visit. We estimate that during a two-day visit it might be possible for a dedicated team of three survey nurses and one supervisor to abstract a total of 20–30 charts.
Validity of documented malaria signs (method 2).
To assess the validity of documented malaria signs, we compared data from patient interviews and HW interviews to those obtained through chart abstraction for 11 patients with a febrile illness. Six patients were < 5 years of age. Altogether 121 signs were assessed (11 patients × 11 signs/patient), of which 47 (39%) were documented either present or absent in the chart (Table 3). Of these documented signs, 45 (96%) were confirmed through patient and HW interviews.
Table 3.
Validity of documented malaria signs (method 2)
| Signs | Presence documented n (%*) | Absence documented n (%*) | Total documented n (%*) | Validated n (%†) |
|---|---|---|---|---|
| History of fever | 8 (73) | 0 | 8 (73) | 8 (100) |
| Measured temperature | 10 (91) | 0 | 10 (91) | 10 (100) |
| Pallor of mucous membranes | 6 (55) | 5 (45) | 11 (100) | 11 (100) |
| Altered consciousness | 2 (18) | 0 | 2 (18) | 2 (100) |
| Convulsions | 3 (27) | 0 | 3 (27) | 3 (100) |
| Difficulty breathing | 1 (9) | 4 (36) | 5 (45) | 3 (60) |
| Jaundice | 1 (9) | 3 (27) | 4 (36) | 4 (100) |
| Inability to eat/drink | 0 | 0 | 0 | 0 |
| Dark colored urine | 2 (18) | 1 (9) | 3 (27) | 3 (100) |
| Diminished urine output | 0 | 0 | 0 | 0 |
| Abnormal bleeding | 1 (9) | 0 | 1 (9) | 1 (100) |
| Total malaria signs, N = 121 | 34 (28‡) | 13 (11‡) | 47 (39‡) | 45 (96†) |
Percent of patients.
Percent of total documented.
Percent of total malaria signs.
Characterizing non-documented malaria signs as normal or abnormal (method 2).
To assess what information patient and HW interviews could add to better understand the documentation gaps (i.e., the true values of the 61% non-documented malaria signs [see Table 3, column 4 showing 39% of signs were documented, thus 61% were non-documented]), we classified the non-documented signs as normal or abnormal according to patient and HW interviews (Table 4). Of the 74 non-documented signs in the charts, patient and HW interviews revealed that 61 (82%) were normal in the patient; and for 12 (16%) of the non-documented signs, the patient had an abnormal condition or a sign of severity (specifically: history of fever, altered consciousness, difficulty breathing, inability to eat/drink, dark urine, diminished urine output, and abnormal bleeding). One (1%) non-documented sign (measured temperature) could not be classified as normal or abnormal because interviews confirmed that no thermometer was available, thus no temperature was measured.
Table 4.
Classification of non-documented malaria signs as normal or abnormal (method 2)
| Sign | Chart | Patient and HW interviews | |
|---|---|---|---|
| Non-documented n (%*) | Identified abnormal condition n (%†) | Confirmed normal condition n (%†) | |
| History of fever | 3 (27) | 2 (67) | 1 (33) |
| Measured temperature | 1 (9) | NA‡ | NA‡ |
| Pallor of mucous membranes | 0 (0) | 0 (0) | 0 (0) |
| Altered consciousness | 9 (82) | 1 (11) | 8 (89) |
| Convulsions | 8 (73) | 0 (0) | 8 (100) |
| Difficulty breathing | 6 (55) | 2 (33) | 4 (67) |
| Jaundice | 7 (63) | 0 (0) | 7 (100) |
| Inability to eat/drink | 11 (100) | 3 (27) | 8 (72) |
| Dark colored urine | 8 (73) | 2 (25) | 6 (75) |
| Diminished urine output | 11 (100) | 1 (9) | 10 (91) |
| Abnormal bleeding | 10 (91) | 1 (10) | 9 (90) |
| Total malaria signs, N = 121 | 74 (61§) | 12 (16†) | 61 (82†) |
Percent of patients.
Percent of non-documented.
HW and patient interviews confirmed that in one instance a thermometer was not available therefore temperature was not measured. We did not classify this event as normal or abnormal.
Percent of total malaria signs.
Feasibility of patient and HW interviews (method 2).
It took one survey nurse ∼0.5–0.75 days per hospital to complete the chart abstraction and patient and HW interviews for this method (total of two charts, two patient interviews, and two HW interviews per hospital). The feasibility of this method was low: challenges included finding eligible patients with a HW available for an interview, language barriers for patient interviews, and a time consuming methodology.
Chart abstraction plus HW interviews of recently discharged patients (method 3).
To understand the underlying reasons for non-documentation, we interviewed HWs regarding missing information in 10 charts from recently discharged patients to determine i) if the HW assessed the patient for the sign but failed to document it (in this case, the HW was asked to recall the value of the malaria sign); or ii) the HW did not assess the sign. Through chart abstraction and interview, a total of 110 malaria signs were evaluated (10 patients × 11 signs/patient) (Table 5). Five charts were from patients < 5 years of age. Half of the malaria signs (N = 55) were documented present or absent in the chart. For the 55 non-documented signs, values for 36 (65%) were gained through the HW interview because the HW reported having assessed the sign but did not document the result in the chart. Of these 36 signs, 32 (89%) were recalled to be normal. The HW interviews identified 4 (11%) abnormal conditions for signs of danger/severity (including: pallor of mucous membranes, difficulty breathing, and inability to eat or drink). Altogether, these results show that the HWs assessed 91 (83%) of 110 malaria signs.
Table 5.
Assessment of malaria signs with chart abstraction and HW interviews (method 3)
| Malaria sign | Non-documented signs n (%*) | Non-documented sign was assessed but not recorded n | Assessed non-documented sign was normal n | Assessed non-documented sign was abnormal n |
|---|---|---|---|---|
| History of fever | 1 (10) | 1 | 1 | 0 |
| Measured temperature | 0 (0) | 0 | 0 | 0 |
| Pallor mucous membranes | 1 (10) | 1 | 0 | 1 |
| Altered consciousness | 5 (50) | 4 | 4 | 0 |
| Convulsions | 7 (70) | 4 | 4 | 0 |
| Difficulty breathing | 4 (40) | 3 | 2 | 1 |
| Pulmonary edema | 5 (50) | 2 | 2 | 0 |
| Jaundice | 4 (40) | 3 | 3 | 0 |
| Inability to eat/drink | 10 (100) | 8 | 6 | 2 |
| Dark colored urine | 8 (80) | 6 | 6 | 0 |
| Diminished urine output | 10 (100) | 4 | 4 | 0 |
| Total malaria signs, N = 110 | 55 (50†) | 36 (65‡) | 32 (89§) | 4 (11§) |
Percent of patients.
Percent of total malaria signs.
Percent of non-documented signs.
Percent of non-documented sign was assessed but not recorded.
Feasibility of HW interviews (method 3).
Chart abstraction and HW interviews took one survey nurse ∼0.5 days to complete per hospital (two chart abstractions and two HW interviews per hospital). Although this method offered the potential advantage that the inpatient admission had been completed and the HW interview could fill potential gaps in documentation, we found that the feasibility of this method was low. It was challenging to identify HWs to interview because those who administered care for recently discharged patients were often not working the day of the assessment because they had worked the preceding day or two.
Quality of care.
Best results from the three methods were used to determine quality of care for inpatients in this evaluation. Supplemental Tables 2 and 3, and Figure 1 show the quality of care assessment.
Discussion
Malaria-control program managers who want to improve the quality of care for inpatients with severe malaria might consider a low-cost survey to evaluate quality using chart abstraction-based methods. However, this approach has limitations, most notably, the uncertain availability and validity of information in patients' medical records. Both interview methods that we examined have potential advantages compared with chart abstraction alone. Interviewing currently admitted patients could confirm the presence or absence of signs and symptoms that are missing from the charts; although, for determining quality of care, this approach is less ideal as the patient's hospital stay is not yet complete. Alternatively, conducting chart abstraction on recently discharged patients and interviewing HWs has the potential benefit of evaluating the quality of patient care for an entire hospitalization, and the HW could fill in gaps in chart documentation. The purpose of this study was to evaluate the feasibility and the limitations of chart abstraction for assessing inpatient malaria-specific care, and the potential benefit of adding patient and HW interviews to chart abstraction-based methods in Benin.
Using probability sampling of patients discharged (or deceased) the previous month, approximately three-quarters of sampled charts were available for abstraction. To improve the sampling success rate, we recommend that if charts at a hospital are poorly organized, the estimation of missing charts should then be increased to improve the likelihood that the target number of abstracted charts is achieved.
Of the information documented in the charts, our analysis suggests that 96% was valid. Thus, the principal concern is interpreting missing information. According to the three methods assessed in this study, 50–62% of all malaria signs were not documented. However, the non-documented signs were not always a problem for assessing quality of care because 74% of charts documented at least one sign of severity, and only one sign is necessary to classify a case as severe malaria.
Although patient and HW interviews from method 2 revealed that the non-documented malaria signs were often normal, 16% of the non-documented signs were abnormal. The HW interviews from method 3 suggested that 17% of malaria signs were never assessed by HWs. These levels of abnormal and non-assessed malaria signs are too high to assume that non-documented signs are always normal.
Unknown malaria test results, which occurred in about one-third of all medical records, are problematic. First, from a methodological perspective, without this information it is not possible to evaluate malaria quality of care; thus, sample size calculations for a national inpatient survey should be designed to take into account this level of missing information. Second, from a quality of care perspective, the lack of a documented test result, by itself, indicates poor quality care that should be improved because it suggests that HWs are making treatment decisions without timely results of malaria diagnostic tests.
We designed this study to be a methodological assessment to examine the feasibility and general data quality attributes of the three abstraction-based methods; our results will assist in planning a national inpatient survey in Benin. It was not possible for us to conduct a study with sufficient sample sizes on the three approaches with the resources available to us; thus, we were limited by small sample size, and the resulting 95% CI presented on the documentation of malaria signs are wide. Although patient and HW interviews were informative to fill in gaps that were missing in the charts, because of feasibility and logistical limitations, we do not recommend adding patient or HW interviews to a large, national inpatient survey (at least in Benin). The HW interviews for recently discharged patients were difficult to coordinate because HWs who cared for these patients were often off-duty during the time of the assessment. In addition, these interviews were lengthy and sometimes uncomfortable because the study team asked about omissions in the chart. Finally, we were not able to assess the extent of recall or social desirability bias. These biases could have resulted in an overestimation of the proportion of malaria signs assessed. These interviews added some missing information but at a substantial cost in survey time.
From this evaluation of chart abstraction-based methods, we conclude that abstraction from inpatient medical charts in Benin is feasible and that the data documented in charts are largely valid. However, in considering the extent of non-documented and non-assessed malaria signs, we recommend introducing and evaluating a standardized medical admission form as an intervention. An integrated, standardized admission form could improve case management for all patients, not just those with malaria. Similar admission forms were used at Queen Elizabeth Central Hospital, Blantyre, Malawi,21,22 and implemented to improve pediatric hospital care in Kenya.10,23 Such tools can serve dual purposes: first, to function as a job aid for HWs, facilitating routine assessment of all signs of severity associated with malaria, thus improving the quality of care for inpatients with malaria. Second, it can improve data quality for inpatient medical records, thereby facilitating long-term monitoring and evaluation for the quality of inpatient malaria care. To improve the quality of inpatient medical care for those patients with malaria, programs should consider investing in strategies to ensure that HWs routinely assess patients for all signs of malaria and severe febrile illness, and improve chart documentation, in particular the documentation of malaria signs and laboratory test results.
Supplementary Material
ACKNOWLEDGMENTS
We gratefully acknowledge the participation of the patients, HWs, and hospital administrators in this assessment. We also thank Bruno Aholoukpe from the National Malaria Control Program, Cotonou, Benin for his assistance in developing the instruments used in this assessment. Finally, we acknowledge the hard work, dedication, and patience of the survey team.
Footnotes
Financial support: Funding for the survey was provided by the United States President's Malaria Initiative and the Government of Benin.
Authors' addresses: Kimberly E. Mace, Abdou Salam Gueye, Michael F. Lynch, and Alexander K. Rowe, Centers for Disease Control and Prevention, Center for Global Health, Malaria Branch, Division of Parasitic Diseases and Malaria, Atlanta, GA, E-mails: igd3@cdc.gov, for6@cdc.gov, wzl4@cdc.gov, and axr9@cdc.gov. Esther M. Tassiba, Population Services International, Cotonou, Benin, E-mail: mtassiba@psi.org.
References
- 1.WHO . World Health Statistics. Geneva: World Health Organization; 2012. http://apps.who.int/iris/bitstream/10665/44844/1/9789241564441_eng.pdf?ua=1&ua=1 Available at. Accessed May 16, 2013. [Google Scholar]
- 2.Biai S, Rodrigues A, Gomes M, Ribeiro I, Sodemann M, Alves F, Aaby P. Reduced in-hospital mortality after improved management of children under 5 years admitted to hospital with malaria: randomized trial. BMJ. 2007;335:862. doi: 10.1136/bmj.39345.467813.80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Rowe AK, de Leon GF, Mihigo J, Santelli AC, Miller NP, Van-Dunem P. Quality of malaria case management at outpatient health facilities in Angola. Malar J. 2009;8:275. doi: 10.1186/1475-2875-8-275. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Leonard K, Masatu MC. Outpatient process quality evaluation and the Hawthorne Effect. Soc Sci Med. 2006;63:2330–2340. doi: 10.1016/j.socscimed.2006.06.003. [DOI] [PubMed] [Google Scholar]
- 5.Leonard KL, Masatu MC. The use of direct clinician observation and vignettes for health services quality evaluation in developing countries. Soc Sci Med. 2005;61:1944–1951. doi: 10.1016/j.socscimed.2005.03.043. [DOI] [PubMed] [Google Scholar]
- 6.Hermida J, Nicholas DD, Blumenfeld SN. Comparative validity of three methods for assessment of the quality of primary health care. Int J Qual Health Care. 1999;11:429–433. doi: 10.1093/intqhc/11.5.429. [DOI] [PubMed] [Google Scholar]
- 7.Reyburn H, Mwakasungula E, Chonya S, Mtei F, Bygbjerg I, Poulsen A, Olomi R. Clinical assessment and treatment in pediatric wards in the north-east of the United Republic of Tanzania. Bull World Health Organ. 2008;86:132–139. doi: 10.2471/BLT.07.041723. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Rowe AK, Onikpo F, Lama M, Deming MS. Evaluating health worker performance in Benin using the simulated client method with real children. Implement Sci. 2012;7:95. doi: 10.1186/1748-5908-7-95. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kahabuka C, Kvale G, Hinderaker SG. Factors associated with severe disease from malaria, pneumonia and diarrhea among children in rural Tanzania: a hospital-based cross-sectional study. BMC Infect Dis. 2012;12:219. doi: 10.1186/1471-2334-12-219. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Ayieko P, Ntoburi S, Wagai J, Opondo C, Opiyo N, Migiro S, Wamae A, Mogoa W, Were F, Wasunna A, Fegan G, Irimu G, English M. A multifaceted intervention to implement guidelines and improve admission pediatric care in Kenyan district hospitals: a cluster randomized trial. PLoS Med. 2011;8:e1001018. doi: 10.1371/journal.pmed.1001018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Achan J, Tibenderana J, Kyabayinze D, Mawejje H, Mugizi R, Mpeka B, Talisuna A, D'Alessandro U. Case management of severe malaria–a forgotten practice: experiences from health facilities in Uganda. PLoS ONE. 2011;6:e17053. doi: 10.1371/journal.pone.0017053. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.English M, Esamai F, Wasunna A, Were F, Ogutu B, Wamae A, Snow RW, Peshu N. Assessment of inpatient pediatric care in first referral level hospitals in 13 districts in Kenya. Lancet. 2004;363:1948–1953. doi: 10.1016/S0140-6736(04)16408-8. [DOI] [PubMed] [Google Scholar]
- 13.Nolan T, Angos P, Cunha AJ, Muhe L, Qazi S, Simoes EA, Tamburlini G, Weber M, Pierce NF. Quality of hospital care for seriously ill children in less-developed countries. Lancet. 2001;357:106–110. doi: 10.1016/S0140-6736(00)03542-X. [DOI] [PubMed] [Google Scholar]
- 14.Djadou KE, Komlangan A, Balaka B, Dokounor D, Gbadoe AD, Atakouma D, Agbere AD, Kessie K, Assimadi K. Management of children severe malaria in Tsevie's (Togo) hospital regional center. Arch Pediatr. 2006;13:1555–1556. doi: 10.1016/j.arcped.2006.09.010. [DOI] [PubMed] [Google Scholar]
- 15.Zeidan Z, Kojal H, Habour A, Nowary K, Hashim F, Awadelkarim M. Clinical and epidemiological features of severe malaria in children in four hospitals in Sudan. East Mediterr Health J. 2006;12:783–791. [PubMed] [Google Scholar]
- 16.Walter ND, Lyimo T, Skarbinski J, Metta E, Kahigwa E, Flannery B, Dowell SF, Abdulla S, Kachur SP. Why first-level health workers fail to follow guidelines for managing severe disease in children in the Coast Region, the United Republic of Tanzania. Bull World Health Organ. 2009;87:99–107. doi: 10.2471/BLT.08.050740. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Ibeh CC, Ele PU, Ukpabi OJ. Assessment of doctors' management of severe malaria in urban centres in Eastern Nigeria. Afr J Med Med Sci. 2004;33:149–153. [PubMed] [Google Scholar]
- 18.Meningitis Weekly Bulletin, Inter-Country Support Team - West Africa, May 30–July 31, 2010. World Health Organization, Regional Office for Africa. http://www.meningvax.org/epidemic-updates.php Available at. Accessed June 24, 2013.
- 19.Politique Nationale De Lutte Contre Le Paludisme et Cadre Strategique de Mise En Oeuvre, Benin . Benin: Programme National de Lutte Contre le Paludisme, Direction Nationale de la Protection Sanitaire. Republique Du Benin: Ministere De La Sante Publique; 2005. [Google Scholar]
- 20.Roethlisberger FJ, Dickson WJ. Management and the Worker: An Account of a Research Program Conducted by Western Electric Company, Hawthorne Works, Chicago. Cambridge, MA: Harvard University Press; 1939. [Google Scholar]
- 21.WHO Improving Quality of Paediatric Care in Small Hospitals in Developing Countries; Report of a meeting, Geneva, 19–21 June 2000, World Health Organization; 2000. Available at. Accessed May 15, 2013. [Google Scholar]
- 22.Taylor TE. Caring for children with cerebral malaria: insights gleaned from 20 years on a research ward in Malawi. Trans R Soc Trop Med Hyg. 2009;103((Suppl 1)):S6–S10. doi: 10.1016/j.trstmh.2008.10.049. [DOI] [PubMed] [Google Scholar]
- 23.Mwakyusa S, Wamae A, Wasunna A, Were F, Esamai F, Ogutu B, Muriithi A, Peshu N, English M. Implementation of a structured pediatric admission record for district hospitals in Kenya–results of a pilot study. BMC Int Health Hum Rights. 2006;6:9. doi: 10.1186/1472-698X-6-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.WHO . Guidelines for the Treatment of Malaria. Second edition. Geneva: World Health Organization; 2010. p. 194.http://whqlibdoc.who.int/publications/2010/9789241547925_eng.pdf Available at. Accessed November 11, 2010. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.