Quality Assurance of Rapid Diagnostic Tests for Malaria in Routine Patient Care in Rural Tanzania

Abstract

Histidine-rich protein II (HRP2)-based malaria rapid diagnostic tests (RDTs) have shown high sensitivity and specificity for detecting Plasmodium falciparum malaria in a variety of study settings. However, RDTs are susceptible to heat and humidity and variation in individual performance, which may affect their use in field settings. We evaluated sensitivity and specificity of RDTs during routine use for malaria case management in peripheral health facilities. From December 2007 to October 2008, HRP2-based ParaHIT-f RDTs were introduced in 12 facilities without available microscopy in Rufiji District, Tanzania. Health workers received a single day of instruction on how to perform an RDT and thick blood smear. Job aids, Integrated Management of Childhood Illness guidelines, and national malaria treatment algorithms were reviewed. For quality assurance (QA), thick blood smears for reference microscopy were collected for 2 to 3 days per week from patients receiving RDTs; microscopy was not routinely performed at the health facilities. Slides were stained and read centrally within 72 hours of collection by a reference microscopist. When RDT and blood smear results were discordant, blood smears were read by additional reference microscopists blinded to earlier results. Facilities were supervised monthly by the district laboratory supervisor or a member of the study team. Ten thousand six hundred fifty (10,650) patients were tested with RDTs, and 51.5% (5,488/10,650) had a positive test result. Blood smear results were available for 3,914 patients, of whom 40.1% (1,577/3,914) were positive for P. falciparum malaria. Overall RDT sensitivity was 90.7% (range by facility 85.7–96.5%) and specificity was 73.5% (range 50.0–84.3%). Sensitivity increased with increasing parasite density. Successful implementation of RDTs was achieved in peripheral health facilities with adequate training and supervision. Quality assurance is essential to the adequate performance of any laboratory test. Centralized staining and reading of blood smears provided useful monitoring of RDT performance. However, this level of QA may not be sustainable nationwide.

Introduction

Prompt, reliable diagnosis of malaria is essential to improving case management and monitoring trends in malaria control. Clinical diagnosis alone is not specific and results in inappropriate use of antimalarial drugs.^1–5 Increasing drug resistance to inexpensive antimalarial drugs and the higher cost of alternative medications has led to an increased focus on malaria diagnosis. Rapid diagnostic tests (RDTs) were developed to improve the quality of malaria diagnosis in resource-limited settings. Initial field trials of HRP2-based RDTs showed sensitivity and specificity of over 90% for Plasmodium falciparum malaria at parasite densities of > 200 parasites/µL.^6,7 However, recent product testing by the World Health Organization (WHO) found that just 6 of the 41 RDTs tested were able to achieve > 90% P. falciparum detection rates† at parasite densities of 200 parasites/µL and few of these products had been developed for large-scale production.⁸ Previously, we showed poor operational sensitivity of 64.8% with Paracheck Pf (Orchid Biomedical Systems, Mumbai, India) in routine use at health facilities in rural Tanzania. Sensitivity varied greatly between health facilities (range 18.8–85.9%) and was difficult to interpret because of poor slide quality from some facilities.⁹ On the basis of this experience, we developed an alternative strategy for RDT implementation and quality assurance (QA).

Current WHO recommendations for RDT QA include the following: 1) RDTs should be purchased from a manufacturer that follows good manufacturing practices, 2) each lot of RDTs should be tested on arrival in the country of use to ensure that the tests were not exposed to extreme temperatures or other conditions that may affect RDT performance, and 3) RDTs should be monitored in field use monthly through comparison of RDT results to reference microscopy.¹⁰ Field use recommendations are that each facility using RDTs should submit blood smears from 20 patients with positive RDT results and 20 patients with negative RDT results every month for QA. However, this may not be possible for health facilities working in areas of low malaria transmission intensity, as they may not have 20 positive RDT results in a month. There is no guidance provided on patient selection for RDT QA. These guidelines were not available when this project began and therefore blood smears were collected with greater frequency than what is currently recommended by WHO.

The Tanzanian National Malaria Control Program (NMCP) wants to expand malaria diagnostic capacity to reduce inappropriate use of the first-line antimalarial, artemether-lumefantrine. Microscopy is available at hospitals and most health centers in Tanzania, but not at the most peripheral health facilities (dispensaries). The NMCP plans to distribute RDTs to dispensaries for routine malaria diagnosis. ParaHIT-f (Span Diagnostics, Surat, India), ICT Malaria Pf (ICT Diagnostics, Cape Town, South Africa), and Paracheck Pf are registered for routine use in Tanzania. The RDTs have been introduced in operational research studies on the mainland and on the islands of Zanzibar. National Integrated Management of Childhood Illness (IMCI) guidelines state that all febrile children less than 5 years of age should be treated for malaria. National guidelines for malaria treatment and diagnosis have recently been revised to advise that patients testing negative by RDT should not receive antimalarial treatment, regardless of age. Because of the lack of local experience in using RDTs and the need to ensure adherence to current malaria treatment and IMCI guidelines, we provided training in RDT use and introduced them in 12 dispensaries in a rural district in Tanzania to gain experience that would help inform national policy regarding the routine use of RDTs in peripheral health facilities.

Materials and Methods

Location.

From December 2007 to October 2008, we introduced an HRP2-based RDT (ParaHIT-f) for suspected malaria cases in 12 peripheral health facilities without microscopy services in Rufiji District, Tanzania. Health facilities were chosen by convenience and location within a demographic surveillance site. Rufiji District is a rural setting with holoendemic malaria transmission located 178 km south of Dar es Salaam on the Indian Ocean. Over 200,000 inhabitants are served by 59 health facilities, including two hospitals. An estimated 89% of the district's population live within 5 km of a health facility and acute febrile illness, including malaria, is one of the leading causes of death in the district.¹¹ There is considerable variation in the transmission intensity in Rufiji District from year to year but the predominant species is P. falciparum with an average entomological innoculation rate (EIR) of 80 to 180 infective bites per year from 2002 to 2004 (Huho B, personal communication). The majority of residents are subsistence farmers.

Training.

Ninety-nine health workers and health facility volunteers were trained to perform RDTs at the 12 health facilities during the first month of implementation. Health facility volunteers were literate members of the community who acted as aides to the health workers on a routine basis and were supervised by the head nurse or another health staff member. Health workers and volunteers received a single day of instruction on how to perform an RDT and thick blood smear. Health workers and health facility volunteers were provided per diem on the day of training but received no other remuneration. Job aids, IMCI guidelines, and national malaria treatment algorithms were reviewed. Because of conflicting messages regarding testing in children less than 5 years of age and national IMCI guidelines, we instructed health workers not to test children under five and treat febrile episodes according to IMCI guidelines for this age group. Health workers were instructed to perform RDTs on all patients 5 years of age and older with fever or history of fever in the last 48 hours or in whom they suspected malaria. Facilities were supervised monthly by the district laboratory supervisor or a member of the study team who used a standardized checklist to monitor health worker RDT performance.

Laboratory procedures.

Health workers and volunteers were trained to perform ParaHIT-f rapid diagnostic tests, according to the manufacturer's instructions. For QA, thick blood smears for reference microscopy were collected for 2 to 3 days per week from patients receiving RDTs; microscopy was not routinely performed at the health facilities. Slides were collected weekly from health facilities by study data collectors. Slides were stained with 10% Giemsa for 30 minutes within 72 hours of collection and read centrally by a reference microscopist blinded to initial RDT results. The reference microscopist counted parasites against 200 white blood cells and examined 100 fields before declaring slides negative. When RDT and blood smear results were discordant, the blood smear was read by two additional reference microscopists blinded to earlier results.

Ethical review.

The evaluation was granted a non-research determination by the Ifakara Health Institute and the Centers for Disease Control and Prevention (CDC) institutional review boards. Ifakara's Institutional Review Board (IRB) requested verbal consent from patients through provision of a fact sheet and one was provided to all tested patients.

Data analysis.

Data and blood smears from each facility were collected weekly. Data collectors also measured the temperature in the stock room where RDTs were stored weekly. Data collectors summarized the number of patients seen, the number of positive and negative RDTs, and the treatments received. Data collectors also retrieved case-based data from log books kept at the health facilities to compare with individual RDT and blood smear results. The RDT results were compared with the reference microscopists' readings of thick blood smears to determine RDT sensitivity and specificity. Data were entered into an EpiInfo version 3.4.1 (Centers for Disease Control and Prevention, Atlanta, GA) database for descriptive analysis. SAS 9.1 (SAS Institute, Cary, NC) PROC GENMOD was used to perform log-binomial regression to model the change in RDT sensitivity with increasing parasite density.

Results

During the 10 months of implementation, 30,195 patients were seen at the 12 health facilities. Among all patients seen, 15,043 (49.8%) were diagnosed with malaria (clinical or laboratory-confirmed diagnosis). Among those with a malaria diagnosis, 9,292 (61.8%) were children less than 5 years of age, and 5,751 (38.2%) were adults and children aged 5 and older. Among 10,737 patients tested with RDTs, test results were recorded for 10,650 (99.2%) patients. Age was recorded for 10,599 (99.5%) of patients with test results, and among them 284 (2.7%) were less than 5 years of age, and 10,315 (97.3%) were aged 5 and older. Overall RDT sensitivity was 90.7% (range by facility 85.7–96.5%) and specificity was 73.5% (range by facility 50.0–84.3%) compared with reference microscopy. When the 44 positive blood smears with parasite densities less than 200 parasites/µL are excluded, sensitivity and specificity increase to 91.1% and 77.9%, respectively. Figure 1 shows monthly RDT sensitivity and specificity, and the percentage of positive RDTs and positive blood smears.

Figure 1.

Sensitivity, specificity, and percent positive blood smear of rapid diagnostic tests (RDTs) by month, Rufiji District, Tanzania, December 2007–October 2008.

Figure 1 also illustrates that there was a seasonal increase in the percentage of positive blood smears following the rainy season from April to July. Blood smear results were available for 3,914 patients, of which 40.1% (1,577/3,914) were positive for P. falciparum malaria. No non-falciparum species were identified by the reference microscopists. The percentage of positive blood smears ranged from a low of 14.4% in December 2007 to a high of 60.5% in June 2008, the peak of transmission. The same seasonal increase seen in blood smears is reflected in the percentage of positive RDTs, which ranged from a low of 22.3% in December 2007 to a high of 72.4% in June 2008. Sensitivity of RDTs increased with increasing parasite density (Table 1). Sensitivity for detection of parasite densities below the 200 parasites/µL was 79.5%, and decreased slightly to 72.0% for parasite densities of the 200 to 800 parasites/µL. Thereafter, RDT sensitivity increased with increasing parasite density and the overall trend was statistically significant (P < 0.001). The predictive value of a positive test was 73.0% with a false positive rate of 26.5%. The predictive value of a negative test was 93.0% with a false negative rate of 9.3%.

Table 1.

Rapid diagnostic test (RDT) sensitivity by parasite density, Rufiji District, Tanzania, December 2007–October 2008 (3,914 blood smears reviewed, 1,577 positive blood smears)

Number of positive blood smears (% of total)	Parasite density per 200 WBC*	Parasite density per µL	Sensitivity (%)
44 (2.8)	1–4	40–160	79.5
182 (11.5)	5–20	200–800	72.0
133 (8.4)	21–50	840–2,000	85.0
147 (9.3)	51–100	2,040–4,000	88.4
357 (22.6)	101–500	4,040–20,000	92.7
714 (45.3)	> 500	> 20,000	96.8
Total = 1,577 (100)	1–> 500	40–> 20,000	90.7

^*WBC = white blood cell.

Patients were also queried about recent use of antimalarial drugs. All tested patients were asked if they had received treatment of malaria in the 3 weeks before being tested. Health workers were instructed to refer patients who had recently been treated for malaria to the nearest referral health facility; however, 585 patients who had received recent treatment with antimalarials were tested with RDTs. Among these 585 patients 454 (78.8%) had a positive RDT result. One hundred forty-nine (25.5%) patients who had recently received treatment also had a blood smear taken for QA. Among these 149, 104 (69.8%) had positive blood smears and 45 (30.2%) had negative blood smears. One hundred (96.2%) of the 104 recently treated patients with positive blood smears had positive RDT results. Twenty-two (48.9%) of the 45 recently treated patients with negative blood smears had false positive RDT results.

There was a high degree of concordance among blood smear and RDT results. Among 3,914 blood smears read by the reference microscopists there were 1,431 true positive and 1,820 true negative RDT results, for 3,251 (83.1%) concordant results. Of the 663 discordant results, 517 (78.0%) were false positives and 146 (22.0%) were false negatives according to reference microscopy. Three hundred forty-five (52.0%) blood smears were reviewed by a second microscopist, and 159 were reviewed by a third reader. Approximately half of the discordant slides were not available for second or third readings because they were lost or damaged in transport. When unavailable for review the first microscopist's reading was kept, as this allowed the most conservative estimation of sensitivity and specificity. Among the 345 blood smears reviewed, there was agreement between the first and second readers on 260 (75.4%). Of the 85 (24.6%) blood smears with discordant readings between the first and second microscopists, a third reader reviewed 51 (60.0%). The third reader agreed with the first microscopist on 7 (13.7%) slides and agreed with the second microscopist on 44 (86.3%) slides.

Temperature was recorded weekly in the area of the health facilities where RDTs were stored. The average temperature recorded during the study period was 31.7°C (range 23–37°C).

Discussion

This project shows that successful implementation of malaria rapid diagnostic tests in peripheral health facilities is possible with adequate training, supervision, and QA. We provided a single day of training to health workers and health facility volunteers, including practical experience with performing RDTs and making blood smears. With appropriate supervision, these volunteers were able to successfully perform RDTs and prepare thick blood smears. Likewise, peripheral health facility staff members were able to successfully perform RDTs and prepare thick blood smears. Sensitivity and predictive value of a negative test were high at all participating health facilities, but specificity was variable and positive predictive value was relatively low. As recommended by national IMCI guidelines, few patients less than 5 years of age were tested with RDTs. In addition, few patients who had been treated with antimalarial drugs in the 3 weeks before presentation at the health facility were tested with RDTs. Strong adherence to national IMCI guidelines for the treatment of febrile children with antimalarials was noted. Supervisory visits found frequent use of job aides and adherence to standard operating procedures including proper disposal of sharps. On the basis of the high sensitivity measured across all sites and proper adherence to IMCI guidelines and testing protocols, this RDT implementation was successful in providing accurate, timely diagnosis of malaria.

Previously, we showed poor mean operational sensitivity of RDTs in health facilities with microscopy.⁹ In this earlier implementation, nine health facilities with microscopy services had highly variable performance in introducing RDTs. Job aides were underused, healthcare workers adhered poorly to testing protocols, blood smears obtained for QA were improperly stained, and many patients with negative test results continued to be treated presumptively with antimalarials based on clinical symptoms. Because laboratory technicians were asked to perform both RDTs and blood smears at facilities with relatively high volumes of patients, laboratorians may have been overly burdened at these facilities leading to poor compliance with testing protocols. Poor compliance with testing protocols and improper staining and storage of blood smears likely contributed to the low sensitivity measured in this initial implementation period. Therefore, in initiating our second attempt at routine implementation of RDTs, peripheral health facilities (dispensaries) were chosen because they did not have laboratory technicians, the facilities generally saw fewer patients per day, and they were the facilities that the NMCP planned to target for RDT distribution. Working with peripheral health facilities without laboratories provided an additional challenge for monitoring RDT use with QA based on microscopy. However, central staining of blood smears by the reference microscopist resulted in improved quality smears that were easier to read and this greatly improved slide readability for QA. Among the slides sent for review by second and third readers, approximately 10% had contaminants or were found to be of poor quality and half of the slides intended for review by second and third readers were lost or damaged. Such difficulties call into question the use of blood smear microscopy on a national scale for RDT QA.

Despite the overall success of this implementation, 585 patients who had received recent treatment with antimalarials were tested with RDTs. This resulted in inappropriate treatment of patients whose parasitemia had cleared but who had persistent HRP2 antigenemia and likely increased exposure to second-line antimalarials and severe malaria drugs, such as intravenous quinine, when they were not indicated. It may also decrease healthcare worker confidence in artemisinin-based combination therapy and other antimalarials if they incorrectly believe there to be a high rate of treatment failures. Healthcare workers must be trained not to use RDTs in patients who have recently been treated with antimalarials, and this must be reinforced during supervisory visits.

Among the 1,577 positive blood smears, there were 146 (9.3%) false negative RDT results. In 80 (54.6%) of these 146 false negative RDT patients parasitemia was ≤ 2,000 parasites/µL, which may indicate that malaria was unlikely to be the source of their acute febrile illness. Our implementation did not include follow-up of patients with false negative RDT results because second and third readings of blood smears were not timely enough to affect patient management. Therefore, we are unable to assess outcomes of patients with false negative RDT results who did not receive antimalarial treatment. Few studies have examined the impact of false negative RDT results on patient outcomes. However, the few published studies have found that the majority of untreated patients came later for treatment of uncomplicated malaria and others cleared their parasitemia without treatment.^12–14 Health workers must be trained to inform patients with negative test results to return to the health facility if their symptoms have not resolved in 2 days. Further research on outcomes of patients with false negative RDT results is warranted.

Although RDTs were successfully introduced in these health facilities, the procedures we used to monitor RDT implementation are unrealistic for national implementation. We used one full-time reference microscopist and two data collectors, retrieved blood smears and data weekly from the health facilities, and provided transportation support for the district laboratory supervisor to monitor 12 health facilities. With over 7,000 health facilities in Tanzania, this level of support is not practical for widespread implementation. Because microscopy-based QA requires functional microscopes, quality stain and staining techniques, technical expertise in reading smears, and appropriate storage and transportation of slides from peripheral facilities, it is unlikely that microscopy-based QA could be implemented on a national scale. Future research in RDT implementation should focus on testing QA strategies that are less labor intensive, costly, and more widely implementable. The WHO recommendation to obtain blood smears with 20 positive and 20 negative RDTs should be tested in a variety of transmission settings and other QA strategies employing positive controls, polymerase chain reaction, or other diagnostic methods should be explored to ensure reliable performance of RDTs in field conditions.

Disclaimer: The findings and conclusions in this report are those of the author(s) and do not necessarily represent the views of the Centers for Disease Control and Prevention.