Benefits of Lot Testing to Improve the Quality of Malaria Rapid Diagnostic Tests in India

ABSTRACT.

Since 2010, malaria rapid diagnostic tests (RDTs) are widely used to detect malaria. The Indian Council of Medical Research–National Institute of Malaria Research performed lot testing (LT) according to WHO procedures since 2016. Lot testing is performed to evaluate the lot-to-lot variation in performance of malaria RDTs. Four sets of positive quality control (QC) panels for P. falciparum (Pf) and P. vivax (Pv) and 10 negative panels tested RDTs. RDTs were reported as pass, failed, or deferred on the basis of WHO criteria. In the past 5 years, 275 lots containing 15,488 RDT kits for malaria diagnosis were subjected to LT. The monovalent RDTs (n = 1,216), based on either Pf histidine rich protein 2 (HRP2) or Pan-Plasmodium lactate dehydrogenase (Pan-pLDH) antigens, showed 90.4% sensitivity and 100% specificity, whereas RDTs based on HRP2 + Pan-pLDH or HRP2 + pLDH (n = 13,924) had sensitivity 95.6% and specificity 99.5%, respectively. RDTs based on PfHRP2 + Pv-pLDH + Pan-pLDH (n = 348) had 100% sensitivity and specificity. In a comparison between HRP2 + pLDH or HRP2 + Pan-pLDH to HRP2 + pLDH + Pan-pLDH RDTs, it was found that the sensitivity of PfHRP2 with Pan-pLDH RDTs (n = 2,382) was only 83%. Of the 275 lots analyzed, 15 lots of PfHRP2 with Pan-pLDH were deferred. The QC panel for Pf revealed a faint Pan band in the tested lots, which is a cause for concern. The results of deferred lots were reported to concerned government agencies. Quality-compromised RDTs may lead to an incorrect diagnosis. It is critical to have a QC system in place for effective malaria management.

INTRODUCTION

Globally malaria was responsible for an estimated 247 million cases in 2021.¹ The WHO South East Asia Region accounts for approximately 2% of the global malaria burden. India accounted for 79% of malaria cases in the region.¹ The National Center for Vector-Borne Disease Control (NCVBDC; formerly known as the National Vector Borne Disease Control Program) in India set a deadline for eliminating malaria by 2030 throughout the country. Early diagnosis and complete treatment are the core interventions.² Diagnosis of malaria by examining peripheral blood smears by light microscopy remains the gold standard in India. However, depending on geographic settings, rapid diagnostic tests (RDTs) and microscopy are used to diagnose malaria.³ In remote areas where microscopy facilities are not available, RDTs are used. Between 2003 and 2004, the NCVBDC introduced large-scale RDT kits for malaria diagnosis in India for areas where microscopy is unavailable. Initially, only monovalent histidine-rich protein 2 (Pf-HRP2) RDTs were introduced for detecting Plasmodium falciparum (Pf).⁴ With the deployment of these Pf-specific RDTs, there was no reduction of microscopy load because microscopy examination was still required for RDT negative results. In addition, these RDTs missed Plasmodium vivax (Pv) cases. Because RDTs aid in the prompt diagnosis for management of malaria cases, the National Malaria Program distributed 267 million bivalent RDTs in 2020.⁵ Many factors may impact on the quality of malaria RDTs, such as coating monoclonal antibodies on immunochromatography strips, packaging, shipping, storage, and transport temperature, which may affect diagnostic performance. In addition, inadequate or unclear instruction in product inserts may lead to errors in test-line interpretation, affecting a test result’s accuracy.⁶ The WHO established a lot testing (LT) program for malaria RDTs. This program serves as an autonomous quality control (QC) system that aims to assess the variability in the performance of malaria RDTs across production lots. The primary objective of this plan is to guarantee the procurement and utilization of malaria RDT lots that adhere to the highest quality standards in countries affected by malaria. The laboratory conducts assessments of RDT batches by the protocols established by the WHO. Quality control samples contain standardized quantities of well-characterized strains of malaria parasites. To respond to the heterogeneous quality of malaria RDTs, the Indian Council of Medical Research–National Institute of Malaria Research (ICMR-NIMR), New Delhi, India, established a Malaria RDT Lot Testing Laboratory in 2009, which was subsequently audited by WHO to assess compliance with recommended WHO malaria RDTs quality assurance procedures in 2016.^7,8

The ICMR-NIMR and the ANDI Center of Excellence for Malaria Diagnosis, University of Lagos, Nigeria, are LT laboratories following WHO procedures for LT to assess the quality of malaria RDTs procured for national deployment in India and Nigeria, respectively. WHO supports LT of malaria RDTs at the Research Institute for Tropical Medicine (RITM), Manila, in the Philippines following the guidelines outlined in the Methods Manual for Laboratory QC Testing of Malaria RDTs.⁹ Lot testing for malaria RDTs evaluates the diagnostic performance of RDTs at antigen concentrations equivalent to parasite densities of 200 parasites/µl and 2000 parasites/µl for both Pf and Pv. The diagnostic performance is assessed, and results noted by laboratory technicians during the testing procedures are documented and reported to concerned authorities if any irregularities are observed. The LT program uses QC panels similar to those used by the product testing program, with differences in the number of reference samples used for both parasite species. The parasite panel set at 200 parasites/μL reflects the low-end parasite density associated with symptoms. A “Fail” result in LT denotes a lot that has not passed the initial QC assessment by a primary institution, and the secondary institution confirms the same results.⁹ The pan test band in the RDTs is paramount because it elucidates the presence of any four Plasmodium (Pf, Pv, P. malariae, and P. ovale) species causing malaria in humans. This study investigated the Pan-Plasmodium lactate dehydrogenase (Pan-pLDH) test band performance of 3,436 Pf combination RDTs received from different Indian manufacturers for pre-procurement and long-term testing. The laboratory evaluation component of malaria RDT prequalification is based on publicly available WHO Foundation for Innovative New Diagnostics standard operating procedures.⁹

MATERIALS AND METHODS

The WHO had supported the ICMR-NIMR to provide LT services to assess the diagnostic performance of malaria RDTs in India since 2016. The institute requests 150 RDT kits for bivalent and 100 for monovalent RDT lots for LT from the requesting manufacturers. The results of these lots are then submitted to government procurement agencies by the manufacturers. Different RDT products in India target different antigens in multiple combinations, such as Pf/Pv/Pan, Pf/Pv, Pan, and Pan/Pf. The turn-around time from receiving the samples to reporting lot test results was 5 working days.

Tests were conducted following the WHO guidelines. Briefly, 34 and 58 RDTs were tested from a single lot for monovalent and bivalent RDTs using four QC panels of Pf, Pv, or both for bivalent tests. The assessment of accessories (buffer ampules, alcohol swabs, lancets, and blood transfer devices) was performed before testing the samples of each lot. QC panels used for LT were selected depending on the target antigen of the kits received. The LT’s QC panels were externally characterized for antigen concentration by the Hospital of Tropical Medicine, London, UK, supported by WHO. Four different positive QC panels were selected for testing monovalent RDTs. For each of the four positive QC samples, testing on six RDTs performed with the 200 parasites/μL QC aliquot(s) (total 24 RDTs), along with 10 negative panels for the LT of one product. If the requested lots were bivalent, then 58 kits were tested, with 24 Pf panels, 24 Pv panels, and 10 negative QC panels. The manufacturer’s instructions were followed during testing. The band intensities were observed and reported according to the color chart with the intensity of test bands provided by the WHO. If the control band did not appear, the test was considered invalid. Problems in test performance, such as incomplete clearing, red background, streaking lines, and split test lines, were included in the report.

The LT report includes the number of RDT samples for each lot tested. The test is reported as negative if the test line is not visible with any intensity band within the lateral flow strip in the time frame as per manufacturers’ instructions. The results are declared either pass, deferred (this RDT lot failed this assessment on quality control dilutions and need to send to a secondary institution for confirmation), or fail for that specific number of RDTs tested. It is considered deferred if the results fail in initial and repeat QC samples in the primary laboratory. If the test results are deferred, the kits are sent to a secondary WHO reference laboratory, RITM, Manila, Philippines, for confirmation. A list of Pan RDTs with different combinations of antigens is listed in Table 1, different manufacturers, corresponding to 275 production lots, are listed in Supplemental Table 1.

Table 1.

Detail of rapid diagnostic tests with Pan and combinations received for lot testing

Manufacturer	Product name	Target antigen for Pf	Target antigen for Pan	Target antigen for Pv	No. of RDTs tested	No. of lot tested
M1	P1 (Pf/Pan (HRP2/pLDH) Ag Combo RDT)	HRP2	Pan-pLDH	NA	58	1
M11	P1 (AG Pf/Pan)	HRP2	Pan-pLDH	NA	348	6
M12	P1 (PAN + PF Card)	HRP2	Pan-pLDH	NA	58	1
M13	P1 (Malaria Antigen Pf/Pan)	HRP2	Pan-pLDH	NA	174	3
M15	P1 (Malaria Antigen Pf/Pan)	HRP2	Pan-pLDH	NA	58	1
M17	P1 (Malaria Pf(HRP-II)/PAN LDH)	HRP2	Pan-pLDH	NA	58	1
M20	P1 (Malaria Ag Pf/Pan)	HRP2	Pan-pLDH	NA	174	3
M23	P1 (Malaria P.f/Pan Ag)	HRP2	Pan-pLDH	NA	348	6
M23	P2 (Malaria/CRP Duo)	HRP2	Pan-pLDH	NA	120	2
M25	P1 (Malaria Ag Pf/Pan)	HRP2	Pan-pLDH	NA	290	5
M26	P1 (Pan Device)	NA	Pan-pLDH	NA	174	3
M26	P2 (Pan/Pf/Pv)	HRP2	Pan-pLDH	Pv-pLDH	174	3
M26	P3 (Pan/Pf Device)	HRP2	Pan-pLDH	NA	174	3
M26	P4 (Malaria Pan Device)	NA	Pan-pLDH	NA	174	3
M27	P1 (Pf/Pan rapid test)	HRP2	Pan-pLDH	NA	348	6
M28	P1 (Pan Device)	NA	Pan-pLDH	NA	150	2
M28	P2 (Pan Dipstick)	NA	Pan-pLDH	NA	208	3
M28	P3 (Pan/Pv/Pf Device)	HRP2	Pan-pLDH	Pv-pLDH	174	3
M28	P4 (Pan/Pf Device)	HRP2	Pan-pLDH	NA	174	3
Total					3,436	58

A total of 3,436 Pan tests were examined, of which 2,382 were Pf+Pan-pLDH, 706 were for Pan-pLDH alone, and 348 were for Pv-pLDH/Pf-HRP2/Pan-pLDH. M = manufacturers of RDT; NA = not available; P = product type.

The long-term testing of RDTs received was conducted on all the lots received until June 2021. For each lot, 40 samples of test kits were stored at 37°C for testing 6 months before the expiration date of RDT. In the long-term testing procedure, two positive QC panels specific to the targeted antigens and two negative panels were selected for testing two kits. Exposure of the desiccant to humidity was recorded for the kit accessories stored for the long term. Changes in the volume and color of the buffer. The list of RDTs and Pan RDTs tested for the long term are listed in Supplemental Tables 2 and 3.

The RDTs detecting pan antigens were also tested with high parasitemia QC panels equivalent to 2,000 parasites/µL of blood. The results for the target antigen are presented in Supplemental Table 4.

STATISTICAL ANALYSES

Statistical analysis was carried out with the help of SAS and Microsoft Excel. Proc-Freq function with the SENSPEC option was used to calculate sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and 95% CIs in SAS. Tables were created using SAS, and bar charts and pie charts were plotted using MS Excel.

RESULTS

From 2016 to June 2021, 15,488 RDTs were evaluated for LT. Figure 1 illustrates the various RDTs that have been assessed for specific antigens. The testing QC panels adhered to the WHO’s antigen concentration specifications and are visually depicted in Supplemental Figure 1A and B. The number of RDTs undergoing LT increased from 2016 until June 2021, as shown in Figure 2A, and the same was observed for Pan and combination RDTs. There were 15,488 RDTs analyzed, 3,436 of which were Pan RDTs from various manufacturers (Figure 2B). The interpretation of test lines was based on the original color chart provided by WHO. Of 3,436 Pan RDTs tested, there were no comments for 2,964 RDTs. However, 249 RDTs results showed +1 test band intensity, 142 with the red background, 57 incomplete clearings with streaking blood, 23 incomplete clearings, and one ghost line (white lines on a stained background, color of test line not developed, noted as a negative RDT results). Observations for the Pan RDT lots tested are shown in Supplemental Figure 2. The Pan band intensity was observed to differ while testing with the two-species QC panels (Supplemental Figure 3A and B). There were 2,382 Pf-Pan products tested between 2016 and 2021 June. The performance of 2,382 RDTs product tested for the Pf-pLDH test band was generally as per the color chart. Of 1,356 Pf-Pan products tested with Pf QC panels, 906 (15 lots of Pf Pan) were deferred. (These RDT lots failed this assessment on QC dilutions and were sent to another institution for confirmation.)

Figure 1.

Rapid diagnostic tests (RDTs) with different antigen combinations received for lot testing between 2016 and 2021 June. The target antigen for Plasmodium falciparum (Pf) is histidine-rich protein 2 (HRP-2), Plasmodium lactate dehydrogenase (pLDH) of Plasmodium vivax (Pv), and Plasmodium lactate dehydrogenase (pLDH) of Plasmodium species for identification of Pan Plasmodium species.

Figure 2.

(A) Total rapid diagnostic tests (RDTs) and lots examined by year. (B) Number of Pan RDTs and lots examined by year.

The Pan RDT’s performance at higher parasitemia (2,000 parasites/µL) was evaluated. A total of 64 RDTs were tested, and among these, 22% (n = 14) had no test band intensity, and 45% (n = 29) had +1 test band intensity, 22% (n = 14) had +2 band intensity, and 11% (n = 7) had +3 band intensity (Supplemental Figure 4).

Total RDT sensitivity and specificity for all combinations tested were 95.3% (94.9–95.7% 95% confidence level [CL]) and 99.6% (99.3–99.8% 95% CL), respectively; PPV and NPV were 99.9% (99.8–99.9% 95% CL) and 82% (80.7–83.3% 95% CL).

There was 3,436 Pan RDTs examined, with 2,382 for Pf+Pan, 706 for Pan alone, and 348 for Pv-pLDH/HRP2/Pan-pLDH, respectively (Table 1).

Pan combination RDTs had a sensitivity 85% (83.7–86.3% 95% CL) and a specificity 98.4% (97.4–99.4% 95% CL), respectively; ROC, Pf+Pan RDTs had a sensitivity 83% (81.3–84.6% 95% CL) and a specificity 97.7% (96.32–99.21% 95% CL). In contrast, Pan RDTs had 84.3% (81.4–87.3% 95% CL) and 100% (100–100% 95% CL) sensitivity and specificity, respectively. In comparison, RDTs containing a combination of PfHRP2 + Pv-pLDH + Pan-pLDH) gives 100% sensitivity and specificity. Of the 15 deferred lots of Pf/Pv/Pan RDTs sent to RITM for secondary confirmation, 87% (n = 13) of Pf/Pan were deferred (Supplemental Table 5). The RDTs (n = 130) reserved for long-term testing with combinations of Pf and Pan combination RDTs showed 94.2% (89.7–98.7% 95% CL), and 100% (100–100% 95% CL) sensitivity and specificity, respectively Supplemental Figure 5.

DISCUSSION

The WHO recommends quality checking for all lots of malaria RDTs before procurement by National Malaria Programs. The evaluation carried out by the LT laboratories was conducted according to the testing protocol provided by the manufacturer and the standard operating procedures of the methods manual for laboratory quality control testing of malaria rapid diagnostic tests.⁸ A total of 6,056 lots were evaluated in the WHO-FIND LT program for malaria RDTs from 2007 to 2017, totaling approximately 1.6 billion RDTs. Of these, 69 (1.1%) lots failed QC, for an estimated 7.9 million poor-quality RDTs. The lot test requesters were advised that failed lots were unsuitable for patient care management.¹¹

Malaria RDTs are commonly manufactured using three target antigens: PfHRP2, Plasmodium aldolase, and pLDH. Individual product performance varies from lot to lot. However, regions with a high Pv prevalence, such as Central and South America and the Indian subcontinent, should consider Pv-LDH/Pan-pLDH or aldolase combinations. HRP2/Pf-pLDH/Pan-pLDH combinations can also benefit other countries heading toward elimination. Nonetheless, pLDH does not have the same gene deletion or prozone effects as PfHRP2, and it can increase PfHRP2 sensitivity independently.¹⁰ In more than 40% of positive tests, the PfHRP2 detecting band shows a positive result even when no positive pan band exists. These were similar to lots tested against low parasite densities.¹⁰

Pf-LDH-detecting RDT performance against wild-type, Pf does not always predict performance against Pf-HRP2-deleted parasites.¹² Of 284 HRP-2 test lines for Pf, 205 samples (72.2%) had strong or medium line intensities, but the Pan-pLDH test lines for all species, notably P. ovale, had less prominent line intensities.¹³ In the LT laboratory from India, 15,488 RDTs were tested from 28 product manufacturers over 5 years (2016–June 2021). Of these, 3,436 RDTs were for Pan combination. Although both the Pan and Pf bands were positive in the past, the intensity of the Pan band was consistently lower than that of the Pf band, despite differences in the parasitemia. At 200 parasites/µL, 65% of the positive Pan bands showed a weak band intensity (+1). This percentage dropped to 16% when tested at samples equivalent to 2,000 parasites/µL.

Faint bands might be challenging to distinguish, resulting in false-negative results for healthcare professionals in low-light circumstances or when health workers have poor vision.¹⁴ Combining Pf-HRP2 and the pLDH antigen on three-band RDTs could increase the diagnostic specificity for malaria parasitemia. It may help differentiate between acute infection and infection that had recently been treated based on the specificity of the pLDH antigen present in the blood. According to a long-term study conducted in Uganda, RDTs that detect HRP2 antigenemia have lower specificity than RDTs that detect Pf pLDH antigenemia and were better at detecting parasites at low densities.¹⁵ To find substandard lots before and after field deployment, the WHO recommends pre-dispatch, post-dispatch, and post-marketing surveillance. The results from LT were relevant because a considerable proportion of RDTs made in India and submitted for LT at ICMR-NIMR did not receive WHO prequalification.

Additionally, to address the QC issues, more uniform packaging of RDTs is needed. It also urges attention to inconsistencies in product information pamphlets and brochures and incorrect labeling of RDT cassettes. There was a need to base procurement decisions on RDT LT and evaluation results. It was necessary to guide initial procurement decisions on RDT quality by requiring manufacturers to meet the LT quality standard as per technical specifications of the WHO.

A strategic approach is urgently needed to ensure the procurement of quality-assured malaria RDTs to reach malaria elimination goals. Our analysis gives comprehensive data on the diagnostic performance of RDTs evaluated at equivalent parasite densities of 200 parasites/µL for both Pf and Pv, as well as on the sensitivity and specificity of HRP2, Pv pLDH, and Pan-pLDH combinations. The concerns regarding the observations of Pan-specific RDTs must be addressed promptly and effectively.

Note: Supplemental material appears at www.ajtmh.org.