Evaluation of the ICT Malaria P.f/P.v and the OptiMal Rapid Diagnostic Tests for Malaria in Febrile Returned Travellers

Abstract

Rapid diagnostic tests (RDTs) are less reliant on expert microscopy and have the potential to reduce errors in malaria diagnosis but have not been extensively evaluated in nonimmune persons or in countries where infection is not endemic. We evaluated the ICT P.f/P.v (ICT-Amrad, Sydney, Australia) and OptiMal (Flow Inc., Portland, Oreg.) assays prospectively for the diagnosis of malaria in 158 specimens from 144 febrile returned travellers in Australia by using expert microscopy and PCR as reference standards. Malaria was diagnosed in 93 specimens from 87 patients by expert microscopy, with 3 additional specimens from recently treated patients testing positive for Plasmodium falciparum by PCR. For the diagnosis of asexual-stage P. falciparum malaria, the sensitivity and specificity of the ICT P.f/P.v assay were 97 and 90%, respectively, and those of the OptiMal assay were 85 and 96%, respectively. The ICT P.f/P.v assay missed one infection with a density of 45 parasites/μl, whereas the OptiMal assay missed infections up to 2,500/μl; below 1,000/μl, its sensitivity was only 43%. For the diagnosis of P. vivax malaria, the sensitivity and specificity of the ICT P.f/P.v assay were 44 and 100%, respectively, and those of the OptiMal assay were 80 and 97%, respectively. Both assays missed infections with parasite densities over 5,000/μl: up to 10,000/μl with the former and 5,300/μl with the latter. Despite the high sensitivity of the ICT P.f/P.v assay for P. falciparum malaria, caution is warranted before RDTs are widely adopted for the diagnosis of malaria in nonimmune patients or in countries where malaria is not endemic.

Although regions with endemic infection incur most of the burden of malaria-associated morbidity and mortality, the infection also presents a considerable problem in other countries. Cases of imported malaria have increased in Western countries over the past several decades (6, 20) and are related to increased international travel and immigration. Given that 80% or more of travelers who contract malaria do not develop symptoms until returning home (2, 13), the diagnosis of malaria generally depends on laboratories in areas where infection is not endemic. However, in such settings, expertise in malaria diagnosis is often limited and, consequently, diagnostic inaccuracies are a well-recognized problem (2, 13, 16). Mortality from imported malaria, primarily due to Plasmodium falciparum, is 0.6 to 3.8%, (5, 20; H. O. Lobel, C. C. Campbell, and J. M. Roberts, Letter, Lancet i:873, 1985) and is strongly associated with delays in diagnosis and institution of appropriate treatment (5, 19, 31). Strategies to improve the accuracy and timeliness of malaria diagnosis, particularly where expertise in microscopy is limited, are thus priorities.

Microscopy remains the diagnostic technique of choice for malaria. It is relatively inexpensive, sensitive to a threshold of 5 to 50 parasites/μl (depending on technical factors such as the expertise of the microscopist) (17, 33), and able to characterize the infecting species and their relative densities. However, it requires considerable technical expertise for optimal blood film preparation, examination, and interpretation. Immunocapture rapid diagnostic tests (RDTs) based on the detection of malarial antigens have been developed to improve the timeliness, sensitivity, and objectivity of malaria diagnosis, through less reliance on expert microscopy. The ParaSight-F (Becton Dickinson, Franklin Lakes, N.J). and ICT P.f (Amrad-ICT, Sydney, Australia) assays detect the histidine-rich protein 2 (HRP2) antigen of the asexual stages and young gametocytes of P. falciparum and have a detection threshold of around 40 to 60 parasites/μl and an overall sensitivity of 84 to 100% for P. falciparum infections (17, 24, 27, 29, 34). Two other recently developed RDTs can detect P. vivax in addition to P. falciparum. The ICT P.f/P.v assay (Amrad-ICT) detects the same P . falciparum-specific HRP2 antigen as the ICT P.f assay and a panmalarial antigen expressed by P. falciparum, P. vivax, and possibly also P. ovale and P. malariae (D. P. Mason, C. Wongsrichanalai, K. Lin, R. S. Miller, and F. Kawamoto, Letter, J. Clin. Microbiol. 39:2035, 2001). The OptiMal assay (Flow Inc., Portland, Oreg.) detects Plasmodium-specific lactate dehydrogenase (pLDH) antigens that are expressed by both sexual and asexual parasitic stages, one P. falciparum specific and one Plasmodium genus specific (23). From limited evaluations, both assays appear sensitive for the detection of P. falciparum infections but variable for non-falciparum infections (1, 3, 7-9, 12, 18, 21, 24, 27, 29).

These RDTs, however, have a number of important limitations, including suboptimal sensitivity at low parasite densities, an inability to accurately identify parasites to the species level or to quantify infection density, and a higher unit cost relative to microscopy. False-positive results occur with HRP2-based assays because of antigen persistence for up to 28 days despite clearance of asexual-stage parasites with chemotherapy (15), and they occur with both HRP2- and pLDH-based assays in the presence of rheumatoid factor (10; A. Bartoloni, G. Sabatinelli, and M. Benucci, Letter, N. Engl. J. Med. 338:1075, 1998; M. P. Grobusch, U. Alpermann, S. Schwenke, T. Jelinek, and D. C. Warhurst, Letter, Lancet 353:297, 1999). Despite these limitations, RDTs are a potentially useful adjunct to microscopy, particularly where expertise is limited, and have been adopted in laboratories in many countries where infection is not endemic. In Australia, 43% of laboratories participating in the Royal Australasian College of Pathologists quality assurance program for malaria diagnosis routinely use RDTs to supplement microscopy (S. Neville, personal communication). The performance characteristics of such assays, however, have not been extensively established for febrile returned travelers.

The objective of this study was to determine prospectively the performance characteristics of two recently developed RDTs (ICT P.f/P.v and OptiMal) in the diagnosis of malaria in febrile patients presenting in New South Wales, Australia. Reference standards used for comparison were expert microscopy and PCR.

(The work cited in this paper was presented in part at the 2001 Australian Society for Microbiology Annual Scientific Meeting, Perth, Australia, 30 September to 5 October 2001 [abstract PP7.1].)

MATERIALS AND METHODS

Study setting.

The study was undertaken at the Institute for Clinical Pathology and Medical Research in Sydney, Australia. The Parasitology Department provides both a primary diagnostic service for locally referred specimens and a reference service for malaria diagnosis for the state of New South Wales.

Study design and specimens.

Consecutive specimens submitted to the Parasitology Department from August 2000 to June 2001 were included in the study; these comprised specimens referred directly from medical practitioners and those referred secondarily from other laboratories in New South Wales for confirmation. Inclusion in the study required the availability of thick blood films and the primary whole-blood specimen (collected in EDTA) stored at 2 to 8°C. Microscopy (J.W.), PCR (E.G.P.), and the RDTs (M.A.) were performed in parallel on all specimens by separate experienced operators blinded to the results of the other assays. All identifying details were removed from specimens after allocation of a study number, and the results were correlated after completion of all assays.

Microscopy.

Thick and thin films were prepared from primarily referred whole-blood specimens and stained with Field's stain by standard procedures within 1 h of receipt (32). Blood films submitted from other laboratories were examined as submitted (if stained) or were stained with Field's stain. A microscopist with more than 30 years of experience examined each thick film for at least 20 min under a 40× oil immersion lens (total magnification, ×500) such that at least 200 to 300 fields were examined. The parasite density was calculated with reference to the white cell count measured from simultaneously drawn blood and was expressed as the number of parasites per microliter (26).

PCR assay.

DNA was extracted from 200 μl of whole blood using the QIAamp DNA mini kit (Qiagen, Clifton Hill, Victoria, Australia) as specified by the manufacturer. Extracted DNA was amplified in a seminested multiplex PCR assay using primers targeting the small ribosomal subunit gene, allowing the simultaneous detection and identification to the species level of malaria species as previously described (25). Bands were visualized on a 2% agarose gel after electrophoresis at 200 V for 40 min. Contamination control measures included the use of dedicated equipment in separate laboratory areas for each assay step as well as other standard measures (14). A negative control was included for each specimen, and a positive control (incorporating a mixture of all four malaria species) was included for each assay run.

Rapid diagnostic tests.

Each whole-blood specimen was tested immediately on receipt by using the ICT P.f/P.v and OptiMal assays as specified by the manufacturers. The RDTs were stored at 4°C until use. Both assays were considered valid if the control line was visible. For the ICT P.f/P.v assay, a visible HRP2 line was interpreted as indicating infection with P. falciparum and a visible panmalarial antigen line was interpreted as indicating infection with P. vivax. Where both the HRP2 and the panmalarial antigen lines were visible, P. falciparum infection was diagnosed, although this is also consistent with a mixed infection with P. falciparum and another malaria species. For the OptiMal test, where both the P. falciparum-specific and the panmalarial pLDH lines were visible, infection with P. falciparum was diagnosed, although this is also consistent with a mixed infection as above. A visible panmalarial pLDH line was interpreted as infection with P. vivax.

Reference standards.

Following validation against microscopy, PCR was used as the reference standard for the diagnosis of all malaria, P. falciparum (all stages) malaria, P. vivax malaria, and non-falciparum malaria. Since PCR cannot differentiate between sexual and asexual P. falciparum parasites, microscopy was used as the reference standard for the diagnosis of asexual-stage P. falciparum infection.

Statistical analysis.

Specimens were classified as true positive, true negative, false positive, or false negative for each test under evaluation compared with the reference standard. Sensitivity, specificity, positive and negative predictive values, and likelihood ratios (LRs) for positive and negative test results were calculated (4). Comparison of independent means was performed using a two-sample t test, and comparison of independent proportions was performed using a χ² test. All significance tests were two tailed. Data were analyzed using EpiInfo version 6 and Minitab for Windows release 12 (Minitab Inc., State College, Pa.).

RESULTS

General characteristics and microscopy of study specimens.

During the study period, 158 specimens from 144 patients were tested by microscopy, PCR, and the two RDTs. Of the study patients, 96 were males, and the median age was 31 years (range, 1 to 75 years).

Microscopy was positive for 93 specimens from 88 episodes of malaria in 87 patients (Table 1). Of the infections, 28 were acquired in the Pacific region (mostly Papua New Guinea), 27 were acquired in Africa, 21 were acquired in Southeast Asia (mostly East Timor), 11 were acquired in southern Asia (mostly from the Indian subcontinent), and 1 was acquired in South America.

TABLE 1.

Comparison of the results of microscopy and PCR^a

Result by microscopy	No. of specimens with following result by PCR:							Total no. of specimens
	PF	PM	PO	PV	PF/PV	PF/PO	NEG
PF (asexual stages)	32							32
PF (sexual stage only)	3							3
PM		2						2
PO			3					3
PV			1	50				51
PFb/PV					1			1
PFc/PO						1		1
NEG	3						62	65
Total no. of specimens	38	2	4	50	1	1	62	158

^aPF, P. falciparum; PM, P. malariae; PO, P. ovale; PV, P. vivax; PF/PV, mixed P. falciparum and P. vivax; PF/PO, mixed P. falciparum and P. ovale; NEG, negative.

^bAsexual-stage ± sexual-stage P. falciparum parasites.

^cSexual-stage P. falciparum parasites only.

P. falciparum parasites were detected in 37 specimens (from 35 patients). Asexual-stage (with or without sexual-stage) parasites were present in 33 specimens (from 31 patients) at densities of 5 to 992,000/μl (median, 4,830/μl; interquartile range, 500 to 27,000/μl); one specimen also contained P. vivax parasites. Four specimens (from four patients) contained only sexual-stage parasites; one also contained P. ovale parasites. P. vivax parasites were detected in 52 specimens (from 49 patients) at densities of 14 to 28,500 parasites/μl (median, 2,970/μl; interquartile range, 690 to 7,080/μl). P. ovale was detected in four specimens (from four patients; densities, 630 to 16,930/μl), and P. malariae was detected in two specimens (from two patients; densities, 340 and 770/μl).

Comparison of PCR with microscopy.

Microscopy and PCR were concordant for 155 (98%) of 158 specimens for the detection of malaria parasites (Table 1). The three discrepant specimens were PCR positive for P. falciparum but microscopy negative. Although no parasites were detected after repeated microscopic examination, these specimens probably represented true-positive results by PCR since they were obtained after initiation of treatment for microscopy-confirmed P. falciparum infection. Microscopy and PCR results were concordant for the identification to the species level of malaria parasites in 92 (99%) of 93 specimens in which parasites were detected by both techniques. The single discrepancy was identified as P. ovale by PCR but P. vivax by microscopy; subsequent blinded review of the blood film by microscopy favored a diagnosis of P. ovale.

Comparison of RDTs with PCR and microscopy.

The performance of the RDTs compared with the reference standards of PCR and microscopy is presented in Tables 2 and 3. The median duration of whole-blood storage at 2 to 8°C until performance of the RDTs was 2 days (range, 0 to 12 days).

TABLE 2.

Comparison of the results of the ICT P.f/P.v and OptiMal assays with PCR^a

Result by PCR	No. of specimens with following result by ICT P.f/P.v:			Total no. of specimens	No. of specimens with following result by OptiMal:			Total no. of specimens
	PF	PV	NEG		PF	PV	NEG
PF	37		1	38	28		10	38
PM			2	2			2	2
PO			4	4		3	1	4
PV		22	28	50	2	40	8	50
PF/PV	1			1	1			1
PF/PO	1			1	1			1
NEG	2		60	62	1		61	62
Total no. of specimens	41	22	95	158	33	43	82	158

^aSee Materials and Methods for interpretation of the ICT P.f/P.v and OptiMal assays. Abbreviations are as in Table 1.

TABLE 3.

Performance characteristics of ICT P.f/P.v and OptiMal compared with PCR and microscopy

Assay	Result by PCR or microscopya	% Sensitivity (95% CI)d	% Specificity (95% CI)	% Positive predictive value (95% CI)	% Negative predictive value (95% CI)	LR for:
						Positive test	Negative test
ICT P.f/P.v assay	Any parasite	63.5 (53.1-73.1)	96.8 (88.8-99.6)	96.8 (89.0-99.6)	63.2 (52.6-72.8)	19.8	0.37
	PFb (all stages)	97.5 (86.8-99.9)	98.3 (94.0-99.7)	95.1 (83.5-99.4)	99.1 (95.3-99.9)	57.4	0.03
	PFb (asexual stages)	97.0 (84.2-99.9)	90.4 (83.8-94.9)	78.1 (62.3-89.4)	99.1 (95.3-99.9)	10.1	0.03
	PVc	44 (30.0-58.7)	100 (96.6-100)	100 (84.6-100)	79.4 (71.6-85.9)	∞	0.61
	Non-PFc	39.3 (26.5-53.2)	100 (96.6-100)	100 (84.6-100)	76.1 (68.2-82.8)	∞	0.61
Optimal assay	Any parasite	78.1 (68.5-85.9)	98.4 (91.3-99.9)	98.7 (92.9-99.9)	74.4 (63.6-83.4)	48.8	0.22
	PFb (all stages)	75.0 (58.8-87.3)	97.5 (92.7-99.5)	90.9 (75.7-98.1)	92.0 (85.8-96.1)	30	0.26
	PFb (asexual stages)	84.8 (68.1-94.9)	96.0 (90.9-98.7)	84.8 (68.1-94.9)	96.0 (90.9-98.7)	21.2	0.16
	PVc	80.0 (66.3-90.0)	97.2 (92.1-99.4)	93.0 (80.9-98.5)	91.3 (84.6-95.8)	28.5	0.21
	Non-PFc	76.8 (63.6-87.0)	100 (96.5-100)	100 (91.8-100)	88.7 (81.4-93.8)	∞	0.23

^aMicroscopy was used as the reference standard for calculating the performance characteristics for asexual-stage P. falciparum infection; PCR was used for all other diagnoses (see Materials and Methods).

^bIncludes mixed infections.

^cExcludes mixed infections; non-PF, all non-falciparum infections.

^dCI, confidence interval.

For the diagnosis of P. falciparum infections (all stages), the ICT P.f/P.v assay was significantly more sensitive than the OptiMal assay (97.5 and 75%, respectively; χ²₁ = 8.54, P = 0.003). It was also more sensitive for asexual-stage P. falciparum infections (97 and 84.8%, respectively), but the difference was not statistically significant (χ²₁ = 2.92, P = 0.09). The only false-negative result with the ICT P.f/P.v assay was a specimen with a parasite density of 45/μl (Table 4). False-negative results, however, occurred with the OptiMal assay up to parasite densities of 2,500/μl; below this level, only 62% of infections were detected, and for infections at less than 1,000/μl, only 43% were detected (Table 4). The ICT P.f/P.v and OptiMal assays were positive for all seven and one of seven specimens, respectively, with only sexual stages of P. falciparum or that were PCR positive but microscopy negative.

TABLE 4.

Sensitivities of the ICT P.f/P.v and OptiMal assays for the diagnosis of asexual-stage P. falciparum and P. vivax infections, stratified by parasite density^a

Infection type and parasite density (parasites/μl)	ICT P.f/P.v				OptiMAL
	No. of specimens			% Sensitivity (95% CI)	No. of specimens			% Sensitivity (95% CI)
	PF	PV	NEG		PF	PV	NEG
P. falciparum (asexual stage)
<100	1	0	1	50 (13-99)	0	0	2	0 (0-84)
100-1,000	5	0	0	100 (48-100)	3	0	2	60 (15-95)
1,000-10,000	11	0	0	100 (72-100)	10	0	1	91 (59-100)
>10,000	15	0	0	100 (78-100)	15	0	0	100 (78-100)
Total	32	0	1	97.0 (84.2-99.9)	28	0	5	84.8 (68.1-94.8)
P. vivax
<100	0	0	3	0 (0-71)	0	1	2	33 (8-91)
100-1,000	0	1	13	7 (0-34)	0	10	4	71 (42-92)
1,000-10,000	0	13	12	52 (31-72)	2	21	2	84 (64-96)
>10,000	0	8	0	100 (63-100)	0	8	0	100 (63-100)
Total	0	22	28	44.0 (30.0-58.7)	2	40	8	80.0 (66.3-90.0)

^aAbbreviations are as in Tables 1 and 3.

The ICT P.f/P.v assay performed poorly for the diagnosis of P. vivax infection (sensitivity, 44%); false-negative results occurred at parasite densities up to 10,000/μl, and below this level only 33% of infections were detected (Table 4). Although the OptiMal assay was significantly more sensitive than the ICT P.f/P.v assay (sensitivity, 80%; χ²₁ = 13.8, P < 0.001), false-negative results still occurred at parasite densities up to 5,300/μl. The ICT P.f/P.v assay failed to detect any of the P. ovale and P. malariae infections, whereas the OptiMal assay detected three of four P. ovale infections but neither of the P. malariae infections.

The ICT P.f/P.v assay produced false-positive P. falciparum results in two PCR-negative specimens, whereas the OptiMal assay produced P. falciparum false-positive results in one PCR-negative specimen and two specimens containing only P. vivax parasites. No specimen gave a false-positive result in both assays. Clinical follow-up did not suggest recent or subsequent P. falciparum malaria in patients with false-positive results.

Based on recorded intervals between specimen collection and testing, there was no evidence that false-negative or false-positive results were related to a delay in performing RDTs.

DISCUSSION

The diagnosis of malaria may be challenging for laboratories in countries where infection is not endemic, such that inaccuracies are relatively common (2, 13, 16). Immunocapture RDTs that detect circulating malaria antigens have the potential to improve the accuracy and timeliness of malaria diagnosis, particularly in areas where expertise in microscopy is limited. Given that delays in the diagnosis and institution of treatment for malaria are important and potentially avoidable contributors to malaria-associated mortality (5, 19, 31), such improvements are important priorities. Although the ICT P.f and ParaSight assays are generally highly sensitive and specific for P. falciparum infection (17, 22, 28, 30, 34), they are unable to detect non-falciparum species. The recently developed ICT P.f/P.v and OptiMal assays can detect both P. falciparum and non-falciparum species but have not been extensively evaluated, particularly in nonimmune persons. We evaluated the performance characteristics of these two assays for malaria diagnosis in febrile travelers returning to a country where infection is not endemic. The study was designed to evaluate consecutive specimens prospectively, using the reference standards of expert microscopy and PCR as reference standards, with independent operators performing each assay while blinded to the other results.

The ICT P.f/P.v assay was highly sensitive (97%) for the diagnosis of clinically significant P. falciparum infections (i.e., those with asexual-stage parasites), consistent with previously published evaluations of the assay in its current and previous formats (17, 24, 27, 29). It demonstrated a low detection threshold, although only two specimens with parasite densities of <100/μl were assessed. However, the assay performed particularly poorly in diagnosing P. vivax malaria (sensitivity, 44%), with false-negative results encountered for infections with densities up to 10,000 parasites/μl. These results are considerably inferior to those in other reports from countries with endemic infection (27, 29), possibly reflecting the lower parasite densities encountered in infected nonimmune patients.

On the other hand, the OptiMal assay demonstrated only moderate sensitivity for both P. falciparum and non-falciparum malaria, generally consistent with previous reports (7, 8, 12, 18, 21). False-negative results occurred with P. falciparum malaria at densities up to 2,500 parasites/μl and with P. vivax malaria at densities up to 5,300/μl.

The specificities of both assays were generally high, despite the occurrence of occasional false-positive P. falciparum detections. In addition, the specificity of the ICT P.f/P.v assay for asexual-stage P. falciparum infections suffered from the detection of all seven clinically insignificant infections (i.e., those that contained only sexual-stage P. falciparum parasites or that were PCR positive and microscopy negative).

LRs can be used to quantitate the impact of either a positive or a negative test result on the pretest disease probability; from knowledge of the LR and an estimate of the pretest probability, the posttest probability can be derived directly or with a nomogram (11). Thus, LRs are of greater practical value for clinicians as performance parameters of diagnostic tests than are sensitivity, specificity, and predictive values. In general, large and generally conclusive changes in disease probability are produced by tests with LRs greater than 10 or less than 0.1 (11). From this study, both assays have LRs for a positive test result greater than 10 for all malarial diagnoses (Table 3); a positive test result thus makes the likelihood of malaria very high across a wide range of pretest probabilities. In contrast, with one exception, the LRs for a negative test result with both assays are greater than 0.2; a negative test result therefore does not substantially alter the pretest disease probability. However, with an LR of 0.03, a negative ICT P.f/P.v assay result would virtually exclude the diagnosis of asexual-stage P. falciparum malaria. This study used expert microscopy and PCR as reference standards. The concordance of these standards was high; the only discrepancies involved three post-treatment specimens in which P. falciparum was detected by PCR but not by microscopy and one misidentification of P. ovale as P. vivax by microscopy. Although PCR is an impractical primary diagnostic technique because of the turnaround time, expense, and technical expertise required, its accuracy makes it an important reference standard for evaluating new diagnostic assays.

RDTs have the potential to improve the accuracy and timeliness of malaria diagnosis, particularly for laboratories in countries without endemic infection, where expertise with microscopy may be limited. Consequently, such assays are becoming increasingly utilized in such settings. Given that malaria in nonimmune returned travelers is generally associated with low-level parasitemias of between 50 and 5,000/μl (17), the relatively poor performance characteristics of RDTs at these densities demands caution regarding their use in nonimmune infected persons. The incremental utility of RDTs over nonreference microscopy in countries without endemic infection has also not been evaluated; such an evidence-based approach is required before their application can be widely recommended. On the basis of the published evaluations of these assays, the most rational potential application of RDTs in countries without endemic infection would be where expert microscopy is unavailable for the detection or exclusion of P. falciparum parasites in suspicious negative blood films or in the presence of a non-falciparum species. However, the latter situation is uncommon in Australia, since mixed infections with P. falciparum constituted only 21 (1.4%) of 1,473 malaria notifications in Australia during 1992 to 1993 (2). Therefore, we think that the results of this study emphasize that microscopy should remain the diagnostic technique of choice for malaria diagnosis in nonimmune patients and that they also emphasize the importance of close links with a reference microscopy service where expertise is limited such as in countries where infection is not endemic. The accuracy and utility of RDTs require further investigation before their widespread adoption in such settings can be recommended.