Clinical Evaluation of a Loop-Mediated Amplification Kit for Diagnosis of Imported Malaria

Abstract

Background. Diagnosis of malaria relies on parasite detection by microscopy or antigen detection; both fail to detect low-density infections. New tests providing rapid, sensitive diagnosis with minimal need for training would enhance both malaria diagnosis and malaria control activities. We determined the diagnostic accuracy of a new loop-mediated amplification (LAMP) kit in febrile returned travelers.

Methods. The kit was evaluated in sequential blood samples from returned travelers sent for pathogen testing to a specialist parasitology laboratory. Microscopy was performed, and then malaria LAMP was performed using Plasmodium genus and Plasmodium falciparum–specific tests in parallel. Nested polymerase chain reaction (PCR) was performed on all samples as the reference standard. Primary outcome measures for diagnostic accuracy were sensitivity and specificity of LAMP results, compared with those of nested PCR.

Results. A total of 705 samples were tested in the primary analysis. Sensitivity and specificity were 98.4% and 98.1%, respectively, for the LAMP P. falciparum primers and 97.0% and 99.2%, respectively, for the Plasmodium genus primers. Post hoc repeat PCR analysis of all 15 tests with discrepant results resolved 4 results in favor of LAMP, suggesting that the primary analysis had underestimated diagnostic accuracy.

Conclusions. Malaria LAMP had a diagnostic accuracy similar to that of nested PCR, with a greatly reduced time to result, and was superior to expert microscopy.

(See the major article by Hopkins et al on pages 645–57.)

Since the 1880s, the standard diagnostic test for malaria has been microscopic examination of peripheral blood smears [1–3]. The development of rapid antigen-detection tests (RDTs) in the early 1990s has improved diagnosis of Plasmodium infection in malaria-endemic countries and targeting of treatment [4, 5]. In countries where malaria is not endemic, where imported malaria cases in travelers can be mistaken for nonspecific viral illness [6, 7], causing delay in diagnosis and in some cases progression to severe illness and death [8–11], RDTs have improved the capacity for rapid malaria diagnosis by nonspecialist health workers [12–15]. While RDTs and expert microscopy are considered adequate for case management in malaria-endemic populations [16], there is growing interest in “improvements in point-of-care tests for case management, and the development of new tests capable of identifying very low parasite densities in asymptomatic individuals in field settings for mass screening and treatment” [17].

Polymerase chain reaction (PCR), in its most sensitive form, has a limit of detection as low as 50 parasites per milliliter of peripheral blood [18, 19], although results take 10–16 hours, whereas expert microscopy is less sensitive but routinely produces results in 60 minutes. Real-time quantitative PCR (qPCR) can reliably detect parasite DNA within 3–5 hours of sample receipt but is also less sensitive than nested PCR [20]. Recent studies show that loop-mediated amplification (LAMP) assays for malaria, which deploy isothermal molecular amplification in a closed system with visual readout, can deliver PCR-level diagnostic accuracy in a little more than 1 hour, with lower laboratory capacity requirements [21–23]. To date, malaria LAMP has not been available in a format suited to routine diagnosis in the clinic. A clinically validated, CE-marked assay would be an attractive alternative to RDT and microscopy in settings where malaria is or is not endemic.

A malaria LAMP kit has been developed as a point-of-care diagnostic test, and after CE marking it was released commercially in mid-2012. This kit comprises a disposable extraction device and tubes containing vacuum-dried and temperature-stable reaction components. We investigated the diagnostic accuracy of this kit for case management in a study of sequential blood samples from suspected imported malaria cases received by a specialist parasitology laboratory in London, United Kingdom, over the first 7 months of 2011. Primary diagnosis was by expert microscopic examination of blood films. An established nested PCR assay was deployed as the reference standard [18]. Diagnostic accuracy of the new kit was superior to that of microscopy and similar to that of nested PCR but with the additional benefits of reduced assay time and ease of operation.

METHODS

Patients and Samples

The study was conducted according to a detailed protocol that conforms to the STARD (Standards for the Reporting of Diagnostic Accuracy Studies) guidelines [24]. A synopsis of the protocol was considered and approved as service improvement activity by the United Kingdom National Research Ethics Service. No patient information was retained other than that routinely collected, and individual patient identifiers were removed from the study database. Clinical staff retained the ability to link study data to patient records.

The target sample size was 866 blood samples (Supplementary Materials). All samples sent for blood parasite testing in the Department of Clinical Parasitology, Hospital of Tropical Diseases, London, between 24 January and 20 July 2011 that originated from either the walk-in clinic, inpatient wards, or the Accident and Emergency Department in the main hospital were eligible for inclusion in the study. Venous blood for malaria diagnosis was collected in anticoagulant tubes containing either ethylenediaminetetraacetic acid (EDTA; for microscopy and nested PCR) or heparin (for the malaria LAMP test). An aliquot of the EDTA blood sample was sent to the University College London Hospital Haematology Department for white blood cell (WBC) counting.

Diagnostic microscopy was performed as per routine standard operating procedures for malaria diagnosis. Each day, a member of staff not involved in performing the malaria LAMP test assigned each heparin blood sample a unique study sample identifier in a predefined random order. A single study researcher performed all the malaria LAMP assays; this individual received the anonymous samples and thus was blinded to all microscopy results.

An aliquot of each EDTA blood sample (200 µL) was stored at −20°C for later extraction of DNA for nested PCR. DNA was prepared in batches of 12 samples, using an automated system as previously described [20]. Parasite density was estimated for positive samples by counting the number of parasites present per 500 WBCs and converting the sum to parasites per microliter of blood, using the actual WBC count for that sample. A fluorescent probe–based qPCR was also used to estimate parasite densities as previously described [20].

Malaria LAMP Assay

Heparinized blood samples were stored at 4°C until processing, which typically occurred within 72 hours. DNA was purified from whole blood using PURE extraction technology comprising bespoke plastic ware, hardware, and reagents supplied by Eiken Chemical (Japan). Full details of kit contents and procedures are provided online (available at: http://www.finddiagnostics.org/programs/malaria-afs/lamp/standard_procedures/index.html). A total of 35 µL of blood was aliquoted into a heating tube containing extraction buffer. The tubes were sealed with a screw-topped lid, shaken by hand, and incubated in a dry block at 75°C for 5 minutes, after which they were briefly cooled. Tubes were then screwed into the top of the PURE extraction tube, releasing the contents into clean-up powder. Buffer and powder were mixed by vigorous shaking. An injection cap supplied with the kit was screwed into the base of the adsorbent tube, and 25 µL of purified DNA solution was squeezed out of the tube, drop by drop, into the reaction tubes. These were volumetrically marked during manufacturing to allow 25 µL to be measured by eye.

Sealed reaction tubes were inverted briefly to reconstitute the reaction mixture vacuum-dried in the cap, including either pan–Plasmodium genus or Plasmodium falciparum–specific primers, before incubation at 65°C for 40 minutes in a real time LA-320 C turbidimeter (Eiken Chemical). Each set of patient samples was run alongside a positive control (purified plasmid DNA) and negative control (nuclease-free water) reaction. Upon termination of the reaction by incubation for 5 minutes at 85°C, the tubes were removed from the turbidimeter. Two readers then independently scored reactions as positive or negative by eye, using a UV backlighting lamp (supplied with the LF-160 Simplified LAMP Reactor, Eiken Chemical) to excite calcein fluorescence within the reaction mix. Individuals scoring the fluorescence assay were blinded to the results of the real-time turbidimetry analysis. The real-time turbidity data were also recorded; this was scored as positive if an increase in turbidity exceeding 0.1 OD units per second was observed.

For comparison with previously published studies, purified DNA was also prepared from heparin samples, using a boil and spin method as described elsewhere [23]. The resultant supernatant was diluted 1:12.5 with sterile H₂O, 25 µL was transferred into a reaction tube containing either lyophilized pan–Plasmodium genus or P. falciparum reagents, and then tested and scored as described above.

Reference Assay

DNA was extracted from 200 µL aliquots of EDTA blood on an automated platform (QiaCube, Qiagen, Germany) as described elsewhere [19]. Five microliters of DNA extract was amplified using nested PCR to detect P. falciparum, Plasmodium ovale subspecies, Plasmodium vivax, and Plasmodium malariae DNA, as per published methods [18, 23], following standard operating procedures. Amplicons were visualized on 2% agarose gels, independently scored by 2 investigators. Both individuals were blinded to the microscopy and LAMP test results.

qPCR Estimation of P. falciparum Parasite Density

P. falciparum density was estimated from 2 µL of EDTA blood–extracted DNA by qPCR as described elsewhere [20, 25]. All other malaria species were recorded only as present or absent by the qPCR assay.

Archive Sample Test Panel

An archive of anonymous samples from patients who presented to the Hospital of Tropical Diseases between September 2008 and January 2011, supplemented by 6 P. falciparum–infected and 17 P. vivax–infected blood specimens collected by the Instituto de Medicina Tropical Alexander von Humbolt, Universidad Peruana Cayetano de Heredia, Peru, as part of the specimen bank for the World Health Organization Foundation for Innovative New Diagnostics RDT product testing program [14], was stored at −20°C. From this collection, 20 parasite-negative, 10 P. ovale subspecies–infected, 30 P. vivax–infected, 33 P. falciparum–infected, and 7 P. malariae–infected blood samples were randomly selected for use in the LAMP study. An operator blinded to relevant parasitological data processed these samples for LAMP-based and PCR-based diagnosis as described above. LAMP results for these samples were based solely on the real-time turbidimetry data.

Data Analysis

The unit of analysis was a single venous blood sample taken at 1 particular time. Some patients contributed multiple samples at different times, each with a separate report form and identifier, but linkage to the primary sample was retained. Patients infected with P. falciparum were admitted for inpatient care as recommended by United Kingdom guidelines [3], and daily samples were taken until cure or discharge [7]. The following routine data collected by clinic staff were recorded: sample date, date of birth, sex, history, fever history, and current oral temperature. Because a number of samples came from other hospital departments, travel history, fever history, and current temperature were not always available. All data were double entered by 2 team members into a password-protected Access database. The staff member performing LAMP assays was not given access to the database, to maintain blinding to microscopy and PCR data for each sample. After resolution of any entry discrepancies, the harmonized database was duplicated and analyzed independently in parallel in London and Geneva, following a previously agreed data analysis plan. For each pairwise comparison between the index test (malaria LAMP) and the reference (nested PCR), the null hypothesis of no difference in performance between the 2 tests was retained for P values of ≥ .05, determined by the McNemar test. Confidence intervals (CIs) were calculated on the basis of the binomial distribution (Clopper-Pearson).

Secondary analyses performed included comparison of malaria LAMP and nested PCR results for a panel of previously archived DNA extracted from patients with malaria and those without malaria and exploratory PCR retesting, in triplicate, of all samples for which the original PCR and malaria LAMP results disagreed.

RESULTS

During the 6-month period of the study, 817 blood samples were received for diagnostic microscopy. Of the blood samples received, 707 comprised both EDTA and heparin collection tubes; the majority of these were suspected malaria cases, but there were also 12 requests to examine for Trypanosoma species, 5 for Babesia species blood-stage parasites, and 1 for Borrelia species. Microscopy, nested PCR, and malaria LAMP tests were completed on 705 samples. Sample numbers and test outcome summaries are presented as a flow diagram in Figure 1. A total of 330 of 664 individual patients (49.7%) contributing samples were female. Mean age was 42.7 years (95% CI, 41.1–44.3). Travel histories varied, often with multiple destinations. The most prevalent single destinations were India (104 journeys for 2 positive cases), Ghana (41 for 2 positive cases), Nigeria (31 for 8 positive cases), Kenya (28 for 2 positive cases), and Thailand (20 for no positive cases).

Figure 1.

Flow chart of study. Results of all 3 tests deployed are summarized. Discrepancies between the gold standard and index tests are shown in red boxes. Triplicate quality control (QC) repeats of the polymerase chain reaction (PCR) test from each sample with discrepant results, performed as a secondary analysis, resolved 4 of the 15 discrepancies. These were 3 PCR-negative, loop-mediated amplification (LAMP)–positive discrepancies (1 of 9 Plasmodium falciparum [Pf] only, LAMP-positive samples and 2 of 3 Pf- and pan–Plasmodium genus [Pg]–positive samples), which were found to be PCR positive in at least 1 of 3 QC replicates, and the single PCR-positive, LAMP-negative sample, which was found to be negative by PCR in all 3 QC replicates. Abbreviation: EDTA, ethylenediaminetetraacetic acid.

All 56 microscopy-positive samples were positive by both PCR and LAMP. Of 648 microscopy-negative samples, 11 were PCR positive for P. falciparum. Seven of these were posttreatment follow-up samples from treated inpatients, 2 were from a single individual who was microscopy negative on 2 different days 1 week apart but PCR positive on both occasions (and was thus almost certainly carrying a persistent low-density parasitemia), and another was from an individual negative by microscopy but PCR positive (performed during the routine diagnostic procedures for this sample). Detailed results for each sample from each positive patient, together with parasite densities estimated from both microscopy and qPCR, are presented in Supplementary Table 1.

Primary Analysis of Test Diagnostic Accuracy

DNA templates prepared from blood by the boil and spin protocol performed in malaria LAMP with statistical equivalence to templates prepared using the test extraction method (PURE) for both primer sets (Supplementary Table 2). Therefore, for comparison against the reference PCR, results using PURE extraction of DNA and visual scoring of LAMP results are presented.

Malaria LAMP displayed diagnostic accuracy similar to that of nested PCR and superior to that of expert microscopy. Table 1 presents sensitivity, specificity, negative predictive value, and positive predictive value for both LAMP tests as compared to nested PCR. Comparison of microscopy with the reference standard is also shown. In this analysis, the malaria LAMP with pan–Plasmodium genus primers was statistically equivalent to PCR for detection of all species (P = .257), whereas statistical nonequivalence with PCR was found for detection of P. falciparum with species-specific primers (P = .002). Microscopy displayed substantially lower sensitivity than PCR and in head-to-head comparison was found to be inferior to LAMP (P < .001, by the McNemar test, for both pan–Plasmodium genus and P. falciparum primer sets).

Table 1.

Diagnostic Accuracy of Malaria Loop-Mediated Amplification (LAMP) as Compared to Gold Standard Nested Polymerase Chain Reaction (PCR), and Blood Film Microscopy in 705 Sequential Malaria Tests

Comparison, Primer Set	Sensitivity		Specificity		PPV, %	NPV, %
	Percentage (95% CI)	Subjects, Proportiona	Percentage (95% CI)	Subjects, Proportionb
Malaria LAMP vs nested PCR
Pan–Plasmodium genus	97.0 (89.6–99.6)	65/67	99.2 (98.1–99.7)	633/638	92.7	99.7
P. falciparum	98.4 (91.5–100)	62/63	98.1 (96.8–99.0)	630/642	83.5	99.8
Microscopy vs nested PCR
Pan–Plasmodium genus	83.6 (72.5–91.5)	56/67	100 (99.4–100)	638/638	100	98.3
P. falciparum	82.5 (70.9–91.0)	52/63	100 (99.4–100)	642/642	100	98.3

Abbreviation: P. falciparum, Plasmodium falciparum.

^a Data are no. of samples with positive results concordant with results of the gold standard/total no. of samples positive by gold standard.

^b Data are no. of samples with negative results concordant with results of the gold standard/total no. of samples negative by gold standard.

Secondary Analysis—Archived Sample Bank

The malaria LAMP was also evaluated on an archived DNA sample set (see Methods), generating high estimates for sensitivity and specificity that were similar to those obtained in the clinical study (Table 2). A single sample from Peru found to harbor P. vivax only by nested PCR tested for both P. falciparum and pan–Plasmodium genus primers; this is presented as a LAMP false-positive result for P. falciparum in Table 2 (but see Discussion).

Table 2.

Malaria Loop-Mediated Amplification (LAMP) Results for 100 Archived Samples in Which Plasmodium Species Were Detected by Gold Standard Nested Polymerase Chain Reaction (PCR)

Nested PCR Result	Samples, No.	LAMP Result
		P. falciparum Test Positive	Pan–Plasmodium Genus Test Positive	Agreement, %
Negative	19	0	0	100
Positive
For P. falciparum	33	33	33	100
For P. falciparum plus P. vivax	1	1	1	100
For P. malariae	7	0	7	100
For P. ovale	10	0	10	100
For P. vivax	30	1a	30	96.7
Overall	100	…	…	99.0

Abbreviations: P. falciparum, Plasmodium falciparum; P. malariae, Plasmodium malariae; P. ovale, Plasmodium ovale; P. vivax, Plasmodium vivax.

^a Positive by both primer sets. It is plausible that this is a bona fide mixed species infection with P. falciparum missed by PCR.

Secondary Analysis—Nested PCR Post Hoc Quality Control

As shown in Figure 1, there was 100% concordance between LAMP and nested PCR results for the 56 microscopy-positive samples. However, there were 14 LAMP-positive samples that tested negative by both microscopy and nested PCR and 1 sample that was PCR positive but negative by both LAMP and microscopy. In a post hoc analysis, nested PCR was repeated in triplicate for all 15 samples with discrepant results, along with 20 randomly selected samples from the study to act as controls, by operators blinded to the original result. All 20 controls gave the same result in triplicate repeat testing as that obtained in the primary analysis. Of the 15 samples with discrepant results, 11 gave the same PCR result as that obtained in the primary analysis. Three samples that were negative by the original nested PCR but positive by LAMP were found to be positive in at least 1 of the triplicate verification tests. qPCR results supported these findings (Supplementary Table 1). We conclude that these are true-positive results but with low parasite density, suggesting a slightly higher sensitivity of malaria LAMP as compared to the reference PCR assay. The single sample that was positive by nested PCR in the primary analysis but negative by LAMP tested negative in all nested PCR verification test replicates (9 PCR reactions in total). We conclude that this sample had a false-positive nested PCR result in the primary analysis.

Factors Contributing to Discrepant Results Between the LAMP Tests and the Gold Standard

There remained 11 blood samples in which a positive result was obtained for either or both LAMP tests, but no parasite DNA was detected by nested PCR. No evidence was found that samples with discrepant results were more common in subgroups of samples defined by parasite density, patient age, sex, or WBC count (data not shown). We also tested for temporal clustering of discrepant LAMP results. During a single 10-day period from 13 to 22 June 2011, false amplification in the negative control tube was observed in some LAMP tests. This had been noted at the time, and simple remedial action was undertaken in the forms of decontamination of the apparatus and better separation between bench areas; false amplification in the negative control was then eliminated. We found that 5 of the 11 samples (45.5%) with discrepant results were tested during this period, compared with 6.4% of concordant samples (odds ratio, 12.27; 95% CI, 2.82–49.96; P < .001, by the Fisher exact test). This suggests transient laboratory contamination as an extrinsic cause for these 5 discrepant results; problems with the laboratory's ventilation system had occurred during the month of June.

DISCUSSION

The commercially manufactured malaria LAMP test evaluated here, which amplifies parasite mitochondrial gene targets, demonstrated good diagnostic accuracy in comparison to the reference nested PCR. Malaria LAMP demonstrated diagnostic sensitivity significantly superior to that of expert microscopy. In the primary analysis, using pan–Plasmodium genus primers, LAMP did not differ in performance from nested PCR (P = .3447). LAMP with P. falciparum primers was found to have significantly different diagnostic accuracy than PCR (P = .004), with a positive predictive value of 83.5% and a negative predictive value of 99.8%. Fifteen samples displayed discrepant results between LAMP and PCR and were checked by triplicate repeat testing with nested PCR. Four of these discrepancies were thus resolved, confirming the original LAMP result as correct in each case. The remaining 11 samples with discrepant results were all negative by PCR, suggesting false-positive LAMP results. Five of these clustered in a single 10-day period within the 6-month study, strongly suggesting a temporary source of DNA contamination in the laboratory. Six apparent false-positive LAMP results remain unexplained. This suggests that, if deployed as the primary diagnostic test, the malaria LAMP test would have correctly identified all 70 malaria infections and resulted in 11 false-positive results among the 638 uninfected samples. By use of a regret theory approach to decision making [26], and given the high sensitivity of the test, this seems an acceptable type I error rate, as regret due to unnecessary treatment will be lower than that associated with untreated Plasmodium infections. In fact, some apparent false-positive discrepant results were resolved, by replicate PCR testing, in favor of LAMP. Thus, the test format used here may be more sensitive than the reference standard nested PCR. The design-locked format tested here thus performs better than the prototype format previously described [23]. This notion cannot be readily tested in the absence of a suitable tie-breaker test with a sensitivity higher than that of nested PCR.

The current format of pan-Plasmodium genus and P. falciparum–specific LAMP is able to identify all P. falciparum–infected individuals. Nonfalciparum Plasmodium species are all identified but not resolved to the species level (Table 2), a limitation shared with most RDT tests. Among previously collected positive blood samples, one sample characterized as P. vivax monospecies infection by nested PCR was found to be positive for P. falciparum by LAMP. This may be a bona fide mixed species infection, with P. falciparum being the minor species at a density at or below the limit of detection for our nested PCR assay: such cryptic mixed infections have been previously identified in Peru, using PCR [27]. Additional species-specific primer sets for P. malariae, P. ovale curtisi, P. ovale wallikeri, P. vivax, and Plasmodium knowlesi may be useful developments for diagnosis of malaria in travelers returning from settings where these species are prevalent. For patient management, however, the key information for rapid decision making is whether malaria is present and whether potentially fatal P. falciparum malaria is present. The LAMP primer sets tested here are adequate in this regard. Further species identification is not required to inform choice of chemotherapy for acute cases; when species identification is required, a PCR test performed afterward would suffice.

The current test fulfills requirements for use as a point-of-care diagnostic test, namely a complete CE-marked sample processing system using temperature-stable DNA-extraction reagents and vacuum-dried reaction mixes contained within the reaction tubes. This results in a simple, rapid test with minimal opportunity for user error [28]. We conclude that the current test format is a useful diagnostic procedure for the case management of Plasmodium infections. In the diagnostic laboratory of a developed country, malaria LAMP provides a number of advantages over microscopy, including superior sensitivity, minimal training requirements, and significantly less operator time. In the current study, a single staff member routinely processed 14 patient samples to final result with both primer sets in 90 minutes. Simple facility and equipment requirements, lower start-up costs, and greatly reduced assay time are also significant advantages over nested PCR, which in our laboratory takes approximately 16 hours from sample receipt to reporting, including 3–4 hours of operator time.

In malaria-endemic countries, low-cost point-of-care lateral flow tests are considered adequate for case management [16, 29]. However, the ability to test finger-stick blood samples with PCR-equivalent diagnostic accuracy in a near-patient facility could enable implementation of more cost-effective screening and treatment strategies in low-transmission and elimination settings [17, 30]. The costs of LAMP reagents are close to those used for nested PCR, but comparison of both techniques in terms of equipment and labor costs reveals that LAMP would be a more affordable option for laboratories in malaria-endemic countries. For similar reasons, the assay may have applications in antenatal screening, drug-efficacy monitoring (as an initial screening step), and large-scale prevalence surveys, particularly if high-throughput 96-well-plate formats can be developed [31, 32]. The accompanying article in this issue of the Journal that describes deployment of the P. falciparum LAMP test at a rural clinic in Tororo, Uganda, demonstrates the potential of LAMP for active case detection in malaria-endemic countries [33].

In conclusion, the malaria LAMP test, evaluated here for the primary diagnosis of malaria in returned travelers, has advantages over other molecular tests in speed, sensitivity, and minimal need for specialist training. Malaria LAMP is a suitable test for diagnosing imported cases of malaria in minimally equipped clinical laboratories. The current test format has potential to replace microscopy in developed country settings and, because of the greater diagnostic accuracy provided, reduce the delay to diagnosis of malaria in returned travelers.

Supplementary Data

Supplementary materials are available at The Journal of Infectious Diseases online (http://jid.oxfordjournals.org/). Supplementary materials consist of data provided by the author that are published to benefit the reader. The posted materials are not copyedited. The contents of all supplementary data are the sole responsibility of the authors. Questions or messages regarding errors should be addressed to the author.