A Diagnostic Accuracy Study to Evaluate Standard Rapid Diagnostic Test (RDT) Alone to Safely Rule Out Imported Malaria in Children Presenting to UK Emergency Departments

Abstract

Background

Microscopy is the gold standard for malaria diagnosis but is dependent on trained personnel. Rapid diagnostic tests (RDTs) form the mainstay of diagnosis in endemic areas without access to high-quality microscopy. We aimed to evaluate whether RDT alone could rule out imported malaria in children presenting to UK emergency departments (EDs).

Methods

UK-based, multi-center, retrospective, diagnostic accuracy study. Included: any child <16 years presenting to ED with history of fever and travel to a malaria-endemic country, between 01/01/2016 and 31/12/2017. Diagnosis: microscopy for malarial parasites (clinical reference standard) and RDT (index test). UK Health Research Authority approval: 20/HRA/1341.

Results

There were 47 cases of malaria out of 1,414 eligible cases (prevalence 3.3%) in a cohort of children whose median age was 4 years (IQR 2–9), of whom 43% were female. Cases of Plasmodium falciparum totaled 36 (77%, prevalence 2.5%). The sensitivity of RDT alone to detect malaria infection due to any Plasmodium species was 93.6% (95% CI 82.5–98.7%), specificity 99.4% (95% CI 98.9–99.7%), positive predictive value 84.6% (95% CI 71.9–93.1%) and negative predictive value 99.8% (95% CI 99.4–100.0%). Sensitivity of RDT to detect P. falciparum infection was 100% (90.3–100%), specificity 98.8% (98.1–99.3%), positive predictive value 69.2% (54.9–81.2%, n = 46/52) and negative predictive value 100% (99.7–100%, n = 1,362/1,362).

Conclusions

RDTs were 100% sensitive in detecting P. falciparum malaria. However, lower sensitivity for other malaria species and the rise of pfhrp2 and pfhrp3 (pfhrp2/3) gene deletions in the P. falciparum parasite mandate the continued use of microscopy for diagnosing malaria.

INTRODUCTION

Malaria is endemic in 87 countries [1] and causes around 400,000 deaths a year, two-thirds of which occur in children under 5 years of age [2, 3]. Malaria cases imported to non-endemic areas globally are estimated at around 30,000 cases annually [4]. Europe accounts for approximately 70% of the global burden of imported malaria [5]. People who travel to endemic countries to visit friends and relatives (VFR) are eight times more likely to be diagnosed with malaria than tourists [6]. UK monitoring estimates that around 10% of imported cases of malaria are in children [7].

Plasmodium falciparum is by far the dominant species in sub-Saharan Africa, causing an estimated 99.7% of cases and deaths in 2021 [2] and accounts for approximately 75% of clinical cases in the UK [7]. P. falciparum is the main cause of severe infection and death but other species can also result in significant morbidity and since signs and symptoms of malaria are non-specific, patients with fever must be screened emergently for the disease when returning from an endemic area [8]. Traditionally, clinical guidelines in well-resourced, non-endemic countries advise the reference standard diagnostic test for malaria is serial thick and thin blood films. These detect and speciate malaria parasites, and up to three negative films are required to rule out malaria [7]. However, thick and thin films sent to a specialist laboratory take time, and, given the need for further visits for repeat testing, put added costs and pressure on busy frontline settings.

To improve diagnostic accuracy and speed, many centers now use rapid antigen detection tests (RDTs) (using lateral flow chromatography on a test strip to bind malaria antigens—a combined detection test uses the HRP2 antigen to detect P. falciparum and pLDH to detect pan-malarial species, eg, Plasmodium vivax) alongside light microscopy [9, 10]. RDTs are now an important adjunct to malaria diagnosis in resource-limited settings, especially where there is no access to high-quality microscopy, as they are quick to process and do not require specialized personnel [11].

However, there are challenges with RDTs, which include: false-positives from histidine-rich protein-2 antigen (HRP2) remaining in the bloodstream after the infection is treated; RDT failure due to assay instability in storage; operator error through the subjectivity of interpretation; parasite densities below the limit of detection; qualitative rather than quantitative results which do not allow the percentage parasitemia in P. falciparum infections to be calculated (an important marker for severe infection); inability to detect non-falciparum malaria by HRP2-only tests [11]; apparent false-negatives caused by actual high-density infections in endemic regions (prozone effect); challenges with diagnosing malaria infections in pregnancy [12, 13]; and P. falciparum parasites containing pfhrp2 and pfhrp3 (pfhrp2/3) gene deletions which also produce false-negatives [14].

Despite these issues, the widespread use of RDTs in endemic countries has prompted health services in non-endemic countries to question the need for serial films [5]. Evidence for the diagnostic accuracy of imported malaria in children is limited [15–17]. In this study, we used a large retrospective cohort to evaluate the diagnostic accuracy of RDT alone to rule out malaria—in particular, P. falciparum malaria—in children presenting to UK emergency departments (EDs), in a non-endemic setting. We also aimed to evaluate whether hemoglobin and/or platelet level played any additional role in screening for malaria.

METHODS

Travel Fever was a multi-center, retrospective diagnostic accuracy study carried out by the Paediatric Emergency Research in the UK and Ireland (PERUKI) network and sponsored by Birmingham Women’s and Children’s NHS Foundation Trust. Standards for the Reporting of Diagnostic Accuracy Studies were followed [18].

For a site to participate, their hematology laboratory should routinely perform both microscopy for malarial parasites (clinical reference standard) and RDT (index test). Participants were any child or young person (CYP) <16 years who had presented to a participating ED with a history of fever and traveled to a malarial area (as designated by the http://travelhealthpro.org.uk/ website) between 1st January 2016 and 31st December 2017 and who underwent malaria screening, regardless of disposition (ie, admitted or sent home from ED). Anonymized, retrospective data were collected on a standardized spreadsheet from patient notes and hospital laboratory result reporting systems, recording: patient demographics; geographical area of travel: result of RDT; result of blood film; how many films in total performed for each patient; parasitemia for P. falciparum cases; the patient’s hemoglobin and platelet count; discharge diagnosis; whether they re-attended the same ED or died in the following 30 days.

Pseudo-anonymized data from which potential identifiers were removed by contributing sites were submitted by sites and collated by CB. Anonymized data were reviewed by CB and TF, with queries sent back to sites by CB for incomplete data. CYP with incomplete data, except for a history of fever (see below) were excluded.

For the index test, 14 sites used a combined detection test for both HRP2 antigen (to detect P. falciparum) and pLDH (plasmodium lactate dehydrogenase, pan-malarial, eg, P. vivax) while one site used a pLDH test only—see Table 1. For the reference test, 13 sites used thick and thin blood films for the detection and speciation of malaria parasites (Giemsa and Field’s stain). One site used thin film only while another site used the quantitative buffy coat method for detection with thin film for speciation and parasite count for P. falciparum. Indeterminate results were sent to the Malaria Reference Laboratory, London, where, if required, samples underwent real-time PCR (screening) and nested PCR (gold standard) for malaria confirmation and species determination.

Table 1.

Characteristics of Rapid Diagnostics Tests Used by Each Site (HRP2—histidine rich protein 2, P. falciparum antigen; pLDH—plasmodium lactate dehydrogenase, pan-malarial antigen)

Make of RDT	Number in Study	Antigen Targets	Detection Bands
CareStart™Malaria HRP2/pLDH Combo Test, Access Bio Inc	11	HRP2/pLDH	3 lines: Control—negative; HRP2—P. falciparum; pLDH—P. vivax, malariae, ovale; all 3 lines positive—either P. falciparum infection or mixed infection
BinaxNOW; Abbott	3	HRP2/pLDH	3 lines: Control—negative; HRP2—P. falciparum; pLDH—P. vivax, malariae, ovale; all 3 lines positive—either P. falciparum infection or mixed infection
Clearview Malaria pLDH, Orgenics	1	pLDH	2 lines: Control—negative; pLDH—P. falciparum, vivax, malariae, ovale

Sample Size

We aimed to obtain data from a minimum of 1,200 patients, assuming a prevalence of malaria of 5% (based on a pilot study), and a sensitivity of the RDT of approximately 95%, (95% confidence interval (CI) for the sensitivity of 86–98%, with a higher prevalence leading to increased precision).

Statistical Analysis

Patient demographics were summarized as percentages and as mean, range, and interquartile range (IQR), as appropriate. Sensitivity, specificity, likelihood ratios, positive predictive value, and negative predictive value were estimated firstly for all participants. The primary analysis assessed the diagnostic accuracy of the RDT with any species of malaria as the target for detection, and a secondary analysis assessed the accuracy to detect P. falciparum alone. For individuals incorrectly classified by RDT (false-positive and false-negative results), we checked the discharge diagnosis with sites to establish whether there was an indication that these were truly malaria cases or not, or reasons why they may have been misclassified by the RDT. Multiple logistic regression was used to determine the additional predictive accuracy of hemoglobin and platelet levels as predictors of malaria infection, adjusting for the RDT result. All estimates are presented with 95% CIs. Statistical analysis used R version 3·6·1 including the epiR and personograph packages [19].

Ethics

The study gained Health Research Authority (HRA) approval on 1st June 2020 (reference: 20/HRA/1341). The study was also discussed at a Public Patient Involvement meeting (National Institute for Health and Care Research Community Healthcare MIC, Oxford, 13/03/2019). The study ensured full data protection safeguards.

Role of the Funding Source

The study funders had no role in the study design, data collection, data interpretation, or writing of the report.

RESULTS

A total of 16 PERUKI sites in England, Scotland, and Wales submitted anonymized data but one site was excluded as RDTs were not routinely used alongside films and so did not meet the study’s inclusion criteria. Out of the total 1,486 cases across the 15 remaining sites, 14 were excluded as patients had not had both RDT and film performed; data from a further 58 individuals were excluded from analysis as they were above the upper age range for inclusion. The use of both film and RDT was 99% across the included sites.

The total number of eligible cases was 1,414, with 615 (43%) females, and a median age of 4 years (IQR 2–9 years). A history or presence of fever was recorded in 84% of cases, with 5.8% having no fever, and the remaining 10% unknown (see Appendix 1 and Discussion section below).

There were two deaths in the cohort, neither of which was attributable to malaria. Of the children discharged, 85 (6%) re-presented with febrile illness within 30 days. Of the six children who re-presented with a diagnosis of malaria, all were diagnosed correctly with the first RDT. Four children were either not tested or did not wait to be seen at the first presentation while the remaining two children were diagnosed with malaria at the first presentation. The reason for the return of the latter two children was unclear from our data.

The prevalence of malaria according to the reference test (microscopy) was 3.3% (47/1,414). The sensitivity of RDT alone to detect malaria infection compared to microscopy was 93.6% (95% CI 82.5–98.7%), specificity 99.4% (95% CI 98.9–99.7%), positive predictive value 84.6% (95% CI 71.9–93.1%) and negative predictive value 99.8% (95% CI 99.4–100.0%) (see Figure 1 for a graphical representation of the results).

Figure 1.

Personogram showing approximate expected numbers of rapid antigen detection test results in a hypothetical sample of 1,000 children. Of the children tested, we would expect 33 to have malaria but 2 of the RDT results would be false-negatives.

Of the three false-negative results, one had Plasmodium malariae (identified on the first film examined), one had Plasmodium ovale (identified on the second film examined), and one had a mixed P. vivax and P. ovale infection (identified on the first film examined) (Table 2). Therefore, none of the three cases had P. falciparum on the basis of the film result.

Table 2.

Comparison of RDT and Film for Malaria Diagnosis

		Film
		Positive	Negative	Total
Rapid antigen detection test	Positive	44	8	52
	Negative	3	1,359	1,362
	Total	47	1,367	1,414

Of the eight false-positive cases (Table 2), four resulted in an eventual discharge diagnosis of malaria, of which four were noted as having been treated or diagnosed prior to presentation (HRP2 antigen can remain in the bloodstream for several weeks after successful treatment). One indicated detection of Plasmodium species antigen on PCR but no parasites on a malaria parasite screen. The remaining three cases were diagnosed as viral infections.

Of the 47 malaria cases, all except one were identified with a combination of RDT and first film (one case of P. ovale from West Africa was identified only with the second film and speciated using PCR).

Out of 47 malaria cases, 36 (77%) were P. falciparum, seven (15%) P. vivax, two (4.3%) P. malariae, one P. ovale, and one mixed infection (P. vivax and P. ovale). There were no cases of Plasmodium knowlesi. Prevalence of P. falciparum was 2.5% (95% CI 1.8–3.5%). The sensitivity of RDT alone to detect P. falciparum infection was 100% (90.3–100%, n = 36/36), specificity 98.8% (98.1–99.3%, n = 1,362/1,378), positive predictive value 69.2% (54.9–81.2%, n = 46/52) and negative predictive value 100% (99.7–100%, n = 1,362/1,362). Of the 36 P. falciparum cases, 34 (94.4%) were from sub-Saharan Africa. In those with a parasitemia level reported, 11 (30.5%) cases had high parasitemia levels (≥2%—ie, severe or complicated malaria), with 24 (67%) found to have parasitemia levels <2%.

Lower levels of hemoglobin were associated with a positive film diagnosis of malaria, but this effect diminished after adjustment for RDT diagnosis. Lower platelet counts were strongly associated with a positive film diagnosis for malaria. The effect of low platelet counts, as defined by the presence or absence of thrombocytopenia (< 150 × 109/L), remained after adjustment for RDT diagnosis (p = 0.01, see Figure 2). The original positive predictive value of 84.6% (based on a positive RDT result alone) increased to 100% (n = 25/25, 95% CI 83.4–100%) among participants who had both a positive RDT result and thrombocytopenia, and dropped to 70.8% (n = 17/24, 95% CI 48.8–86.6%) among participants who had a positive RDT result without thrombocytopenia, based on participants with available platelet count data.

Figure 2.

Distribution of hemoglobin and platelet count by film malaria diagnosis.

DISCUSSION

Standard malaria RDTs were 100% sensitive in detecting P. falciparum malaria in CYP with fever and travel to an area endemic for malaria, with 98.8% specificity and a negative predictive value of 100% (99.7–100%). Sensitivity was lower (93.6%) and specificity higher (99.4%) for all malaria species combined (the WHO recommends a sensitivity threshold of 95% for malaria-endemic settings). These results should be treated with caution as there were three false negatives out of 11 non-falciparum malaria infections—previous studies have shown lower diagnostic accuracy for non-falciparum species [20]. Thrombocytopenia (< 150 × 10⁹/L) increased the RDT’s positive predictive value from 84.6% alone—malaria prevalence was low in this cohort—to 100%.

At the prevalence of malaria observed in this study, we might expect the RDT to return a false-negative result once per 470 individuals tested.

Comparison with Other Literature

Previous malaria diagnostic accuracy studies in non-endemic countries have either enrolled only adult patients, or combined adult/child populations with the number of children not specified. A meta-analysis of 21 studies containing 5,747 mostly adult patients concluded that RDTs, while a useful adjunct, should not replace microscopy [21].

A more recent prospective study in eight French hospitals in mostly adult patients showed RDTs detecting the HRP2 antigen had high sensitivity (96%) and specificity (97%) but as RDTs with a pan-malarial antigen had a low negative predictive value (52–64%) for other Plasmodium species, the authors concluded that “all negative results must be confirmed by the reference diagnostic method when clinical, biological, or other factors are highly suggestive of malaria” [20]. Our study’s use of combination RDTs had a higher negative predictive value of 99.8% (95% CI 99.4–100.0%) for all Plasmodium species.

A prospective study of 852 returning travelers in the USA evaluated RDTs against microscopy and found combination RDTs had an overall sensitivity of 97% for malaria and 100% for P. falciparum, outperforming microscopy. It was unclear how many children were enrolled and the authors did not advocate RDT use alone for preliminary diagnosis [22].

Confirming previous studies, thrombocytopenia was strongly associated with malaria infection. While early diagnosis of thrombocytopenia—usually available ahead of malaria diagnosis by microscopy—is an important marker for possible malaria infection, other imported infections such as dengue and leptospirosis can also show a drop in platelet counts [23] and a full blood count is not routinely point-of-care in UK EDs.

Our study did not record the clinical severity of infections and no child from our cohort died from malaria. Death from imported malaria in children is rare but severe illness less so [24]. One 20-year observational study of mortality from imported P. falciparum malaria in the UK found no deaths in the under 5 s and one death among 5–18 year-olds [17, 25, 26]. A retrospective study of imported P. falciparum malaria in France (1996–2005) recorded 851 cases of severe malaria out of a total of 4,150 cases (20.5%), with one death [26].

While standard RDT appeared to perform well in detecting P. falciparum in children in our study, pfhrp2 and pfhrp3 (pfhrp2/3) gene deletions in the P. falciparum parasite, which lead to false-negatives in HRP2-based RDTs, place a question mark over the long-term reliability of HRP2-based RDTs. First found in the Amazon basin, these deletions are now found in sub-Saharan Africa, are predicted to increase, and are not uniformly distributed across the globe [27, 28]. While some brands of RDTs using pan-LDH antigens can still correctly diagnose falciparum malaria, the authors of one study concluded that new targets for P. falciparum detection need to be identified urgently, given the reliance on HRP2-based tests in areas endemic for malaria [29].

The use of loop-mediated isothermal amplification of DNA (LAMP), a simple and relatively inexpensive technology, has been explored to aid malaria screening. One French study of 319 suspected cases showed LAMP had 100% sensitivity (95.5–100%) and 93.64% specificity (89.55–96.48%) compared to PCR, with four false-positives [30]. LAMP is now used in that setting to confirm a negative RDT, with microscopy reserved for either positive RDT or LAMP tests only [31].

Strengths

This study is the first to date to report the diagnostic accuracy of malaria RDTs alone to rule out imported malaria infection specifically in children. The multi-center, pragmatic design in a setting with a high number of imported cases supports the applicability of its findings in European, non-endemic settings with relatively large numbers of imported malaria.

LIMITATIONS

Retrospective study designs have an inherent bias—diagnostic algorithms, data collection, data entry, and quality were not planned ahead of time [32] and data collected was based on referral for a test rather than clinical symptoms at the presentation—but the design ensured we were able to capture enough cases to adequately power the study [24].

The standard practice still advocates that up to three films are required to rule out malaria but our study found that 46 out of 47 confirmed cases were diagnosed with a combination of first film and RDT (one case was diagnosed on the second film). The lack of three films for all cases (only 15% of our patients had three films performed, with several of these to monitor treatment) could have missed an infection that was presented elsewhere. The use in one center of the quantitative buffy coat for malaria, and in another, use of thin film only (without a thick film), might also have affected the results. Our study shows that ED clinicians, for reasons that are not clear, rarely follow traditional guidance on three films, as reflected in our results. Our study, like Stauffer et al’s US-based, real-world diagnostic accuracy study evaluating malaria RDTs [22], was based on current UK practice.

The use of anonymized data and limited resources meant follow-up of cases was limited to children re-presenting to the same ED, with the potential to overestimate RDT performance. We found no cases of imported P. knowlesi, endemic in Borneo and Malaysia, which can be difficult to diagnose [33]. In this retrospective study, 5.8% were recorded as having no fever, with the remaining 10% unknown and while we decided to include these cases in the analysis—we believe that a child with no fever at presentation could have been recorded erroneously as having no fever, for example, two cases of malaria in our cohort were recorded as “fever—unknown”—this could have affected the results.

Implications for Clinical Practice and Future Research

Simple, accurate, inexpensive diagnostic tests such as RDTs would be a boon for busy EDs as they are quick and easy to perform and are relatively non-invasive. A recent study in the Democratic Republic of Congo—considered an area where pfhrp2 and pfhrp3 (pfhrp2/3) gene deletions are likely to increase [28]—found single deletions in pfhrp2 in 3/1109 (0.27%) of samples, the authors concluding that HRP2-based RDTs could still be used appropriately in the area studied [34]. However, current RDT’s lower sensitivity for other malaria species and the potential for a rise in pfhrp2 and pfhrp3 (pfhrp2/3) gene deletions in the P. falciparum parasite means that UK EDs must still send blood samples for microscopy.

Earlier diagnosis could be made in the majority of cases if RDTs were used in ED rather than being sent on to the hospital lab as they are currently in the UK [35]. The additional finding of low platelets sent on an accompanying full blood count could help alert clinicians to possible malaria infection or another tropical differential diagnosis (eg, Dengue) before the result of microscopy is known.

A prospective study in countries non-endemic for malaria but with high numbers of imported malaria cases would be ideal to test RDT and platelet count alone against standard microscopy alongside LAMP technology and the new generation RDTs in children while at the same time ensuring the performance of diagnostics is maintained in busy ED settings with multiple operators.

CONCLUSION

Standard RDTs were 100% sensitive in detecting P. falciparum malaria in children with fever and travel to an area endemic for malaria, with false-positives in half of the cases due to prior treatment of infection, but sensitivity was lower (93.6%) for all malaria species, with a high number of false negatives for non-falciparum species. A prospective study should be carried out to confirm external validity and evaluate the use of new malaria diagnostics for use in acute pediatric services in non-endemic settings to aid earlier diagnosis in children and ensure the performance of diagnostics is maintained in busy ED settings with multiple operators.

Appendix 1.

Baseline characteristics for Travel Fever study. Data are shown as n (%) or median [range; interquartile range].

	Positive for malaria (n = 47)	Negative for malaria (n = 1,367)	Total (n = 1,414)
Male sex	25 (53%)	774 (57%)	799 (57%)
Age	9 [0–15; 6–12]	4 [0–15; 2–9]	4 [0–15; 2–9]
Fever
Yes	45 (96%)	1,140 (88%)	1,185 (84%)
No	-	82 (6.0%)	82 (5.8%)
Unrecorded	2 (4.3%)	145 (11%)	147 (10%)
Parasite count (%)	0.5 [0.01–10.4; 0.3–4.1]	-	0.5 [0.01–10.4; 0.3–2.4]
Number of films
1	13 (28%)	1,041 (76%)	1,054 (75%)
2	6 (13%)	145 (11%)	151 (11%)
3+	28 (60%)	181 (13%)	209 (15%)
Hemoglobin at presentation (g/L)	109 [60–136; 98–123]	120 [36–202; 112–129]	120 [36–202; 112–129]
Platelet count at presentation (109/L)	134 [14–486; 56–177]	295 [4–1,580; 234–364]	291 [4–1,580; 228–361]
Died
Yes	-	2 (0.1%)	2 (0.1%)
No	44 (94%)	1312 (96%)	1356 (96%)
Unrecorded	3 (6.4%)	53 (3.9%)	56 (4.0%)
Re-presented with febrile illness within 30 days
Yes	6 (13%)	79 (5.8%)	85 (6.0%)
No	38 (81%)	1209 (88%)	1247 (88%)
Unrecorded	3 (6.4%)	79 (5.8%)	82 (5.8%)

Financial support. This work was supported by the National Institute for Health and Care Research (NIHR) Community Healthcare MedTech and In Vitro Diagnostics Co-operative at Oxford Health NHS Foundation Trust (grant number MIC-2016-018). TF receives funding from the NIHR Applied Research Collaboration Oxford and Thames Valley at Oxford Health NHS Foundation Trust.

Potential conflicts of interest. Christopher Bird—no conflict; Gail N Hayward—no conflict; Philip J Turner—no conflict; Vanessa Merrick—no conflict; Mark D Lyttle—no conflict; Niall Mullen—no conflict; and Thomas R Fanshawe—no conflict.