Conventional and High-Sensitivity Malaria Rapid Diagnostic Test Performance in 2 Transmission Settings: Haiti 2017

Abstract

Accurate malaria diagnosis is foundational for control and elimination, and Haiti relies on histidine-rich protein 2 (HRP2)–based rapid diagnostic tests (RDTs) identifying Plasmodium falciparum in clinical and community settings. In 2017, 1 household and 2 easy-access group surveys tested all participants (N = 32 506) by conventional and high-sensitivity RDTs. A subset of blood samples (n = 1154) was laboratory tested for HRP2 by bead-based immunoassay and for P. falciparum 18S rDNA by photo-induced electron transfer polymerase chain reaction. Both RDT types detected low concentrations of HRP2 with sensitivity estimates between 2.6 ng/mL and 14.6 ng/mL. Compared to the predicate HRP2 laboratory assay, RDT sensitivity ranged from 86.3% to 96.0% between tests and settings, and specificity from 90.0% to 99.6%. In the household survey, the high-sensitivity RDT provided a significantly higher number of positive tests, but this represented a very small proportion (<0.2%) of all participants. These data show that a high-sensitivity RDT may have limited utility in a malaria elimination setting like Haiti.

Field-deployable diagnostic tests for malaria serve to allow reliable confirmation of infection and appropriate malaria case management [1]. A Plasmodium falciparum species-specific antigen, histidine-rich protein 2 (HRP2), is the most common rapid diagnostic test (RDT) antigenic target in P. falciparum–endemic settings due to its high expression level and multiepitope avidity [2, 3], though it is known to linger for weeks to months in human circulation following clearance of parasites [4]. Recent improvements to malaria RDTs include the enhanced sensitivity of antigen detection as well as new antigenic targets more useful for identifying non–falciparum malaria infections [5].

As a nation approaches malaria elimination, a higher proportion of all infections are asymptomatic and do not exhibit treatment-seeking behavior, and monitoring changes in true infection prevalence over time (or changes due to an intervention) becomes particularly difficult [6–9]. The nation of Haiti is currently focused on interruption of local malaria transmission, and identification of a high proportion of all infections is paramount (see www.malariazeroalliance.org) [10]. Within this endeavor, the recently developed high-sensitivity RDT (hsRDT; SD Bioline) is being evaluated in direct comparison with a conventional RDT (cRDT) produced by the same manufacturer. Both tests exclusively detect the P. falciparum HRP2 antigen, and performance characteristics for these 2 tests have been considered in other areas of the world from previous studies: Myanmar [11, 12], Uganda [12], and Tanzania [13].

As part of ongoing elimination efforts, 3 surveys were completed in Haiti in 2017. Two of these occurred in different transmission settings in the country and utilized an easy-access group (EAG) sampling design within schools, health facilities, and churches. A third survey was performed in a low-transmission setting using a simple random sample of households from a census-based sampling frame. In total, 32 506 persons who were enrolled from these 3 surveys received both cRDT and hsRDT for presence of P. falciparum HRP2 antigen. We report here the concordance in test results between these 2 RDTs, and evaluation of test performance of a subset of these samples in comparison with a bead-based laboratory assay for the HRP2 antigen and 18S rDNA photo-induced electron transfer polymerase chain reaction (PET-PCR) assay for P. falciparum DNA.

METHODS

Human Subjects

For all surveys, laboratory staff did not have access to personal identifiers. Persons consented for diagnostic tests and blood sample assays for markers of malaria. Activity did not constitute engagement in human subjects research as determined by the US Centers for Disease Control and Prevention (CDC) Center for Global Health Human Subjects office (number 2016–135a), and field activities received approval from the Haitian Ministry of Public Health and Population Bioethics Committee (Comité National de Bioéthique) and the institutional review boards (IRBs) of Tulane University and the London School of Hygiene and Tropical Medicine.

Participant Enrollment

Information regarding participant consent for surveys is presented in the Supplementary Data. EAG surveys received approval from the Haitian Ministry of Public Health and Population Bioethics Committee (number 1516–30), and the IRBs of Tulane University (number 794709) and the London School of Hygiene and Tropical Medicine (number 10393). A single finger-prick was performed on consenting participants to collect capillary blood for cRDT (SD Bioline Malaria Antigen P.f., 05FK50, Standard Diagnostics), hsRDT (also known as ultrasensitive RDT, Alere Malaria Ag P.f., 05FK141, Standard Diagnostics), and blood spots on filter paper (Whatman 903, GE Healthcare). Blood was dried on filter paper overnight, and each filter paper was stored in an individual baggie with desiccant at ambient temperature protected from light. On a weekly basis, samples were shipped to the Haitian national laboratory where they were stored at 4°C until laboratory processing. Individuals with a positive cRDT result received free treatment as per the national policy in Haiti.

For the household survey, blood was collected for RDTs and prepared on filter papers in the same manner as the EAG surveys. The household survey was approved by the Haiti Ministry of Public Health and Population National Bioethics Committee (numbers 1516–29 and 1617–31) and the IRBs of the CDC (number 6821) and the London School of Hygiene and Tropical Medicine (number 10466).

Selection of Samples for HRP2 Detection and DNA Analysis

Due to the inherent differences in sampling design, analyses for the 2 EAG surveys are presented together, and household survey estimates presented separately. All the participants from the EAG and household surveys with a positive cRDT or hsRDT were selected for quantification of HRP2 antigen by bead-based assay and parasite density determined by PET-PCR. For the EAG studies, selection of samples from persons not RDT positive were biased toward the venues where infections had been found, with 80% of samples selected from “high risk” venues (meaning at least 1 RDT positive was identified). Additional samples from persons sampled in the “low risk” venues (no RDT positives) were randomly selected until a sample size of 300 for Artibonite EAG and 750 for Grand Anse EAG was reached.

Only RDT-positive persons were selected from the household survey for further laboratory analysis. Of 161 persons in whom either RDT was positive (and results were available for both RDT types), 153 (95.0%) had a blood sample available for antigen detection and PET-PCR.

Laboratory Assays for HRP2 and Parasite DNA

Presence and quantification of antigens utilizing the bead-based Luminex-based MAGPIX platform (Luminex Corporation) [14] and parasite DNA detected by PET-PCR [15] were performed as described in the Supplementary Data.

Statistical Analysis

A logistic regression model was fit to the HRP2 concentration vs RDT result dose-response data, and was used to estimate the HRP2 concentrations at which 50%, 75%, 90%, and 95% of the RDTs would be expected to be positive in the study population [16]. Both locally estimated scatterplot smoothing (LOESS) and logistic regression curves with 95% confidence intervals were created using R software version 3.3.0, “stats” package (R Foundation for Statistical Computing). Characteristic of test performance through receiver operating characteristic curve (ROC) analysis was performed through the SAS PROC LOGISTIC command with the ROC statement (SAS version 9.4 software). The Youden J statistic as a measure of informedness was calculated for each RDT for each of the 2 EAG surveys by the equation J = (sensitivity + specificity) − 1 [17]. Testing for statistical significance between estimated parasite densities of cRDT vs hsRDT results was performed by the PROC TTEST procedure in SAS. All Cohen κ statistics were generated in SAS. McNemar χ² statistic with 1 degree of freedom was generated for age categories within the household-based survey, but modified by the Edwards continuity correction due to low numbers of discordant results [18], and test statistic generated by the equation:

$${\chi }^2=\frac{{(\left|b-c\right|-1)}^2}{b+c}$$

to approximate the exact binomial test with b and c indicating cells with discordant results.

RESULTS

Presentation of Results by Separate Sampling Designs

The majority of RDT positives in the 2 EAG surveys were found in the treatment-seeking population, with 74.5% (301/404) of all RDT-positive persons enrolled at health facilities. Conversely, all persons enrolled from the household survey were at their residence at time of enrollment, so not seeking standard medical treatment by definition. Due to this inherent difference between the EAG and household survey participant enrollment, and the fact that symptomatic malaria infections will skew blood antigen concentrations higher, results are presented separately for these 2 survey types.

RDT Concordance and Test Results Relative to Blood HRP2 Concentration

Five additional HRP2-positive persons were found by the hsRDT for the Artibonite EAG, and 14 additional positives in the Grand Anse EAG survey (Figure 1A and 1B). Additional hsRDT positives represented a 10.2% increase in RDT-positive numbers in Artibonite and 3.9% increase in Grand Anse, with only the augmented numbers in the Grand Anse EAG significantly differing. For both surveys, agreement was strong between the reported results for the 2 RDTs, with a Cohen κ coefficient of 0.946 for the Artibonite EAG and 0.978 for the Grand Anse EAG. Of 1001 blood samples selected for further laboratory analyses (RDT concordance for these in Supplementary Figure 2), both parametric and nonparametric regression models provided a similar dose-response relationship with high convergence for both survey sites and RDTs (Figure 1C and 1D). One visible deviation among the LOESS and logistic curves occurred due to a single blood sample found to have high levels of HRP2, but that participant’s RDT results were called negative (Figure 1C). Estimates are provided for each RDT and EAG study site for test sensitivity at 50%, 75%, 90%, and 95% confidence of a positive test result at a given concentration of HRP2 antigen in the person’s blood sample (Table 1). At 50% confidence, all estimates of HRP2 concentration for the cRDT and hsRDT from both EAG survey locations were similar and had overlapping confidence intervals, but estimates began to diverge by survey location as level of confidence increased. At the 95% confidence level, estimates for HRP2 concentration were significantly different for the cRDT between the Artibonite and Grand Anse EAGs (2.84 vs 14.61 ng/mL, respectively). No significant differences were observed between the cRDT and hsRDT estimates within the same survey area for any of the confidence levels.

Figure 1.

Concordance between test results for the 2 types of rapid diagnostic tests (RDTs) in Artibonite and Grand Anse easy-access group surveys and the dose-response relationship between histidine-rich protein 2 (HRP2) antigen concentration and RDT result. Panels are shown for the Artibonite and Grand Anse study sites for the conventional and high-sensitivity RDTs (cRDT/hsRDT, respectively). Two-by-two tables for the 2 RDTs comparing test concordance for all persons enrolled in the Artibonite (A) and Grand Anse (B) surveys with Cohen κ agreement. *McNemar test to indicate statistical significance of discordant test results between the 2 RDTs was modified to adjust for low numbers of discordant results as described in the Methods. Logistic and locally estimated scatterplot smoothing regression of probability of RDT positivity by antigen concentration in study participants for Artibonite (C) and Grand Anse (D) surveys. Outputs for the logistic regression are shown in Table 1.

Table 1.

Modeled Concentrations of Histidine-Rich Protein 2 Detection by Probability of a Positive Test for the Conventional and High-Sensitivity Rapid Diagnostic Tests at the 2 Easy-Access Group Study Sites

	HRP2 Concentration, ng/mL (95% CI)
Sensitivity	Artibonite, cRDT	Artibonite, hsRDT	Grand Anse, cRDT	Grand Anse, hsRDT
50%	0.318 (.18–.66)	0.213 (.12–.46)	0.458 (.32–.66)	0.326 (.23–.47)
75%	0.719 (.33–1.5)	0.542 (.24–1.2)	1.659 (1.0–2.6)	1.216 (.77–1.9)
90%	1.623 (.52–3.6)	1.379 (.42–3.3)	6.012 (3.2–11)	4.538 (2.4–8.1)
95%	2.849 (.65–6.4)	2.607 (.54–6.4)	14.61 (6.5–27)	11.19 (4.8–21)

Abbreviations: CI, confidence interval; cRDT, conventional rapid diagnostic test; HRP2, histidine-rich protein 2; hsRDT, high-sensitivity rapid diagnostic test.

Histograms showing the entire range of HRP2 concentrations for the samples designated for laboratory analysis are shown in Supplementary Figure 3, and many more persons selected from Grand Anse were positive for HRP2 (n = 322 [45.9%]) than from Artibonite (n = 50 [16.7%]). As the difference in dose-response regression estimates for HRP2 concentration at the 95% confidence level was minor between the 2 RDT types, few additional positives would be predicted to be discovered by the high-sensitivity test (as reflected by the actual counts in Figure 1A and 1B).

In direct comparison with the bead-based HRP2 laboratory assay, both RDT types performed well at both survey sites with ROC area under the curve values of ≥0.93 when modeling for RDT result based on HRP2 antigen positivity (Figure 2). If considering the laboratory assay to be the gold standard for HRP2 detection, the cRDT had sensitivities of 86.3% and 95.0% at the Artibonite and Grand Anse sites, respectively, and specificities of 99.6% and 92.9%. The hsRDT had slightly higher sensitivities of 88.2% and 96.0% at the Artibonite and Grand Anse sites, but slightly lower specificities of 98.0% and 90.0%.

Figure 2.

Performance of the 2 rapid diagnostic tests (RDT) in comparison with the laboratory histidine-rich protein 2 bead assay. Two-by-two tables, receiver operating characteristic curves, and performance measures are shown for the 2 easy-access group (EAG) study sites for the 2 types of RDTs employed. Abbreviations: AUC, area under the curve; cRDT, conventional rapid diagnostic test; hsRDT, high-sensitivity rapid diagnostic tests; NPV, negative predictive value; PPV, positive predictive value.

RDT Result Relative to Estimated Parasite Density at Time of Sampling

Though the only target for these RDTs was the HRP2 antigen, comparison to P. falciparum parasite densities provides valuable information regarding the utility of the tests to identify active infections. Most individuals (97.5%) who tested RDT negative in the EAG surveys did not have detectable P. falciparum parasite DNA, and persons with positive RDT results had evidence of P. falciparum DNA >80% of the time for both surveys and both tests (Figure 3 and Table 2). No statistically significant differences were observed in comparing the mean estimated parasite densities among all 4 RDT-negative categories ([2 RDT categories] × [2 survey sites]), or among all 4 RDT-positive categories.

Figure 3.

Estimated Plasmodium falciparum parasite densities by rapid diagnostic test (RDT) result. Panels are shown for the Artibonite and Grand Anse easy-access group study sites and categories indicate negativity or positivity to the conventional RDT (cRDT) or high-sensitivity RDT (hsRDT). Horizontal bars show median parasite density as estimated by photo-induced electron transfer polymerase chain reaction (PCR) for the respective category.

Table 2.

Polymerase Chain Reaction Positivity and Estimated Parasite Density by Rapid Diagnostic Test Result

RDT Result by Site	Plasmodium falciparum DNA Positive, %	Estimated Parasite Density of DNA Positives, Parasites/μL
Artibonite
cRDT
Positive	88.4	511.4
Negative	4.5	6.7
hsRDT
Positive	85.4	463.2
Negative	3.3	10.3
Grand Anse
cRDT
Positive	84.0	1009.9
Negative	6.6	15.7
hsRDT
Positive	81.6	998.1
Negative	5.6	15.6

Abbreviations: cRDT, conventional rapid diagnostic test; hsRDT, high-sensitivity rapid diagnostic test; RDT, rapid diagnostic test.

Dose-response modeling of cRDT and hsRDT result as a function of estimated parasite density is shown in Supplementary Figure 4. Since so few RDT-negative persons were found to harbor P. falciparum DNA, the logistic sigmoidal curve did not reach a lower asymptote, and estimates for lower levels of RDT performance could not be derived. For both RDT tests at both survey sites, reliability of a positive RDT test in identifying active P. falciparum infections was shown to decrease only at the lowest estimated parasite densities (<100 parasites/μL).

Household Survey cRDT and hsRDT Concordance and Differences in HRP2 Antigen and Parasite Densities

In the Artibonite household survey, 21 517 persons were administered both types of RDTs, and agreement between the cRDT and hsRDT was strong for all age categories: 0–5, 6–15, and >15 years (Figure 4A). Persons tested cRDT negative but hsRDT positive in all 3 age categories, with increasing test statistic of discordant results as age increased. Among the 3 age categories, only slight variation was observed in the distribution of HRP2 concentrations among the RDT-positive persons (Supplementary Figure 5), and positive correlation was observed for antigen concentration and parasite density (Supplementary Figure 6). One person tested positive by cRDT and negative by hsRDT, but a blood sample was not available for investigation of HRP2 antigen or parasite presence. Overall, the hsRDT identified 36 more HRP2-positive persons (28.8% more positives than the cRDT alone), but this only represented 0.17% of the total study population. Highly significant differences were seen in HRP2 concentrations between the blood samples of those who tested positive by both RDTs (median, 23 577 pg/mL) or the hsRDT only (median, 1168 pg/mL) (Figure 4B). Additionally, highly significant differences were observed in PET-PCR estimated parasite density in persons testing positive by both RDTs (median, 8.64 [range, 0.0–2045.8] parasites/μL) or the hsRDT alone (median, 0.90 [range, 0.0–11.4] parasites/μL).

Figure 4.

Concordance of conventional rapid diagnostic test (cRDT) and high-sensitivity rapid diagnostic test (hsRDT) for large household-based survey in Artibonite. A, Two-by-two tables are separated by age categories: 0–5, 6–15, and >15 years of age, with estimates for Cohen κ statistic for agreement between the 2 types of RDTs. *McNemar test to indicate statistical significance of discordant test results between the 2 RDTs was modified to adjust for low numbers of discordant results as described in the Methods. Differences in histidine-rich protein 2 (HRP2) antigen levels (B) and photo-induced electron transfer polymerase chain reaction–estimated parasite density for blood samples from persons testing positive for both RDTs, or high-sensitivity RDT (hsRDT) only. Antigen levels are log-transformed and normal and kernel distributions overlaid. Boxes are 25% and 75% percentile with the horizontal line displaying the median. Whiskers display minimum and maximum values within 1.5 interquartile range and circles are outliers beyond this range.

cRDT to hsRDT Comparison Summary Across All 3 Surveys

In total, 32 506 persons had both a cRDT and an hsRDT performed. Of these, 565 (1.74%) tested positive by any RDT. Of these 565 positives, 509 (90.1% of all positives, 1.57% of total population) were positive by both RDTs, 55 (9.73% of all positives, 0.17% of total population) were positive by only the hsRDT, and 1 person was positive by only the cRDT.

DISCUSSION

A nation faces many challenges as it progresses from very low malaria to no malaria, and traditional field diagnostic tests of microscopy and cRDTs were not designed for detecting low-density infections [7, 19]. Newer field-deployable diagnostics have undergone testing in field settings with promising results in their ability to detect low-density P. falciparum infections by molecular [20–23] and antigen detection [11, 12] methods. Though these high-sensitivity field diagnostic tests consistently detect more infections than their conventional counterparts of microscopy or cRDT, the significance of detected very low-density infections is still being investigated with respect to potential transmissibility and overall population-based estimates [24, 25].

Transmission of P. falciparum in Haiti is seasonal and heterogeneous [26–28], with some regions of the country estimated to have parasite prevalence >5%, whereas other regions have a complete absence of reported malaria [10, 29]. This current study included 2 sites in Haiti representing 2 different transmission strata: Grand Anse as the area of the nation currently reporting the most clinical cases, and Artibonite as a setting of lower P. falciparum prevalence ([10] and internal data from the national malaria program). A previous Artibonite prevalence estimate of P. falciparum infection was approximately 3% with most of these individuals being asymptomatic carriers at the time of the survey [28]. For this current study, EAG test positivity rates for any RDT in Artibonite and Grand Anse confirmed the disparity in number of P. falciparum infections, with 0.83% (49/5876) of participants in the Artibonite EAG and 7.1% (355/5014) of participants in the Grand Anse EAG surveys testing RDT positive. If assessing the added number of positives provided by the hsRDT in these 2 surveys, 0.08% more of the sampled population in Artibonite (5 persons) and 0.28% more in Grand Anse (14 persons) had levels of HRP2 antigen low enough to be missed by the cRDT, but detectable by the hsRDT. This finding has been observed previously, with a low overall percentage of the study population found to harbor HRP2 antigen levels in the “opportunistic” zone of detection between the sensitivities of the 2 tests [12, 25]. However, studies in some higher-transmission settings have found considerably greater percentages of the population that tested positive with an hsRDT alone [11, 12]. Even within a country of heterogeneous transmission patterns, the hsRDT may have value for some areas in providing more accurate (and substantial) prevalence estimates, yet not be significantly advantageous over a cRDT in other settings. Importantly, most (75%) of the RDT-positive persons from the Haiti EAG surveys came from treatment-seeking individuals enrolled in health facilities, and higher parasite and antigen densities would negate any benefit of detecting low levels of the HRP2 antigen [13].

A key factor of RDT performance is overall HRP2 carriage in the population, which itself is determined by the level of P. falciparum infection [30], recent infection (lingering HRP2 in host blood [4, 31]), and potential accumulation of HRP2 from tandem and/or frequent infections. In the lower-transmission Artibonite setting, there was only a small number of individuals with HRP2 levels near RDT detection limits. However, in Grand Anse, where many more infections were found (and higher likelihood of P. falciparum exposure in the preceding few months), a much higher percentage of persons had intermediate and low levels of HRP2 antigen—giving the hsRDT an opportunity to be the sole detector of these low HRP2 concentrations. Antigen persistence potentially explains our finding in the higher-transmission Grand Anse setting: 18.4% of hsRDT-positive persons were not found to have P. falciparum DNA. However, some of these “false positive” tests may have actually been very low-density infections undetectable by the PET-PCR assay as this nucleic acid assay is less analytically sensitive than the hsRDT [15, 30]. Additionally, sampling from this single point in time does not take into account fluctuations of parasites through replication cycles or parasite load from the recent past. Though being a “false positive” in the sense of no active parasitemia, detection of the HRP2 antigen following a P. falciparum infection could still be a useful indicator for a malaria program as a clear proxy of current or recent P. falciparum exposure at the individual and population levels [14].

Among both tests and both EAG sites, overall sensitivity and specificity estimates were high (≥86.3%) when defining the laboratory HRP2 detection assay test as the gold standard. Though cRDT and hsRDT performance within a single study site was basically equivalent, consistent differences were seen between surveys with both types of RDTs providing a lower sensitivity in Artibonite, but higher specificity. The Youden J statistic (as a measure of a probability of making an informed decision with 1.0 being perfect decision making [17, 32]) was strong for all scenarios with values of 0.86 or higher. In Grand Anse, multiple samples were positive by the HRP2 laboratory assay only, but an even higher number of persons were RDT positive only for both RDT types. The RDT positives only would lower specific estimates, but this is potentially an artificial depression due to inherent differences in the RDT and laboratory tests. The field RDT uses fresh, undiluted blood immediately drawn from a participant, whereas the sample type for the laboratory test is dried blood on filter paper. To rehydrate blood and remove it from filter paper, an intrinsic dilution of the blood sample is needed [14]. Additionally, antigen degradation is possible from drying of blood, storage conditions, or length of storage. Even with the increased detection limit of the laboratory test, some very low HRP2 concentrations have the potential to be detected by field tests but missed in the laboratory.

Active P. falciparum infections detected from the household survey in Artibonite were typically found to be low density, with 80.3% <100 parasites/μL and 54.9% <10.0 parasites/μL. Of 21 591 persons in the household survey, 0.72% of 0- to 5-year-olds, 1.1% of 6- to 15-year-olds, and 0.59% of >15-year-olds tested positive by any RDT. Concordance between the cRDT and hsRDT was very good, alluding to the high sensitivity and specificity in detecting HRP2 antigen in participants’ blood. Though the hsRDT detected a statistically higher number of positives in all age categories, this was 0.2% or less from any category, and 0.17% overall regardless of age. In endemic settings, younger persons are at greater risk for high-density P. falciparum infections, whereas older individuals are more able to suppress parasite replication [33, 34], but our study found little difference in the distribution of HRP2 concentrations among age categories. Comparison of persons testing positive to both RDTs vs the hsRDT alone found significantly lower antigen (and parasite) levels in persons detected by the hsRDT alone, but there was simply just a low number of Artibonite residents with these low antigen levels at the time of sampling.

In the malaria elimination setting of Haiti, both conventional and high-sensitivity HRP2-based RDTs performed well, and high concordance between the 2 tests was observed in 2 different transmission zones. In comparison with the cRDT, statistically higher numbers of positive tests were observed with the hsRDT, supporting the increased HRP2 detection capacity of this test, but this was a very small percentage of the overall population sampled. These findings are in line with other studies in malaria elimination settings that have noted more positives when employing the hsRDT but that these slightly higher estimates do not change the overall prevalence for relevant programmatic purposes. Though likely of limited utility in the Haitian setting, further evaluation of novel antigen-based tests is warranted to investigate utility in different transmission settings, P. falciparum genotypes, and human populations. As innovative diagnostics are introduced, increased performance of a novel test in a population will also need to be weighed against the increased costs for that test.