Evaluation of 2 Lateral Flow Rapid Tests in the Diagnosis of Chagas Disease in the Washington Metropolitan Area

Abstract

We compared the accuracy of the Stat-Pak and Chagas Detect Plus with a latent class analysis. Sensitivity values of 89.7% and 91.9% and specificities of 97.1% and 80.3%, respectively, were seen in the serodiagnosis of Chagas disease in Hispanic immigrants, revealing the limitations of these tests in diverse populations.

The diversity of Hispanics in the United States from varying Chagas-endemic countries provides challenges in the diagnosis of Trypanosoma cruzi infection (TcI) [1, 2]. The anti–T. cruzi antibody levels and sensitivity detected by Food and Drug Administration (FDA)–cleared tests are lower in Mexicans and Central Americans as compared with South Americans [1, 2]. These variations align with the geographic distribution of T. cruzi discrete typing units (DTUs) [2], with TcI being predominant in Mexico, Central America, and Northern South America, and TcII/V/VI in Southern South America [3].

The inadequate specificity of some FDA-cleared tests leads to a high ratio of false positives to true positives in this low-prevalence population [1]. These limitations create the need for further follow-up in this population, which is mainly uninsured and is usually taken care of by nonprofit organizations or community clinics.

The use of rapid tests that can be performed in a community setting facilitates the detection and follow-up of hard-to-reach populations. Most rapid tests are performed in 15–20 minutes, so potentially infected individuals can be informed of their results during their first contact, which increases adherence to follow-up testing of rapid test–positive individuals [4]. As the World Health Organization (WHO) recommends the use of at least 2 different tests, researchers in low-resource settings have suggested the parallel use of 2 rapid tests based on different antigens to facilitate detection [5].

The Chagas Detect Plus (CDP, InBios International, Inc., WA, USA) is based on a multiepitope fusion antigen ITC 8.2 (TcF, SAPA, Pep30, Pep36, KMP11, KMP11, Pep1) and was cleared by the FDA in 2016. Studies in the United States suggest that CDP has suboptimal specificity but adequate sensitivity (≥97.0%), even for infections acquired in Mexico and Central America [1].

The Stat-Pak (Chembio Diagnostic Systems, NY, USA) is not FDA-cleared but has been extensively evaluated in Southern South America [6]. The assay uses recombinant proteins (B13, 1F8, and H49/JL7). In a meta-analysis, Stat-Pak sensitivity in Bolivians was 97.0% (95% CI, 87.6%–99.3%) and specificity was 99.4% (95% CI, 98.6%–99.8%) when compared with at least 2 different diagnostic tests [6]. Studies in TcI-predominant areas found sensitivity values ranging from 95.0% to 100.0% in repository samples from Central Americans [7, 8] and 62.5% in umbilical cord blood samples in Mexico [9], but evaluations were predominantly in comparison with only 1 reference test [7, 9].

Due to the observed geographic variability in test performance, which is likely due to the differences in the geographic distribution of T. cruzi DTUs, we conducted this evaluation to generate more evidence of test accuracy in at-risk immigrants. To determine whether the use of 2 different rapid tests can facilitate early diagnosis as recommended by the WHO, we evaluated the CDP and the Stat-Pak in serum samples of Latin Americans living in the Washington Metropolitan Area (WMA).

METHODS

A random sample of seronegatives (n = 350) was selected from a cross-sectional study that enrolled healthy Hispanic adults from any of the 21 Chagas-endemic countries via recruitment in churches, community centers, consulate events, and health fairs in the WMA [2]. Individuals were classified as being from TcI-predominant (Mexicans and Central Americans, n = 16) and TcII/V/VI-predominant areas (Bolivians, n = 21) [3]. Asymptomatic Latin American immigrants from any of the 21 Chagas-endemic countries were enrolled (Supplementary Methods) [2].

The 2 rapid tests were compared against the conditional probability of class membership to determine Chagas status using a modified version of a latent class analysis (LCA), which has been previously described (Supplementary Methods, Supplementary Table 1) [2]. The modified LCA used area of origin and the results of 2 FDA-cleared immunoassays: Hemagen Chagas kit (Hemagen Laboratories, MD, USA), Chagatest recombinant, version 3.0 (Wiener Laboratories SAIC, Argentina), and the IgG-TESA-blot (a Western blot that uses the Trypomastigote Excretory-Secretory Antigen) (Supplementary Methods, Supplementary Table 2) [2, 10, 11]. The 3 assays are based on different antigens and were run in parallel on all samples (Supplementary Table 3). The statistical software Mplus, version 8 (Muthén & Muthén, Los Angeles, CA, USA), was used for LCA.

The assays were read by 2 observers blinded to each other’s reading and the LCA specimen classification. The weighted kappa index comparing positive and negative results between the 2 readers was 0.87 (P < .001) for the CDP and 0.89 (P < .001) for the Stat-Pak. Samples with any reactions in the test line were considered positive.

Band intensities were recorded at the end of the incubation time by the 2 observers using an intensity card elaborated by our research group (Supplementary Figure 1); the mean value of band intensities was used for future analysis. To determine time-to-positivity, results were also read every 5 minutes after the addition of the sample and reagents and until the incubation time (15 and 20 minutes for the Stat-Pak and the CDP, respectively).

The sensitivity, specificity, and Youden Index (YI) of the test were calculated using 2 × 2 tables against the results of the LCA. The chi-square test and the Kruskal-Wallis test were used to determine statistical associations. We used a receiver operating characteristics curve (ROC) to determine the cutoffs of band intensities that provide the highest YI. A 2-sided P value of <.05 was considered significant. These statistical analyses were done using Stata, version 15 (StataCorp, TX, USA).

Patient Consent Statement

The Institutional Review Board of the Johns Hopkins School of Public Health approved the protocol. All participants provided written informed consent.

RESULTS

CDP sensitivity was lower in TcI-predominant (87.5%) compared with TcII/V/VI-predominant (95.2%; P = .39) areas, but the difference was not statistically significant, which could be due to the small number of seropositives. CDP specificity was low in both endemic regions (80.3%, 241/300). Significantly lower Stat-Pak sensitivity was observed in TcI-predominant compared with TcII/V/VI-predominant areas (75.0% vs 100.0%; P = .03) (Table 1).

Table 1.

Performance of the 2 Lateral Flow Assays, Chagas Detect Plus (InBios International) and Stat-Pak (Chembio Diagnostic Systems), for the Diagnosis of Chronic Chagas Disease in 2 Endemic Areas

	Both Areas	TcI-Predominant	TcII/V/VI-Predominant	P Value (TcI- vs TcII/V/VI- Predominant
CDP
Sensitivity, % (n/N)	91.9 (34/37)	87.5 (14/16)a	95.2 (20/21)b	.39
Specificity, % (n/N)	80.3 (241/300)	79.8 (142/178)	81.2 (99/122)	.77
Time-to-positivity, min
Seropositives, median (IQR)	5.0 (5.0–5.0)	5.0 (5.0–5.0)	5.0 (5.0–5.0)	.35
False positives, median (IQR)	10.0 (5.0–10.0)	10.0 (5.0–10.0)	10.0 (5.0–10.0)	.93
P value (difference between seropositives and false positives)	<.001	.010	<.001	-
Band intensity, score
Seropositives, median (IQR)	5.0 (4.0–6.0)	4.0 (3.0–5.5)	5.5 (4.8–6.0)	.02
False positives, median (IQR)	2.0 (1.0–2.5)	2.0 (1.0–2.3)	2.0 (1.5–3.0)	.23
P value (difference between seropositives and false positives)	<.001	<.001	<.001	-
Stat-Pak
Sensitivity, % (n/N)	89.7 (26/29)	75.0 (9/12)c	100.0 (17/17)b	.03
Specificity, % (n/N)	97.1 (340/350)	96.2 (230/239)	99.1 (110/111)	.13
Time-to-positivity, min
Seropositives, median (IQR)	5.0 (5.0–5.0)	5.0 (5.0–5.0)	5.0 (5.0–5.0)	.49
False positives, median (IQR)	7.5 (5.0–15.0)	5.0 (5.0–15.0)	15.0 (15.0–15.0)	.25
P value (difference between seropositive and false positives)	.03	.29	.02	-
Band intensity, score
Seropositives, median (IQR)	4.0 (3.5–4.5)	3.5 (2.5–4.0)	4.0 (3.5–5.0)	<.01
False positives, median (IQR)	2.0 (1.5–2.0)	2.0 (1.5–2.0)	2.0 (2.0–2.0)	.85
P value (difference between seropositive and false positives)	<.001	.005	.09	-

Bold values denote statistical significance.

Abbreviations: CDP, Chagas Detect Plus, InBios International; IQR, interquartile range; TCI, Trypanosoma cruzi infection.

^aParticipants in this group were from El Salvador (n = 11), Guatemala (n = 3), Honduras (n = 1), and Mexico (n = 1).

^bAll participants in this group were from Bolivia.

^cParticipants in this group were from El Salvador (n = 7), Guatemala (n = 3), Honduras (n = 1), and Mexico (n = 1).

False positives in the 2 rapid tests had higher time-to-positivity and lower band intensities compared with true positives. No differences were observed in time-to-positivity and band intensities between seropositives in TcI-predominant and TcII/V/VI-predominant areas in either of the 2 tests, possibly because our seropositives from TcI-predominant areas were mainly from Central America, where reactivity seems to be lower than South America but higher than Mexico [1].

A very low YI was observed for the Stat-Pak in TcI-predominant areas (YI at a cutoff ≥2 in TcI-predominant areas: 0.77; vs TcII/V/VI-predominant areas: 0.99) (Table 2).

Table 2.

Sensitivity, Specificity, and Youden Index of Different Cutoffs of Band Intensities Observed With the 2 Rapid Tests (Chagas Detect Plus and Stat-Pak) in the 2 Geographic Areas

	Chagas Detect Plus			Stat-Pak
Cutoffa	Sensitivity	Specificity	Youden Index	Sensitivity	Specificity	Youden Index
	Both areas (n = 335)			Both areas (n = 259)
	ROC, 0.94 (95% CI, 0.88–0.99)			ROC, 0.96 (95% CI, 0.91–1.00)
(≥1)	0.92	0.82	0.74	0.93	0.93	0.87
(≥2)	0.89	0.89	0.78	0.93	0.96	0.89
(≥3)	0.86	0.96	0.82	0.79	1.00	0.79
(≥4)	0.69	0.99	0.68	0.52	1.00	0.51
(≥5)	0.53	1.00	0.52	0.17	1.00	0.17
(≥6)	0.33	1.00	0.33	0.10	1.00	0.10
	TcI-Predominant Areas (n = 193)			TcI-Predominant Areas (n = 141)
	ROC, 0.90 (95% CI, 0.80–1.00)			ROC, 0.90 (95% CI, 0.88–1.00)
(≥1)	0.87	0.81	0.68	0.83	0.91	0.74
(≥2)	0.87	0.89	0.75	0.83	0.94	0.77
(≥3)	0.80	0.96	0.76	0.50	0.99	0.49
(≥4)	0.47	0.99	0.46	0.25	0.99	0.24
(≥5)	0.27	0.99	0.26
(≥6)	0.20	1.00	0.20
	TcII/V/VI-Predominant Areas (n = 142)			TcII/V/VI-Predominant Areas (n = 118)
	ROC, 0.96 (95% CI, 0.90–1.00)			ROC, 1.00 (95% CI, 1.00–1.00)
(≥1)	0.95	0.83	0.78	1.00	0.97	0.97
(≥2)	0.90	0.88	0.79	1.00	0.99	0.99
(≥3)	0.90	0.95	0.86	1.00	1.00	1.00
(≥4)	0.86	0.99	0.85	0.71	1.00	0.71
(≥5)	0.71	1.00	0.71	0.29	1.00	0.29
(≥6)	0.43	1.00	0.43	0	1.00	0.00

Bold values denote the cutoff point with the highest Youden index.

Abbreviations: ROC, receiving operating characteristics; TCI, Trypanosoma cruzi infection.

^aThe cutoff values represent the score of band intensity in the test line using our band intensity card.

DISCUSSION

CDP specificity in serum was lower in our study than previously reported in US blood donors (UBD; 87.5%–92.3%) [1]. A high CDP sensitivity (≥97.0%) without significant differences among Mexicans and Central and South Americans was reported in UBD [1]. The CDP sensitivity in serum was also lower than our estimate (94.7%; 95% CI, 92.4%–95.0%) in whole blood in community settings [2]. A study in Bolivia also showed slightly lower CDP specificity in serum than whole blood (96.9%; 95% CI, 94.2%–98.6%; vs 98.8%; 95% CI, 95.9%–99.9%) [12]. Serum samples are evaluated under controlled laboratory conditions, where health workers may be more likely to observe faint bands. Health workers in areas such as Bolivia, where high antibody levels prevail [2], might be accustomed to high band intensities and more likely to miss faint bands, resulting in higher specificities in TcII/V/VI-predominant countries. However, a recent study in Bolivia demonstrated suboptimal specificity (91.9%; 95% CI, 88.6%–94.5%), similar to the current data as well as the UBD evaluation [13]. The differences in the results of the CDP specificity between the oldest and recent publications in Bolivia could be due to the differences in the methodology between studies or differences in the performance of the test over time.

The low Stat-Pak sensitivity is similar to the one reported in Mexicans when the test was compared only with the Chagatest recombinant, version 3.0 [9]. The Stat-Pak specificity is comparable with that calculated in a meta-analysis in Bolivians (99.4%; 95% CI, 98.6%–99.8%). However, cross-reactions of the Stat-Pak with leishmaniasis (22.2%, 2/9) and hepatitis B (18.2%, 2/11) have been reported [8]. Assay validation only in TcII/V/VI-predominant areas, where most seropositives have high antibody levels, may lead to the use of low antigen concentrations that are enough to achieve good sensitivity and specificity in TcII/V/VI-predominant areas, but when the test is used in TcI-predominant areas, a lower sensitivity could be observed without affecting the specificity.

One limitation of this study is the small sample size of seropositive samples, especially from Mexicans, who tend to have low antibody levels [1]; this could result in even lower assay sensitivity. Our results cannot conclude that test performance is due to the differences in geographic distribution of DTUs because we have not conducted genotyping analysis.

While the Stat-Pak had high specificity, its variable sensitivity limits its utility for screening in community settings with highly diverse populations. However, its accuracy is high in TcII/V/VI-predominant areas where its use could be justified. Conversely, the low specificity of the CDP in serum samples leads to a high ratio of false positives to true positives in both areas. For both rapid tests, standardized intensity scoring and validation of a borderline category for low-intensity scores could provide a better balance of sensitivity and specificity; however, the need for a second test to confirm or refute borderline results will remain (Table 2). Effective screening of low-prevalence populations will require availability of assays with better performance characteristics.

Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.