Comparative Performance of Latest-Generation and FDA-Cleared Serology Tests for the Diagnosis of Chagas Disease

ABSTRACT

Confirmed diagnosis of chronic Chagas disease (CD) requires positive results by two different IgG serology tests. Variable sensitivity has been reported among tests and in different geographic regions. Inadequate specificity presents a particular challenge in low-prevalence settings such as the United States. This study provides a direct comparison of the latest-generation IgG serology assays with four previously assessed FDA-cleared tests. Seven hundred ten blood donor plasma specimens were evaluated by Wiener Lisado and Wiener v.4.0 enzyme-linked immunosorbent assays (ELISAs) and Abbott PRISM Chagas chemiluminescent assay (ChLIA). Sensitivity and specificity were assessed relative to infection status as determined by the original blood donation testing algorithm. All three latest-generation assays demonstrated 100% specificity (95% confidence interval [CI], 98.6 to 100.0). Wiener Lisado, Wiener v.4.0, and Abbott PRISM had sensitivities of 97.1% (95% CI, 95.1 to 98.4), 98.9% (95% CI, 97.4 to 99.6), and 95.5% (95% CI, 93.2 to 97.3), respectively. As with previously evaluated FDA-cleared tests, all three assays had the highest reactivity and sensitivity in samples from donors born in South America and lowest reactivity and sensitivity in specimens from those born in Mexico, with intermediate results in specimens from Central American donors. Wiener v.4.0 had the highest diagnostic sensitivity in all comparisons. Our findings suggest that the latest-generation CD serology tests could improve diagnostic sensitivity without affecting specificity.

INTRODUCTION

Chagas disease (CD) is responsible for the highest public health burden of any parasitic disease in the Americas, with an estimated 6 million people infected and 1.2 million associated cases of cardiomyopathy (1–3). While rare, autochthonous, vector-borne, transplant, transfusion, and congenital transmissions have been reported in the United States, most of the estimated 300,000 U.S. residents with CD acquired the disease in Latin America (4–8). For testing purposes in U.S. populations, birth or residence for greater than 6 months in an area of endemicity has been utilized as an easily screened risk factor (9).

Confirmation of chronic Trypanosoma cruzi infection requires concordant positive results by at least two distinct IgG serological tests; no single assay is considered to have sufficiently high sensitivity and specificity to be relied on alone (10–15). Because assays use cultured or recombinant parasite proteins as antigen sources, regional parasite genetic diversity is hypothesized to affect test performance (16–18). Discordance between assay results has been reported, especially in populations from Mexico (19).

Four assays have U.S. Food and Drug Administration (FDA) 510(k) clearance for diagnostic use: Hemagen Chagas’ kit enzyme-linked immunosorbent assay (ELISA) (Hemagen Diagnostics, Inc., Columbia, MD), Ortho T. cruzi ELISA test system (Ortho Clinical Diagnostics, Raritan, NJ), Wiener Chagatest Recombinante v.3.0 ELISA (Wiener Laboratories, Rosario, Argentina), and InBios Chagas Detect Plus (CDP) rapid test lateral flow assay (LFA) (InBios International, Inc., Seattle, WA). Ortho ELISA and Abbott PRISM Chagas chemiluminescent assay (ChLIA) (Abbott Laboratories, Abbott Park, IL) are licensed for blood and organ donor screening through a separate FDA approval process (20, 21). While Ortho ELISA has 510(k) clearance for diagnostic use, it is currently marketed only for blood donor testing.

In a previous study from our group, the four FDA-cleared diagnostic assays showed a range of performance characteristics (17). All assays exhibited their lowest sensitivity in samples from donors born in Mexico, who make up the largest proportion of Latin American immigrants in the United States (17, 22). We extended this analysis to include two of the latest-generation serology assays from Wiener Laboratories, Chagatest Recombinante v.4.0 ELISA (Wiener v.4.0) and Chagatest Lisado ELISA (Wiener Lisado), and the Abbott PRISM assay, currently licensed only for donor screening. This direct comparison provides novel data on CD diagnostic test performance across diverse populations living in the United States.

MATERIALS AND METHODS

Ethical approval.

This research was approved by the American Red Cross (ARC) institutional review board and was deemed exempt by the Human Research Protection Program at the University of California, San Francisco (UCSF).

Sample selection and preparation.

Blood donor plasma samples were selected from lists of seropositive and seronegative specimens provided by ARC. From the list of 1,091 seropositive specimens, all specimens with country-of-birth data available were included; the remainder of 500 seropositive specimens were selected at random. From the list of 3,938 seronegative specimens, a random sample of 300 specimens, frequency matched by region of donation to the seropositive specimen set, was selected. Serological status was determined at the time of blood donation (September 2006 to June 2018) (17, 23). Final infection status was defined by ARC testing algorithms, which require repeat reactivity by the screening test (Ortho ELISA or Abbott PRISM) followed by positive results on a confirmatory assay (radioimmunoprecipitation assay or Abbott enzyme strip assay) (23). Three hundred sixty-eight confirmed positive specimens were initially screened by Ortho ELISA (September 2006 to August 2011), while 137 specimens were screened positive by Abbott PRISM (starting September 2011). Matrix effects of blood donor plasma and frozen storage time of specimens in this analysis were previously reported and deemed to have minimal effect on specimen quality (24). Additional details regarding specimen selection are available in our previous publication (17).

Plasma samples were initially frozen at −20°C within 24 h of collection and then thawed in a temperature-controlled water bath, aliquoted, and refrozen at −20°C under research protocol. Aliquots tested by Hemagen, Wiener v.3.0, and InBios assays at UCSF were thawed and refrozen only once (March 2019); those tested by Wiener v.4.0 and Wiener Lisado were thawed and refrozen twice (March 2019 and December 2019). The Ortho ELISA and Abbott PRISM were thawed and refrozen twice for testing at Innovative Blood Resources (Minneapolis, MN) and Abbott Laboratories (Abbott Park, IL), respectively. All specimens run at outside laboratories were tested in a blinded manner.

A total of 800 specimens were thawed and spun at 2,300 relative centrifugal force for 10 min, and 1 ml was aliquoted to randomized positions in 96-deep-well plates. During testing, one randomized sample plate (n = 90) had failed runs by Wiener Lisado and Wiener v.4.0, and there was insufficient reagent to retest; these specimens were excluded from analysis. The final data set comprised 710 plasma samples.

Serology testing.

Specimens were tested and interpreted in accordance with package inserts using the kit reagents, an ELx405 Select microplate washer (BioTek, Winooski, VT), and a SpectraMax Plus microplate reader (Molecular Devices, San Jose, CA) with an optical density at 450 nm (OD₄₅₀). Signal-to-cutoff ratios (S/CO) were utilized for quantitative analyses and determined by dividing the OD₄₅₀ readout by the positive cutoff as described in the package insert. InBios CDP results were scored on a semiquantitative scale of 0 to 6, with a score of 1 or greater considered a positive result, as previously described (17). Deviation from package insert protocols were the use of plasma for Hemagen tests (the manufacturer recommends use of serum) and counting tests with indeterminate zones (Hemagen and Wiener ELISAs) as positive. Indeterminate results were counted as positive because they would require confirmatory testing in clinical settings.

Data analysis.

Test characteristics were assessed using ARC donor status as the reference. Sensitivity, specificity, and receiver operating characteristics (ROC) curves were generated for each test compared to the designated status, and binomial exact confidence intervals were used for all resulting estimates. Analyses were conducted in STATA 14.2 statistical software.

RESULTS

Specimen characteristics and demographics.

Of 710 donor samples analyzed, 448 (63%) were classified as seropositive and 262 (37%) as seronegative by the ARC donor algorithm at time of donation (Table 1). Data on region of birth were available for 252 (56%) seropositive donors, 86 (34%) from Mexico, 77 (31%) from Central America, and 65 (26%) from South America.

TABLE 1.

Characteristics of blood donors whose specimens were used in the evaluation

Characteristic	No. (%) of donors by Trypanosoma cruzi infection statusa
	Positiveb	Negative
Total	448 (100)	262 (100)
Sex
Male	222 (49.5)	109 (42.6)
Female	226 (50.5)	153 (58.4)
Ethnicityc
Hispanic	181 (40.4)	32 (12.2)
Non-Hispanic	15 (3.4)	225 (85.9)
Unknown	252 (56.3)	5 (1.9)
Region of birth
United States	24 (3.4)	ND
Mexico	86 (12.1)	ND
Central America	77 (17.2)	ND
South America	65 (14.5)	ND
Unknown	196 (43.8)	262 (100)
Region of blood centerd
California	178 (39.2)	100 (38.2)
West	44 (9.8)	30 (11.5)
Midwest	28 (6.3)	14 (5.3)
Northeast	47 (10.5)	26 (9.9)
Southeast	151 (33.7)	92 (35.1)

^aTrypanosoma cruzi infection status as determined by American Red Cross screening and confirmation algorithm at the time of donation (23). ND, no data available.

^bMedian age, 43 years; interquartile range, 33 to 50.5 years.

^cBlood donors with positive test results were significantly more likely to report Hispanic ethnicity (P < 0.001).

^dSeronegative specimens were frequency matched to seropositive specimens by donation region.

Serology test performance.

All three of the latest-generation assays had 100% specificity (95% confidence interval [CI], 98.6 to 100.0) in this specimen set. Sensitivity was 95.5% (95% CI, 93.2 to 97.3) for Abbott PRISM, 97.1% (95% CI, 95.1 to 98.4) for Wiener Lisado, and 98.9% (95% CI, 97.4 to 99.6) for Wiener v.4.0. The four FDA-cleared tests had specificity estimates from 91.2% to 100% and sensitivity estimates from 87.7% to 97.5% in this set (Table 2). The area under the curve (AUC) figures derived from ROC analyses showed the best overall performance for Wiener v.4.0 and Wiener Lisado.

TABLE 2.

Performance characteristics of latest-generation and FDA-cleared IgG serological assays compared to Trypanosoma cruzi infection status determined by American Red Cross screening and confirmation algorithm at the time of donation

Assaya	Among 448 ARC-positive specimens		Among 262 ARC-negative specimens		AUC (% [95% CI])c
	No. positive	Sensitivity (% [95% CI])b	No. negative	Specificity (% [95% CI])b
ELISA
Wiener Lisado	435	97.1 (95.1–98.4)	262	100 (98.6–100.0)	0.99 (0.97–0.99)
Wiener v.4.0	443	98.9 (97.4–99.6)	262	100 (98.6–100.0)	0.99 (0.99–1.00)
Wiener v.3.0	422d	94.2 (94.6–96.2)	261	99.6 (97.9–100.0)	0.97 (0.95–0.98)
Hemagen	393e	87.7 (84.3–90.6)	262	100 (98.6–100.0)	0.94 (0.92–0.95)
Ortho	416	92.9 (90.1–95.1)	262	100 (98.6–100.0)	0.96 (0.95–0.98)
LFA
InBios CDP	437	97.5 (95.6–98.8)	239	91.2 (87.1–94.4)	0.94 (0.92–0.96)
ChLIA
Abbott PRISM	428	95.5 (93.2–97.3)	262	100 (98.6–100.0)	0.98 (0.96–0.99)

^aELISA, enzyme-linked immunosorbent assay; LFA, lateral flow assay; ChLIA, chemiluminescent assay.

^bBinomial exact 95% confidence interval.

^cAUC, area under the curve of receiver operating characteristics.

^dData include 5 specimens with indeterminate results. Indeterminate results were classified as positive results for the purpose of these analyses (see text for explanation).

^eData include 9 specimens with indeterminate results.

Sensitivity by region of birth.

All three latest-generation assays showed similar patterns to those seen for the FDA-cleared tests in our earlier evaluation, with the highest sensitivity in specimens from those born in South America, intermediate for Central America, and lowest for Mexico (Table 3). Wiener v.4.0 had the highest point estimate of sensitivity across all regions. As with the FDA-cleared tests (17), S/CO values were highest in samples from donors of South American origin, intermediate in those from Central America, and lowest in samples from Mexican-born donors (Fig. 1).

TABLE 3.

Sensitivity of latest-generation and FDA-cleared IgG serological assays compared to Trypanosoma cruzi infection status determined by American Red Cross screening and confirmation algorithm at the time of donation, by donor region of birth

Assaya	South America (n = 65)		Central America (n = 77)		Mexico (n = 86)
	No. pos	Sensitivity (% [95% CI])b	No. pos	Sensitivity (% [95% CI])b	No. pos	Sensitivity (5 [95% CI])b
ELISA
Wiener Lisado	65	100 (94.5–100.0)	74	96.1 (89.0–99.1)	80	93.0 (85.4–97.3)
Wiener v.4.0	65	100 (94.5–100.0)	77	100 (95.3–100.0)	85	98.8 (93.7–100.0)
Wiener v.3.0	64	98.5 (91.7–100.0)	74	96.1 (89.0–99.2)	79c	91.9 (83.9–96.6)
Hemagen	60	92.3 (83.0–97.5)	68d	88.3 (79.0–94.5)	71e	82.6 (72.9–89.9)
Ortho	63	96.9 (89.3–99.6)	75	97.4 (90.9–99.7)	74	86.0 (76.9–92.6)
LFA
InBios CDP	64	98.5 (91.7–100.0)	77	100 (95.3–100.0)	84	97.7 (91.9–99.7)
ChLIA
Abbott PRISM	64	98.5 (91.7–100.0)	74	96.1 (89.0–99.2)	78	90.7 (82.5–95.9)

^aELISA, enzyme-linked immunosorbent assay; LFA, lateral flow assay; ChLIA, chemiluminescent assay.

^bBinomial exact 95% confidence intervals.

^cData include 2 specimens with indeterminate results. Indeterminate results were classified as positive results for the purpose of these analyses (see text for explanation).

^dData include 1 specimen with indeterminate results.

^eData include 5 specimens with indeterminate results.

FIG 1.

Comparison of optical density values for latest-generation IgG serological assays by region of birth. Wiener Lisado displayed a median S/CO of 6.95 (interquartile range [IQR], 6.05 to 7.38) for samples of donors born in South America, 4.89 (IQR, 3.96 to 5.65) for samples of donors born in Central America, and 2.68 (IQR, 2.28 to 3.34) for samples of donors born in Mexico. Wiener v.4.0 displayed a median S/CO of 5.78 (IQR, 5.43 to 6.72) for samples of donors born in South America, 5.20 (IQR, 4.69 to 6.04) for samples of donors born in Central America, and 3.67 (IQR, 3.02 to 4.07) for samples of donors born in Mexico. Abbott PRISM displayed a median S/CO of 8.92 (IQR, 7.64 to 9.71) for samples of donors born in South America, 7.97 (IQR, 6.71 to 8.59) for samples of donors born in Central America, and 5.09 (IQR, 4.42 to 6.60) for samples of donors born in Mexico. Bar and whiskers represent median with interquartile range. ns, P ≥ 0.05; *, P < 0.05; **, P < 0.01; ***, P < 0.001; ****, P < 0.0001.

DISCUSSION

The approach to Chagas disease management in the United States has changed markedly over the last several years (2). Prior to 2018, antitrypanosomal drugs were available only through the Centers for Disease Control and Prevention (CDC) Drug Service under investigational protocols (25). CDC provided both serological testing and treatment drugs directly to physicians, and every drug release required a diagnosis confirmed by a CDC laboratory and discussion with an epidemiologist with expertise in CD (2). Now, diagnosis of chronic T. cruzi infection relies primarily on commercial laboratories, with CDC providing confirmatory testing only if initiated by health care providers. Benznidazole, the first-line treatment for CD (2), was approved by the FDA in 2018. With approval of nifurtimox in 2020, access to antiparasitic treatment has shifted fully to the commercial sector. These developments are likely to accelerate a transition of CD diagnosis and management into the mainstream of U.S. infectious diseases practice. However, accurate diagnosis and appropriate treatment depend on understanding performance characteristics of serological assays, especially in the low-prevalence environment of the United States.

Community screening of high-risk populations in the United States generally yields seroprevalence in the 1% to 2% range, with higher prevalence in older persons (26). Screening that targets younger individuals, for example, pre- or postnatal screening of mothers to identify infants at risk of congenital CD, results in prevalence figures below 1% (27). In such low-prevalence settings, the number of false-positive results will inevitably outnumber the true-positive results unless the clinical specificity of the screening test is higher than 98% to 99%. At the same time, the low sensitivity of many assays in specimens from those infected in Mexico and, to a lesser extent, Central America, will result in missing true infections (17, 19, 28, 29). Among the FDA-cleared assays, we observed the common trade-off sensitivity and specificity, with the most sensitive test being the least specific and vice versa (17). The latest assays, especially Wiener v.4.0, appear to offer a significant advance in performance, with improved sensitivity and highest specificity.

Like the FDA-cleared Wiener v.3.0, Wiener v.4.0 uses recombinant antigens 1, 2, 13, 30, 36, and secreted acute-phase antigen (SAPA) (30, 31). The major difference between the two Wiener recombinant assays is the replacement of the v.3.0 polyclonal conjugate secondary antibody with a monoclonal anti-human IgG antibody in v.4.0. This improvement appears to have resulted in more-accurate reproducible results. Wiener Lisado relies on the same kit reagents and secondary antibody as the Wiener v.4.0 but uses lysate of the Tulahuen T. cruzi strain as the antigen source. This strain belongs to the TcVI genotype, which is predominant in Argentina, Bolivia, and neighboring countries (32). Nevertheless, Wiener Lisado sensitivity remained reasonably high in specimens from TcI-predominant Mexico and Central America. Currently, the Wiener v.4.0 and Wiener Lisado assays are not FDA cleared for use in the United States.

Abbott PRISM is licensed by the FDA for CD screening in blood and organ donors but is not cleared for diagnostic use through 510(k) submission. Unlike more manual ELISAs, Abbott PRISM employs a semiautomated chemiluminescent immunoassay, which enables high-throughput testing. In our analysis, Abbott PRISM demonstrated 95.5% sensitivity, the median figure among the seven assays evaluated, with no false-positive results. Although several assays had higher sensitivity in our data, a high-throughput format greatly increases the capacity for screening at-risk populations. While it is unlikely to be practical to perform clinical testing on the PRISM platform, the same reagents are currently marketed outside the United States for use on Abbott Architect instruments, a widely utilized platform for routine laboratory testing. Successful 510(k) clearance of these reagents could provide a readily adoptable solution for CD testing across hospital systems and reference laboratories.

As with our earlier evaluation, all three assays examined in this study had highest sensitivity in samples from donors born in South America, intermediate in those from Central America, and lowest in those from Mexico (17). This pattern correlates closely with the regional distributions of reactivity and is hypothesized to result from geographic differences in the predominant T. cruzi genotypes (17, 29). From central Brazil southward, the related lineages TcII, TcV, and TcVI predominate, while TcI is the predominant genotype in northern South America, Central America, and Mexico (32). However, TcI is highly heterogeneous and may elicit a range of immune responses (18). These differences may contribute to decreased affinity or lack of antibody response to antigens from heterologous strains (33). Other factors that may affect antibody response could include levels of parasitemia and host-specific immune responses (34).

To overcome the practical challenge of amassing an adequate set of specimens for multiassay evaluations, we used blood donor plasma. However, blood donors tend not to represent the underlying clinical populations. In the United States, donors tend to be younger and healthier and are less likely to be Hispanic than the general population (35, 36). The generalizability of our results could be impaired if age and comorbidities impact immunological response in CD; however, aside from immunosuppression, this has not been reported in the literature. Furthermore, two assays included in this analysis (Ortho ELISA and Abbott PRISM) were used in blood donor screening algorithms, introducing the potential for selection bias. This was unavoidable given the absence of other sources of robust numbers of seropositive specimens incorporating the diversity of Latin American populations in the United States. Specimens from each screening algorithm were included in this analysis as detailed in Materials and Methods. Additional studies including clinical populations and prospective evaluation in higher prevalence populations are needed to better approximate real-world test performance.

Serology assays with simultaneous high sensitivity and specificity hold the potential to advance CD diagnosis in the United States. Clearance through the FDA 510(k) pathway and local distribution will be needed to facilitate widespread diagnostic use (37). Lower reactivity among donors born in Mexico compared to those born in Central and South America was observed among all assays and remains a challenge for CD diagnostic tests. However, the latest-generation of assays show better overall and regional performance than those currently cleared by FDA. Our data provide evidence to support their introduction to improve CD diagnosis in the United States.