Correlation of Treponemal Immunoassay Signal Strength Values with Reactivity of Confirmatory Treponemal Testing

ABSTRACT

Automated treponemal immunoassays are used for syphilis screening with the reverse-sequence algorithm; discordant results (e.g., enzyme immunoassay [EIA] reactive and reactive plasma reagin [RPR] nonreactive) are resolved with a second treponemal test. We conducted a study to determine automated immunoassay signal strength values consistently correlating with reactive confirmatory treponemal testing. We conducted a cross-sectional analysis of four automated immunoassays (BioPlex 2200 microbead immunoassay [MBIA], Liaison chemiluminescence immunoassay [CIA], Advia-Centaur CIA, and Trep-Sure EIA) and three manual assays (Treponema pallidum particle agglutination [TP-PA], fluorescent treponemal antibody absorption [FTA-ABS] test, and Inno-LIA line immunoassay). We compared signal strength values of automated immunoassays and positive and negative agreement. Among 1,995 specimens, 908 (45.5%) were true positives (≥4/7 tests reactive) and 1,087 (54.5%) were true negatives (≥4/7 tests nonreactive). Positive agreement ranged from 86.1% (83.7 to 88.2%) for FTA-ABS to 99.7% (99.0 to 99.9%) for Advia-Centaur CIA; negative agreement ranged from 86.3% (84.1 to 88.2%) for Trep-Sure EIA to 100% for TP-PA (99.6 to 100%). Increasing signal strength values correlated with increasing reactivity of confirmatory testing (P < 0.0001 for all automated immunoassays by Cochran-Armitage test for trend). All automated immunoassays had signal strength cutoffs corresponding to ≥4/7 reactive treponemal tests. BioPlex MBIA and Liaison CIA had signal strength cutoffs correlating with ≥99% and 100% TP-PA reactivity, respectively. The Advia-Centaur CIA and Trep-Sure EIA had signal strength cutoffs correlating with at least 95% TP-PA reactivity. All automated immunoassays had signal strength cutoffs correlating with at least 95% FTA-ABS reactivity. Assuming that a 95% level of confirmation is adequate, these signal strength values can be used in lieu of confirmatory testing with TP-PA and FTA-ABS.

INTRODUCTION

The traditional algorithm for syphilis screening has involved use of a manual nontreponemal test (e.g., rapid plasma reagin [RPR]) followed by confirmation with a treponemal test (e.g., Treponema pallidum particle agglutination [TP-PA]). Nontreponemal tests are inexpensive, but require significant hands-on time by laboratory personnel and produce subjective results. High-volume laboratories are increasingly using a reverse-sequence algorithm, using partially or fully automated treponemal tests (e.g., enzyme immunoassay [EIA], chemiluminescence immunoassay [CIA], or multibead immunoassay [MBIA]), followed by a nontreponemal test (e.g., RPR or VDRL test) if reactive (1). Potential advantages of the reverse-sequence algorithm include reduced labor requirements due to automation, higher throughput with walk-away capability, and objective results (2).

Several notable clinical issues have emerged with increasing use of the treponemal immunoassay for reverse-sequence syphilis screening. Treponemal tests often remain reactive for life, limiting their utility as a screening test in high-prevalence populations; performance of nontreponemal testing is still necessary to detect whether an individual is currently infected. Additionally, reverse-sequence screening identifies patients with discordant treponemal and nontreponemal test results (e.g., EIA reactive and RPR nonreactive) that were previously unrecognized under the traditional RPR-based algorithm and may indicate an early infection. In these cases, the Centers for Disease Control and Prevention (CDC) currently recommend performance of a second treponemal test to resolve the discrepancy (3). The use of a second treponemal test leads to increased costs, but may avoid unnecessary treatment of patients with initial false-positive treponemal immunoassay results (1, 4).

Several analyses have concluded that treponemal immunoassay signal strength values may be used in lieu of confirmatory testing with a second treponemal test: two studies of a microbead immunoassay (MBIA), one of a CIA, and another of an EIA (5–8). To confirm these prior results and evaluate other assays currently used for syphilis screening and diagnosis in the United States, we conducted a cross-sectional analysis of four treponemal automated immunoassays (BioPlex 2200 microbead immunoassay [MBIA], Liaison chemiluminescence immunoassay [CIA], Advia-Centaur CIA, and Trep-Sure EIA) and three manual assays (Treponema pallidum particle agglutination [TP-PA], fluorescent treponemal antibody absorption [FTA-ABS] test, and Inno-LIA line immunoassay). Our objective was to determine signal strength cutoffs associated with 90, 95, 99, and 100% reactive confirmatory testing (TP-PA or FTA-ABS). For the four automated immunoassays, we also attempted to establish a signal strength cutoff associated with a consensus of at least 4/7 reactive treponemal tests (e.g., for Liaison CIA, the cutoff value at which Liaison CIA plus at least three other tests were reactive). The findings from this study will inform national laboratory guidelines on use of confirmatory treponemal testing for patients with initially discordant serology.

MATERIALS AND METHODS

Serum specimens.

Deidentified banked serum samples (n = 1,995) collected between January 2009 and March 2013 were sent to the CDC Syphilis Reference Laboratory from the following institutions: Kaiser Permanente Northern California (KPNC; n = 584, 29.3%), Kaiser Permanente Southern California (KPSC; n = 1,290 [64.7%]), and the San Francisco Department of Public Health (SFDPH; n = 121 [6.1%]). Serum samples from SFDPH (n = 45/121) were subjected to two freeze-thaw cycles, while the remainder and all specimens from KPNC and KPSC were subjected to a single freeze-thaw cycle. KPNC and KPSC are large managed health care organizations, each with approximately 4 million members (14), and both KPNC and KPSC regional laboratories utilize reverse-sequence screening. Overall seroprevalence has previously been reported as approximately 2% at each institution (1). Specimens from SFDPH were from patients presenting to the city's municipal sexually transmitted disease clinic, which uses the Venereal Disease Research Laboratory (VDRL) test for screening, and were all from patients diagnosed with primary and secondary syphilis. CDC technicians were blind to clinical or prior serologic results associated with these specimens.

Treponemal testing.

All sera were tested with seven treponemal assays. Characteristics of the assays, including ranges of signal strength values, are listed in Table 1. All assays were performed on the same freeze-thaw cycle. Each week, 139 samples were thawed and testing for all assays completed within 7 days on samples to avoid extra freeze-thaw cycles. Of the seven assays, BioPlex 2200 syphilis G, Advia Centaur syphilis, Liaison Treponema screen, and Trep-Sure EIA are characterized as automated immunoassays, and the Treponema pallidum particle agglutination assay (TP-PA) and fluorescent treponemal antibody absorbed test (FTA-ABS DS) are manual treponemal tests, while Inno-LIA is a manual line immunoassay. All testing was performed according to the manufacturer's instructions in package inserts.

TABLE 1.

Positive and negative agreement of seven treponemal assays with a consensus of the test panel

Assay and resulta	Test panel consensus (no. of samples)b		% agreement (95% CI)		κ value
	Positive	Negative	Positive	Negative
BioPlex MBIA
Positive	870	16	95.8 (94.3–97.0)	98.1 (97.1–98.8)	0.95
Negative	31	1,066
Indeterminate	7	5
Centaur CIA
Positive	905	43	99.7 (99.0–99.9)	96.0 (94.7–97.1)	0.95
Negative	2	1,044
Indeterminate	1	0
FTA-ABS
Positive	782	8	86.1 (83.7–88.2)	99.0 (98.2–99.5)	0.91
Negative	82	1,076
Indeterminate	44	3
Inno-LIA
Positive	841	16	92.6 (90.7–94.2)	98.5 (97.6–99.1)	0.92
Negative	67	1,071
Liaison CIA
Positive	879	39	96.8 (95.4–97.8)	96.0 (94.6–97.0)	0.94
Negative	20	1,043
Indeterminate	9	5
TP-PA
Positive	756	0	83.3 (80.7–85.6)	100 (99.6–100)	0.86
Negative	137	1,087
Indeterminate	15	0
Trep-Sure EIA
Positive	902	149	99.3 (98.5–99.7)	86.3 (84.1–88.2)	0.85
Negative	5	938
Indeterminate	1	0

^aMBIA, microbead immunoassay; EIA, enzyme immunoassay; LIA, line immunoassay; TP-PA, Treponema pallidum particle agglutination assay; FTA-ABS, fluorescent treponemal antibody absorbed test; CIA, chemiluminescence immunoassay.

^bConsensus was defined as ≥4/7 tests positive or ≥4/7 tests negative (n = 1,995).

(i) Liaison Treponema screen.

The Liaison Treponema screen (Diasorin, Stillwater, MN) is a fully automated single-step CIA (total antibody, IgM, and IgG) in which recombinant antigens are coated onto magnetic particles linked to isoluminol antigen conjugate. T. pallidum-specific antibodies form complexes with the antigen conjugate. Starter reagents produce a flash chemiluminescent reaction when added, which is measured as relative light units (RLU). RLU are directly related to the total T. pallidum antibody concentration present in samples, controls, and calibrators. A signal strength value of ≥1.1 (range, 0 to 70.0) is considered reactive.

(ii) BioPlex 2200 syphilis IgG assay.

The BioPlex 2200 syphilis IgG assay (Bio-Rad Laboratories, Hercules, CA) is a multibead immunoassay (IgG). Patient samples, diluent, and the antigen bead reagent are combined in a reaction vessel followed by an incubation step. T. pallidum-specific antibodies bind to bead reagents. After the addition of conjugate, the bead mixture is incubated, washed, and passed through a detector that detects the intensity of the fluorescence in the beads as a result of phycoerythrin (PE). This is measured in terms of relative fluorescent intensity (RFI). A signal strength value of ≥1.1 (range, 0 to 8.0) is considered reactive.

(iii) ADVIA Centaur syphilis assay.

The Advia Centaur syphilis assay (Siemens Healthcare Diagnostics, Inc., Newark, DE) is a direct sandwich CIA (IgG). Acridinium ester-labeled T. pallidum antigens are added to samples or controls. The solid-phase reagent, containing biotinylated T. pallidum recombinant antigen preformed to streptavidin-coated magnetic latex particles, is then added to the sample mix. A light signal measured in RLU is produced. A signal strength value of ≥1.1 (range, 0 to 45.0) is considered reactive.

(iv) Trep-Sure EIA.

The Trep-Sure EIA (Phoenix Biotech, Mississauga, Ontario, Canada) is a qualitative EIA (IgG or IgM). Patient samples (serum or plasma) and controls are added to wells in microplates precoated with immobilized specific recombinant treponemal antigens to form complexes if Treponema-specific antibodies are present. Conjugated treponemal antigens are added to the wells, which react with the antigen-antibody complexes. After incubation and a wash step, tetramethylbenzidine (TMB) substrate is added to the wells to create a chromogenic reaction. The intensity of color produced is measured spectrophotometrically (450 nm) and reported as optical density (OD) values. A signal strength value of ≥1.1 (range, 0 to 15.5) is considered reactive.

(v) Treponema pallidum particle agglutination assay.

The Treponema pallidum particle agglutination (TP-PA) assay (Fujirebio, Inc., Malvern, PA) is a manual assay (total antibody) in which T. pallidum antigens are used to sensitize gelatin particles. Serum that contains Treponema-specific antibodies binds to the antigens on the particles, forming a smooth mat of particles on the bottom of the microtiter plate well. The unsensitized particle control well for each serum sample should also form a compact button on the bottom of the well, indicating the absence of nonspecific agglutination.

(vi) Inno-LIA.

Inno-LIA (Fujirebio, Inc., Malvern, PA) is a line immunoassay (IgG) that utilizes three recombinant proteins, one synthetic peptide, and four control lines that are coated on each test strip. Serum samples and controls were incubated with the antigen-coated test strips. Conjugate labeled with alkaline phosphatase was added to allow binding to any syphilis antigen-antibody complexes formed. After incubation with the substrate BCIP/NBT (5-bromo-4-chloro-3-indolylphosphate–nitroblue tetrazolium), a dark brown color is produced that is proportional to the amount of specific antibody present in the sample. This assay was made automatable with the use of the Auto-LIA 48 instrument.

(vii) FTA-ABS test.

The fluorescent treponemal antibody absorbed test (FTA-ABS DS; Zeus Scientific) is a modification of the standard FTA-ABS DS test system, which employs nonviable T. pallidum (Nichols strain) cells as a substrate (antigen). These substrate cells are reacted with specially treated patient sera. Treponema-specific total antibodies when present in test sera form an antigen-antibody complex with substrate cells. Anti-human globulin labeled with rhodamine is added to the T. pallidum substrate complex, followed by the addition of an antitreponemal fluorescein isothiocyanate (FITC)-labeled globulin used as a counterstain reagent. The substrate cells are then examined with a fluorescence incident illuminator microscope. The FITC selection of filters is used first to read the FITC reaction to determine the presence or absence of treponemes without the use of a dark-field condenser.

Statistical analyses.

Specimens with ≥4/7 tests reactive were considered true positives, and those with ≥4/7 tests nonreactive were considered true negatives. Positive and negative agreement against this gold standard was calculated for each assay with 95% confidence intervals (95% CI) using the binomial distribution. The κ statistic was calculated as a secondary measure of correlation. The agreement of the results by κ values was categorized as near perfect (0.81 to 1.0), substantial (0.61 to 0.8), moderate (0.41 to 0.6), fair (0.21 to 0.4), slight (0 to 0.2), or poor (0) (9). SAS version 9.3 (Cary, NC) was used for statistical analysis.

The relationship between increasing signal strength value and TP-PA or FTA-ABS reactivity was analyzed using the Cochran-Armitage test for trend; a P value of <0.05 was considered significant. This analysis includes a subset of the data (1,849/1,995 samples) that represents patients with either reactive or nonreactive results on all seven tests (excluding indeterminate results).

RESULTS

Among 1,995 specimens tested, 908 (45.5%) were categorized as true positives and 1,087 (54.5%) as true negatives using a consensus of the test panel as the gold standard (see Materials and Methods for case definitions). Table 1 includes results for positive agreement, negative agreement, and κ statistics for the seven assays. Positive agreement was lowest for FTA-ABS (86.1%; 95% confidence interval [CI], 83.7 to 88.2%) and highest for Centaur EIA (99.7%; 95% CI, 99.0 to 99.9%). Positive agreement for the automated immunoassays (BioPlex MBIA, Centaur CIA, Liaison CIA, and Trep-Sure EIA) and Inno-LIA was higher than that of the FTA-ABS and TP-PA (confidence intervals did not overlap). Negative agreement was lowest for Trep-Sure EIA (86.3%; 95% CI, 84.1 to 88.2%) and highest for TP-PA (100%; 95% CI, 99.6 to 100%). The other automated immunoassay and Inno-LIA demonstrated high negative agreement (>96%). The κ values ranged from 0.85 (Trep-Sure) to 0.95 (BioPlex); all assays demonstrated a high coefficient of correlation (κ= 0.81 to 1.0).

Table 2 illustrates the signal strength cutoffs corresponding to 90, 95, 99, or 100% reactivity with TP-PA or FTA-ABS, as well as the value corresponding to ≥4/7 reactive treponemal tests. All automated immunoassays demonstrated signal strength values corresponding to ≥4/7 reactive treponemal tests and at least 95% TP-PA or FTA-ABS reactivity. BioPlex EIA and Liaison CIA demonstrated signal strength cutoffs (BioPlex, 7.4; Liason, 17.1) that correlated to 99% and 100% TP-PA reactivity, respectively. None of the assays had signal strength cutoffs that correlated to 99 or 100% FTA-ABS reactivity.

TABLE 2.

Correlation of signal strength cutoffs of four treponemal immunoassays to percentage of reactivity of TP-PA and FTA-ABS assay or ≥4/7 reactive treponemal tests

Assay	Signal strength range	Signal strength cutoff based on:
		% reactive								≥4/7 treponemal tests reactive
		TP-PA				FTA-ABS
		90.0–94.9%	95.0–98.9%	99.0–99.9%	100%	90.0–94.9%	95.0–98.9%	99.0–99.9%	100%
BioPlex MBIA	1.1–8.0	0.8	2.4	7.5		0.8	2.8			5.2
Liaison CIA	1.1–70	1.1	1.8	7.5	17.1	1.1	2.4			2.5
Advia-Centaur CIA	1.1–45	1.8	6.2			1.5	8.0			6.6
Trep-Sure EIA	1.1–15.5	15.1	15.3	15.4	15.4	13.0	15.3	15.4	15.4	15.3

Table 3 illustrates the relationship between increasing automated immunoassay signal strength values and increasing reactivity of confirmatory treponemal testing. For all four automated immunoassays, increases in signal strength value were correlated with increases in percentage of reactivity for both TP-PA and FTA-ABS (P < 0.0001 for all automated immunoassays by Cochran-Armitage test for trend).

TABLE 3.

Correlation of increasing signal strength value of four treponemal immunoassays to increasing reactivity with TP-PA or FTA-ABS

Assay and index valuea	n	No. (%) reactive (n = 1,849) by:
		TP-PA	FTA-ABS
BioPlex MBIA
0–<1.1	1,065	12 (1)	20 (2)
1.1–<3.7	100	54 (54)	58 (58)
3.7–<7.9	124	109 (88)	110 (89)
7.9–8.0	560	555 (99)	551 (98)
Z statistic		−39.85	−39.07
P		<.0001	<.0001
Advia-Centaur CIA
0–<1.1	1,029	1 (<1)	8 (<1)
1.1–<18.0	207	129 (62)	135 (65)
18.0–<44.0	194	188 (97)	185 (96)
44.0–45.0	419	412 (98)	411 (98)
Z statistic		−38.34	−37.61
P		<.0001	<.0001
Liaison CIA
0–<1.10	1,047	7 (<1)	19 (2)
1.1–<16.0	269	192 (71)	200 (74)
16.0–<60.0	304	302 (99)	296 (97)
60.0–70.0	229	229 (100)	224 (98)
Z statistic		−36.52	−35.56
P		<.0001	<.0001
Trep-Sure EIA
0–<1.3	935	1 (<1)	8 (<1)
1.3–<9.0	122	20 (16)	25 (20)
9.0–<12.0	118	89 (75)	86 (73)
12.0–15.5	674	620 (92)	620 (92)
Z statistic		−34.04	−33.60
P		<.0001	<.0001

^aP values were determined by Cochran-Armitage test for trend.

DISCUSSION

In our analysis, the signal strengths obtained from the automated treponemal immunoassays were found to be comparable to those of reactive traditional treponemal tests (FTA-ABS and TP-PA). BioPlex MBIA, Centaur CIA, and Liaison CIA demonstrated higher levels of positive agreement and a similar level of negative agreement to traditional tests when using a consensus of the test panel as the gold standard. The exception was the Trep-Sure EIA, which demonstrated lower negative agreement compared with the other assays. Studies utilizing the Trep-Sure EIA have demonstrated that between 18.6 and 25.2% of Trep-Sure-positive specimens were not confirmed when tested with a second treponemal test (1, 2); however, other investigators have found higher specificity with Trep-Sure EIA (95.7%) using a gold standard similar to that used in this study (≥4/7 tests negative) (9).

In establishing automated immunoassay signal strength values correlating to TP-PA reactivity, our current findings with the Liaison CIA were comparable to those previously published by our group, where we found that a signal cutoff greater than 12.00 correlated with 100% TP-PA reactivity (6). We also found that a BioPlex MBIA signal strength of 7.4 (range, 0 to 8.0) had >99% correlation with TP-PA reactivity, which supports findings by Loeffelholz et al., who found that a signal strength of 8.0 correlated to >99% TP-PA reactivity for both low- and high-risk cohorts of patients (5). However, the investigators did note that cutoffs may need to differ when screening low-prevalence versus high-prevalence populations. Dai et al. also established signal strength cutoffs with another automated immunoassay (not included in the present study) that correlated with 100% TP-PA reactivity, implying that cutoffs can likely be established for other currently available treponemal automated immunoassays (10). Of note, values for the individual assay cannot be correlated or compared with one another as all are measured differently and have different ranges. However, for each automated immunoassay in this study, we demonstrated that increasing signal strength values (reflecting increasing levels of antigen-antibody complexes) consistently correlated to increasing reactivity with TP-PA and FTA-ABS.

We did not find a signal strength value for Trep-Sure EIA correlating to 99% or 100% TP-PA/FTA-ABS reactivity. This contrasts with prior findings from Wong et al., who found that Trep-Sure EIA specimens with signal strengths of 8.0 or greater had nearly perfect correlation (99.6%) with TP-PA (7). Although our specimens did include those from the same setting as that of Wong et al. (a public health sexually transmitted disease [STD] clinic), we utilized previously frozen specimens and also included specimens from two low-seroprevalence populations in our current analysis. Our findings with the Centaur CIA were similar to that of Trep-Sure, where correlation with at least 95% TP-PA/FTA-ABS reactivity was observed. We are not aware of other comparable studies of the Centaur CIA assay.

None of the four automated immunoassays had signal strength values that correlated perfectly with FTA-ABS reactivity. Although our objective was to determine values associated with ≥95% correlation to TP-PA and FTA-ABS reactivity, an acceptable level of correlation has not yet been established in national guidelines. For hepatitis C virus antibody testing, laboratories do not typically perform supplemental testing if initial signal/cutoff ratios correlate with 95% reactivity on confirmatory testing (11). Assuming that a minimum 95% level of confirmation is adequate, all four automated immunoassays met this threshold. However, in low-prevalence populations (e.g., prenatal screening), a higher threshold of correlation may be desired before foregoing confirmatory testing.

Some investigators have found that the presence of discordant results using the reverse-sequence algorithm test (e.g., EIA reactive and RPR nonreactive) leads to extra testing and higher costs compared to the traditional RPR-based algorithm (11), related in part to higher false-positivity rates in low-prevalence populations (12). However, it has also been argued that the reverse algorithm could identify cases of very early infection when the nontreponemal antibodies are not yet detectable or latent cases of syphilis when the nontreponemal antibodies are no longer detectable (4, 12).

For laboratories that chose reverse screening, the use of signal strength values in lieu of confirmatory treponemal testing may mitigate some costs related to adjudication of discordant specimens. One laboratory that performs 47,000 tests per year using BioPlex MBIA estimated that eliminating confirmatory testing on specimens with high signal strength would result in savings of $4,800 U.S. per year (13). For commercial laboratories or large health systems with even higher testing volumes, greater cost savings are possible from eliminating unnecessary confirmatory testing. Laboratories would need to weigh the effort needed to create separate workflows for specimens with signal strengths above and below the cutoff value for confirmatory testing.

The limitations of this study included the use of previously frozen samples that were subjected to a single freeze-thaw cycle prior to testing. According to the manufacturer's inserts for the seven treponemal tests, it is recommended that fresh samples (serum or plasma) be used for testing. It is unclear if the use of fresh versus frozen specimens affects automated immunoassay signal strength values, and we did not have access to fresh specimens in patients with/without syphilis to compare our current findings. Since we conducted this study, there are now more treponemal automated immunoassays that are FDA approved and commercially available. Our results are not generalizable to these assays, and further studies are needed to establish signal strength cutoffs for newer automated immunoassays.

In conclusion, for treponemal automated immunoassays such as the Liaison CIA and BioPlex MBIA, signal strength values can be used in lieu of confirmatory testing with TP-PA. Given the conflicting data with Trep-Sure EIA and little available data for the Centaur CIA, more studies are needed to determine whether signal strength can be reliably used for these assays. Guidelines with published cutoffs for each of the FDA-approved treponemal immunoassays should be available to microbiology laboratory directors to facilitate use of signal strength values. Finally, consensus on the acceptable level of correlation with confirmatory TP-PA or FTA-ABS results should be established to guide laboratories as to when it may be appropriate to forego confirmatory testing.