Analytical Comparison of the Architect Syphilis TP and Liaison Treponema Automated Chemiluminescent Immunoassays and Their Performance in a Reverse Syphilis Screening Algorithm

LETTER

The incidence of syphilis has increased in the United States since 2000 (1, 2) and remains a global public health concern (3, 4). A tiered serology-based testing approach, historically starting with nontreponemal testing, is recommended by the Centers for Disease Control and Prevention (CDC) (5). To increase throughput and improve sensitivity for latent syphilis, laboratories are switching to a testing algorithm that starts with an FDA-cleared automated treponeme-specific chemiluminescent immunoassay (CLIA), followed by confirmation with a nontreponemal test and/or a different treponeme-specific test (6–11).

In this study, we compared the recently FDA-cleared Architect Syphilis TP (Architect; Abbott Laboratories, Abbott Park, IL) CLIA to the Liaison Treponema (Liaison; DiaSorin, Stillwater, MN) CLIA by testing 1,028 consecutive sera submitted for syphilis screening to the University of North Carolina Hospitals (UNCH) Clinical Immunology Laboratory. Serum was stored for no longer than 3 months at −80°C. All specimens were remnant specimens from UNCH patients over the age of 18, and approval was obtained from the University of North Carolina Institutional Review Board. Initial screening identified 976 nonreactive and 47 reactive specimens that were concordant by both methods (Table 1). Of the five discordant specimens, four were Architect reactive and Liaison nonreactive, while one specimen was Liaison reactive and Architect nonreactive, providing positive, negative, and total percent agreements of 97.9%, 99.6%, and 99.5%, respectively (kappa coefficient = 0.947; 95% confidence interval [95% CI], 0.901 to 0.993).

TABLE 1.

Liaison and Architect result concordance^a

Architect result	No. of specimens with Liaison result:		Total
	RX	NR
RX	47	4	51
NR	1	976	977
Total	48	980	1,028

^aNR, nonreactive; RX, reactive. The positive and negative percent agreements were 97.9% and 99.6%, respectively. The kappa coefficient was 0.947 (95% CI, 0.901 to 0.993).

Comparing each assay's screening results to reverse-algorithm-verified positive results (see the materials and methods in the supplemental material), the Architect Syphilis TP assay identified 45 of 45 verified positive results, while five specimens were Architect reactive, ASI rapid plasma reagin (RPR) (Arlington Scientific, Inc., Springville, UT) nonreactive, and Serodia Treponema pallidum particle agglutination (TP-PA) (Fujirebio Inc., Malvern, PA) nonreactive, for a false-positivity rate of 0.5% (Fig. 1). No verified-positive specimens were missed by Architect, providing a sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of 100% (95% CI, 92.1 to 100%), 99.5% (95% CI, 98.8 to 99.8%), 90.0% (95% CI, 78.2 to 96.7%), and 100.0% (95% CI, 99.6 to 100.0%), respectively (Table 2). The Liaison Treponema assay also identified 45 of 45 verified positives, while two specimens were positive by the Liaison assay but RPR and TP-PA nonreactive, for a false-positivity rate of 0.2% (Fig. 1). The Liaison assay also had no false-negative specimens, for a sensitivity, specificity, PPV, and NPV of 100.0% (95% CI, 92.1 to 100.0%), 99.8% (95% CI, 99.3 to 100.0%), 95.7% (95% CI, 85.5 to 99.5%), and 100.0% (95% CI, 99.6 to 100.0%), respectively (Table 2). One specimen was falsely positive by both the Architect and Liaison assays, while an additional specimen that was positive by both assays was RPR nonreactive but TP-PA indeterminate (Table 3). The levels of inter- and intra-assay precision of both assays were comparable, with the coefficients of variation of all reactive TP-PA specimens being equal to or less than 3.5% (Table S1).

FIG 1.

Reverse syphilis testing algorithm. RX, reactive; NR, nonreactive; RPR, rapid plasma reagin.

TABLE 2.

Reverse-algorithm comparison^a

Test and result	No. (%) of indicated positive specimens that were verified to be:		Total (%)	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
	RX	NR
Liaison
RX	45 (4.4)	2 (0.2)	47 (4.6)
NR	0 (0.0)	980 (95.4)	980 (95.4)
Totals	45 (4.4)	982 (95.6)	1,027	100.0	99.8	95.7	100.0
Architect
RX	45 (4.4)	5 (0.5)	50 (4.9)
NR	0 (0.0)	977 (95.1)	977 (95.1)
Totals	45 (4.4)	982 (95.6)	1,027	100.0	99.5	90.0	100.0

^aRX, reactive; NR, nonreactive; PPV, positive predictive value; NPV, negative predictive value.

TABLE 3.

Discordant-specimen results^a

Specimen	Liaison result	Architect result	ASI RPR result	TP-PA result	Interpretation
610	NR	RX	NR	NR	Architect false positive
948	NR	RX	NR	NR	Architect false positive
1002	NR	RX	NR	NR	Architect false positive
1025	RX	RX	NR	NR	Dually false positive
1128	NR	RX	NR	NR	Architect false positive
1282	RX	NR	NR	NR	Liaison false positive
1493	RX	RX	NR	IND	Undefined

^aNR, nonreactive; RX, reactive; IND, indeterminate.

Considering that the reverse-testing algorithm involves one additional test compared to the traditional algorithm, strategies to identify potential false positives and reduce unnecessary tests have been employed by using chemiluminescent signal strengths to set a cutoff for confirmatory testing (12–14). The optimal Architect assay signal to cutoff (S/CO) and Liaison assay index values used to predict TP-PA confirmatory testing results were determined using receiver operating characteristic (ROC) curves. An S/CO value that provided 100% specificity was determined for the Architect Syphilis TP assay, as all specimens with an S/CO value greater than 6.0 (40/40; P = 0.0001) were confirmed by TP-PA testing (Fig. 2A). For the Liaison Treponema assay, an index value greater than 3.0 provided 100% specificity, as all specimens testing higher than 3.0 (45/45) were confirmed by TP-PA testing (Fig. 2B) (P = 0.005). Adopting a testing approach where subsequent testing is based on CLIA signal strengths might significantly reduce costs for a laboratory (13); however, a larger sample size which includes more CLIA-reactive but RPR- and TP-PA-nonreactive specimens will be needed to determine reliable S/CO and index values to use clinically.

FIG 2.

ROC analysis to predict truly positive (TP-PA-confirmed) specimens. Signal/cutoff (S/CO) and index values providing 100% specificity were selected for the Architect Syphilis TP assay (manufacturer's reactive S/CO, ≥1.00) (A) and Liaison Treponema assay (manufacturer's equivocal index, 0.90 < 1.10, and positive index, ≥1.10) (B) for which all specimens with higher values resulted in reactive TP-PA results. P was 0.0001 (A) and P was 0.005 (B) by Fisher's exact test. AUC, area under the curve.

Similar studies performed outside the United States have evaluated the Architect Syphilis TP assay, with reported sensitivities of 99.5 to 100.0% and specificities of 54.5 to 100.0%, depending on patient population (15, 16).

In conclusion, the Architect Syphilis TP assay exhibited sensitivity and specificity that were as high as those of the Liaison Treponema assay and is suitable as a syphilis screening assay in a clinical laboratory setting.