Performance of treponemal tests for the diagnosis of syphilis

Abstract

Background:

Treponemal immunoassays are increasingly used for syphilis screening with the reverse sequence algorithm. There are little data describing performance of treponemal immunoassays compared to traditional treponemal tests in patients with and without syphilis.

Methods:

We calculated sensitivity and specificity of seven treponemal assays: 1) ADVIA Centaur (chemiluminescence immunoassay-CIA), 2) Bioplex 2200 (microbead immunoassay-MBIA), 3) fluorescent treponemal antibody absorbed test (FTA-ABS), 4) INNO-LIA (line immunoassay), 5) LIAISON CIA, 6) TP-PA (Treponema pallidum particle agglutination assay), and 7) Trep-Sure (enzyme immunoassay-EIA), using a reference standard combining clinical diagnosis and serology results. Sera were collected between May 2012 to January 2013. Cases were characterized as: 1) current clinical diagnosis of syphilis: primary, secondary, early latent, late latent 2) prior treated syphilis only, 3) no evidence of current syphilis, no prior history of syphilis and at least 4/7 treponemal tests negative.

Results:

Among 959 participants, 262 had current syphilis, 294 had prior syphilis, and 403 did not have syphilis. FTA-ABS was significantly less sensitive for primary syphilis [78.2% (65.0–88.2)], compared to the immunoassays or TP-PA (94.5–96.4%) (all p≤0.01). All immunoassays were 100% sensitive for secondary syphilis and 95.2–100% sensitive for early latent disease, but were less sensitive in late latent disease (86.8–98.5%). TP-PA had 100% specificity (99.0–100).

Conclusion:

Treponemal immunoassays demonstrated excellent sensitivity for early syphilis. Sensitivity of FTA-ABS in primary syphilis was poor compared to the immunoassays and TP-PA. Given its high specificity and superior sensitivity, TP-PA is a better test to adjudicate discordant results with the reverse sequence algorithm than the FTA-ABS.

Introduction

Syphilis is currently increasing at epidemic rates across the United States (US), particularly among men who have sex with men, women, and newborns. [1] Diagnosis of syphilis has traditionally involved use of non-specific, non-treponemal serology directed against lipoidal antigens (e.g., rapid plasma reagin-RPR) with confirmation of reactive results by a specific treponemal test (e.g. Treponema pallidum particle agglutination assay-TP-PA, fluorescent treponemal antibody absorbed-FTA-ABS). While non-treponemal tests are inexpensive and useful for monitoring response to treatment, they require significant hands-on time and are not as sensitive as treponemal tests in primary syphilis. [2] They are also associated with biologic false positive results among injection drug users and in various chronic diseases, including autoimmune conditions and HIV. [3]

In the past decade, a shift has occurred in the syphilis testing paradigm; high volume laboratories are increasingly utilizing treponemal immunoassays for syphilis screening and diagnosis, including the enzyme immunoassay (EIA), chemiluminescence immunoassay (CIA), and microbead immunoassay (MBIA), among others. These assays can be automated, reducing labor and turnaround time. Employing a reverse sequence algorithm, a treponemal immunoassay is performed first, followed by reflex non-treponemal testing (e.g. RPR) on initially reactive specimens. [4]

Patients with discordant serology (e.g. EIA-reactive, RPR-non-reactive) present diagnostic and treatment challenges for clinicians, because these results may reflect either a false positive treponemal EIA, prior syphilis, or very early syphilis prior to development of a reactive RPR. [5–8] Analyses including early generation EIAs demonstrated 31% of reactive EIA specimens were non-reactive when tested with TP-PA. These isolated EIA-reactive specimens could reflect false positive results, but definitive interpretation is difficult without a laboratory gold standard. [9] Currently the CDC recommends performance of a TP-PA to adjudicate discordance between the immunoassay and RPR. [4] There are few studies comparing head-to-head performance of treponemal tests in clinically characterized sera, stratified by stage of syphilis. A study by Lam et al found FTA-ABS was less sensitive than TP-PA for primary, secondary and latent syphilis of unknown duration. [10] Another study compared performance of an immunoassay (Captia IgG EIA) versus FTA-ABS by stage of syphilis: performance was similar, but sample sizes in each stage were small (n <21). [11]

The objective of this study was to compare the sensitivity and specificity of newer automated treponemal tests (e.g. EIA, CIA, MBIA) and manual treponemal tests (e.g. FTA-ABS, TP-PA) in patients with a clinical diagnosis of syphilis (by stage), and in those without evidence of syphilis. The findings from this study will help inform the selection of the most appropriate second treponemal test for patients with initially discordant treponemal and non-treponemal serology, and selection of an automated treponemal test when the reverse sequence algorithm is used for a laboratory diagnosis of syphilis.

Materials and Methods

Study population:

A convenience sample of de-identified remnant serum samples (n=1995) prospectively collected between May 2012 and March 2013 were frozen and sent to the CDC Syphilis Reference Laboratory for testing. Samples were from Kaiser Permanente Northern California (KPNC), Kaiser Permanente Southern California (KPSC) and San Francisco Department of Public Health (SFDPH).

KPNC and KPSC are large managed healthcare organizations, each with approximately 4 million members. [12] Both KPNC and KPSC regional laboratories utilized reverse sequence screening; KPNC utilized the LIAISON CIA and KPSC utilized the TrepSure EIA as the initial screening test, and reflex tested all reactive CIA or EIA specimens with the RPR. Seroprevalence has previously been reported as approximately 2% at each institution. [9] Specimens from KPNC and KPSC were a combination of screening and diagnostic specimens.

Specimens from SFDPH were from consecutive patients presenting to the city’s municipal sexually transmitted disease clinic with reactive serology and diagnosed with primary or secondary syphilis. SFDPH utilizes a point-of-care RPR in the clinical setting; all specimens are tested by and the Venereal Disease Research Laboratory test followed by the TP-PA in the laboratory.

Treponemal testing

All participants’ sera were tested with seven treponemal assays. CDC investigators were blinded to clinical characteristics or original serologic results when performing the laboratory testing. Assays were performed on the same freeze-thaw cycle. Testing was performed according to manufacturer’s instructions in package inserts. A more complete description of the assays and testing methods for this study has been published previously. [13]

ADVIA Centaur Syphilis (Siemens Healthcare Diagnostics Inc., Newark DE) is a CIA that measures IgG.

Bioplex 2200 Syphilis IgG (Bio Rad Laboratories, Hercules, CA) is a MBIA that measures IgG.

INNO-LIA (Fujirebio, Inc Malvern, PA) is a manual line immunoassay that measures IgG.

Fluorescent Treponemal Antibody Absorbed test – FTA-ABS DS (Zeus Scientific) is a manual indirect fluorescence assay that measures IgG and IgM.

LIAISON Treponema Screen (Diasorin, Stillwater, MN): LIAISON is a CIA that measures IgG and IgM.

Treponema pallidum Particle Agglutination assay (TP-PA - Fujirebio, Inc Malvern, PA) is a manual agglutination assay that measures IgG and IgM.

TrepSure EIA (Phoenix Biotech, Mississauga, Ontario, Canada) is an EIA that measures IgG and IgM.

Of the seven assays, Bioplex 2200-MBIA, Centaur CIA, LIAISON CIA and TrepSure EIA are automated immunoassays typically used for initial screening with the reverse sequence algorithm. Treponema pallidum Particle Agglutination assay (TP-PA), Fluorescent Treponemal Antibody Absorbed test – (FTA-ABS) are manual treponemal tests typically used to confirm reactive non-treponemal tests or for adjudication of discordant serology. INNO-LIA is a manual line immunoassay used by some laboratories to adjudicate discordant results with the reverse sequence algorithm (e.g., EIA-reactive, RPR-non-reactive).

Case definitions:

Investigators performed chart review of the electronic medical record to characterize patient specimens as having current syphilis, prior syphilis only, or not having syphilis. Two investigators (IP, JC) were aware of CDC laboratory testing results, as one of the case definitions required ≥4/7 negative treponemal tests. All other investigators performing chart review/data abstraction and staging were blind to CDC laboratory results. Among the 1995 specimens, 1036 (52%) were excluded due to insufficient information to clinically characterize the cases, or insufficient serum volume for testing with all of the assays.

Patients with current syphilis included those with a diagnosis of primary, secondary, early latent, or late latent disease. Syphilis stage was determined through combined analysis of serology results from the initial visit (plus past results, if available), physical findings, clinical presentation, plus darkfield microscopy findings (for SFDPH patients). Chart review included determination of symptoms/signs on the day of testing, gender of sex partners, HIV status, serologic test results, recent contact to a case of early syphilis, past history of treated syphilis, and clinical diagnosis associated with initial visit.

Primary syphilis:

presence of an anogenital chancre or lesions and 1) presence of spirochetes on darkfield microscopy plus reactive non-treponemal or treponemal serology, or 2) negative darkfield (or darkfield not performed) with reactive treponemal and non-treponemal serology.

Secondary syphilis:

mucocutaneous lesions including presence of a rash (trunk, scrotum, palms/soles) and/or patchy alopecia, mucous patches, and/or condyloma lata with reactive non-treponemal and treponemal serology.

Early latent syphilis:

absence of symptoms and either 1) reactive non-treponemal and treponemal serology, or 2) two reactive treponemal tests (e.g., EIA-reactive, RPR-non-reactive, TP-PA reactive) AND prior sexual contact to a case of early syphilis within the past 12 months, OR prior non-reactive serology within the past 12 months.

Late latent syphilis:

absence of symptoms and 1) reactive non-treponemal and treponemal serology, or 2) two reactive treponemal tests (e.g., EIA reactive, RPR non-reactive, TP-PA reactive), no prior history of syphilis, no serologic test results in the prior 12 months, no sexual contact to a case of early syphilis in the prior 12 months.

Prior treated syphilis only:

syphilis history documented in the patient chart, but no signs or symptoms of syphilis on the day of specimen collection and no subsequent diagnosis of syphilis in the 6 months after the day of specimen collection.

No syphilis:

no diagnosis of syphilis on the day of testing or in the 6 months after the day of specimen collection, no syphilis in the past medical history, no reactive prior syphilis serology, (all available lab records reviewed), and least 4/7 treponemal tests were negative (after testing by CDC reference laboratory).

Data analysis:

Sensitivity and specificity by stage of syphilis were calculated with 95% confidence intervals using the binomial distribution. The t-test was used to compare means, the chi square test was used to compare proportions, a p value of 0.05 was considered statistically significant.

The institutional review boards at the California Department of Public Health (CDPH), Kaiser Permanente Southern California (KPSC), Kaiser Permanente Northern California (KPNC), University of California-San Francisco (UCSF), and the Centers for Disease Control and Prevention (CDC) approved this study.

Results:

Of 959 patients, n=262 had current syphilis (all stages), n=294 had prior syphilis only, and n=403 did not have syphilis. The demographic characteristics of all patients and a comparison of patients with current syphilis and those who did not have syphilis are included in Table 1. Patients with current syphilis were older, more likely to be male, a man who has sex with men, and HIV positive compared to those who did not have syphilis. (all p<0.05).

Table 1:

Demographic characteristics of patients with current clinical diagnosis of syphilis, prior treated syphilis and those without syphilis, n=959

Characteristics	Current syphilis n=262 (%)	Prior treated syphilis only n=294	No syphilis N=403 (%)	p value*
Mean age (SD)	43.3 years (13.7)	47.7 years (12.6)	40.5 years (18.4)	0.03
Male gender	233 (88.9%)	248 (84.4%)	204 (50.6%)	<0.0001
Man who has sex with men (MSM)	172 (65.6%)	145 (48.3%)	51 (12.7%)	<0.0001
Pregnant	3 (1.2%)	6 (2.0%)	11 (2.7%)	0.16
HIV Positive	136 (51.9%)	181 (61.6%)	12 (3.0%)	<0.0001

^*Comparing patients with current syphilis vs those without syphilis

The sensitivity and specificity of the assays associated with a clinical diagnosis of syphilis is included in Table 2. For syphilis (all stages combined), all assays demonstrated greater than 95% sensitivity, with the exception of FTA-ABS [90.8% (86.7–94.0)] which was significantly less sensitive than TP-PA (p=0.038) and the other immunoassays (all p<0.001). There was greater variability in specificity among the seven assays. The Trep-Sure EIA demonstrated significantly lower specificity than the other assays [82.5% (78.4–86.1)] (p <0.0001). TP-PA had 100% specificity (99.0–100).

Table 2:

Sensitivity and specificity of treponemal assays for diagnosis of current syphilis, all stages (n=262) and for those with without syphilis (n=403).

Assay	Sensitivity N=262	Specificity N=403
FTA-ABS	90.8%* (86.7–94.0)	98.0% (96.1–99.1)
TP-PA	95.4% (92.1–97.6)	100% (99.0–100)
LIAISON CIA	96.9% (94.1–98.7)	94.5% (91.8–96.5)
Bioplex MBIA	96.9% (94.1–98.7)	96.7% (94.4–98.2)
INNO-LIA	96.9% (94.1–98.7)	98.5% (96.8–99.5)
Centaur CIA	97.3% (94.6–98.9)	95.5% (93.0–97.3)
TrepSure EIA	98.5% (96.1–99.6)	82.6%† (78.4–86.1)

^*FTA-ABS less sensitive than TP-PA (p=0.038) or other immunoassays (all p<0.001)

^†TrepSure EIA was significantly less specific than all other assays (all p<0.0001)

In examining sensitivity by stage of syphilis (Table 3), FTA-ABS had the lowest sensitivity among the seven assays for primary syphilis (all p ≤0.01). It was also significantly less sensitive for secondary syphilis (p=.007). All other assays demonstrated sensitivity ranging between 94.5%−96.4% for primary syphilis and sensitivity of 100% for secondary syphilis.

Table 3:

Sensitivity of treponemal assays for detection of syphilis, by stage (n=262)

Assay	Primary (n=55)	Secondary (n=98)	Early Latent (N=41)	Late Latent (N=68)
FTA-ABS	78.2%* (65.0–88.2)	92.8%* (85.7–97.0)	100% (90.7–100)	92.6% (83.7–97.6)
TP-PA	94.5% (84.9–98.9)	100% (96.2–100)	100% (90.7–100)	86.8%† (76.4–93.8)
Centaur CIA	94.5% (84.9–98.9)	100% (96.2–100)	100% (90.7–100)	94.1 % (85.6–98.4)
TrepSure EIA	94.5% (84.9–98.9)	100% (96.2–100)	100% (90.7–100)	98.5% (92.1–99.9)
LIAISON CIA	96.4% (94.5–98.2)	100% (96.2–100)	97.6% (87.4–99.9)	92.6% (83.7–97.6)
Bioplex MBIA	96.4% (94.5–98.2)	100% (96.2–100)	95.1% (83.8–99.4)	94.1 % (85.6–98.4)
INNO-LIA	96.4% (94.5–98.2)	100% (96.2–100)	100% (90.7–100)	91.1% (81.7–96.7)

^*FTA-ABS was less sensitive than other assays for primary syphilis (all p ≤0.01) and secondary syphilis (p=0.007)

^†TP-PA significantly less sensitive than TrepSure EIA for late latent syphilis (p=0.009), all other comparisons were not statistically significant.

Among patients with early latent syphilis (n=41), all seven assays demonstrated high sensitivity, ranging from 95–100%, with no statistically significant differences. Sensitivity for those with late latent disease (n=68) was lower, ranging from 86.8–98.5%. TP-PA was significantly less sensitive than TrepSure EIA (86.8% vs 98.5%, p=0.009); there were no other statistically significant differences.

Persistence of treponemal antibody in patients with prior treated syphilis is described in Table 4). Reactivity of FTA-ABS and TP-PA were both less than 95% and were not statistically different from each other (p>0.05), but FTA-ABS was less likely to be persistently reactive than the immunoassays (p<0.001). The immunoassays all demonstrated reactivity of 95.9–99.3%.

Table 4:

Persistent reactivity of treponemal assays among patients with prior syphilis (n=294)

Assay	Reactive N (%)	Non-reactive N(%)	Indeterminate N(%)
FTA-ABS	258 (87.8)*	33 (11.2)	3 (1.0)
TP-PA	272 (92.5)	22 (7.5)	0
Bioplex MBIA	282 (95.9)	12 (4.1)	0
LIAISON CIA	283 (96.3)	8 (2.7)	3 (1.0)
INNO-LIA	284 (96.6)	1 (0.3)	9 (3.1)
TrepSure EIA	289 (98.3)	5 (1.7)	0
ADVIA Centaur CIA	292 (99.3)	2 (0.7)	0

^*FTA-ABS more likely to revert to non-reactive than the immunoassays (p<0.001) but not significantly different from TP-PA (p>0.05).

Discussion:

This study demonstrates that the four immunoassays routinely used for screening (LIAISON CIA, ADVIA Centaur CIA, TrepSure EIA, Bioplex 2200 MBIA) all demonstrated high sensitivity for primary, secondary and early latent syphilis, with sensitivities comparable to traditional manual tests such as TP-PA. The FTA-ABS demonstrated poor sensitivity, particularly for primary syphilis. The FTA-ABS was previously used as the gold standard in studies of syphilis test performance in the 1990s. [14, 15] During this time, the CDC conducted quality control (QC) on FTA-ABS reagents manufactured in the United States. These QC activities are no longer being performed. FTA-ABS also produces subjective results and thus requires microbiologist expertise for optimal performance/interpretation. The current data call into question whether FTA-ABS should continue to be used, particularly given the availability of immunoassays and/or TP-PA that demonstrate better performance and provide objective results.

Specificity was high for both the traditional treponemal tests and most immunoassays. In particular, the TP-PA demonstrated 100% specificity, which supports the current recommendation for its use as a second test adjudicate discordant specimens (e.g., EIA-reactive, RPR-non-reactive). [4] The exception was the TrepSure EIA, which demonstrated significantly lower specificity than the other assays.

Prior studies of TrepSure’s specificity have yielded mixed results. In studies by Wong et al and Busse et al (using TP-PA and FTA-ABS as the reference standard) TrepSure demonstrated 99% and 94% specificity, respectively. [16, 17] Prior analyses by the CDC in both high and low prevalence populations demonstrated 18.6–25.2% of TrepSure EIA-reactive specimens were subsequently TP-PA non-reactive, possibly reflecting false positive results. [9] The specificity estimate of 82.8% in this study using a clinical reference standard is in line with this prior CDC analysis. However, further studies with clinically well-characterized specimens in both high and low prevalence populations are needed to better define TrepSure EIA’s performance.

Clinicians and laboratories utilizing reverse sequence screening with immunoassays should ensure that a highly specific treponemal test such as TP-PA be performed on all EIA-reactive, RPR-non-reactive specimens. This is key in prenatal screening of populations with low rates of syphilis among women of reproductive age and low rates of congenital syphilis. Another option for adjudication would be INNO-LIA, a manual line immunoassay. Although it is not FDA-cleared, it has been validated to meet Clinical Laboratory Improvement Amendments (CLIA) and used by some commercial laboratories in the United States. In this analysis, specificity of INNO-LIA was 98.5%; Centaur CIA and Bioplex MBIA demonstrated specificities of >95%. Any of these could be a reasonable alternative to TP-PA as a second treponemal test in high risk populations such as men who have sex with men. In populations with a low prevalence of syphilis, using tests with lower specificity may lead to false positive results which could cause harm and emotional distress due to an incorrect syphilis diagnosis. [18] Larger studies are needed to better define the specificity of treponemal tests in high and low prevalence populations. Until then, this study supports use of TP-PA as the most specific, FDA-cleared test available to adjudicate discordant treponemal/non-treponemal results, especially in low prevalence populations.

Treponemal test positivity generally persists after prior treated infection, although positivity may wane for patients treated in the primary stage of syphilis as well as those with advanced HIV disease. [19–21] Among patients with prior treated syphilis, assay positivity was greater than 90% overall, and the immunoassays were significantly more likely to remain reactive compared to the FTA-ABS. In this study, stage of disease at the time of prior diagnosis/treatment was not known, so it is possible that further differences in reactivity would be observed if the population with a history of treated syphilis could be further stratified by stage. Given the high proportion of reactivity of the immunoassays among patients with a prior history of treated syphilis, laboratories in high seroprevalence settings (e.g., STD clinics) would likely have to perform greater numbers of costly confirmatory tests under a reverse sequence screening algorithm compared to the traditional screening algorithm. [9]

There are several limitations to this study. Testing was performed on frozen samples that were subjected to a single freeze-thaw cycle. According to the manufacturers’ inserts for the seven treponemal tests, it is recommended that fresh samples (serum or plasma) be used for testing. It is unclear if use of frozen specimens affects test performance. Further studies with fresh specimens in patients with/without syphilis are needed to compare the current findings. Additional treponemal immunoassays are FDA approved and commercially available for syphilis screening (Abbot Architect CIA, Lumipulse G-TP CIA); the current results are not generalizable to these assays. Finally, all patients with primary syphilis included in this study had reactive serologies when they were diagnosed; however both treponemal and non-treponemal tests can be negative early in the course of primary syphilis. [2] Therefore, the true sensitivity of these treponemal assays in primary syphilis is likely lower than what was observed in this study.

Serology alone is not sufficient for diagnosis of syphilis. Serology results must be interpreted in the context of the patient’s sexual history, prior syphilis history, and current symptoms/findings. CDC guidelines recommend presumptive treatment for patients with syphilis-related symptoms and risk factors for syphilis even before serology results are available. [4]

In conclusion, treponemal immunoassays and TP-PA demonstrated excellent sensitivity for early syphilis. Sensitivity of FTA-ABS for diagnosis of syphilis was inferior, particularly in primary and secondary syphilis. Other treponemal tests (EIA, CIA, MBIA, TP-PA) would be preferred for confirming non-treponemal tests using the traditional algorithm. TP-PA is preferred to adjudicate cases of discordant serology with the reverse sequence algorithm in populations with both low and high prevalence of syphilis. If CLIA-validation and appropriate quality assurance was in place, INNO-LIA would be an acceptable alternative. Centaur CIA and Bioplex MBIA would also be acceptable alternatives to TP-PA as a second treponemal test in high prevalence populations.