Development of a syphilis serum bank to support research, development, and evaluation of syphilis diagnostic tests in the United States

Abstract

The Centers for Disease Control and Prevention’s (CDC) Division of STD Prevention, in collaboration with the Association of Public Health Laboratories (APHL), is developing a nationally available syphilis serum repository for research of Food and Drug Administration (FDA)-cleared or investigational syphilis diagnostic assays in the United States. State and local public health laboratories (PHL) submitted de-identified residual sera with information on collection date, volume, storage conditions, freeze-thaw cycles, PHL serology results, reported syphilis stage and demographic details if available. Previous test results were blinded and sera (N=152 reported syphilis stage, N = 131 unknown status) were tested at CDC using five FDA-cleared and one investigational syphilis tests. Treponemal and nontreponemal test sensitivity ranged from 76.3–100% and 63.2–100%, respectively, among staged specimens. The conventional treponemal assays showed high concordance of 95.4%. By providing syphilis stage and comprehensive serological test data, developed repository may serve as a valuable resource for diagnostic test validation studies.

1. Introduction

Treponema pallidum subspecies pallidum is the etiological agent of syphilis and is transmitted sexually, vertically from an infected mother to her child, and rarely by blood transfusion or other nonsexual contact (Goh, 2005). Untreated syphilis progresses through primary, secondary, early/late latent, and tertiary disease stages. Recent national surveillance reports from the Centers for Disease Control and Prevention (CDC) have shown primary and secondary syphilis as being on the rise in the United States (US) (Workowski and Bolan, 2015). A total of 30,644 primary and secondary syphilis cases were reported in 2017, with men who have sex with men (MSM) accounting for the majority of cases (Centers for Disease Control and Prevention, 2017b). An increase in the rate of congenital syphilis has also been reported, with 918 congenital syphilis cases being recorded (including 64 still births, and 13 infant deaths) in 2017 at a national rate of 23.3 cases per 100,000 live births. These data indicate a 43.8% increase from 2016 (16.2 cases per 100,000 live births) and a 153.3% increase from 2013 (9.2 cases per 100,000 live births) (Centers for Disease Control and Prevention, 2017b).

The diagnosis of syphilis involves serological techniques and direct detection methods, along with patient history and clinical symptoms (Henao-Martinez and Johnson, 2014; Morshed and Singh, 2015; Ratnam, 2005). The use of direct detection techniques is however often limited to research or field settings, with diagnostic laboratories primarily using serological tests that include nontreponemal and treponemal categories. Nontreponemal antibodies are produced during active syphilis in response to the lipoidal moieties released from damaged host cells and possibly also from the treponemes during infection (Jost et al., 2013; Larsen and Johnson, 1998; Morshed and Singh, 2015). Venereal Disease Research Laboratory (VDRL), Rapid Plasma Reagin (RPR), Unheated Serum Reagin (USR) and Toluidine Red Unheated Serum Test (TRUST) are examples of the nontreponemal syphilis tests that are capable of detecting immunoglobulin (Ig) G and IgM classes of antibody to cardiolipin, lecithin, and cholesterol in serum or plasma (Larsen et al., 1995). However, false reactive nontreponemal tests may be associated with hepatitis, viral infections, malaria, leprosy, intravenous drug use, pregnancy or linked to connective tissue diseases such as systemic lupus erythematous (Binnicker et al., 2011; Larsen et al., 1995; Morshed and Singh, 2015). It is therefore recommended that a reactive nontreponemal result be followed by a treponemal test that typically includes whole bacteria or highly purified treponemal peptides/proteins as target antigen(s) in the assay design. Treponemal tests detect antibodies specific for T. pallidum antigen (s) which result from active or previously treated T. pallidum infection. T. pallidum Particle Agglutination (TP-PA), T. pallidum Hemagglutination Assay (TPHA), Fluorescent Treponemal Antibody-Absorption (FTA-ABS), Trep-Sure Enzyme Immunoassay (EIA), INNO-LIA Syphilis Score (Line Immunoassay, LIA), automated LIAISON treponema assay (Chemiluminescence Immunoassay, CIA) are examples of treponemal tests. Recently, the Food and Drug Administration (FDA) cleared the use of a rapid immunochromatographic test, Syphilis Health Check (Rapid Syphilis Test, RST), for T. pallidum specific antibody detection (Matthias et al., 2016; Pereira et al., 2018). This test is also CLIA-waived (Clinical Laboratory Improvement Amendments) and can easily be performed by a trained non-laboratorian, while being cognizant of quality control, competency, training, and proficiency testing.

Although many syphilis serological assays are commercially available, deciding which to use as a screening or confirmatory test can be challenging for a laboratory due to factors such as testing volume and frequency, turnaround time, accuracy and cost involved. The serological diagnosis of syphilis follows one of two testing algorithms in the US. The traditional algorithm begins with a nontreponemal test as a screening test, and if reactive, followed by a treponemal test as mentioned above (Loeffelholz and Binnicker, 2012). This practice is still in use as a standard algorithm in many laboratories for syphilis screening. Conversely, the reverse algorithm utilizes a treponemal test (EIA/CIA, automated) for initial screening, and if reactive, is followed by a nontreponemal test (Loeffelholz and Binnicker, 2012). To manage discordant test results (e.g. treponemal reactive and non-treponemal non-reactive), reflex testing with another treponemal test, TP-PA, is recommended (Centers for Disease and Prevention, 2011; Park et al., 2019). The reverse algorithm is increasingly being applied due to the high throughput and improved work flow associated with automated treponemal tests, and it also potentially has higher sensitivity for primary and latent stage syphilis detection compared to nontreponemal tests (Donkers et al., 2014).

To support the field’s advancement of syphilis diagnostic tests in the US, CDC and the Association of Public Health Laboratories (APHL) collaborated to collect, characterize, and provide syphilis specimens as a resource for research, public health, clinical or commercial institutions that perform syphilis test validation and/or development. This work aligns with a goal highlighted in 2016 CDC syphilis summit (Kersh and Lukehart, 2018) and the 2017 National Call to Action (Centers for Disease Control and Prevention, 2017a) that envisioned development of a characterized specimen repository. Described herein are the methods used to acquire and test submitted specimens, using syphilis serological tests to comprehensively characterize the panels for the repository. The nontreponemal RPR test and the treponemal TP-PA, EIA, LIA, and CIA were selected for specimen testing based on the routine application of these assays in syphilis test algorithms and the availability of previously reported performance data (Hagedorn et al., 2002; Jost et al., 2013; Larsen and Johnson, 1998; Zhang et al., 2012). Since these tests are all conventional laboratory assays, Syphilis Health Check, was also included to obtain RST data for these specimens and to shed light on its performance as a treponemal assay in relation to syphilis disease stage, given that it is a relatively new test in the field of syphilis diagnostics.

2. Materials and methods

2.1. Specimens

Association of Public Health Laboratories (APHL) members from 11 state and/or local public health laboratories provided 464 human serum samples to the CDC through a joint collaborative effort. Approvals in accordance with federal regulations, state laws, ethics guidelines, and CDC regulatory policies were obtained prior to specimen collection and CDC laboratory testing. These sera were collected by the PHL from years 2012–2016 in response to a solicitation by the APHL to its membership, and are intended to facilitate serological test validation and development in the syphilis field, with specimen panels prepared, banked and provided by the CDC. Contributing laboratories submitted anonymized specimens that were unlinked from personally identifying information (PII) and were assigned an APHL study number, with an additional de-linking step performed at the CDC. Limited information was provided with each specimen and included state of origin, with the source being either a state or local public health laboratory, and non-identifying data such as age, sex, reported syphilis stage (when available from the test request form or as reported by the program following CDC case definitions), diagnostic test utilized, and previous test results (Table 1). Of the 464 specimens provided to the CDC, 283 met the minimum 2 mL sample volume requirement that would be sufficient for both downstream assays and archiving. The specimens were further divided based on the inclusion of syphilis status where 152 specimens had reported syphilis disease stage, 109 reported as being unknown for syphilis stage, and 22 being reported as serofast status. Out of 464 specimens, 183 had insufficient specimen volume (< 2 ml) and were excluded for this study, but were archived for internal quality control studies at the CDC. All specimens were shipped frozen on dry ice and stored at −80 °C until tested. Only one freeze thaw cycle was reported by the submitters.

Table 1.

Summary of specimen data received from the Association of public health laboratories.

Location of State Public Health Laboratory	Specimens (n)	Sex		Mean Age (Years)		Reported syphilis stage/ status*
		M	F	M	F	Primary	Secondary	Early Latent	Late latent	Unclassified latent	Sero-fast	Unknowna
Arizona	3	1	2	42	43.5	0	0	1	2	0	0	0
Arkansas	3	1	2	24	41.5	0	0	0	0	0	3	0
California	25	10	15	32.6	35.5	2	2	4	1	1	0	15
Indiana	42	35	7	33.6	36.3	0	1	0	0	1	0	40
Michigan	137	122	15	35.3	33.9	14	31	13	29	0	0	50
New Hampshire	15	10	5	34	29	4	3	3	2	2	0	1
New York	80	59	21	36.9	38.6	5	12	5	22	0	0	36
Oklahoma	30	16	14	35.2	36	4	2	7	2	0	0	15
Texas	83	55	28	36.7	33.6	6	12	13	8	15	23	6
Utah	11	10	1	44.4	34	0	0	0	0	0	0	11
Virginia	35	29	6	32.2	37	5	9	14	6	0	0	1
Total (n)	464	348	116	-	-	40	72	60	72	19	26	175

n = number

^aUnknown = Reported as unknown for syphilis stage

^*As reported by the laboratory from data obtained from the test request form or as reported by program following CDC case definitions.

2.2. Serological assays

The diagnosis of syphilis is dependent on the use of nontreponemal and treponemal tests, hence both test types were included in our evaluation. The 283 specimens that met the minimum volume requirement were tested with nontreponemal RPR (ASI RPR card test, Arlington Scientific, UT, USA), and treponemal-TP-PA (Serodia TP-PA, Fujirebio, Japan), EIA (Trep-Sure EIA, Trinity Biotech, NY, USA), LIA (INNO-LIA syphilis score, Fujirebio, Japan), an automated CIA (LIAISON treponema assay, DiaSorin, Italy), and RST (Syphilis Health Check, Trinity Biotech NY, USA). CDC laboratorians performing the serological tests were initially blinded to the previously reported test results and reported syphilis stage or status that were provided by the APHL member public health laboratories. All testing and result interpretations were performed according to respective manufacturer’s directions. After completion of tests, residual specimens were prepared as 0.2 ml aliquots and archived for repository purposes.

2.3. Data analysis

Diagnostic sensitivity of the evaluated tests using PHL-reported syphilis staged specimens (primary, secondary, early latent, late latent and unclassified latent) were calculated using standard formula as follows (Larsen and Johnson, 1998):

%Sensitivity =[True positive/(True positive + False negative)] × 100

Where, true positive represents syphilis staged specimens found reactive by a given test, and false negative is staged specimens nonreactive by that same test.

A comparative analysis was performed among four conventional laboratory serological assays. Results of three treponemal tests EIA, CIA and TP-PA were analyzed to get percentage concordance/discordance in the context of syphilis staged specimens. RPR results simultaneously obtained were compared to treponemal data. The RST and LIA were excluded for this analysis due to their relatively new and/or investigational status, and most PHL settings use conventional laboratory serological assays for specimen testing.

3. Results

3.1. Repository content and accessibility

A summary of data for all specimens received thus far from the APHL member public health laboratories are shown in Table 1, which indicate specimen source, clinical information and patient demographics. Concurrent with specimen accrual and characterization, a public website, https://www.cdc.gov/std/syphilis/lab/serumbank.htm, was developed by the CDC to provide general information about the repository and the process to request specimens, which involves submission of a brief proposal via the website. Upon approval, a contract of agreement and detailed inventory of specimen panels along with CDC-characterized test results will be provided to the requestor for selection and procurement of a limited number of specimens per annum.

3.2. Repository characterization

Since diagnostic testing was performed by multiple laboratories across the US, there was variability in the type of test and/or algorithm applied, with a given laboratory using any one or combination of RPR, VDRL or USR for nontreponemal testing, and/or TP-PA, EIA, FTA-ABS or Captia syphilis G for treponemal testing. Test results and clinical status information for the specimens were reported by the PHL and are documented as is by our laboratory at the CDC. Thus, in order to better characterize and provide a degree of standardization across specimens, comprehensive testing was performed at our laboratory using five FDA-cleared and one investigational serological assay. For the 152 specimens that included reported syphilis stage information, testing using the six serological assays showed overall sensitivities ranging from 85–98% as shown in Table 2. The nontreponemal RPR yielded sensitivity of >90% for primary, secondary, early latent, and unclassified latent staged specimens and showed sensitivity of 63.2% for the late latent specimens. Treponemal assays (TP-PA, EIA, LIA and CIA) showed sensitivities in the range of 90–100% across all reported stages of syphilis. The RST, Syphilis Health Check, showed sensitivity in the range of 76–96%. Testing of 131 unstaged syphilis specimens had yielded sensitivity of 70–100% among which specimens with serofast status (n=22) had a sensitivity of 100% for both RPR and TP-PA, and of 95.4% for EIA, LIA, and CIA. With the same specimens, RST showed a sensitivity of 90.9% (Table 2). Out of 22 serofast specimens, 18 had RPR titers ≤1:8 or less (data not shown). Testing of specimens categorized as unknown under unstaged syphilis showed a sensitivity of >90% for TP-PA, EIA, LIA, and CIA, while being 70.6% and 78.9% for RPR and RST respectively (Table 2).

Table 2.

Performance of serological tests with reported syphilis staged and unstaged specimens.

Specimens (n = 283)	Serological test sensitivity, n (%)
	Nontreponemal test	Treponemal tests
	RPR	TP-PA	EIA	LIA	CIA	RST
Reported syphilis stage specimens (n= 152)
Primary (n = 21)	20 (95.2)	21 (100)	20 (95.2)	19 (90.4)	21 (100)	18 (85)
Secondary (n = 46)	42 (91.3)	46 (100)	46 (100)	45 (97.8)	46 (100)	44 (95.6)
Early Latent (n = 30)	27 (90)	29 (96.6)	30 (100)	29 (96.6)	29 (96.6)	29 (96.6)
Late Latent (n = 38)	24 (63.2)	36a (94.7)	38 (100)	37b (97.3)	35c (92.1)	29 (76.3)
Unclassified Latent (n = 17)	17 (100)	16 (94.1)	16 (94.1)	16 (94.1)	16 (94.1)	16 (94.1)
Total	130 (85.5)	148 (97.4)	150 (98.7)	146 (96.1)	147 (96.7)	136 (89.5)
Unstaged syphilis specimens (n = 131)
Serofast (n = 22)	22 (100)	22 (100)	21 (95.4)	21 d (95.4)	21 (95.4)	20 (90.9)
Unknown (n = 109)	77 (70.6)e	99 (90.8)e	105 (96.3)e	100 (91.7)e,f	99 (90.8)e	86 (78.9)e

n = number, % = Percentage, RPR = Rapid Plasma Reagin, TP-PA = T. pallidum Particle Agglutination, EIA = Enzyme Immunoassay, LIA = Line Immunoassay, CIA = Chemiluminescence Immunoassay, RST = Rapid syphilis test. “Staged” refers to the information reported by the laboratory from data obtained from the test request form or as reported by program following CDC case definitions.

^a= Excludes one sample with indeterminate result

^b= Excludes one sample with indeterminate result

^c= Excludes two samples with indeterminate result

^d= Excludes one sample with indeterminate result

^e= Represents percentage positive, not test sensitivity

^f= Excludes three samples with indeterminate result.

Table 3 shows comparative analysis of four conventional FDA-cleared syphilis serology laboratory tests in the context of disease stage. Among the three conventional treponemal tests, a concordance of 95.4% was noted. Four specimens had discordant treponemal results (2.6%), with two of the three treponemal tests showing either reactive or nonreactive results for a given specimen. One unclassified latent staged specimen (0.6%) showed nonreactive results by all three treponemal tests, but was reactive by RPR (minimal reactivity, Rm), while 16 specimens (10.5%) across all stages showed the opposite outcome. Three late latent staged specimens (2.0%) yielded indeterminate results with CIA and/or TP-PA and were excluded for this analysis. All four conventional assays showed reactive results with 84.2% of specimens tested.

Table 3.

Comparative analysis of conventional treponemal and nontreponemal tests with reported syphilis staged specimens.

Laboratory serological assays				Reported syphilis stage												% Concordance/discordance among treponemal tests (n = 152)
Treponemal			Nontreponemal
EIA	CIA	TP-PA	RPR	Primary (n=21)		Secondary (n=46)		Early Latent (n=30)		Late Latent (n=38)a		Unclassified Latent (n=17)		Total (n=152)
				n	%	n	%	n	%	n	%	n	%	n	%		n	%
+	+	+	+	19	90.5	42	91.3	27	90.0	24	63.2	16	94.1	128	84.2
+	+	+	-	1	4.8	4	8.7	2	6.7	9	23.7	0	0.0	16	10.5	Concordance	145	95.4
-	-	-	+	0	0.0	0	0.0	0	0.0	0	0.0	1	5.9	1	0.6
+	+	-	-	0	0.0	0	0.0	0	0.0	1	2.6	0	0.0	1	0.6
+	-	-	-	0	0.0	0	0.0	1	3.3	0	0.0	0	0.0	1	0.6
																Discordance	4	2.6
+	-	+	-	0	0.0	0	0.0	0	0.0	1	2.6	0	0.0	1	0.6
-	+	+	+	1	4.8	0	0.0	0	0.0	0	0.0	0	0.0	1	0.6

n = number, % = Percentage, RPR = Rapid Plasma Reagin, EIA = Trep-Sure Enzyme Immunoassay, CIA = LIASON treponema assay, TP-PA = Treponema pallidum Particle Agglutination assay

⁺=reactive

^-= non-reactive

^a= Excludes three specimens; two were with equivocal status in CIA and an additional one with indeterminate status in TP-PA. “Staged” refers to the information reported by the laboratory from data obtained from the test request form or as reported by program following CDC case definitions.

4. Discussion

A national syphilis serum repository has been developed at the CDC consisting of staged and unstaged (serofast or unknown status) syphilis specimens with comprehensive laboratory-characterized serological test data. The repository is intended to support research, public health, clinical or commercial institutions that develop and/or validate novel and existing syphilis diagnostic assays for use and/or FDA clearance in the US. Since the repository may be used not only to develop novel syphilis assays, but also to evaluate the performance of new tests relative to existing assays, the characterization of sera for the repository was not limited to determining reactivity status alone; performance data for an array of currently available tests was also assessed to provide a frame of reference for laboratories that may conduct similar assays using specimens from this repository. Importantly, the performance data described in this study lends credence to the quality of the repository specimens, as the overall level of sensitivity yielded for nontreponemal RPR and treponemal tests including RST are consistent with previous studies (Castro et al., 2003; Hagedorn et al., 2002; Pereira et al., 2018; Sena et al., 2010).

The repository development initiative provided a unique research opportunity to evaluate test results in the context of reported syphilis stage. The investigational treponemal LIA showed sensitivity comparable to the standard treponemal assays for staged specimens, while the EIA showed the highest sensitivity among the other tests considered in this study. However, there was generally high concordance among the treponemal assays tested, which is consistent with prior findings (Jost et al., 2013; Larsen and Johnson, 1998). The data obtained also reiterates the higher sensitivity of treponemal tests for detecting antibodies in syphilis specimens from all stages (Park et al., 2019). However, discordant and inconclusive results were noted among the treponemal tests that were evaluated, mainly with primary, early latent, and late latent staged specimens. The intrinsic differences in treponemal assay design or platform could contribute to the observed discrepancies as the analytical sensitivity varies among tests (Jost et al., 2013). Of note, one unclassified latent staged specimen showed nonreactive treponemal results but a minimal reactive result in RPR, suggesting a false positive case (Ratnam, 2005), though additional clinical history related to the specimen would be needed to guide further interpretation of these findings.

The discrepancies observed between treponemal results and those obtained by RPR testing suggest a lower sensitivity of RPR for the staged specimens tested in this study (Table 2). The sensitivity of RPR is reduced during early or late syphilis (Larsen and Johnson, 1998). In addition, prior successful treatment for syphilis could account for diminished RPR sensitivity or reversion to nonreactive status (Larsen and Johnson, 1998; Ratnam, 2005). In the case of secondary staged specimens that yielded nonreactive RPR and reactive treponemal test results, the prozone effect was considered due to the presence of high levels of antibodies during this stage of infection (Morshed and Singh, 2015). However, this possibility was excluded by semi-quantitative RPR testing for all four RPR nonreactive-secondary staged specimens. In addition to parameters discussed above, the presence of concomitant sexually transmitted infections such as HIV and the potential impact of co-infections on the outcome of various test types cannot be ruled out (Larsen et al., 1995; Morshed and Singh, 2015). However, our laboratory is not privy to information on prior or current treatment history, co-infections, or other disease conditions linked to this specimen set. It is also important to note that the current study’s evaluation is based on the syphilis stage reported by the PHL that provided the specimens, and these diagnoses could not be independently verified by us. Furthermore, frozen specimens were used for analysis when most assay manufacturers recommend use of fresh specimens. However, freeze-thaw cycles were kept to a minimum and a previous CDC study has shown that as many as ten freeze-thaw cycles have negligible impact on treponemal antibody reactivity when using EIA (Castro and Jost, 2013).

In summary, we demonstrated methods used for development of a syphilis serum repository through a collaborative initiative with APHL. The performance data of syphilis tests described herein will broaden insight on assay’s sensitivity in the context of syphilis stage. Indeed, there are limited number of studies that validate syphilis test performance – nontreponemal or treponemal-in relation to disease stage and this work may contribute new data in that respect. Our study is not intended to alter previously issued recommendations for a sequence algorithm(s) of laboratory-based serological tests to confirm ambiguous results. Indeed, these data collectively underscore the need to confirm or verify results with an additional test type(s), as recommended by current algorithms that are used to guide syphilis diagnosis. Factors that include clinical history and geographical prevalence should also be taken into consideration for syphilis diagnosis (APHL, 2015; Binnicker et al., 2012; Centers for Disease and Prevention, 2011; Park et al., 2019). Efforts to expand the CDC serum repository in collaboration with APHL member public health laboratories are ongoing.