Evaluation of the HISCL Anti-Treponema pallidum Assay as a Screening Test for Syphilis

Abstract

The resurgence of syphilis in recent years has become a serious threat to public health worldwide, and the serological detection of specific antibodies against Treponema pallidum remains the most reliable method for laboratory diagnosis of syphilis. This study examined the performance of the recently launched HISCL anti-Treponema pallidum (anti-TP) assay as a screening test for syphilis in a high-volume laboratory. The HISCL anti-TP assay was tested in 300 preselected syphilis-positive samples, 704 fresh syphilis-negative samples, 48 preselected potentially interfering samples, and 30 “borderline” samples and was compared head to head with the commercially available Lumipulse G TP-N. In this study, the HISCL anti-TP assay was in perfect agreement with the applied testing algorithms with an overall agreement of 100%, comparable to that of Lumipulse G TP-N (99.63%). The sensitivity and specificity of the HISCL anti-TP assay were 100% (95% confidence interval [CI], 98.42% to 100%) and 100% (95% CI, 99.37% to 100%), respectively. Considering the excellent ease of use and automation, high throughput, and its favorable sensitivity and specificity, the HISCL anti-TP assay may represent a new choice for syphilis screening in high-volume laboratories.

INTRODUCTION

Syphilis is a sexually transmitted disease caused by Treponema pallidum subsp. pallidum, which can also be passed from a mother to her fetus during pregnancy. Currently, infectious syphilis has become a significant public health problem throughout the world with an estimated 36.4 million people affected according to the WHO (1). China is among the countries experiencing a reemergence of syphilis with infection cases increasing by 30% per year (2, 3). Syphilis seroreactivity rates are high, especially in some population groups, such as men who have sex with men (11.9%) (2).

The course of syphilis infection may lead to various clinical presentations, which can be classified into early (infectious) and late (noninfectious) stages (4). In the early stages of syphilis, many individuals may be unaware that they are infected because symptoms are usually mild and atypical and may be missed or mistaken for other conditions, which may also result in a high risk of transmission via sexual intercourse or during pregnancy (4, 5). If left untreated, the infection may eventually progress to the more serious symptomatic late stages, with significant complications such as cerebral and vascular involvement (6). However, syphilis can be treated successfully and inexpensively with antibiotics, particularly if it is diagnosed in its early stages (5). Diagnosis is, therefore, crucial so that treatment may be initiated earlier to improve outcomes and prevent transmission (5).

Despite recent technological advances, the diagnosis of syphilis remains a challenging enterprise (7). As T. pallidum cannot be cultured in vitro, the serological detection of specific antibodies against T. pallidum is of particular importance (4, 8, 9). Serologic tests often fall into two categories: nontreponemal and treponemal assays. Nontreponemal tests detect the host's antibody response to a nontreponemal (cardiolipin lecithin) antigen and include the VDRL test, the rapid plasma reagin (RPR) test, the toluidine red unheated serum test, and unheated serum reagin tests. In contrast, treponemal tests allow for the detection of specific treponemal antibodies and include the T. pallidum hemagglutination assay (TPHA), the T. pallidum particle agglutination assay (TPPA), the fluorescent treponemal antibody absorption (FTA-ABS) test, Western blot, enzyme immunoassay (EIA), and more recently the chemiluminescent immunoassay (CLIA) (9–11). Even though many diagnostic tests are available for syphilis, there is still no gold standard, and diagnosis usually relies upon a combination of tests (12). The traditional approach to the diagnosis of syphilis begins with a nontreponemal assay, either the VDRL test or, more commonly, the RPR test, which detects anticardiolipin antibodies (13). Since these antibodies are not specific for syphilis, reactive nontreponemal assay results must be confirmed with an assay that detects antibodies produced against T. pallidum, such as the FTA-ABS test, TPHA, or TPPA (13). As the nontreponemal tests are unsuitable for automation, labor-intensive, and have a lack of sensitivity in primary syphilis and the late stages of infection, more and more clinical laboratories have shifted to screening patients with a treponemal assay such as EIA and CLIA, followed by a quantitative nontreponemal test to assess disease activity and monitor response to treatment (i.e., the “reverse algorithm”) (5, 10).

With the heavier burden of syphilis and the fast increase of test volumes, there is still a critical demand for simple, rapid, and high-throughput detection of syphilis. To solve these issues, the HISCL anti-Treponema pallidum (anti-TP) assay (Sysmex Corporation, Kobe, Japan) was recently launched. The HISCL anti-TP assay is a CLIA and displays excellent ability to automate testing in high-throughput instrumentations, which is also possible for other infectious disease testing.

In this study, we evaluated the performance of the HISCL anti-TP assay as a screening test for syphilis in a high-volume clinical laboratory in West China Hospital (a university-affiliated hospital with 4,300 beds). The HISCL anti-TP assay was compared with the commercially available and routinely used Lumipulse G TP-N, with the Mikrogen Syphilis Immunoblot as the confirmatory test.

MATERIALS AND METHODS

Samples.

During September and December 2014, a total of 1,082 samples were tested, including 300 preselected samples from patients with medically diagnosed syphilis (no more than one sample per patient). The staging of the disease was done following the accepted clinical and laboratory criteria (5, 14). In particular, 32 samples were from patients with primary syphilis, 21 were from patients with secondary syphilis, 12 were from patients with tertiary syphilis, 81 were from patients with latent syphilis, and 154 were from patients with unknown syphilis stages. Also included were 704 syphilis-negative fresh plasma samples tested by the commercially available and routinely used Lumipulse G TP-N and 48 preselected samples from patients with diseases or conditions known to potentially cross-react with syphilis assays to yield false-positive results (potentially interfering samples), including 5 hemolytic samples, 5 samples with high bilirubin, 10 samples from patients with autoimmune diseases (5 from patients with rheumatoid arthritis and 5 from patients with systemic lupus erythematosus), 24 samples from patients with other infection (5 samples from patients with HIV infections, 5 samples from patients with hepatitis B virus [HBV] infections, 5 samples from patients with hepatitis C virus [HCV] infections, 5 samples from patients with Epstein-Barr virus [EBV] infections, and 4 samples from patients with yeast infections), and 4 samples from patients with alcoholic cirrhosis. Furthermore, 30 samples with signal-to-cutoff (s/co) ratios ranging from 0.75 to 3.0, determined using the InTec enzyme-linked immunosorbent assay (ELISA) kit for TP, were collected as “borderline” samples. All were leftover plasma samples from routine requests and were anonymized for further analysis. The preselected samples were stored at −20°C prior to testing.

Assays.

The HISCL anti-TP assay (Sysmex Corporation, Kobe, Japan) is a two-step double-antigen sandwich (DAGS) qualitative chemiluminescence enzyme immunoassay (CLEIA) performed on fully automated analyzers (HISCL-5000; Sysmex Corporation, Kobe, Japan), which uses the recombinant treponemal TpN15, TpN17, and TpN47 antigens to simultaneously detect the anti-treponemal IgG and IgM antibodies in human serum or plasma. Samples were tested according to the manufacturer's instructions. The total assay time was 17 min, and the results were expressed as a cutoff index (COI), with a COI of <1.00 indicating a negative result and a COI of ≥1.0 indicating a positive result.

The Lumipulse G TP-N (Fujirebio Diagnostics, Inc., Tokyo, Japan) is a two-step DAGS CLIA performed on a fully automated system Lumipulse G1200 (Fujirebio Diagnostics, Inc. Tokyo, Japan), which utilizes the recombinant treponemal antigens (TpN15 to TpN17 and TpN47) to qualitatively detect the IgG and IgM antibodies to T. pallidum in human serum or plasma. The Lumipulse G TP-N showed good performance compared to that of TPPA (15) and was employed as a reliable test for syphilis detection (16). Samples were tested according to the manufacturer's instructions. The results were expressed as a COI, and the total assay time was 25 min. According to the manufacturer's instructions, a COI of <1.00 indicated a negative result while a COI of ≥1.0 indicated a positive result.

The InTec ELISA kit for TP (InTec Products, Inc., Xiamen, China) is a two-step DAGS ELISA that utilizes the recombinant treponemal antigens (TpN15, TpN17, and TpN47). The antigens are coated on the microplate wells and as a conjugate labeled with horseradish peroxidase to qualitatively detect the IgG and IgM antibodies to T. pallidum in human serum or plasma. Samples were tested according to the manufacturer's instructions. The results were expressed as an s/co ratio, and the total assay time was 120 min. According to the manufacturer's instructions, an s/co of <1.0 indicated a negative result, while an s/co of ≥1.0 indicated a positive result.

The Mikrogen Syphilis Immunoblot (Mikrogen Diagnostic, Martinsried, Germany) is a recombinant immunoblot assay, which allows the separate lining up of individual recombinantly produced antigens to determine specific IgG and IgM antibodies to individual T. pallidum antigens. Samples were tested according to the manufacturer's instructions. A test was considered positive when at least two of the six diagnostic bands corresponding to Tp47, TmpA, Tp257 (Gpd), Tp453, Tp17, or Tp15 were clearly recognized.

Methods and analyses.

The intra-assay variation of the HISCL anti-TP assay was evaluated by replicate measurements (n = 20) of patient samples at 3 levels (negative, weak positive, and strong positive) in 5 days (4 times a day).

To evaluate its sensitivity, preselected syphilis-positive samples were blindly tested using the HISCL anti-TP assay and the Lumipulse G TP-N on the same day. The HISCL anti-TP assay was also assessed for specificity in syphilis-negative samples and potentially interfering samples. As syphilis testing and testing algorithms vary from country to country, there is no gold standard for syphilis diagnosis (in fact, no definitive reference standard has been defined for syphilis testing) (12, 17). A “true positive/true-negative” result was either clinically confirmed or defined as a sample positive/negative by at least two of the three treponemal assays used in the study. The testing algorithms are shown in Fig. 1.

FIG 1.

The testing algorithms. Each true-positive or true-negative result was either clinically confirmed or defined as positive or negative, respectively, by at least two of the three treponemal assays used in the study.

For “borderline” samples, testing was also blindly performed using the HISCL anti-TP assay and the Lumipulse G TP-N. For testing algorithms, refer to Fig. 1.

To evaluate the overall clinical performance of this novel method, the HISCL anti-TP assay was also compared head to head with the commercially available and currently used Lumipulse G TP-N in terms of statistical positive and negative agreement.

Statistical analysis and representation were performed using SPSS 19.0 and GraphPad Prism 5.0. In addition to percent agreement, kappa coefficients were calculated as a secondary measure of agreement. The agreement of the results by kappa values is categorized as near perfect (0.81 to 1.0), substantial (0.61 to 0.8), moderate (0.41 to 0.6), fair (0.21 to 0.4), slight (0 to 0.2), or poor (<0) (18). The COI of the two assays were assessed using Pearson's correlation test. The statistical significance was defined as P being <0.05.

RESULTS

In the present study, a total of 1,082 samples were tested for treponemal antibodies to T. pallidum, and the HISCL anti-TP assay was in perfect agreement with Lumipulse G TP-N (κ = 0.98; 95% confidence interval [CI] 0.97 to 0.10). The intra-assay variation of the HISCL anti-TP assay was evaluated using patient samples at 3 levels, negative, weak positive, and strong positive, with the coefficient of variation (CV) value of <3% (Table 1).

TABLE 1.

Intra-assay variation of the HISCL anti-TP assay

Replicate measurement	No. of replicates	Mean COI	SD	CV (%)
Negative	20	0.00	0.00
Weak positive	20	3.53	0.07	1.98
Strong positive	20	35.48	0.86	2.42

The sensitivities of the HISCL anti-TP assay and Lumipulse G TP-N were assessed in 300 clinical and laboratory-characterized syphilitic samples. The results are shown in Table 2. The HISCL anti-TP assay and the Lumipulse G TP-N each showed 100% (95% CI, 98.42% to 100%) sensitivity for patients at different stages of syphilis.

TABLE 2.

Sensitivity of the HISCL anti-TP assay and Lumipulse G TP-N in samples from patients at different stages of syphilis

Syphilis stage and assay type	Total samples (no.)	True positive (no.)	False negative (no.)	Sensitivity (% [95% CI])
Primary
HISCL anti-TP	32	32	0	100 (86.66–100)
Lumipulse G TP-N	32	32	0	100 (86.66–100)
Secondary
HISCL anti-TP	21	21	0	100 (80.76–100)
Lumipulse G TP-N	21	21	0	100 (80.76–100)
Tertiary
HISCL anti-TP	12	12	0	100 (69.87–100)
Lumipulse G TP-N	12	12	0	100 (69.87–100)
Latent
HISCL anti-TP	81	81	0	100 (94.36–100)
Lumipulse G TP-N	81	81	0	100 (94.36–100)
Unknown stages
HISCL anti-TP	154	154	0	100 (96.97–100)
Lumipulse G TP-N	154	154	0	100 (96.97–100)
Overall
HISCL anti-TP	300	300	0	100 (98.42–100)
Lumipulse G TP-N	300	300	0	100 (98.42–100)

A total of 704 syphilis-negative samples and 48 potentially interfering samples were tested using the HISCL anti-TP assay and Lumipulse G TP-N. From the two groups, 704 and 47 samples yielded concordantly negative results, respectively. One potentially interfering sample yielded concordantly positive results and was confirmed to be from an HIV-positive patient coinfected with syphilis. The specificities were assessed with negative samples from the two groups and are shown in Table 3. The HISCL anti-TP assay and the Lumipulse G TP-N each showed 100% (95% CI, 99.37% to 100%) specificity.

TABLE 3.

Specificity of the HISCL anti-TP assay and Lumipulse G TP-N using samples from different patient populations

Sample and assay type	Total samples (no.)	True negative (no.)	False positive (no.)	Specificity (% [95% CI])
Syphilis negative
HISCL anti-TP	704	704	0	100 (99.32–100)
Lumipulse G TP-N	704	704	0	100 (99.32–100)
Potentially interfering
HISCL anti-TP	47a	47	0	100 (90.59–100)
Lumipulse G TP-N	47a	47	0	100 (90.59–100)
Overall
HISCL anti-TP	751	751	0	100 (99.37–100)
Lumipulse G TP-N	751	751	0	100 (99.37–100)

^aOne sample with a confirmed positive result was excluded from the analysis of specificity.

In 30 samples with borderline anti-TP results (s/co ratios ranging from 0.75 to 3.0, as previously measured using the InTec ELISA kit for TP), 22 samples yielded concordantly positive results and 8 samples yielded discordant results. The Mikrogen Syphilis Immunoblot assay was performed for the 8 samples, and the results are listed in Table 4. Of those, 4 samples with negative immunoblot results were confirmed to be “true negative,” and 4 samples with indeterminate immunoblot results were subsequently excluded from the study (refer to Fig. 1). Compared with the testing algorithms, the accuracies of the HISCL anti-TP assay and the Lumipulse G TP-N were 100% (95% CI, 83.98 to 100) and 84.62% (95% CI, 64.27 to 94.95), respectively.

TABLE 4.

Mikrogen syphilis immunoblot assay results for 8 borderline samples with discordant results between the HISCL anti-TP assay and Lumipulse G TP-N

Assay type	Result for sample no.
	1	2	3	4	5	6	7	8
HISCL anti-TP (COI)a	0.547	0.797	0.735	0.887	0.85	0.826	0.72	0.668
Lumipulse G TP-N (COI)a	1	1.2	1.3	1.6	1.4	1.3	1.4	1.1
Mikrogen syphilis immunoblotb	I	N	N	N	N	I	I	I

^aA COI value of <1 indicates a negative result, and a COI value of ≥1 indicates a positive result for each assay.

^bI, indeterminate; N, negative.

For the 1,078 samples, the agreements of the HISCL anti-TP assay and the Lumipulse G TP-N were compared with the testing algorithms, and the results are shown in Tables 5 and 6. The HISCL anti-TP assay and the Lumipulse G TP-N showed 100% (95% CI, 99.56% to 100%) and 99.63% (95% CI, 98.98% to 99.88%) agreement, respectively.

TABLE 5.

Comparison of positive and negative samples via the HISCL anti-TP assay or Lumipulse G TP-N with those of applied testing algorithms^a

Assay and result	Testing algorithm result
	Positive	Negative
HISCL anti-TP
Positive	323	0
Negative	0	755
Lumipulse G TP-N
Positive	323	4
Negative	0	751

^aSee Fig. 1.

TABLE 6.

Agreement of results of HISCL anti-TP assay or Lumipulse G TP-N and those of applied testing algorithms^a

Assay and result	No. agreed/no. total	Agreement (% [95% CI])	Kappa value
HISCL anti-TP
Overall agreement	1,078/1,078b	100 (99.56–100)	1.0
Positive agreement	323/323c	100 (98.53–100)
Negative agreement	755/755d	100 (99. 37–100)
Lumipulse G TP-N
Overall agreement	1,074/1,078b	99.63 (98.98–99.88)	0.99
Positive agreement	323/323c	100 (98.53–100)
Negative agreement	751/755d	99.47 (98.55–99.83)

^aSee Fig. 1.

^bA total of 1,082 samples were tested in this study, and 4 samples that had indeterminate immunoblot results were excluded.

^cThe total number of positive samples consisted of 300 preselected syphilis-positive samples, 1 confirmed positive sample from the potentially interfering group, and 22 confirmed positive samples from the borderline group.

^dThe total number of negative samples consisted of 704 syphilis-negative samples, 47 confirmed negative samples from the potentially interfering group, and 4 confirmed negative samples from the borderline group.

To further assess the HISCL anti-TP assay, the distribution of COIs of all samples was analyzed and is shown in Fig. 2. For the 300 syphilis-positive samples, the COI values obtained using the HISCL anti-TP assay or the Lumipulse G TP-N correlated well, with a Pearson's correlation coefficient of 0.95 (P < 0.05) (Fig. 3).

FIG 2.

Distribution of the COI values of samples determined with the HISCL anti-TP assay. NEG, negative; POS, positive.

FIG 3.

Correlation of the COI values between HISCL anti-TP assay and Lumipulse G TP-N in syphilis-positive samples (n = 300).

DISCUSSION

The resurgence of syphilis in recent years has become a serious threat to public health worldwide and led to the research and development of a number of serologic tests for syphilis diagnosis. Among the various tests, CLIA is most popular and widely employed for its ease of use, high sensitivity, and suitability of automation. Compared with the traditional CLIA, the HISCL anti-TP assay has several new features.

The HISCL anti-TP assay is suited to provide rapid diagnosis in laboratories with high test volumes, with the first results available within about 17 min and the capability of providing up to 200 results per hour (Table 7). In this novel test, the minimum volume for a sample is 20 μl, and antigens first react with antibodies in liquid phase to improve the response speed. In the process of bound/free separation, microparticles will be released after the first wash and then captured again before the next wash to make it more thorough. The HISCL anti-TP assay includes a disposable tip and a specific membrane filter to wipe the sample suction mouth after sampling to minimize the potential for carryover. Furthermore, the sensitivity of the chemiluminescent substrate (CDP-Star) employed in HISCL is 4 times as high as that of the traditional 3-(2-spiroadamatane)-4-methoxy-4-(3-phosphoryloxy)-phenyl-1,2-dioxetane (AMPPD). In addition, the HISCL anti-TP assay is very convenient to replenish reagents at any time without interrupting measurements.

TABLE 7.

Performance of the HISCL anti-TP assay compared with other treponemal assays based on CLIA

Assay	Antigen(s)	Antibody target(s)	Sample vol (μl)	Assay time (min)	Throughput time (h)
HISCL anti-TP	TpN15, TpN17, TpN47	IgG, IgM	20	17	200
Elecsys syphilis (17)	TpN15, TpN17, TpN47	IgG, IgM	10	18	170
Architect syphilis TP (4, 27, 28)	TpN15, TpN17, TpN47	IgG, IgM	30	29	200
Liaison Treponema pallidum specific (24, 29)	TpN17	IgG, IgM	80	40	180
Chemclin CLIA (30)	TpN15, TpN17, TpN47	naa	na	≥60	na
Immulite syphilis screen (9)	Tp17	na	100	35	na

^ana, not applicable.

With all these new features, the HISCL anti-TP assay stands out as a new choice for syphilis testing. More effort is needed to explore its performance in areas with different syphilis prevalence rates and different patient populations.

In the present study, the HISCL anti-TP assay showed favorable reproducibility (intra-assay CV, <3%), and the overall sensitivity and specificity were 100% (95% CI, 98.42% to 100%) and 100% (95% CI, 99.37% to 100%), respectively. The HISCL anti-TP assay was in perfect agreement with the applied testing algorithms (Table 6).

For syphilis serology, sensitivities vary depending on the type of test and stage of infection, with lower sensitivities in primary syphilis and late syphilis (10). In the present study, 300 clinical and laboratory-characterized syphilitic plasma samples from patients with different stages of syphilis were collected to assess sensitivities. The HISCL anti-TP assay and Lumipulse G TP-N each showed 100% sensitivity with samples at all stages of infection, including primary and late syphilis. The use of three recombinant antigens (TpN15, TpN17, and TpN47) was reported to contribute to the excellent sensitivity (19) (TpN16 was also used in Lumipulse G TP-N). TpN47 is highly immunogenic and activates endothelial cells, and TpN17 and TpN15 induce antibody responses, all of which are essential in mediating inflammatory effects of T. pallidum and are detectable throughout the disease course (10, 20). The response against TpN47 is one of the earliest detectable during the disease course (21). The assays are based on the DAGS principle, where treponemal antibodies, including IgM, produced within approximately 2 weeks after exposure can be effectively captured and detected (10, 22). However, there is still some controversy over the clinical relevance of IgM antibody testing. Except for the increased sensitivity in the early stages of syphilis, IgM antibody testing is also considered to be a source of problems due to false-positive results (7). In some treponemal tests, such as BioElisa Syphilis 3.0, CAPTIA Syphilis-G, and Trep-Check IgG EIA, only the IgG antibody is tested to avoid the potential false-positive result caused by the IgM antibody (10, 19, 23). As early detection is crucial in syphilis diagnosis to improve outcomes and prevent transmission, the employment of IgM antibody testing is worthwhile in syphilis screening tests to reduce delayed or missed diagnoses. In addition, confirmatory tests are essential to reduce false-reactivity.

Leftover routine syphilis-negative samples were collected to assess for specificity. As syphilis assays are often characterized by a number of false-positive reactions, 48 samples (including hemolytic samples, samples with high bilirubin, and samples from patients with autoimmune disease and other infections) were included, which represented populations with reportedly high rates of false-positive serological testing results (5, 10, 20, 24–26). The HISCL anti-TP assay demonstrated an overall specificity of 100% in different patient populations (Table 3), comparable to that of Lumipulse G TP-N (100%). The HISCL anti-TP assay performed very well as a screening test for patients with autoimmune disorders and other infections. Viral infections are common in syphilitic patients, especially HIV. T. pallidum can produce open lesions, which may result in an increased risk of acquiring HIV. In addition, HIV can adversely affect the serologic response to syphilis and may affect the detection of infection (24, 25). In the present study, no cross-reactivity was observed. The HISCL anti-TP assay and Lumipulse G TP-N can each detect syphilis infection correctly with the present interference.

In the clinical setting, we are often faced with the problem that some samples with borderline results are difficult to define, which may result in a waste of time and money for duplicate detection. In the present study, 30 “borderline” samples determined with a commercially available ELISA were included to assess the accuracy of the HISCL anti-TP assay and Lumipulse G TP-N. Compared with the testing algorithms, the HISCL anti-TP assay showed perfect performance in distinguishing the “borderline” samples with an accuracy of 100%. As for the Lumipulse G TP-N, there was a higher false-positive rate (15.38%). The use of the treponemal antigens TpN15, TpN17 and TpN47 was most common in treponemal tests (Table 7). Except for that, TpN16 was additionally included in Lumipulse G TP-N, which may guarantee sensitivity but may also account for the high false-positive rate. The distribution of COI values in all testing samples, including 30 preselected borderline samples, was assessed (Fig. 2). The HISCL anti-TP assay was efficient in distinguishing between positive and negative results.

A limitation of the study is that, although the test performed well for adult patients with primary, secondary, tertiary, and latent syphilis, patients with congenital syphilis were not included. As with all other treponemal assays, the HISCL anti-TP assay cannot distinguish among recent, remote, and previously treated infections. Therefore, its interpretation must be combined with clinical findings and results from other tests for optimal use.

In conclusion, considering the ease of use and automation, high throughput, and its good sensitivity and specificity, the HISCL anti-TP assay may represent a new choice for the screening of syphilis in high-volume laboratories, but the results should be carefully evaluated in association with other laboratory and clinical findings.