Retrospective Review of Treponema pallidum PCR and Serology Results: Are Both Tests Necessary?

ABSTRACT

There has been a resurgence of syphilis diagnoses in Australia. We investigated whether our Treponema pallidum PCR test provides any additional diagnostic information over syphilis serology (chemiluminescence immunoassay [CMIA], Treponema pallidum particle agglutination [TPPA] assay, and the rapid plasma reagin [RPR] flocculation test). A retrospective audit of all T. pallidum PCR requests that came through our laboratory from January 2010 to June 2017 was conducted; data collected included age, gender, site of swab, and results from T. pallidum PCR, syphilis serology, and herpes simplex virus 1 (HSV-1) and HSV-2 PCRs. A total of 441 T. pallidum PCR tests were performed; on average, 3 T. pallidum PCRs per month were requested in 2011, and this rate increased to 17.2 requests per month in 2017. A total of 323 patients had both T. pallidum PCR and syphilis serology performed, with 67% of swabs taken from the genitals. T. pallidum PCR gave positive results for 61/323 (19%) patients; of these 61 patients, 59 (97%) also had positive syphilis serology results (T. pallidum PCR sensitivity, 68%; specificity, 99%; positive predictive value, 97%; negative predictive value, 89%). Syphilis serology was positive for 91/323 patients (28%); of these 91 patients, 61 (66%) were also T. pallidum PCR positive (syphilis serology sensitivity, 97%; specificity, 88%; positive predictive value, 60%; negative predictive value, 99%). The Cohen's kappa value was 0.74, indicating substantial agreement between the two tests. Our results show that most patients with positive T. pallidum PCR results also had positive syphilis serology. Therefore, T. pallidum PCR adds little clinical value over serology for the diagnosis of syphilis in certain clinical settings.

INTRODUCTION

Syphilis is a sexually transmitted disease caused by the spirochete Treponema pallidum. It has varied disease manifestations and is classically divided into three distinct stages: primary, secondary, and tertiary syphilis (1). There has been a resurgence of syphilis in Australia. This has occurred largely in the metropolitan areas, predominantly among men who have sex with men (MSM), but now syphilis is also increasing in regional indigenous communities, particularly among adolescents and young adults (2–4).

The diagnosis of syphilis remains a challenge. Serology has emerged as the mainstay of syphilis diagnosis and is divided into treponemal and nontreponemal tests. The nontreponemal tests are the rapid plasma reagin (RPR) and Venereal Disease Research Laboratory (VDRL) tests. These define disease activity but suffer from poor specificity. Treponemal tests detect antibodies to treponema species and are used to confirm the presence of a treponemal infection. These include the chemiluminescence immunoassay (CMIA), Treponema pallidum hemagglutination (TPHA) assay, and Treponema pallidum particle agglutination (TPPA) assay. Their sensitivities and specificities are not affected by the activity of the infection, and these tests remain positive for life (5, 6).

Although serological tests are very sensitive at diagnosing secondary syphilis, some studies have estimated the sensitivity of serology for diagnosing primary syphilis at 86% (5). This is due to a window period whereby the humoral response to syphilis develops 1 to 4 weeks after the chancre forms in primary syphilis (7). PCR tests have been developed to aid in the diagnosis of early, primary syphilis. These tests have proven to be both sensitive and specific for primary syphilis but less sensitive for secondary syphilis (8, 9). Further studies have also indicated that PCR may in fact be more sensitive than serology in early, primary syphilis and suggest that PCR is a necessary adjunct to serological testing to ensure that no patients in the “window period” are missed (10).

Our laboratory introduced an in-house T. pallidum PCR test in 2001, but this test was widely adopted only in 2010. Serology is usually performed concurrently with the PCR test. The T. pallidum PCR test is performed in the central laboratory in Brisbane, Australia, 1,400 km away from our facility, increasing turnaround time and costs associated with the transport of specimens. Serology is performed in our local Townsville laboratory. This duplication in testing (i.e., serology plus PCR), in addition to the increased specimen handling and transport, more than doubles the cost of syphilis screening and diagnosis, from AUS$25 to AUS$60, on conservative estimates based on the typical costs billed by our facility. For this reason, and because of increasing requests for syphilis PCR, we decided to conduct a study looking at whether the addition of the PCR test resulted in any extra diagnoses of primary syphilis, or whether positive patients would have been captured regardless with positive serological tests.

MATERIALS AND METHODS

Study population.

The Pathology Queensland (QLD) laboratory at the Townsville hospital is one of a network of 35 public hospital laboratories. In addition to providing laboratory services to the 600-bed Townsville hospital, the laboratory also provides diagnostic services to hospitals and health care facilities across North Queensland, in the northeast of Australia. Some PCR tests, including the T. pallidum PCR, are performed by the central Pathology QLD laboratory in Brisbane. Clinical samples in this study came from hospitals and sexual health services in the city of Townsville, as well as the surrounding towns of Ayr, Palm Island, Mornington Island, Hughenden, Mount Isa, Charters Towers, and Doomadgee.

Study design and clinical definitions.

A retrospective review of the results of T. pallidum PCR requests from 2010 to 2017 was performed. We used an extended search of our results database, “AUSLAB,” with requests for T. pallidum PCR from 2010 to 2017 as the primary filter. Patients <18 years old, and those whose swabs were from placentas, were excluded. The results of serology performed within 14 days of the T. pallidum PCR, as well as age, gender, the site of the swab, and the results of herpes simplex virus 1 (HSV-1) and HSV-2 PCRs performed on the same swab were also collected.

Prospective approval for this audit was granted by the Townsville Hospital and Health Service Human Research Ethics Committee (HREC/17/QTHS/87).

T. pallidum PCR testing.

Swabs of lesions were sent either in dry sterile containers or in vials containing viral transport medium. If a swab was sent for T. pallidum PCR, it was assumed that there was a clinical lesion suggestive of a chancre. PCR testing was performed at the Pathology QLD laboratory in Brisbane. DNA was extracted from swab samples by use of the MagNA Pure 96 system (Roche Applied Science, Indianapolis, IN). The PCR targeted the T. pallidum 47-kDa integral membrane lipoprotein gene and was based on a previously described method (11). Amplification and detection were achieved on the Rotor-Gene Q real-time PCR instrument (Qiagen, Victoria, Australia).

Syphilis serology testing.

Syphilis serology was performed on sera at the Pathology QLD laboratory at the Townsville hospital. The initial test was a Treponema pallidum chemiluminescence immunoassay (CMIA) performed on the commercially available Architect analyzer (Abbott Diagnostics, Lake Bluff, IL) as per the manufacturer's instructions. If the result was positive by CMIA, a Treponema pallidum particle agglutination (TPPA) assay was performed with the Serodia-TP-PA test kit (Fujirebio, Tokyo, Japan) according to the manufacturer's instructions to confirm the result, and a quantitative rapid plasma reagin (RPR) flocculation test was performed with the BD Macro-Vue RPR card test kit (Thermo Fisher Scientific, Waltham, MA, USA) as per the manufacturer's instructions. The syphilis serology was considered positive and consistent with a new diagnosis of syphilis if the patient had positive reactive treponemal tests (CMIA and TPPA), with or without a positive RPR test result. Serology was considered positive and consistent with new infection if the patient had positive CMIA and TPPA results recorded previously and the RPR was 4-fold higher than that previously recorded (5, 7). If the CMIA result was positive and the TPPA and RPR results were negative, we attached a comment stating that this might be a false-positive result and suggesting repeat testing in 2 to 4 weeks.

Statistical analysis.

Descriptive and regression statistics were performed using Stata 13 (StataCorp, College Station, TX, USA). The sensitivities, specificities, positive predictive values, and negative predictive values of both the T. pallidum PCR and syphilis serology tests were evaluated by using each test as the gold standard for the other.

A kappa value was calculated to measure the level of agreement between the two tests, T. pallidum PCR and syphilis serology. Categorical data were tested using the chi-square test for dependence; a P value of <0.05 was considered significant.

RESULTS

There were 516 requests for T. pallidum PCRs, on 493 unique patients, between January 2010 and May 2017. Of these, 75 were excluded. Reasons for exclusion were an age of <18 years (based on the recommendations of our local Ethics Committee) (51 patients), cancellation of the test prior to performance (11 patients), and performance of the test to investigate congenital syphilis (13 patients). There were, on average, 3 requests for T. pallidum PCR per month in 2011, and this rate increased to 17.2 per month in 2017 (Fig. 1).

FIG 1.

Average monthly rate of T. pallidum PCR requests by year.

Of the 441 patients who had swabs tested for T. pallidum PCR, 323 (73%) also had syphilis serology performed within 14 days. These were the patients included in our analysis.

The median patient age was 29 years (interquartile range [IQR], 23 to 37 years), and 197/323 (61%) were male. The swabs were taken predominantly from the genital region (Table 1). A summary of the T. pallidum PCR, serological, and herpes simplex virus 1 (HSV-1) and HSV-2 PCR results is provided in Table 2.

TABLE 1.

Sites of swabs taken for T. pallidum PCR

Site	No. (%) of swabs
Genital	216 (67)
Anal	21 (7)
Oral	23 (7)
Othera	9 (3)
Ulcer/lesion with no site specified	40 (12)
No clinical information	14 (4)
Total	323 (100)

^aIncludes thighs, groin, arms, face, and eyes.

TABLE 2.

Summary of all results for the 323 patients tested both by T. pallidum PCR and by serology

Pattern of results and no. of samples	Result by:
	Serologya				PCR
	CMIA	TPPA	RPR	Consensusb	T. pallidum	HSV-1	HSV-2
T. pallidum PCR positive,  serology positive
1	Pos	Pos	NR	Pos	Pos	N/P	N/P
3	Pos	Pos	NR	Pos	Pos	Neg	Neg
11	Pos	Pos	Pos	Pos	Pos	N/P	N/P
39	Pos	Pos	Pos	Pos	Pos	Neg	Neg
7	Pos	Pos	Pos	Pos	Pos	Neg	Pos
T. pallidum PCR positive,  serology negative (2)	Neg	N/P	N/P	Neg	Pos	Neg	Neg
T. pallidum PCR  negative, serology  positive
2	Pos	Pos	NR	Pos	Neg	N/P	N/P
4	Pos	Pos	NR	Pos	Neg	Neg	Neg
7	Pos	Pos	Pos	Pos	Neg	N/P	N/P
17	Pos	Pos	Pos	Pos	Neg	Neg	Neg
T. pallidum PCR  negative, serology  negative
42	N/A	N/A	N/A	Neg	Neg	N/P	N/P
119	N/A	N/A	N/A	Neg	Neg	Neg	Neg
15	N/A	N/A	N/A	Neg	Neg	Pos	Neg
53	N/A	N/A	N/A	Neg	Neg	Neg	Pos
1	N/A	N/A	N/A	Neg	Neg	Pos	Pos

^aCMIA, chemiluminescence immunoassay; TPPA, Treponema pallidum particle agglutination; RPR, rapid plasma reagin; Pos, positive; Neg, negative; NR, nonreactive; N/P, not performed; N/A, not applicable (syphilis serology was assessed as negative due to a nonreactive CMIA, or serology that was reactive but unchanged from a previous test result).

^bSyphilis serology assessed as positive or negative based on previous results.

T. pallidum PCR was positive for 61/323 (19%) samples. Syphilis serology was positive for 59 (97%) of these 61 PCR-positive samples. One of the two patients with a positive T. pallidum PCR result and a negative syphilis serology (CMIA) result at baseline had positive CMIA, equivocal TPPA, and negative RPR results when these tests were performed 2 months later. The other patient had follow-up serology (CMIA) performed at 1 month, which remained negative. Two months after this, the patient's repeat CMIA was positive, but TPPA and RPR remained negative. The T. pallidum PCR test had a sensitivity of 68%, a specificity of 99%, a positive predictive value of 97%, and a negative predictive value of 89% by use of syphilis serology as the gold standard (Table 3).

TABLE 3.

Syphilis serology versus T. pallidum PCR results

Syphilis serology result	No. of patients:
	With the following T. pallidum PCR result		Total
	Positive	Negative
Positive	61	30	91
Negative	2	230	232
Total	63	260	323

Of 323 patients, 91 (28%) had positive syphilis serology, and 61 of these 91 patients (66%) were also positive by the T. pallidum PCR (Table 3). Syphilis serology had a sensitivity of 97%, a specificity of 88%, a positive predictive value of 60%, and a negative predictive value of 99% by use of T. pallidum PCR as the gold standard. The Cohen's kappa value was 0.74, indicating substantial agreement between the two tests.

Of the 30 patients who were serology positive and PCR negative, 17 (57%) had an RPR titer of ≥16, compared with 49 (80%) of the 61 patients who were serology positive and PCR positive (P = 0.023) (Fig. 2).

FIG 2.

Rapid plasma reagin (RPR) values for all the patients with positive serological results. NR, nonreactive.

HSV-1 and HSV-2 PCR tests were performed for 260 of the 323 patients (80%) (Table 2). Overall, 5% of the patients were positive for HSV-1 and 20% for HSV-2. One patient was positive for both HSV-1 and HSV-2, and seven patients were positive by both the HSV-2 and T. pallidum PCRs.

DISCUSSION

The prevalence of syphilis has been increasing in recent times, which has resulted in the need for our pathology service to provide syphilis diagnostic tests with accuracy and precision that return results in a timely manner (2). The syphilis PCR test was introduced in addition to syphilis serology in order to increase sensitivity in diagnosing primary syphilis. This was due to the “window period” of the serological tests in early disease (7); the sensitivity of syphilis serology is estimated at 86% for primary syphilis (5). As can be seen from Fig. 1, the use of T. pallidum PCR in our laboratory has been increasing. However, our results indicate that in our setting, the syphilis PCR test does not add significant diagnostic value. Syphilis serology demonstrated an excellent negative predictive value (99%) in our study. Only 2/61 patients (3%) with positive syphilis PCR results had negative serology at the time of diagnosis. One of these patients had an equivocal TPPA result when serum was retested 2 months later, which could be considered a positive syphilis serology result. It is reasonable, therefore, to consider that this patient was in the serological window period at the time of initial diagnosis and that the syphilis PCR yielded a true-positive result. However, the second patient had a negative result by repeat CMIA 1 month after initial testing. Two months after this, the patient's repeat CMIA was positive but the TPPA and RPR remained negative. This patient's syphilis serology never became truly positive, and it is possible that the original positive T. pallidum PCR result was false positive.

In contrast, other published studies, including a meta-analysis, have shown the value of PCR in addition to serology, reporting that as many as 8% of patients with positive T. pallidum PCR results had negative syphilis serology at the time of diagnosis (8, 10). We postulate that our lower rate may be due to patients presenting later in the natural history of their disease than in other settings. We provide pathology services to many surrounding communities in which socioeconomic disadvantage and difficulty accessing health care have been shown to correlate with the prevalence of sexually transmitted infections (12).

Syphilis serology showed poor positive predictive value (60%) in our study: 30 patients had positive syphilis serology and negative T. pallidum PCR results. This resulted in a calculated sensitivity of 67% for the syphilis PCR test by use of syphilis serology as the gold standard, which is lower than the 78.4% for ulcers in primary syphilis reported from a recent meta-analysis (10). We postulate that this disagreement between the two tests occurred for the following reasons. To confirm that the positive syphilis serology did indeed represent new infection, we could only check for past syphilis results on our local pathology database. Theoretically, some of the positive results could be due to past infection if previous positive serology was performed with a different pathology service. In addition, we believe that since we had very little corresponding clinical data, some of these serology-positive, PCR-negative results may represent secondary syphilis, for which T. pallidum PCR testing has been shown to have lower sensitivity (8, 9), or late latent infection. The RPR values were significantly lower (Fig. 2) (P = 0.023) for serology-positive, PCR-negative patients than for serology-positive, PCR-positive patients, also indicating less acute infection (5).

Our study also enabled us to view how clinicians are using the syphilis PCR test in their diagnostic algorithms. A quarter of patients (26%) who had a syphilis PCR test performed had no syphilis serology requested, including two patients with positive PCR results. Australian and international guidelines recommend performing syphilis serology at baseline so that the response to treatment can be monitored (https://www.cdc.gov/std/syphilis/treatment.htm), and no guideline, to our knowledge, recommends using syphilis PCR alone to diagnose primary syphilis. This suggests that we may need to increase education for health professionals regarding this issue.

There has been a steady increase in the requests for syphilis PCR coming through our laboratory (Fig. 1). At a cost of AUS$35 per test (not including transport and processing costs), this means we now spend, on average, ca. AUS$7,000 a year on this test, an amount that could increase significantly if the rate of requests continues to increase.

HSV-1 and -2 were shown to be important differentials for primary syphilis, with 25% of samples testing positive for HSV-1 or -2. This finding is consistent with other reported rates and confirms the need to perform HSV-1 and -2 PCRs on genital lesions as well as investigating for primary syphilis (8). Twenty percent of our patients did not have HSV-1 or -2 PCR tests requested.

We also noted that 7/61 (11%) T. pallidum PCR-positive patients had HSV-2–syphilis coinfections. High rates of coinfection have been reported previously, particularly in cohorts of men who have sex with men, and occur due to shared routes of sexual transmission (13).

The major limitation of our study is the lack of clinical information with which we could correlate our laboratory results, including human immunodeficiency virus (HIV) status, whether the patients had acquired syphilis more than once, and whether any had received prior antibiotic therapy. We assumed, for the purposes of this study, that a swab of a lesion sent with a request for syphilis PCR was an investigation for a chancre in primary syphilis. For this study, we had no means of assessing what these lesions looked like, and how much they resembled a typical syphilitic lesion. We also had no information on how long the lesion or symptoms had been present. Such information would have allowed us to stage the condition as primary or secondary and to understand whether patients were indeed presenting late in their illness. A further limitation of this study is that we did not explore whether differences in PCR methodology affected the results. For example, previous studies have found that nested PCR, or use of the polA gene (rather than the 47-kDa target used here), may enhance PCR sensitivity (10).

In conclusion, we believe that the results of this study indicate that the addition of syphilis PCR to serology in the diagnostic algorithm for primary syphilis may not be necessary in our setting. We had a very low rate of positive syphilis PCR tests that were not accompanied by positive syphilis serology. Since syphilis serology is recommended to be performed at baseline to allow for monitoring of the response to treatment, it seems reasonable to reconsider the need to perform a second diagnostic test, especially in this age of increasing awareness of the need for pathology stewardship (14). However, we acknowledge that other studies have shown different results, and we advocate for individual health services to assess whether the syphilis PCR is necessary in addition to serological testing.