Alternatives to the Cerebrospinal Fluid Venereal Disease Research Laboratory Test for Neurosyphilis Diagnosis

Abstract

Background:

The diagnosis of neurosyphilis relies in large part on the cerebrospinal fluid (CSF) Venereal Disease Research Laboratory (VDRL) test, which is diagnostically specific but not sensitive.

Methods:

We determined the sensitivity and specificity of three CSF tests in addition to the CSF-VDRL in participants with syphilis enrolled in a research study: detection of T. pallidum rRNA, Treponema pallidum Particle Agglutination (TPPA) titer, and chemokine (C-X-C motif) ligand 13 (CXCL13) concentration. Neurosyphilis was defined as asymptomatic or symptomatic meningitis: CSF white blood cells (WBCs) >10/ul without or with neurological symptoms, including new vision or hearing loss.

Results:

CSF-VDRL, CSF T. pallidum rRNA detection, and CSF-TPPA titer ≥1:640 were specific (89–96%) but not sensitive (12–48%). In contrast, diagnostic sensitivity of CSF CXCL13 thresholds established from receiver operating characteristic curves using Youden’s index was 78–83% and specificity was 76–81%. In individuals with nonreactive CSF-VDRL, neurosyphilis diagnosis could be confirmed by CSF-CXCL13 concentration in 69–75%.

Conclusions:

Further studies of CSF CXCL13 should include CSF samples from multiple cohorts and countries and should use standard neurosyphilis definitions to establish uniform thresholds for diagnosis.

Summary

In a study of individuals with syphilis, CSF-VDRL, CSF T. pallidum rRNA detection, and CSF-TPPA titer ≥1:640 were specific (89–96%) but had low sensitivity (12–48%) for neurosyphilis diagnosis. In contrast, diagnostic sensitivity of CSF CXCL13 was 78–83% and specificity was 76–81%.

Introduction

The diagnosis of neurosyphilis relies in large part on cerebrospinal fluid (CSF) nontreponemal antibody tests, including the CSF Wassermann test in the previous century, and the current Venereal Disease Research Laboratory (VDRL) test. The latter is noted to be imperfect because, while specific, meaning that those without neurosyphilis almost always have a negative test, it lacks sensitivity, meaning that those with neurosyphilis don’t always have a positive test (1). Many have lamented that, in the era of specific and sensitive molecular diagnostic tests for other sexually transmitted infections, syphilis diagnostics have not advanced beyond what has been used for over a century.

There are two easily identified reasons to explain this failure. The first is predictable: many studies have compared “new” diagnostic tests to the CSF-VDRL as the gold standard. Such an approach cannot identify a test that is better than the CSF-VDRL. The second reason is more complicated. We know from studies conducted in the early 1900s that, unlike other bacteria that invade the CSF and cause fulminant meningitis, T. pallidum subspecies pallidum (hereafter T. pallidum) can be cleared from CSF without specific therapy and without causing persistent central nervous system (CNS) infection. Moreover, when invasion does lead to infection, diagnostic test performance may vary based on the duration of infection or the clinical manifestations of the infection. A good example of this is provided by Merritt and colleagues in their textbook on neurosyphilis, where the CSF-Wassermann was reported to be reactive in 84% of individuals with asymptomatic neurosyphilis, 91% of those with symptomatic meningitis (both early forms of neurosyphilis), and in 100% of those with syphilitic dementia (a late form of neurosyphilis) (2).

In this manuscript, we provide data on three tests in addition to the CSF-VDRL: CSF reverse transcriptase polymerase chain reaction (RT-PCR) for detection of T. pallidum 16S ribosomal RNA (rRNA), CSF-T. pallidum particle agglutination with a threshold of ≥1:640, and CSF chemokine (C-X-C motif) ligand 13 (CXCL13) concentrations (≥66.7 and ≥184.2) for diagnosis of neurosyphilis using two clinical gold standards in a large cohort of individuals enrolled in a study of CSF abnormalities in syphilis. We previously published data on all of these tests but used different gold standards (3–5); this analysis includes additional samples, and, although it is based on an oral presentation at the 2020 STD Prevention Conference, additional statistical analyses are included.

Methods

Participant Characteristics

We conducted a study of CSF abnormalities in individuals diagnosed with syphilis between 1996 and 2014. Details of the study have been published elsewhere (6). Briefly, eligibility for enrollment included clinical or serological evidence of syphilis and concern for neurosyphilis by the referring provider or by the patient. Reasons for referral to the study included, but were not restricted to, 1) neurological symptoms or signs, particularly vision or hearing loss; 2) serum Rapid Plasma Reagin (RPR) titer ≥1:32, or 3) in persons living with (PLWH), peripheral blood CD4+ T cell count ≤350/ul. Participants could re-enroll in the study with subsequent episodes of syphilis. The study protocol was reviewed and approved by the University of Washington institutional review board, and written informed consent was obtained from all study participants.

Laboratory Methods

Cerebrospinal fluid WBC enumeration and CSF-Venereal Disease Research Laboratory (VDRL) test reactivity were determined in a Clinical Laboratory Improvement Amendments-certified hospital clinical laboratory. Identification of T. pallidum 16S ribosomal RNA in CSF was performed using RT-PCR as previously described (3). Cerebrospinal fluid-TPPA and CXCL13 assessments were performed according to manufacturers’ instructions for serum using cell free CSF (4, 5).

Statistical Analysis

Beginning in 2004, all participants were categorized as no neurosyphilis, asymptomatic meningitis, ocular syphilis, otosyphilis or symptomatic neurosyphilis based on CSF and clinical findings. For the purposes of the analysis, asymptomatic meningitis was defined as no neurological symptoms and with CSF white blood cells (WBCs) >10/ul, and symptomatic meningitis as neurological symptoms, including new vision loss and hearing loss, and CSF WBCs >10/ul.

All available data were included in the analysis of CSF-VDRL and detection of CSF T. pallidum by RT-PCR. Data from convenience samples were included in the analysis of CSF-TPPA and CSF-CXCL13 (4, 5).

Descriptive statistics are expressed as number (percent) or median (interquartile range [IQR]). Sensitivity and specificity were calculated using standard formulae. Differences in sensitivity and specificity were compared using the two sample test of proportions, Stata version 11.2 (StataCorp, College Station, TX, USA). Youden’s index was derived from receiver operating characteristic curves to identify optimal sensitivity and specificity for CSF CXCL13 concentrations using SPSS version 27 (IBM Corporation, Armonk, NY, USA).

Results

Participant characteristics

Seven-hundred fifty-one participants experienced 929 syphilis episodes. Most participants were men (734, 97.7%), 150 (20%) were non-white, 699 (95.4%) of 733 were men who have sex with men, and 587 (78.3%) of 750 were PLWH. Overall, there were 116 episodes of asymptomatic syphilitic meningitis, and 54 episodes of symptomatic syphilitic meningitis.

CSF-VDRL

The sensitivity and specificity of a reactive CSF-VDRL for the two diagnoses of neurosyphilis were determined in 751 individuals with 929 syphilis episodes (Table 1). As expected, specificity was high, ranging from 89.1–91.6% for the group as a whole, but sensitivity was low (45.7–48.1). These estimates were almost identical if the analysis was restricted to first enrollments for each participant (specificity 89.0–91.6%, sensitivity 45.5–46.0%). There were no significant differences in sensitivity and specificity between participants without HIV and participants who were PLWH.

Table 1.

Sensitivity and Specificity of CSF-VDRL Reactivity for Neurosyphilis Diagnosis

	Sensitivity Proportion			Specificity Proportion
	All	No HIV	PLWH	All*	No HIV	PLWH
Asymptomatic meningitis	45.7% 53/116	48.1% 13/27	44.9% 40/89	91.6% 745/813	89.1% 123/138	92.1% 621/674
Symptomatic meningitis	48.1% 26/54	42.9% 6/14	50.0% 20/40	89.1% 780/875	85.4% 129/151	89.9% 650/723

^*HIV status was missing for one participant

CSF-RT-PCR

The sensitivity and specificity of detection of T. pallidum rRNA by RT-PCR for the two diagnoses of neurosyphilis was determined in 746 individuals with 914 syphilis episodes (Table 2). Similar to the CSF-VDRL, specificity was high, ranging from 91.7–91.9%, but sensitivity was lower in asymptomatic patients (25.4%), but similar in symptomatic patients (42.6%). As with the CSF-VDRL, these estimates were almost identical if the analysis was restricted to first enrollments (specificity 90.6–90.7%, sensitivity 26.3–44.0%). There were no significant differences in sensitivity and specificity between participants without HIV and participants who were PLWH.

Table 2.

Sensitivity and Specificity of Detection of T. pallidum rRNA by CSF-RT-PCR for Neurosyphilis Diagnosis

	Sensitivity Proportion			Specificity Proportion
	All	No HIV	PLWH	All*	No HIV	PLWH
Asymptomatic meningitis	25.4% 29/114	19.2% 5/26	27.3% 24/88	91.9% 735/800	91.1% 123/135	92.0% 611/664
Symptomatic meningitis	42.6% 23/54	50.0% 7/14	40.0% 16/40	91.7% 789/860	93.2% 137/147	91.4% 651/712

^*HIV status was missing for one participant

CSF-TPPA titer ≥1:640

The sensitivity and specificity of a CSF-TPPA titer ≥1:640 for the two diagnoses of neurosyphilis were determined in 467 individuals, each with a single syphilis episode (Table 3). Specificity was high, ranging from 94.6–96.0%, but sensitivity was very low (12.8–17.4%). There were no significant differences in sensitivity and specificity between participants without HIV and participants who were PLWH.

Table 3.

Sensitivity and Specificity of CSF-TPPA ≥1:640 for Neurosyphilis Diagnosis

	Sensitivity Proportion			Specificity Proportion
	All	No HIV	PLWH	All	No HIV	PLWH
Asymptomatic meningitis	17.4% 12/69	19.0% 4/21	16.7% 8/48	96.0% 382/398	94.7% 72/76	96.3% 310/322
Symptomatic meningitis	12.8% 5/39	16.7% 2/12	11.1% 3/27	94.6% 405/428	92.9% 79/85	95.0% 326/343

CSF-CXCL13 concentration

Cerebrospinal fluid-CXCL13 concentrations were available for 377 individuals with 379 episodes of syphilis. We derived receiver operating characteristic curves for CSF-CXCL13 for our two neurosyphilis definitions, and we used Youden’s index to derive the best cut-offs, which were 66.7 pg/ml for asymptomatic syphilitic meningitis, and 184.2 pg/ml for symptomatic syphilitic meningitis. The sensitivity and specificity of these thresholds for neurosyphilis diagnosis is shown in Table 4. Too few individuals without HIV are included in the analysis (29, 7 with asymptomatic meningitis and 3 with symptomatic meningitis) to allow us to evaluate test performance by HIV status. While specificity was lower than for the other tests we studied (76.3–81.1%), sensitivity was higher (77.8–82.8%).

Table 4.

Sensitivity and Specificity of CSF-CXCL13 Thresholds for Neurosyphilis Diagnosis

	Sensitivity Proportion	Specificity Proportion
Asymptomatic meningitis, threshold ≥66.7 pg/ml	77.8% 49/63	76.3% 241/316
Symptomatic meningitis, threshold ≥182.4 pg/ml	82.8% 24/29	81.1% 284/350

Is our definition of asymptomatic neurosyphilis too imprecise?

We previously defined asymptomatic meningitis as no neurological symptoms and CSF WBCs >10/ul. To address whether this definition might be too imprecise, we assessed the performance of the CSF TPPA titer ≥1:640 and the CSF CXCL13 cut offs for diagnosis of asymptomatic neurosyphilis using a more rigorous laboratory definition as the gold standard: reactive CSF-VDRL or detection of T. pallidum in CSF by RT-PCR. We included all individuals except those with symptomatic neurosyphilis in the analysis. Using the revised gold standard, the sensitivity and specificity of CSF TPPA ≥1:640 did not differ (97.5% [344/353] and 18.7%, [14/75] compared to the previous 96.0% and 17.4%). Similarly, the sensitivity of the CSF CXCL13 cut off for asymptomatic neurosyphilis did not differ (72.6% [53/73] vs. 77.8%) but the specificity was slightly higher (83.3% [230/276] vs. 76.3%, p=0.04). These data support the use of our original gold standard for diagnosis of asymptomatic neurosyphilis.

Which, if any, alternative tests add value beyond the CSF-VDRL?

To address whether any of the alternative tests examined above could add clinical value to the CSF-VDRL test, the analysis was restricted to samples from individuals with asymptomatic or symptomatic meningitis (using our original definitions), but non-reactive CSF-VDRL. Figure 1 shows the proportion of these samples that were above the thresholds for CSF-TPPA or CSF-CXCL13 or that were positive by RT-PCR. CSF-CXCL13 concentrations were above the set thresholds in 75.0% of samples from individuals with asymptomatic and 68.8% of individuals with symptomatic neurosyphilis. These percentages were 16.1% and 35.7% for CSF-RT-PCR and 4.7% and 4.8% for CSF-TPPA.

Figure 1.

Results of three CSF measures in samples from individuals with asymptomatic or symptomatic syphilitic meningitis whose CSF-VDRL was nonreactive. The stacked bars show the number of CSF samples at or above set thresholds (gray) vs. below the set thresholds (black) for CSF-TPPA titer or CSF-CXCL13, and in which T. pallidum rRNA was (gray) vs. was not (black) detected by RT-PCR.

Discussion

The “gold standard” for neurosyphilis diagnosis has been the CSF-VDRL, which, while specific, is not sensitive. In this study, we used two definitions for neurosyphilis that did not require CSF-VDRL reactivity, and we confirmed its poor sensitivity, regardless of HIV status. We also demonstrated that detection of T. pallidum rRNA in CSF and identification of high-titer CSF TPPA antibody had high specificity, again regardless of HIV status, but even lower sensitivity than the CSF-VDRL.

Marks and colleagues (7) reviewed published literature on molecular detection of T. pallidum in CSF for neurosyphilis diagnosis. These studies included far fewer samples than in our analysis, and used variable neurosyphilis definitions, most of which included CSF-VDRL reactivity. Sensitivity varied from 40–100% and specificity from 61–100%. The authors concluded that low power and variable gold standards limited conclusions, and that multicenter studies are necessary to clarify the role of molecular tests for neurosyphilis diagnosis. Our results in a large sample, including our finding that an additional 16–36% of neurosyphilis diagnoses could be confirmed by RT-PCR in instances in which the CSF-VDRL was nonreactive, suggest that molecular diagnosis may indeed have a future role, but will still suffer from low sensitivity.

We previously reported that the CSF-TPPA was sensitive and the CSF-TPPA with a threshold of ≥1:640 was specific for laboratory and clinically defined neurosyphilis (5); previous work had suggested that this threshold for the CSF-Treponema pallidum hemagglutination (TPHA) test was both sensitive and specific for neurosyphilis diagnosis (8). In our current analysis we confirm the high specificity of the CSF-TPPA threshold using different neurosyphilis diagnoses, but we did not show that this test offered much benefit beyond the CSF-VDRL; only 5% of neurosyphilis diagnoses could be confirmed in instances in which the CSF-VDRL was nonreactive.

In contrast to detection of T. pallidum or high titer TPPA antibody in CSF, we were able to derive thresholds for CSF CXCL13 concentrations that had higher sensitivity, although these would still not be considered good (defined as ≥90%). Notably, our cut-offs were different for asymptomatic and symptomatic meningitis. Several studies have shown that CSF CXCL13 concentrations are higher in individuals (both without and with concomitant HIV) who have asymptomatic and symptomatic neurosyphilis than in controls with syphilis but without neurosyphilis (4, 9–14). While these studies used the same commercial assay that we used, the criteria for neurosyphilis diagnosis varied, and, when reported, diagnostic thresholds differed.

Limitations of our work should be acknowledged. While our analysis of CSF-VDRL and molecular detection of CSF T. pallidum included all available samples, our analyses of CSF-TPPA and CSF CXCL13 used convenience samples. Our neurosyphilis definitions did not include CSF-VDRL reactivity, but we used a low threshold for CSF WBCs, which is appropriate for individuals without HIV and PLWH who are on antiretroviral therapy (ARV). However, this low cut-off might lead to overdiagnosis of neurosyphilis in PLWH not on ARV, and lower sensitivity estimates. However, if this were the case, it would also apply to our CXCL13 estimates, which were higher than that of the other tests. Additionally, we repeated our analysis of the sensitivity and specificity of the CSF-TPPA and CSF CXCL13 thresholds for diagnosis of asymptomatic neurosyphilis using reactive CSF-VDRL or molecular detection of T. pallidum in CSF as the gold standard, and our estimates were largely unchanged. Our analysis of CSF CXCL13 in participants without HIV was limited, and we were not able to compare performance of the assay by HIV status.

Our data, in combination with that of others, suggests that CSF CXCL13 assessment holds the greatest promise for improving the diagnostic accuracy of neurosyphilis. The assay is commercially available and simple to perform. Further studies of CSF CXCL13 should include CSF samples from multiple cohorts and countries and use standard neurosyphilis definitions to establish uniform thresholds for diagnosis.