A DUAL PLATFORM POINT OF CARE TEST FOR NEUROSYPHILIS DIAGNOSIS

Abstract

Background

The diagnosis of neurosyphilis relies on cerebrospinal fluid (CSF) abnormalities (pleocytosis, elevated protein) and CSF- VDRL test. In resource-limited settings the CSF-VDRL test may not be widely available.

Methods

We optimized a commercial immunochromatographic strip test, the DPP^® Chembio syphilis assay, for performance with CSF and tested centrifuged CSF samples of 71 patients with syphilis (35 with neurosyphilis and 36 without neurosyphilis). A CSF dilution of 1:4 was chosen based on agreement with CSF pools with documented CSF-VDRL and CSF-FTA-ABS results. Using an electronic reader, we obtained unit values of treponemal and nontreponemal antibodies for all study samples and generated a receiver operating characteristic (ROC) curve; using Youden’s index, we established diagnostic cutoffs with optimal sensitivity and specificity.

Results

Diagnostic sensitivity of the nontreponemal test was 80% (95% CI 63%−92%) and specificity was 97% (95% CI 85–100%) for neurosyphilis diagnosis using a reactive CSF-VDRL that improved following neurosyphilis therapy as a gold standard.

Conclusions

In this small study, the DPP^®Chembio test showed promising results for neurosyphilis diagnosis. Further studies are needed to assess its performance in resource-limited settings.

Short summary

We optimized the DPP Chembio syphilis assay on CSF for diagnosis of neurosyphilis, and found that this point of care test has a diagnostic performance that is similar to a clinico-serological definition of neurosyphilis

INTRODUCTION

In 2016, a World Health Organization (WHO) bulletin¹ estimated the global syphilis prevalence at 19.9 million cases for adults 15–49 years of age; 6.3 million were incident cases. Low-income and lower middle income countries² and territories represented 44.9% of incident cases globally; these same regions have been disproportionately affected by the HIV epidemic.³ While in low-income and lower middle income countries, syphilis has remained endemic (1.1% prevalence in Asia to 4–6.5% across Africa⁴) it continues to persist in high-income countries, where its incidence is rising⁵ (6.1 new cases per 100,000 population in EU countries in 2016)⁶. Neurosyphilis is a complication of syphilis that can be asymptomatic or symptomatic. If symptomatic, it can lead to strokes, meningitis, cognitive impairment, paralysis, and vision or hearing loss. People living with HIV, especially men who have sex with men, are more commonly affected with neurosyphilis than those without HIV.⁷

The diagnosis of neurosyphilis relies on identifying cerebrospinal fluid (CSF) abnormalities (pleocytosis, elevated protein concentration) and a reactive CSF-nontreponemal test, ideally the Venereal Disease Research Laboratory test (VDRL). A reactive CSF-VDRL confirms the diagnosis of neurosyphilis, but a non-reactive test does not rule it out due to the suboptimal sensitivity⁸. The CSF-VDRL requires specialized glass plates and a light microscope, and takes about 25–30 minutes to perform by trained personnel.⁸ CSF treponemal tests such as the Treponema pallidum particle agglutination assay (TPPA) can be used to exclude a diagnosis of neurosyphilis, and some advocate using a titer ≥1:640 to establish the diagnosis of neurosyphilis when the CSF-VDRL is non-reactive.^9,10

As part of a global effort to improve access to and accuracy of syphilis diagnostic testing, several rapid treponemal immunochromatographic strip tests have been developed for syphilis diagnosis.¹¹ To date, the only immunochromatographic strip test studied for the point of care diagnosis of neurosyphilis is the Syphicheck-WB test (Qualpro Diagnostics, Goa, India). A small study showed that this test, which only tests for treponemal antibodies, had similar performance to the CSF-VDRL for the clinical and laboratory diagnosis of neurosyphilis if tested undiluted and diluted.¹²

The DPP^® Syphilis Screen & Confirm test (Chembio, Medford, NY), introduced in 2010, is an immunochromatographic strip test that simultaneously detects treponemal and nontreponemal IgG and IgM antibodies from blood and yields results in 15–20 minutes.^13,14 Several publications have reported good test performance of the Chembio test on blood for the diagnosis of latent syphilis^14–16. We sought to test whether the Chembio test could be used on CSF to diagnose neurosyphilis in a cohort of patients with confirmed neurosyphilis when compared to controls with syphilis but not neurosyphilis.

MATERIALS AND METHODS

Clinical Methods

Individual CSF specimens were selected for testing from a study of CSF abnormalities in syphilis conducted in Seattle, WA, USA¹⁷. Thirty-five patients with neurosyphilis (cases) and thirty-six patients with syphilis but without neurosyphilis (controls) (n=71) were selected. Confirmed neurosyphilis cases (n=35) had a reactive CSF-VDRL with or without CSF pleocytosis and their CSF abnormalities normalized after CDC-recommended neurosyphilis therapy.¹⁸ Controls had syphilis but had ≤5 white blood cells per microliter CSF (WBC/μL) and a nonreactive CSF-VDRL and were not treated for neurosyphilis. The two groups were matched by age (+/− 5 years), syphilis stage and rapid plasma reagin (RPR) titer (≤1 dilution factor difference); HIV-infected participants were matched by CD4⁺ T-cell count (≤ 50 cells/μL difference) measured within 90 days of the the date of the lumbar puncture (table 1).

Table 1.

Participant characteristics

Characteristic	Neurosyphilis Cases (n=35)	Syphilis Controls (n=36)	P-values
Men	35 (100%)	36 (100%)	N/A
Age	39 (33–47)	39 (31–44)	0.203
PLWH	25 (72%)	27 (75%)	0.943
Syphilis stage
Primary	1 (3%)	1 (3%)	0.834
Secondary	16 (46%)	15 (42%)
Early latent	9 (26%)	13 (36%)
Late latent	9 (26%)	7 (19%)
Serum RPR titer	1:256 (1:64–1:512)	1:128 (1:64–1:256)	0.140
CSF WBC/μl	67 (42–131)	2 (2–4)	<0.001
CSF-VDRL titer	2 (2–8)	0 (0–0)	<0.001
CSF-FTA-ABS reactive	32 (100%)a	8 (44%)a	<0.001
T pallidum detected in CSF by RT-PCR	20 (57%)	3 (8%)	<0.001

Results are expressed as number (n), percent (%) or median (interquartile range)

^aCSF-FTA-ABS results were available for 32 case samples and 18 control samples

^*CSF=cerebrospinal fluid; FTA-ABS=fluorescent treponemal antibody absorption; PLWH=people living with HIV; RPR=rapid plasma reagin; RT-PCR, reverse transcriptase polymerase chain reaction; VDRL=venereal diseases research laboratory test; WBC=white blood cells

Standard Laboratory Methods

Determination of plasma HIV RNA concentrations, CD4⁺ T-cell count, CSF-VDRL, and enumeration of CSF WBC were performed in a Clinical Laboratory Improvement Amendments (CLIA)-approved hospital clinical laboratory (supplemental materials). Serum RPR tests were performed in a single research laboratory according to standard methods.¹⁹ RT-PCR for T pallidum were performed according to previously published methods.²⁰

Optimization of DPP^® Chembio Syphilis Screen & Confirm test for CSF

Chembio kits were purchased from the manufacturer. Based on a previous study,¹² we hypothesized that the conditions for the test in blood or plasma may not be optimal for CSF. As such, the total volumes of CSF and running buffer were modified while maintaining the total volume consistent with the manufacturers’ recommendations. With each Chembio kit, a proprietary disposable dropper for samples and buffer is provided; the microliter equivalents to the drop amount are also provided in the kit manual. Standard procedure per the manufacturer’s protocol consists of adding 10 μL of sample (blood, plasma) into the sample well, followed by 50 μL of running buffer into the same well. After 5 minutes, 135 μL of running buffer are added to the buffer well. Results are read after 15 minutes using an electronic reflectance reading device (DPP^® Micro Reader). The electronic reader captures an image of the test strip surface, verifies the presence and intensity of the control line, and measures the line intensity for each of the test lines (treponemal and nontreponemal); results are interpreted using a proprietary algorithm and are displayed in a liquid crystal display (LCD) screen after approximately 3 seconds. The value displayed is equivalent to the reflectance of the test line, which in turn is proportional to the amount of bound antibody. Three CSF reference pools (CSF-VDRL nonreactive and CSF-FTA-ABS nonreactive, pool 1; CSF-VDRL nonreactive and CSF-FTA-ABS reactive, pool 2; CSF-VDRL reactive and CSF-FTA-ABS reactive, pool 3); were tested once each to determine the optimal CSF dilution (supplemental methods; supplemental table 1). Dilution of CSF 1:4 in sample buffer yielded 100% qualitative agreement with the sample pool results for the nontreponemal and treponemal tests based on the cutoffs used for blood (supplemental table 2); dilution was necessary to eliminate false positive treponemal test results. Subsequent testing of case and control samples used this dilution. A single operator read the test results for the optimization samples and for the case and controls samples using the electronic reader and was blinded to the source of the samples.

Statistical Methods

Categorical variables are expressed as number and proportions, and continuous variables as means (standard deviation) or medians (interquartile ranges [IQR]). Proportions were compared using the χ² test or Fisher’s exact test when values were small. Independent group t tests were used for the comparison of means for continuous variables that were normally distributed; Mann-Whitney U test was used for continuous variables not normally distributed. A receiver operator characteristic (ROC) curve was derived from the results of the case and control samples (figure 1) using reactive CSF-VDRL as the gold standard¹⁹. Using Youden’s index, which captures in a single statistic the optimal performance of a dichotomous diagnostic test, a cutoff of ≥89 units for the treponemal and ≥7.7 units for the nontreponemal test was determined. In comparison, a result of ≥9 units for either test is considered reactive per the manufacturer’s manual. Sensitivity and specificity were calculated using standard formulae and expressed as percent. Kappa was calculated to measure the agreement between different testing methods. Two-sided P-values of less than 0.05 were considered statistically significant. SPSS version 20.0 (IBM Company, Armonk, NY, USA) was used to perform all statistical analyses.

Figure 1. Receiver operator characteristic curves.

A. Receiver operator characteristic curve of treponemal test results of Chembio test (area under the curve is 0.894). B. Receiver operator characteristic curve of non-treponemal test results of Chembio test (area under the curve is 0.910). The diagnosis of neurosyphilis was based on a reactive CSF-VDRL that normalized after neurosyphilis therapy

RESULTS

Chembio kit for neurosyphilis diagnosis

Participants were predominantly HIV-infected (n=52, 73%) men (n=71, 100%). Using the derived cutoffs, we detected CSF treponemal antibodies in 30/35 cases and 5/36 controls, and CSF nontreponemal antibodies in 28/35 cases and 1/36 controls. This resulted in a sensitivity of 86% (95% CI 70%−95%) and specificity of 86% (95% CI 71%−95%) for the treponemal test and sensitivity of 80% (95% CI 63%−92%) and specificity of 97% (95% CI 85–100%) for the nontreponemal test for neurosyphilis diagnosis. The kappa value between the CSF treponemal and nontreponemal test was a 0.60, which is indicative of a moderate level of agreement between the tests. The kappa value of the treponemal test and the nontreponemal test as compared to CSF-VDRL was 0.71 and 0.77 respectively, representing a moderate and strong-to-moderate level of agreement for each.

False negative treponemal and nontreponemal test results were more common in individuals with lower serum RPR and CSF-VDRL titers and lower WBC (Table 2). CSF-VDRL titers were significantly lower in individuals with false negative nontreponemal Chembio test results.

Table 2.

False negatives and true positives of the DPP test for the diagnosis of neurosyphilis

	Treponemal test			Nontreponemal test
	False negatives 5/35 (14%)	True positives 30/35 (86%)	P-values	False negatives 7/35 (20%)	True positives 28/35 (80%)	P-values
Median serum RPR titer (IQR)	1:128(1:128–1:512)	1:256(1:64–1:1024)	0.810	1:128(1:64–1:256)	1:256(1:128–1:512)	0.154
Median CSF-VDRL (IQR)	2(1–4)	4(2–8)	0.151	2(1–2)	4(2–8)	0.005
Median CSF WBC (IQR)	58(28–213)	69(44–138)	0.604	31(29–349)	72(49–124)	0.522

Results are expressed as number (n), percent (%) or median (interquartile range)

^*CSF=cerebrospinal fluid; RPR=rapid plasma reagin; VDRL=venereal diseases research laboratory test; WBC=white blood cells

DISCUSSION

The CSF-VDRL remains the gold standard for the diagnosis of neurosyphilis and its availability may be limited in resource limited settings. A previous study examined the use of a treponemal immunochromatographic strip test, the CSF-Syphicheck, at the point of care for diagnosing neurosyphilis. The diagnostic sensitivity of a reactive test and the diagnostic specificity of a titer ≥ 1:4 was equivalent to published estimates for the CSF-VDRL.¹² However, limitations of the study included the use of uncentrifuged CSF, which worked less well than cell-free CSF, and that the results obtained at the point of care were less accurate than results obtained in a research laboratory.

The Chembio kit was originally developed for use with whole blood or plasma. We optimized the method by diluting the CSF to limit false positive treponemal results, and developed cutoffs specific to CSF. While the sensitivity and specificity of the treponemal component was somewhat lower than that reported for other CSF treponemal tests²¹, the nontreponemal component showed moderate to strong agreement with the CSF-VDRL. False negative results for both tests were more common with lower serum RPR, lower CSF-VDRL titer and fewer CSF WBCs; this may be a consequence of the CSF dilution. A major strength of our study is the use of an electronic reader, which removed subjectivity and allowed for the calculation of new cutoffs for CSF. Additionally, our definition of neurosyphilis as a reactive CSF-VDRL that improved after standard neurosyphilis therapy is both pragmatic and of high diagnostic certainty. Furthermore, our control group had syphilis, which avoids falsely elevated specificity. Limitations of our study are acknowledged. The FTA-ABS reactivity of Pool 2 (CSF-VDRL nonreactive, CSF-FTA-ABS reactive) was lower than expected based on its components. As such, our dilution factor for CSF for the Chembio kit could have been underestimated. However, the chosen sample dilution was the same for Pool 1 (nonreactive CSF-VDRL and nonreactive CSF-FTA-ABS), arguing against this error. Our sample is small, and we tested cell free CSF, which may not be possible at the point of care. As such, our study needs to be performed in a larger number of samples and in field conditions, including without centrifugation of CSF, before it can be widely applied.

This study constitutes an initial step towards the repurposing of syphilis point-of-care tests for diagnosis of neurosyphilis, and our methodology provides a useful template for future studies in resource limited settings. The possibility of developing an accurate and reliable handheld test that can be applied to the bedside without need for refrigeration could significantly improve the diagnosis of neurosyphilis in the developing world.