The Application of Cytokine Expression Assays to Differentiate Active From Previously Treated Syphilis

Abstract

To investigate the role of serum cytokine assays to distinguish between active from treated syphilis among serofast patients, we recruited individuals into a prospective cohort study. Participants underwent routine syphilis screening. We selected specimens from a majority cohort of serofast participants with treated and active syphilis. We analyzed specimens with a 62-cytokine multiplex bead-based enzyme-linked immunosorbent assay. Cytokines, brain-derived neurotrophic factor and tumor necrosis factor β, were most predictive. We built a decision tree that was 82.4% accurate, 100% (95% confidence interval, 82%–100%) sensitive, and 45% (18%–75%) specific. Our decision tree differentiated between serum specimens from serofast participants with treated syphilis versus active syphilis.

Among a prospective cohort with a majority of serofast participants, we built a highly accurate and sensitive 2-cytokine decision tree that could differentiate between serum specimens from participants with treated versus active syphilis; however, the test had poor specificity.

Syphilis is an identifiable and curable infection caused by Treponema pallidum pallidum [1, 2]. Current diagnostics for syphilis mainly rely on serologic assays testing for reactivity to nontreponemal (cardiolipin) and treponemal antigens [3]. Treponemal assays are primarily used to test for antibodies produced due to prior or current syphilis, whereas nontreponemal assays are used to monitor the “activity” of infection [4]. Direct detection methods, such as direct fluorescent antibody testing, dark-field microscopy, nucleic acid amplification testing, and examination of cerebrospinal fluid, can also be used; however, those methods are often insensitive and difficult to access [5].

Among those treated for syphilis, 15%–41% of persons remain seropositive (based on nontreponemal serologic titer) after syphilis treatment (serofast) [6]. The optimal clinical management for those individuals is challenging because serologic assays cannot differentiate syphilis reinfection from the serofast state. The serofast phenomenon is due to the prolonged half-lives of the host antibodies (anticardiolipin), which nontreponemal assays detect [6, 7]. False-positive nontreponemal test results are common and can be due to other infections or autoimmunity [8]. Owing to limitations of serologic testing, diagnostic algorithms using nontreponemal and treponemal assays have been created to increase the sensitivity and specificity syphilis tests [4]. However, those algorithms are not always able to detect treated, early active syphilis, and late latent syphilis. owing to nonreactive nontreponemal assays. Current diagnostic algorithms are unable to distinguish previously treated syphilis from active syphilis. Studies of specific inflammatory cytokines as potential markers of infection resolution among patients with syphilis have found promising results [9].

In our prior study, we recruited and enrolled an observational cohort of 401 men who have sex with men and male-to-female transgender women who engaged in behavior that put them at risk for syphilis in Lima, Peru, from 2013 to 2015 [9–11]. Among the cohort, 42 participants had active syphilis. From the total 1113 serum samples that were collected, 101 samples with active and resolving syphilis were tested for cytokine concentrations. We identified 23 cytokines (brain-derived neurotrophic factor [BDNF]; CD40L; epidermal growth factor; epithelial cell–derived neutrophil attractant 78; eotaxin; Fas ligand; interferon β and γ; interleukin 12P40, 12P70, 1RA, 22, 27, and 6; leptin; monocyte chemoattractant protein 1; monokine inducd by interferon γ; RANTES [regulated on activation of normal T cells expressed and secreted]; stem cell factor; tumor necrosis factor β (TNF-β); TNF-related apoptosis-inducing ligand [TRAIL]; vascular cellular adhesion molecule 1; and vascular endothelial growth factor D) that were associated with changes in rapid plasma reagin (RPR) titer among participants with active syphilis [9]. In addition, the differences in serum cytokine profiles between active and treated syphilis were independent of human immunodeficiency virus (HIV) coinfection [9]. In the current study, to investigate the potential role of serum cytokines in clinical decision making among serofast individuals, we analyzed the above-mentioned cytokines among study participants with active or treated syphilis who had persistently elevated RPR titers.

METHODS

Participants attended a sexually transmitted infection clinic every 3 months for a period of 2 years in Lima, Peru. During study visits, participants were interviewed, received routine syphilis and HIV infection testing, and had clinical specimens collected. Those found to have primary and secondary syphilis were treated during the study visit with a single dose of 2.4 million units of intramuscular benzathine penicillin G.

Syphilis Testing Among Participants

Participants were tested for syphilis with a serum RPR test (BD Macro-Vue RPR Card Test Kit; Becton Dickinson) and Treponema pallidum particle agglutination (TPPA) test (Serodia; Fujirebio Diagnostics). If RPR test results were positive, titer values were measured with the highest value recorded. A cutoff value of ≥1:80 was used to categorize TPPA tests as reactive. Participants were also tested for HIV infection (Alere Determine HIV 1/2 [Alere] and NEW LAV BLOT I [Bio-Rad]).

Selection of Samples for Cytokine Analysis

Active syphilis was defined as an RPR titer ≥1:8 among syphilis-naive participants or a 4-fold increase from the prior RPR titer, and a positive TPPA result, among syphilis-exposed participants. Serofast syphilis was defined as a <4-fold decline in RPR titer 6 months after treatment [7]. Treated syphilis was defined as a ≥4-fold reduction in RPR titer within 6 months after syphilis treatment. A longitudinal series of samples were selected from participants with active, treated, or serofast syphilis [9]. Sample analysis using a multiplex bead-based enzyme-linked immunosorbent assay (62 cytokines and chemokines) (eBioscience/Affymetrix) with Assya Chex control beads (Radix Biosolutions) was conducted at the Human Immune Monitoring Center at Stanford University, using a Luminex FlexMap3D instrument (Luminex Corporation) [9]. Individual cytokines were measured as the average median fluorescence intensity (MFI) in each sample [9]; MFI is a measure of quantification of cytokine levels on Luminex instrument.

Statistical Methods

In the current study, we identified serum samples from participants with treated or active syphilis, both with moderate-range RPR titers (1:8 or 1:16). We defined 3 equal frequency tertiles or “bins” (low, medium, and high), using all serum samples for each of the 23 cytokines identified in our prior study. This established low, medium, and high levels for each cytokine in the context of a larger group of samples with a wide range of RPR titers. Next, we used a random forest method to select the most predictive cytokines to differentiate samples from participants with active syphilis from those without syphilis (R package; v.surf) [12]. To classify the samples using the most predictive variables, we built a decision tree using the R package; rpart. To better calibrate the accuracy of this decision tree, we performed 2 permutation tests; the first test considered trees built with all other 252 possible combinations of 2 cytokines, and the second considered 10 000 trees built using random permutations of the class labels (treated and active).

Ethical Considerations and Availability of Data

The institutional review board at Universidad Peruana Cayetano Heredia and Alberto Barton Health Center, in Lima, Peru, provided ethical approval and oversight of the study (reference no. SIDISI 59996). The University of California Los Angeles institutional review board determined that analysis of deidentified data was exempt from human subjects considerations. The data sets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

RESULTS

Among the collected serum samples from participants with active or treated syphilis, 34 samples had RPR titers of 1:8 or 1:16. Of those samples, 11 were from participants with persistently elevated RPR titers after treatment with penicillin, and 23 were from participants with active syphilis, defined as a 4-fold increase in the RPR titer with a positive TPPA test results.

The random forest method identified that compared to all other studied cytokines, BDNF and TNF-B could best distinguish between groups. A 2-cytokine decision tree using BDNF and TNF-β was created. The cytokine decision tree correctly identified 5 of 11 samples with treated syphilis, and 23 of 23 with active syphilis (Figure 1A). The accuracy of the decision tree was 82.4%, with a sensitivity of 100% (95% confidence interval, 82%–100%) and a specificity of 45% (18%–75%).

Figure 1.

A, Decision tree for differentiation between incident and treated syphilis. Leaf nodes are shown as bars, with light gray indicating the proportion of samples corresponding to active infection and dark gray indicating treated infections. The number of samples reaching each leaf node is indicated in parentheses above the node. The confusion matrix is shown in the upper right corner. B, Observed values of tumor necrosis factor β (TNF-β) and brain-derived neurotrophic factor (BDNF), measured as average median fluorescence intensity (MFI) with boundaries. Each point represents a single sample. Gray lines indicate the boundaries between tertile bins, and colors distinguish the different leaf nodes. Circles represent active infection, and squares, treated infection..

The accuracy of the BDNF–TNF-β tree was significantly better than that of trees built with all other combinations of 2 cytokines (P = .02). The accuracy of the original tree was higher than that of 9474 of 10 000 BDNF–TNF-β trees built with random permutations of the class labels (treated and active; P = .03). Figure 1B shows the observed values of TNF-β and BDNF measured in average MFI for each sample. The ranges of observed average MFI values for BDNF bins were as 886–8251.5 for low, 8259.5–11 204 for medium, and 11 349–16 038 for high; and the respective ranges for TNF-β bins were 95–164, 166–209, and 210–691.

Discussion

This study explored the potential clinical utility of cytokine assays in the differentiation between patients with previously treated syphilis and those with active syphilis, with moderate RPR titers. Using samples from participants with treated or active syphilis, 2 candidate cytokines were identified as having predictive value. The BDNF–TNF-β decision tree correctly identified samples from patients with active syphilis, independent of HIV coinfection [9], among our cohort that had a majority of serofast patients. The decision tree had high sensitivity and low-moderate specificity, suggesting that a cytokine testing method could potentially be used to guide the clinical management of serofast patients. Two different permutation approaches provide additional evidence for the potential value of cytokine assays in clinical management.

The decision tree correctly identified all cases of active syphilis. The decision tree had a high number of false-positive results (n = 6) that were classified from the first 2 nodes. Notably, all samples that were identified as treated with the final TNF-β node were correctly identified. Although samples identified as treated with the stepwise decision tree had many false-positives, there were no false-negative results in our cohort. Clinically, this suggests that if an individual has BDNF and TNF-β values in the middle tertile (ie, average MFI of 8259.5–11 204 for BDNF and of 166–209 for TNF-β), the participant does not have active syphilis.

We were unable to find similar studies for a comparison of our findings. A study conducted among a small cohort found that recurrent syphilis presented with an attenuated immune response [13]. Specifically, those authors found lower concentrations of interferon α and chemokine ligand 4 among persons with repeated syphilis, but their study did not measure BDNF or TNF-β. Our prior study conducted among the same cohort also identified differences in concentrations of BDNF and TNF-β between samples from participants with active versus treated syphilis [9].

The TNF superfamily of cytokines has been characterized to regulate inflammation and cell death [14]. This superfamily also plays key roles in innate and adaptive immunity. Macrophages are a large physiologic source of TNF. It has also been suggested that macrophages can be induced by TNF to express inflammatory cytokines. Historically, BDNF is characterized as a member of the neurotrophic family of cytokines [15]. However, more recently BDNF has been shown to regulate expression in the host inflammatory response. Proinflammatory factors, which include endotoxin and oxidative stress, have been reported from in vitro experiments to up-regulate BDNF in immune cells. In our findings, treated individuals had midrange values for both BDNF and TNF-β, which may suggest that midrange values correspond to a normal homeostasis between inflammatory cytokines, and movement away from the midrange may indicate different patterns of response to active syphilis for different people. This is aligned with results we presented previously, which illustrated different patterns of cytokine response to active syphilis [9].

Strengths of the current study included the longitudinal series of samples, which allowed us to use neighboring samples to characterize those with titers of 1:8 or 1:16 as active, treated, and serofast. Our prior study done in the same cohort found that there was no difference in cytokine expression between participants with HIV infection and without HIV infection [9]. In addition, we compared cytokine data from our study cohort with data from 2 other study cohorts to assess for intercohort variation and found that we had signal validity for our cytokines. One weakness of the current study is the small number of serofast samples. Our study was conducted in a single population among a small cohort. The findings will need to be repeated in larger heterogeneous cohorts to validate our results. In addition, further studies need to be conducted to better define cytokine concentration thresholds for BDNF and TNF-β.

There are barriers to clinical implementation of cytokine assays in clinical settings. Currently, many cytokine assays are expensive, manufacturer dependent, and technically difficult to perform. Furthermore, the platforms that are needed to run those assays are not commonly available in nonacademic clinical settings. Although those barriers can be overcome, clinical performance of cytokine assays needs to be proved, assay performance needs to be standardized, and assay costs need to be reduced.

In conclusion, samples from participants with treated syphilis versus active syphilis could be differentiated using 2 cytokines, BDNF and TNF-β. The stepwise BDNF–TNF-β decision tree had high sensitivity and low-moderate specificity in differentiating active from treated syphilis. Our findings show that cytokine assays could possibly be a clinically useful tool to guide management of serofast individuals. Our results need to be replicated in larger and heterogeneous cohorts.

Notes

Acknowledgments. The authors acknowledge the staff and patients at Epicentro and the Barton Health Center for hosting and participating in the study in Lima, Peru.

Author contributions. Project conception: C. F. C. and J. D. K. Project design: S. R. L., S. K. V., K. A. K., C. F. C., and J. D. K. Data acquisition: H. M., Y. R. H., S. R. L., S. K. V., and K. A. K.. Data analysis: J. C. S. Interpretation of analysis: N. K., J. C. S., H. M., and J. D. K. Drafting of manuscript: N. K. Revision of drafted manuscript: all authors.

Disclaimer. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health (NIH) or the University of California Global Health Institute.

Financial support. This study was funded by the National Institute of Allergy and Infectious Diseases (NIAID) (grants 1R01AI099727 and 5R01AI139265-02; grant AI028697 to J. D. K., through the UCLA Center for AIDS Research), the NIH (grant P30MH058107 to J. D. K., through the Center for HIV Identification, Prevention, and Treatment Services); the Fogarty International Center, NIH (grant D43TW009343 to N. K.); and the University of California Global Health Institute (N. K.).

Potential conflicts of interest. All authors: No reported conflicts. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.