A Dual Point-of-Care Test Shows Good Performance in Simultaneously Detecting Nontreponemal and Treponemal Antibodies in Patients With Syphilis: A Multisite Evaluation Study in China

A multisite evaluation of a dual point-of-care syphilis test with 3 types of specimens (N = 3134) in China indicates good overall sensitivity and specificity: 95%–97% and 99%–100% in detecting treponemal antibodies, and 86%–88% and 94%–96% in detecting nontreponemal antibodies.

Abstract

Background. Rapid point-of-care (POC) syphilis tests based on simultaneous detection of treponemal and nontreponemal antibodies (dual POC tests) offer the opportunity to increase coverage of syphilis screening and treatment. This study aimed to conduct a multisite performance evaluation of a dual POC syphilis test in China.

Methods. Participants were recruited from patients at sexually transmitted infection clinics and high-risk groups in outreach settings in 6 sites in China. Three kinds of specimens (whole blood [WB], fingerprick blood [FB], and blood plasma [BP]) were used for evaluating sensitivity and specificity of the Dual Path Platform (DPP) Syphilis Screen and Confirm test using its treponemal and nontreponemal lines to compare Treponema pallidum particle agglutination (TPPA) assay and toluidine red unheated serum test (TRUST) as reference standards.

Results. A total of 3134 specimens (WB 1323, FB 488, and BP 1323) from 1323 individuals were collected. The sensitivities as compared with TPPA were 96.7% for WB, 96.4% for FB, and 94.6% for BP, and the specificities were 99.3%, 99.1%, and 99.6%, respectively. The sensitivities as compared with TRUST were 87.2% for WB, 85.8% for FB, and 88.4% for BP, and the specificities were 94.4%, 96.1%, and 95.0%, respectively. For specimens with a TRUST titer of 1:4 or higher, the sensitivities were 100.0% for WB, 97.8% for FB, and 99.6% for BP.

Conclusions. DPP test shows good sensitivity and specificity in detecting treponemal and nontreponemal antibodies in 3 kinds of specimens. It is hoped that this assay can be considered as an alternative in the diagnosis of syphilis, particularly in resource-limited areas.

Syphilis is a chronic infectious disease caused by the spirochaete Treponema pallidum. Syphilis is usually transmitted through sexual contact or from mother to fetus or newborn infant. The World Health Organization (WHO) estimated that 11 million new cases of syphilis occurred in adults in 2005, the majority of them in developing countries [1]. Traditionally, the laboratory diagnosis of syphilis is based on a sequence of screening the serum with a nontreponemal test such as rapid plasma reagin, toluidine red unheated serum test (TRUST), or Venereal Disease Research Laboratory test, followed by a treponemal-specific confirmatory test such as T. pallidum particle agglutination (TPPA) assay, fluorescent treponemal antibody absorption test, or enzyme immunoassay [2, 3]. However, the equipment and personnel requirements, such as refrigeration for storing reagents, centrifugation for separating serum, electric rotator for operating flocculation test, and experienced laboratory staff for conducting and interpreting the tests, are often barriers to conducting the tests in low-resource settings in developing countries. In addition, the time required to report the testing results is another concern, usually resulting in a delay in diagnosis of the infection and, consequently, missed opportunity for treatment and interventions, including partner notification.

In recent years, a number of simple, point-of-care (POC) rapid treponemal tests have become commercially available [4–6], and encouraging results from evaluation studies on some of the tests provide potential solutions to the above limitations [5, 6]. These POC tests have been successfully adopted by many countries into their prenatal screening programs for detecting syphilis during pregnancy and preventing its adverse outcomes [7]. However, the current POC tests to detect treponemal antibody are unable to differentiate between currently active and historically infected (or treated) infection because this treponemal specific antibody may remain detectable for life [6, 8], which limits the application of these tests to guide treatment in persons from high-risk groups, who usually have a high lifetime exposure to syphilis. Facing this limitation, an innovative dual treponemal/nontreponemal POC (dual POC) test for the simultaneous detection of reagin and treponemal antibodies (DPP Syphilis Screen & Confirm) has been developed and evaluated with archived serum samples at one laboratory in the United States, indicating the encouraging results [9].

The main objective of this study was to evaluate, in multiple sites in China, the performance of the DPP Syphilis Screen & Confirm test in 3 types of specimens using the TPPA test and TRUST as reference standards.

MATERIALS AND METHODS

Study Sites and Population

Based on several parameters, including local commitment to the study, collection of enough sample size for evaluation, and proficiency in quality control, a total of 6 study sites in China (Nanchang, Nanjing, Guangzhou, Jiangmen, Chengdu, and Yinchun) participated in the evaluation study. This was a prospective study to estimate sensitivity and specificity of an evaluated test. Recruitment of eligible participants took place between June and September 2011. All patients attending the clinics and subjects at outreach settings (such as sex work venues and gay bars) were assessed for eligibility for inclusion in the study. Exclusion criteria included use of antibiotics in the past week, age <16 years, and previous participation. In order to have more positive TPPA and/or TRUST specimens for evaluation, the subjects deemed at higher risk for syphilis were oversampled. After witnessed verbal informed consent was obtained, participants were given a unique code to participate anonymously. Brief information about demographic and behavioral characteristics was collected using a structured questionnaire.

Test for Evaluation

The test for evaluation is the Dual Path Platform (DPP) Syphilis Screen & Confirm test, developed and manufactured by Chembio Diagnostic Systems, Inc (Medford, New York). The test is based on Chembio's patented DPP technology. The DPP incorporates an immunochromatographic device containing 2 nitrocellulose membrane strips perpendicular to each other in a T formation. Details about the principle of the dual POC test have been described elsewhere [9]. The evaluation of the treponemal line (T1) was conducted by using the T1 positivity identified by naked eye to compare with that of TPPA while the evaluation of the nontreponemal line (T2) was conducted by using the T2 positivity identified by naked eye or automatic reader at a cutoff value of ≥30, which was proposed by the manufacturer, to compare with the results of TRUST.

Specimen Collection and Testing Procedures

Based on the expected sensitivity and specificity of 95% [9], in which a precision of 4% was used, and the estimated TPPA and TRUST prevalence of 25% in the study population, a sample size of at least 461 subjects would allow a reasonable estimate of the overall sensitivity and specificity in our study. Almost all the subjects who were invited to participate in the study provided venous blood specimens, but 33.4% of them in clinic settings and 51.4% in outreach settings agreed to provide fingerprick blood as well. A venous or fingerprick blood sample was collected according to normal practices; when fingerprick blood was collected, the first drop of blood was wiped away after pricking the finger and the second drop was collected. The testing procedures were based on the manufacturers’ instructions for fingerprick whole blood and venipuncture whole blood specimens. In brief, a laboratory technician collected the second drop of fingerprick blood with a disposable sample loop or a 5-μL lab pipette, transferred the blood from the loop or pipette into the center of the well 1 of the device, added 2 drops of running buffer to well 1, added 4 drops of running buffer to well 2 after 5 minutes, and read the test result after 10–15 minutes. For venous whole blood or plasma specimens, we directly added 5 μL of specimen into well 1, followed by the same procedures as those for the above fingerprick blood specimens. If there was no control line after 10 minutes from addition of running buffer, the test was considered invalid. One line in the control area with no lines in the T1 or T2 test areas was interpreted as negative for treponemal or nontreponemal antibodies. One line in the control area with a line in the T1 and/or T2 areas was interpreted as positive for treponemal and/or nontreponemal antibodies. The laboratory technician who conducted the test and read the testing results was blinded to the results determined by other assays or based on other specimens.

Data Analysis

The sensitivity and specificity of the dual POC test and their 95% confidence intervals (CIs) were calculated by comparing T1 read by naked eye, and T2 read by naked eye and automatic reader, respectively, with each other against the following reference standards: (1) TPPA (Fujirebio Inc, Tokyo, Japan) on plasma for evaluation of T1; (2) TRUST (Rongsheng Biotech Co, Ltd, Shanghai, China) on plasma for T2; and (3) a combination of positive TPPA and high titer TRUST (titer ≥ 1:4) for T2. In view of the clinical and public health imperative to identify and treat as many cases as possible of active syphilis, a TRUST titer of ≥1:4 rather than ≥1:8 was used to evaluate sensitivity of the POC test. In order to determine whether the automatic reader could improve the performance of T2, the sensitivity and specificity determined by the reader were compared with those determined by naked eye. McNemar test was applied to compare the differences in sensitivities or specificities between subgroups. Statistical significance is reported at the 5% level.

The overall performance of T1 and T2 in the dual POC test compared with TPPA and TRUST, respectively, was evaluated with the use of receiver operating characteristic (ROC) analyses according to standard procedures [10]. Based on the ROC analysis, the best cutoff value was used to estimate sensitivity and specificity of T2 and compare them with those determined at a cutoff value of ≥30, which was proposed by the manufacturer.

Analyses were performed with SPSS software package version 19.0 (SPSS Inc, Chicago, Illinois) and MedCalc software package version 12.2.1 (MedCalc Software, Mariakerke, Belgium). The study was reviewed and approved by the Medical Ethics Committee of the Chinese Academy of Medical Sciences Institute of Dermatology and the National Center for Sexually Transmitted Disease Control in Nanjing.

RESULTS

Study Population and Specimens

In total, 1323 individuals (52% males and 48% females) were included in the study. The mean age was 35.6 years (SD, 11.7 years; range, 17–82 years). Regarding geographic distribution, 193 (14.6%) were from Chengdu, 242 (18.3%) from Guangzhou, 156 (11.8%) from Jiangmen, 140 (10.6%) from Nanchang, 395 (29.9%) from Nanjing, and 197 (14.9%) from Yinchun. Approximately 80% (n = 1070) of patients were recruited from the clinics and another 20% (n = 253) from the outreach settings. One hundred forty-eight (11.2%) women reported having provided commercial sex while 54 (4.1%) men admitted to homosexual behaviors.

Among the participants, 1323 plasma, 1323 venous whole blood, and 488 fingerprick blood specimens were collected and used for evaluation.

Performance of Dual POC Test

Of the 1323 plasma samples tested, 498 (37.6%) were reactive in both the TRUST and TPPA assays, while 146 (11.0%) were reactive in the TPPA assay alone and 3 (0.2%) in the TRUST assay alone. The remaining 676 (51.1%) were nonreactive in both the TRUST and TPPA assays. A comparison of the T1 and T2 results determined that in the dual POC test with TPPA and TRUST results, the overall sensitivities were 94.6% (95% CI, 92.5%–96.1%) for plasma, 96.7% (95% CI, 95.1%–97.9%) for venous whole blood, and 96.4% (95% CI, 93.5%–98.0%) for fingerprick blood for detection of treponemal antibodies; their corresponding specificities were 99.6% (95% CI, 98.7%–99.9%), 99.3% (95% CI, 98.3%–99.7%), and 99.1% (95% CI, 96.6%–99.7%). Based on the comparisons of T2 in the dual POC test with TRUST, the overall sensitivities and specificities were 88.4% (95% CI, 85.3%–90.9%) and 95.0% (95% CI, 93.3%–96.3%), respectively, for plasma; 87.2% (95% CI, 84.0%–89.9%) and 94.4% (95% CI, 92.6%–95.8%), respectively, for venous whole blood; and 85.5% (95% CI, 80.4%–89.4%) and 96.1% (95% CI, 92.9%–97.9%), respectively, for fingerprick blood. The 3 types of specimens had similar sensitivities and specificities for detecting treponemal and nontreponemal antibodies except that sensitivity of fingerprick blood to detect nontreponemal antibody was lower than that in plasma or venous whole blood specimens in clinic settings (Table 1).

Table 1.

Sensitivity and Specificity of Dual Path Platform (DPP) Treponemal Line as Compared With Treponema pallidum Particle Agglutination Assay, and DPP Nontreponemal Line as Compared With Toluidine Red Unheated Serum Test Determined by Naked Eye

Setting by Specimen	Sample Size	DPP T1				DPP T2
		Sensitivity (%) (95% CI)	P Valuea	Specificity (%) (95% CI)	P Valuea	Sensitivity (%) (95% CI)	P Valuea	Specificity (%) (95% CI)	P Valuea
Clinic
Whole blood	1070	96.2 (94.1–97.5)	.14	99.3 (98.2–99.7)	.68	87.6 (83.9–90.5)	.82	93.8 (91.7–95.4)	.41
Fingerprick	358	94.9 (90.7–97.3)	.78	98.9 (96.0–99.7)	.52	80.9 (73.6–86.5)	.04	97.7 (94.7–99.0)	.12
Plasma	1070	94.0 (91.6–95.8)	…	99.6 (98.7–99.9)	…	88.4 (84.7–91.2)	…	94.9 (93.0–96.3)	…
Nonclinic
Whole blood	253	98.6 (95.1–99.6)	.44	99.1 (94.9–99.8)	.48	86.2 (79.0–91.2)	.71	97.7 (93.4–99.1)	.50
Fingerprick	130	99.0 (94.5–99.8)	.74	100.0 (89.0–100.0)	.50	92.5 (85.3–96.3)	.47	86.5 (72.0–94.1)	.12
Plasma	253	96.6 (92.2–98.5)	…	99.1 (94.9–99.8)	…	88.6 (81.8–93.1)	…	95.4 (90.3–97.9)	…
Total
Whole blood	1323	96.7 (95.1–97.9)	.08	99.3 (98.3–99.7)	.72	87.2 (84.0–89.9)	.63	94.4 (92.6–95.8)	.66
Fingerprick	488	96.4 (93.5–98.0)	.31	99.1 (96.6–99.7)	.74	85.5 (82.4–89.4)	.31	96.1 (92.9–97.9)	.61
Plasma	1323	94.6 (92.5–96.1)	…	99.6 (98.7–99.9)	…	88.4 (85.3–90.9)	…	95.0 (93.3–96.3)	…

Abbreviations: CI, confidence interval; DPP, Dual Path Platform; T1, treponemal line; T2, nontreponemal line.

^a Comparison with results from plasma specimens.

Among specimens with a positive TPPA and a TRUST titer ≥ 1:4, the overall sensitivities of T2 to detect nontreponemal antibodies increased to 99.6% for plasma, 100.0% for venous whole blood, and 97.8% for fingerprick blood, and ranged from 96.3% to 100.0% in specimens collected from clinics and 100.0% from nonclinic settings. The sensitivities among specimens with higher TRUST titer (≥1:4) were significantly higher than those with lower titer (≤1:2) for all types of specimens (P ≤ .05; Table 2). Using the combination of T1/T2 results (ie, the ability of the DPP to detect treponemal and nontreponemal antibodies) against the combination of a TPPA/high-titer TRUST in venous whole blood, the sensitivity of T1/T2 was 99.6% (95% CI, 98.0%–99.9%) and specificity was 82.3% (95% CI, 79.9%–84.5%). The sensitivities of T2 were not improved by using the automatic readers as compared with the naked eye (P > .05) when the cutoff value was set as T2 ≥ 20 or ≥30 (Table 3).

Table 2.

Sensitivity of Dual Path Platform Nontreponemal Line as Compared With Toluidine Red Unheated Serum Test (TRUST) by TRUST Titer Determined by Naked Eye

Setting by Specimen	TRUST Titer ≤ 1:2a			TRUST Titer ≥ 1:4			P Valuec
	Sample Size	Sensitivity (%)	P Valueb	Sample Size	Sensitivity (%)	P Valueb
Clinic
Whole blood	160	71.3	.71	217	100.0	1.00	<.01
Fingerprick	59	59.3	.06	82	96.3	.11	<.01
Plasma	160	73.8	…	217	99.5	…	<.01
Nonclinic
Whole blood	58	70.7	.67	65	100.0	1.00	<.01
Fingerprick	37	81.1	.73	56	100.0	.47	<.01
Plasma	58	75.9	…	65	100.0	…	.05
Total
Whole blood	218	71.1	.52	282	100.0	1.00	<.01
Fingerprick	96	67.7	.29	138	97.8	.20	<.01
Plasma	218	74.3	…	282	99.6	…	<.01

Abbreviation: TRUST, toluidine red unheated serum test.

^a Excluding those negative for TRUST.

^b Comparison with results from plasma specimens.

^c Comparison between TRUST titer.

Table 3.

Comparison in Sensitivity of Dual Path Platform Nontreponemal Line With Whole Blood Between Results Determined by Naked Eye and Automatic Reader

Reading Method			Sensitivity				Specificity
	Sample Size	%	P Valuea	P Valueb	Sample Size	%	P Valuea	P Valueb
Naked eye	501	87.2	…	…	822	94.4	…	…
Reader
Criterion T2 ≥ 20	501	88.8	.50	…	822	93.9	.75	…
Criterion T2 ≥ 30	501	84.2	.21	.04	822	96.8	.02	<.01

Abbreviation: T2, nontreponemal line.

^a Comparison with results determined by naked eye.

^b Comparison between criteria T2 ≥ 20 and T2 ≥ 30.

Based on the ROC curve, the maximum area under the curve was obtained at T1 ≥ 24.5 (sensitivity 96.2% and specificity 98.8%; Figure 1) and T2 ≥ 20.5 (sensitivity 89.5% and specificity 94.4%; Figure 2).

Figure 1.

Receiver operating characteristic analysis for venous whole blood specimens positive and negative for Treponema pallidum particle agglutination (TPPA) assay shows the plot for Dual Path Platform treponemal line as compared with the TPPA assay. The cutoff point of 24.5 or above was found best to classify the positivity.

Figure 2.

Receiver operating characteristic analysis for venous whole blood specimens positive and negative for toluidine red unheated serum test (TRUST) shows the plot for Dual Path Platform nontreponemal line as compared with TRUST. The cutoff point of 20.5 or above was found best to classify the positivity.

DISCUSSION

Successful control of syphilis is mainly based on behavioral interventions and medical care. Early diagnosis and timely treatment of infected cases are key elements in the medical care. Serological testing remains the method most frequently used to establish the diagnosis. However, introduction of the traditional testing assays into syphilis screening programs is often hampered by operational and technical difficulties, particularly in developing countries and in high-risk population settings [11–13]. In many developing countries, treatment is often not provided owing to the failure of patients to return to the clinic for test results in a timely manner [14]. To provide same-day testing and treatment, a number of rapid POC tests have been developed and made commercially available in recent years [15]. However, the only commercially available POC serological tests for syphilis detect treponemal antibodies, which limits their use for interpretation of the disease status for treatment and interventions, because they cannot distinguish between active and past or treated infection even though they have good performance as compared with TPPA [5, 6]. Ongoing use of these tests, particularly in populations with a high prevalence of syphilis (eg, female sex workers, men who have sex with men) would result in repeated treatment and counseling or frequent exclusion of the syphilis diagnosis when they are tested with a nontreponemal test later on. These issues often result in mistrust between the testing providers and target populations (Chen et al, unpublished data). In contrast, use of the dual POC test could make an immediate indication of the necessity for treatment, ensuring same-day testing and treatment and the effectiveness of syphilis control efforts.

To our knowledge, this is the first evaluation study on performance of a dual POC test simultaneously using 3 kinds of specimens in clinic and nonclinic settings. The test evaluated in the current study performed well in terms of sensitivity and specificity relative to the reference standard TPPA and TRUST tests, particularly among those specimens with high TRUST titer of ≥1:4 (sensitivity >96%). In comparison with the combination of a TPPA/high-titer TRUST, which can be considered as the marker for “true disease,” the T1/T2 combined results reveal an excellent sensitivity, Interestingly, the performance of the test in outreach settings was as good as that in clinic settings and the performance was not different between 3 types of the specimens when it was used in outreach settings. The fact that the test can be performed to a high standard on site with different types of specimens, in clinic and nonclinic settings, and by clinical staff or outreach team members, means that it can be used in clinics and outreach services to high-risk groups. Its greatest value is likely to be in increasing the coverage of syphilis case finding and the proportion of cases treated, particularly among mobile and hard-to-reach communities such as female sex workers and men who have sex with men, in whom return rates for follow-up treatment are low [14]. The TRUST assay was used as the reference nontreponemal test for this evaluation because this assay is commonly used in clinical practice in the study areas. There is a good qualitative and quantitative agreement between TRUST and the rapid plasma reagin test, which is widely used in other areas [16].

In addition to visual reading of the results, an automatic reader was used to measure the density of the test lines and objectively provide either the quantitative results or the qualitative results by setting a cutoff criterion. As the performance of T1 (treponemal antibody) results read by the naked eye was perfect, we did not evaluate the sensitivity and specificity of T1 identified by the automatic reader and compare the results with those identified by the naked eye. Previous study indicates that the reader appears to be slightly more sensitive than visual reading, but our data did not show any improvement in performance of the test when the reader was used. However, as indicated in the results from ROC analyses, there is a trend toward an inverse relationship between sensitivity and specificity. Thus, for a diagnosis such as syphilis that can lead to further transmission of the disease if the patient is underdiagnosed, or carry a risk of unnecessary stigma for the patient if he or she is overdiagnosed, there may be an advantage in using the numeric data from the reader to balance the sensitivity and specificity for different populations by using a specific cutoff value. In addition, measurement by the automatic reader is superior to the visual reading because the former has the potential to monitor the changes in density of nontreponemal antibodies (corresponding to the titer on nontreponemal test) for evaluating a treatment response or determining a new infection. However, when the automatic reader is introduced, a range of cutoffs needs to be examined in order to find the optimal cutoff. In addition, electricity requirements may be a barrier to introducing this automatic reader in resource-limited settings in developing countries.

Study on use of the dual POC test to prevent adverse pregnancy outcomes in sub-Saharan Africa has shown that the test is cost-effective as it can save costs in resource-limited settings where disease is prevalent and loss to follow-up is high [17]. For introduction of the dual POC test into national or regional syphilis control programs, more operational studies on feasibility from societal, health facility, and patients’ perspectives, translation into policy, and evaluation of impact and cost-effectiveness are needed.

CONCLUSIONS

The dual POC test evaluated in this study shows good sensitivity and specificity to determine treponemal and nontreponemal antibodies in 3 kinds of specimens collected from patients in sexually transmitted infection clinics and high-risk groups in outreach settings in China. Given the simplicity in operating the test and interpreting the testing results, it is likely to be useful in the control of syphilis in primary healthcare settings and outside traditional clinical settings as well.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online (http://www.oxfordjournals.org/our_journals/cid/). Supplementary materials consist of data provided by the author that are published to benefit the reader. The posted materials are not copyedited. The contents of all supplementary data are the sole responsibility of the authors. Questions or messages regarding errors should be addressed to the author.