Clinical Evaluation of Two Point-Of-Care Lateral Flow Tests for the Diagnosis of Syphilis

Abstract

A diagnostic performance study comparing the only Food and Drug Administration (FDA)-approved, point-of-care (POC) treponemal test (Syphilis Health Check) and the World Health organization (WHO)-pre-qualified SD Bioline POC treponemal test against a treponemal hemagglutination test (TPHA) and a sequential algorithm of non-treponemal rapid plasma reagin (RPR) and TPHA found both POC tests had >85% sensitivity compared with the TPHA and >85% sensitivity and >95% specificity compared with the RPR and TPHA standards.

Single-step, rapid, point-of-care (POC) tests provide results in under 30 minutes and improves access to care for many infectious diseases including syphilis [1]. Development and validation of newer POC tests may accelerate timely treatment since RPR testing requires a working laboratory with electricity. Syphilis testing is especially important for pregnant women where 53.4–81.8% had adverse outcomes including stillbirth and congenital syphilis in a recent meta-analysis.[2] The WHO has recommended the use of POC rapid tests for syphilis [3, 4]; in countries with a high prevalence (3–5%) as in Uganda, syphilis tests should be more than 85% sensitive and more than 95% specific. [4].

A number of POC rapid tests for syphilis are now available in Uganda on the open market including the WHO pre-qualified SD Bioline (Yonghi Cho, Korea Kyogghi Do, Korea), the ABON Syphilis Ultra Rapid test strip (Wellkang Ltd, London UK), rapid chromatographic immunoassays for the qualitative detection of treponemal antibodies (IgG and IgM) in whole blood, serum or plasma. The recently FDA-approved, CLIA-waived Syphilis Health Check (Trinity Biotech, USA) is a qualitative, rapid treponemal membrane immuno-chromatographic lateral flow assay. Prior evaluations of the Syphilis Health Check have only been done in US populations, so more data on the performance of this syphilis POC test in African populations with higher prevalence is needed.

We sought to investigate the performance of the FDA-approved Syphilis Health Check and the SD Bioline test compared to TPHA (Syphilis TPHA liquid Test Human GmbH Wiesbaden, Germany) reference standard and also compared to the sequential diagnostic algorithm (non-treponemal RPR followed by treponemal testing) recommended for use by the Uganda National STD Treatment Guidelines in sexually transmitted disease (STD) and antenatal clinic (ANC) attendees where syphilis testing is conducted routinely.

From February 2012 to June 2013, we conducted a cross-sectional study among outpatient attendees ≥18 years of age at the STD and the ANC clinics of Mulago National Tertiary Referral Hospital to compare the performance of the Syphilis Health Check and SD Bioline POC tests for the diagnosis of syphilis. Using trained research assistants, we approached the directors in the respective clinics and obtained their consent for involvement of their clinics in the recruitment of study participants. Written informed consent was sought and given by all participants. Persons who either presented with signs and symptoms suggestive of genital ulcer disease to the two clinics or women who were pregnant were tested for syphilis and screened using a standard of care testing algorithm of an non-treponemal rapid plasma reagin (RPR) test (Carbon, Cypress Diagnostics, Langdorp Belgium) and treponemal antibody (TPAb, ABON Syphilis Ultra Rapid Test)[5] confirmation. Participants received pre- and post-test counseling for syphilis. A standardized pretested questionnaire was used to collect basic socio-demographic data including age, sex and a history of symptoms related to genital ulcer disease and other sexually transmitted infections. Data were transmitted to a central data base using the DataFax data management system. Participants consented to syphilis testing using all the tests under evaluation, but received a clinical syndromic diagnosis as per national guidelines which was confirmed by a laboratory result based on the RPR test and ABON TPAb confirmatory test in the clinic. Participants with a positive confirmed test were provided treatment according to National Treatment Guidelines at the STD clinic. They were also asked to bring in their partners for treatment. Partner and subsequent treatment follow-up was conducted at the STD clinic. Pregnant participants were treated according to guidelines specific for pregnancy in the National Treatment guidelines. The study was reviewed by the Joint Clinical Research Center (JCRC) Internal Review Board (IRB), the Johns Hopkins University IRB, and the Uganda National Council of Science and Technology. Participants received compensation for their time as approved by the IRBs.

The first 100 persons negative for syphilis as per testing protocol were consecutively enrolled and confirmed by TPHA. Persons who tested positive by RPR and ABON TPAb confirmatory test from each of the clinics were also consecutively enrolled in an effort to reach a target of 100 positive participants. Blood samples from the enrolled participants were also sent to the Makerere University Medical Microbiology Laboratory for the TPHA test and to the Infectious Diseases Institute Translational Laboratory for the Syphilis Health Check and the SD Bioline test, the syphilis POC tests. The tests were performed by laboratory technologists who were blinded to the results from the other syphilis tests. All participants were assigned a unique participant identified that was used to match the routine STD lab testing with the testing done in the reference labs.

Data were analyzed using the STATA statistical package (STATACorp. 2011. Stata Statistical Software: Release 12. College Station, TX: StataCorp LP). Demographic characteristics were presented as proportions or medians with interquartile ranges. Sensitivity and specificity of the 2 POC syphilis lateral flow treponemal tests (SD Bioline and the Syphilis Health Check) were calculated and compared with the TPHA test result and also against the sequential algorithm.

A total of 215 persons were enrolled at the STD or ANC clinics who provided both questionnaire data and blood samples for the tests; 144 participants (66.9%) were women. The median age of participants was 26 years (IQR 22, 32). The majority of participants (59%) had signs and symptoms of genital ulcers, 24% were referred by a medical worker and 9% had come for a clinical check-up visit.

The sensitivity of the Syphilis Health Check (Trinity Biotech) against the TPHA reference standard was 89.8% (95% CI; 82.0–95.0) and the specificity of the test was 92.3% (95% CI; 85.9–96.4). The sensitivity of the SD Bioline rapid test compared to the TPHA Gold standard was 91.8% (95%CI; 84.6 – 96.4) and the specificity of the test was 82.9% (95% CI: 74.8 – 89.2) (Table 1)

Table 1.

Number (N) of participants who tested positive using the SD Bioline and Syphilis Health Check tests compared to the TPHA reference standard

		TPHA					TPHA
SD Bioline		Reactive	Non- reactive	Total	Syphilis Health Check		Reactive	Non- reactive	Total
	Positive	90	20	110		Positive	88	9	97
	Negative	8	97	105		Negative	10	108	118
	Total	98	117	215		Total	98	117	215

The sensitivity of the Syphilis Health Check and the SD Bioline against the clinical diagnostic algorithm of the sequential RPR and TPHA test was 95.3% (95% CI; 88.4 – 98.7) and 98.8% (95% CI; 93.6 – 99.9), respectively. The specificity of the Syphilis Health Check and the SD Bioline against the RPR/TPHA testing algorithm was 98.1% (95% CI; 93.3 – 99.8) and 91.4% (95% CI; 84.3 – 96.0), respectively (Table 2). The sensitivity and specificity of each test stratified by genitourinary symptoms against the TPHA, and the TPHA+RPR reference standards are shown in the Supplemental Tables.

Table 2.

Number (N) of participants who tested positive using the SD Bioline and Syphilis Health Check tests compared to the RPR Test and TPHA reference testing algorithm

		RPR +TPHA					RPR+TPHA
SD Bioline		Reactive	Non- reactive	Total	Syphilis Health Check		Reactive	Non- reactive	Total
	Positive	84	9	93		Positive	81	2	83
	Negative	1	96	97		Negative	4	103	107
	Total	85	105	190		85	105	190	85

In our study, we found that the Syphilis Health Check (Trinity Biocheck), a rapid FDA approved POC test for syphilis, had an acceptable sensitivity of 89.8% and specificity of 92.3% compared to the TPHA treponemal reference standard in this population in a resource-limited setting and met the standard set by the WHO for acceptable performance of a test (>85% sensitivity) [4]. Our data support the use of these tests as a screening tool in this environment. We found that the SD Bioline rapid test, a WHO pre-qualified POC treponemal syphilis test, had a higher sensitivity than the Syphilis Health Check test of 91.8% but had a lower specificity of 82.9% compared to the Syphilis Health Check when the TPHA test was used as a reference standard. Similar sensitivities were found in several previous studies of the SD Bioline test of 88% in whole blood and 99% in serum samples [6–8], however, they found higher specificities of 98.5–99.4%. Specificity of only 90% was found in another study from KwaZulu Natal, South Africa [9]. The Syphilis Health Check had higher sensitivity than the RPR test. Earlier studies had found sensitivity of RPR at 77.5% in a rural population in Africa.[10] In our study, both tests showed acceptable sensitivity, but lower specificity than the WHO recommended standard.

When compared to a sequential diagnostic algorithm, the recommended testing algorithm in the National STD treatment guidelines in Uganda, both the Syphilis Health Check and the SD Bioline met the WHO standards for sensitivity and specificity set by the WHO (>85% sensitivity and >95% specificity)[4]. Further analysis showed that both tests had higher specificity when used among symptomatic patients compared to asymptomatic patients in this sample. Among symptomatic patients, the Syphilis Health Check showed high specificity compared to the SD Bioline when used against the RPR and TPHA algorithm though the estimate is quite unstable due to small sample size. Further studies need to be done for confirmation of this finding. Both tests performed well against the sequential RPR/TPHA syphilis testing algorithm in this environment [1] and would, therefore, provide an accurate and rapid alternative to this recommended diagnostic algorithm. This data could inform the potential use of the Syphilis Health Check test alongside such tests as the SD Bioline as rapid POC tests for syphilis in sub-Saharan Africa especially in pregnant women and STD clinic attendees. Use of these tests could contribute greatly to easier screening and treatment of syphilis in pregnancy and to the prevention of neonatal syphilis.

Limitations of the study include the fact that the study only used serological tests as a reference standard and did not include dark field microscopy which is not available in our setting. Therefore, we could have missed early infections and under reported true positives. Further studies to look at symptoms among individuals with a positive RPR and a negative TPHA are warranted. Our study only included positives consecutively sampled after the first 100 consecutively samples negatives of the antenatal and STD clinic populations. Therefore we cannot make generalizable conclusions on the performance of the tests in larger populations or on associated risk factors for syphilis in the population under study. In addition we are unable to report results by study clinic.

In conclusion, both the rapid tests, Trinity Syphilis Health Check and SD Bioline, met acceptable standards for use as a diagnostic test among at-risk populations such as STD clinic attendees and pregnant mothers in Uganda. Rapid POC tests could improve access to rapid diagnosis and treatment of syphilis.