Evaluation of Rapid Syphilis Tests in KwaZulu‐Natal

Abstract

Background

Despite the considerable decrease in the seroprevalence of syphilis in South Africa, with an estimated prevalence of 1.5% in 2010, the disease remains a threat particularly to pregnant women, hence there is a need for a rapid, reliable, and affordable screening and diagnostic test. A laboratory evaluation study was conducted in response to a call by the KwaZulu‐Natal (KZN) Provincial Department of Health that is considering using rapid point‐of‐care syphilis tests.

Methods

The performances of the Hexagon and the SD Bioline syphilis tests were compared with the Treponema pallidum hemagglutination assay (TPHA) reference test using 297 (142 positive and 155 negative) serum specimens.

Results

Both assays demonstrated good performance with negative and positive concordance of 97 and 94% for the Hexagon assay and 98 and 90% for SD Bioline assay, respectively, when compared to the TPHA. The Hexagon test was quicker and easier to read than the SD Bioline test.

Conclusion

Although the rapid syphilis tests performed favorably, a number of issues need to be considered prior to their use for syphilis screening in the public sector of South Africa.

BACKGROUND

South Africa has seen a decrease in the prevalence of syphilis over the years from 11.2% in 1997 to 1.5% in 2010 among the pregnant women attending antenatal clinics. This is particularly pronounced in the KwaZulu‐Natal (KZN) province, which despite having the highest HIV prevalence in the country showed the lowest syphilis prevalence of 0.3% in 2010. This decrease has been attributed to the routine screening and treatment of pregnant women for syphilis as well as the syndromic management of genital ulcer disease 1.

Syphilis is caused by the spirochete Treponema pallidum that can neither be cultured in vitro 2 nor seen using under the standard bright field microscopy using direct light, and for this reason, serology has been the primary mode of diagnosis in routine laboratories. Serological diagnosis is based on detecting nontreponemal (nonspecific) antibodies and treponemal (specific) antibodies in the blood of patients infected with the spirochete 3. Examples of nontreponemal include Rapid Plasma Reagin (RPR), Venereal Disease Research Laboratory and treponemal tests include T. pallidum hemagglutination assay (TPHA), fluorescent treponemal antibody absorption test, and enzyme immunoassays (EIA). In South Africa, nontreponemal tests have traditionally been used for the syphilis screening because they can distinguish between an active infection and a past infection. They are also less technically demanding than treponemal tests. However, because of the false‐positive reactions that can occur with nontreponemal tests in the context of various medical diseases 4, treponemal tests are still needed to confirm the positive result. Although highly sensitive and specific, treponemal tests are technically very demanding and they remain positive after the successful treatment or time of the syphilis infection 5.

All the above‐mentioned tests are laboratory based, which means results are not available at point of care, and thus treatment may be delayed, leading to patients lost to followup, continued transmission, and complications. Most of these patients are generally not unwell and would benefit from an on‐site diagnosis and treatment where indicated. In a study conducted in South Africa, Bronzan et al. reported poor return of patients associated with off‐site syphilis screening 6. The rapid syphilis point‐of‐care tests, which are immunochromatographic assays used for detecting antibodies against T. pallidum, offer the advantage of quick on‐site results over the traditional methods. The South African Department of Health is currently considering using these assays for syphilis screening.

The KZN Department of Health together with the National Health Laboratory Service held a technical evaluation meeting where syphilis rapid test kits, which had been submitted for the tender evaluation, were assessed. The kits had to meet technical criteria in order to be considered for the tender. The requirements included manufacturer's sensitivity and specificity of at least 99%; detection of all isotypes of antibodies IgM, IgG, and IgA; no additional equipment should be required (all necessary supplies must be part of the test kit); provision of a validation certificate and listing by the World Health Organization; clear and easy‐to‐follow standard operating procedure; rapid results (within 15–20 min); easy performance (less than 4 min); easy interpretation; a minimum shelf life of 18 months at 20–30°C; a maximum of 100 test kits per cartoon for easy storage and be able to use whole blood specimen. It was decided that the kits that met the technical criteria would then be further evaluated in the laboratory and only two kits qualified for this, namely, the Hexagon syphilis (Human GmbH, Germany) and SD Bioline rapid syphilis tests (SD Standard Diagnostics, Inc., Korea).

There are now more than 20 commercially available rapid syphilis point‐of‐care tests and several of these tests have been evaluated including the SD Bioline syphilis 7, 8, 9; however, only a few have been evaluated in our setting where there is a low syphilis prevalence in the presence of a very high HIV prevalence and there is also no published data on the performance of the Hexagon syphilis test. The aim of this study was to determine the concordance of each of the two rapid tests against TPHA as a reference test.

MATERIALS AND METHODS

Study Design

The study consisted of an evaluation of the Hexagon syphilis and SD Bioline syphilis rapid point‐of‐care tests with the retrospective laboratory analysis of stored sera for syphilis testing against TPHA used as the reference standard.

Study Setting

Prince Street laboratory is a reference serology laboratory located in Durban at KZN province with approximately 10,000 samples tested for syphilis every month. Samples come from different health facilities across the province. The testing is done using RPR (BD Macro‐Vue RPR card test, Maryland, USA) as a screening tool, and if it becomes reactive, the sample is serially diluted to determine the lowest titer. Confirmation of a positive RPR result is done using the TPHA (Plasmatech, UK) antibody test. However, the laboratory also receives specimens from peripheral labs that perform RPR only, in which case, the reference laboratory will only do the confirmatory TPHA antibody test. Specimens from patients in peripheral health facilities, where early or late syphilis is suspected, will have a TPHA test done irrespective of the RPR result.

Sample Size

A sample size of 297 was chosen because at least 139 positive and 139 negative specimens were required to estimate the sensitivity and specificity of 5% with a probability of 95%, and assuming 90% for both sensitivity and specificity.

Methods and Procedures

Consecutive serum samples sent for routine testing of syphilis were kept aside daily and stored at −20°C until a total of 297 was reached (142 positive, 155 negative), most of which also had an RPR result, however, TPHA was used as a reference standard. The samples were then tested using both the Hexagon syphilis and the SD Bioline syphilis rapid tests for each specimen. The rapid tests were performed by different technologist except the one performing the RPR and TPHA tests. The RPR and TPHA were performed according to the Standard International Procedures. The rapid tests were performed according to the manufacturer's instructions. When the results were unclear or difficult to read, the test was repeated.

The SD Bioline syphilis 3.0 is a solid‐phase immunochromatographic assay that qualitatively detects antibodies of isotypes IgG, IgM, and IgA against T. pallidum, and can be performed with whole blood, plasma, or serum. It contains a membrane strip that is precoated with recombinant T. pallidum antigens (17 and 15 kDa) on the test band region. The test was performed according to the manufacturer's instructions. According to these instructions, when the test sample and assay diluents are added to the sample well, they move along with the recombinant T. pallidum antigens‐colloidal gold conjugate to the test region where they form a visible line as the antigen‐antibody‐antigen gold particle complex forms. In the absence of T. pallidum antibodies in the sample, no visible color band appears in the test region. A control band is included on the left side of the result window to indicate that the test is working properly.

The Hexagon syphilis is also an immunochromatographic rapid test for detecting T. pallidum antibody isotypes IgG, IgM, and IgA in whole blood, serum, or plasma. It is based on the sandwich technology, where recombinant T. pallidum antigens (15, 17, and 47 kDa) are fixed in the test line and are also conjugated to colloidal gold in the mobile phase and anti‐T. pallidum antibodies (goat) in the control line. The instructions indicate that the tests are performed according to the manufacturer's instructions. A test sample is added to the specimen window, where it then flows through the absorbent pad, and human anti‐T. pallidum antibodies are bound by the recombinant T. pallidum dye conjugate to form an immunocomplex. This binds to the recombinant T. pallidum antigen in the test line and produces a red‐violet test line. Excess conjugate reacts in the control line with the anti‐T. pallidum antibodies (goat) forming a second red‐violet line to demonstrate the correct function of the reagents.

A form was prepared and used to collect data, which included the test date, sample number, TPHA results, RPR results, Hexagon syphilis results, SD Bioline syphilis results, and any additional comments where applicable. The technologist performing the test completed the form.

Data Analysis

The information collected was captured onto excel and analyzed in Stata V12. TPHA was used as the reference standard. Positive and negative concordances with 95% confidence limits were reported for each rapid test. Exact confidence limits for a binomial proportion were used.

RESULTS

A total of 297 specimens were tested (Fig. 1) and the positive and negative concordance of the two assays were more than 90% in each case, see Tables 1 and 2. The Hexagon syphilis and BD Bioline syphilis showed 8 and 14 false‐negative samples, respectively, and all but one of the false‐negative Hexagon syphilis rapid tests were also false negative with the SD Bioline syphilis test (Table 3). All the false‐negative rapid tests had very low RPR titer (≤2), except for one sample, which had an RPR titre of >32. Of the 155 TPHA negative samples, all had corresponding RPR results, and 43 (28%) of these were reactive, with a titer of <8 except for one, which had a titer of 8.

Figure 1.

Flow diagram showing TPHA and RPR test results

Table 1.

Positive and Negative Concordance of the Hexagon Syphilis Rapid Test Compared to TPHA

	TPHA positive (142)		TPHA negative (155)
	Number positive	Positive concordance (95% CI)	Number negative	Negative concordance (95% CI)
Hexagon syphilis	134	94% (0.89–0.97)	151	97% (0.93–0.99)

Table 2.

Positive and Negative Concordance of the SD Bioline 3.0 Syphilis Rapid Test Compared to TPHA

	TPHA positive (142)		TPHA negative (155)
	Number positive	Positive concordance (95% CI)	Number negative	Negative concordance (95% CI)
SD Bioline 3.0 syphilis	128	90% (0.84–0.94)	152	98% (0.94–0.99)

Table 3.

TPHA and RPR Results of False‐Negative Rapid Test Results

Sample	TPHA	RPR	SD Bioline	Hexagon
1	Positive	Reactive 1:1	Negative	Positive
2	Positive	Reactive 1:1	Negative	Positive
3	Positive	Reactive 1:2	Negative	Positive
4	Positive	Reactive 1:1	Negative	Positive
5	Positive	Reactive 1:1	Negative	Positive
6	Positive	Reactive >1:32	Negative	Positive
7	Positive	Reactive 1:2	Positive	Negative
8	Positive	Reactive 1:1	Negative	Negative
9	Positive	Reactive 1:1	Negative	Negative
10	Positive	Reactive 1:1	Negative	Negative
11	Positive	Reactive 1:2	Negative	Negative
12	Positive	Nonreactive	Negative	Negative
13	Positive	Nonreactive	Negative	Negative
14	Positive	Nonreactive	Negative	Positive
15	Positive	Not available	Negative	Negative

The performance of the tests was also calculated by RPR status. The positive concordance was higher in RPR reactive patients, 96 and 91% compared to RPR nonreactive patients, 67 and 50% for Hexagon and Bioline tests, respectively (Table 4). Of the 117 RPR positive samples, 19 (16%) had an RPR titer of ≥8.

Table 4.

Positive Concordance of Hexagon Syphilis and SD Bioline 3.0 Syphilis Rapid Tests Compared to RPR

	TPHA positive (142)
	RPR reactive (117)		RPR Nonreactive (6)
	Number	Concordance	Number	Concordance
Hexagon syphilis	112	96%	4	67%
SD Bioline 3.0 syphilis	107	91%	3	50%

There were ten (3.4%) unreadable results from the SD Bioline syphilis and none from the Hexagon syphilis. These showed poor visibility, which improved on repeat testing, and the results of the second test were used for analysis. All the unreadable results were on the samples with an RPR titer ≤ 2. While the results were clearer and quicker with the Hexagon syphilis than with the SD Bioline syphilis kit, the latter's time fell within the recommended time of 15–20 min.

DISCUSSION

Despite the drastic decrease in the seroprevalence of syphilis in South Africa, the disease still remains a threat, particularly to pregnant women, hence the need for a rapid, reliable, and affordable test for screening and diagnosing the disease. Both assays performed well when compared to the TPHA. The Hexagon syphilis assay showed higher positive concordance compared to the SD Bioline 3.0 syphilis assay and all but one false‐negative Hexagon results were also negative by the Bioline assay (in addition to the seven results that are false negative by SD Bioline only). However, the study was not designed as a superiority trial and thus not sufficiently powered to detect difference between the two assays.

The negative concordance was very good for both assays. The unreadable results shown by the SD Bioline are a cause for concern. These demonstrated a pink color in the reading window, which made it impossible to read the results forcing the technologist to perform another test. Although the visibility improved with repeat testing, this increases the cost and time spent on performing the test. In addition to this, if this happens in serum samples, which should be much easier to read, a lot more could be expected to occur in whole blood samples. The use of serum samples has also been shown to have a better sensitivity than whole blood specimens 10.

The major disadvantage of the syphilis rapid tests is that, like all treponemal tests, they remain positive following successful treatment, hence there is need for a confirmatory RPR test. When we stratified our results according to the RPR results, only 5% were negative on RPR and the rapid tests demonstrated poor sensitivity with this subset of samples. Even though there was no clinical information to determine the cause for these discordant results, the poor performance of the rapid test in this setting is concerning. The RPR titer of ≥8 has generally been accepted as an indicator for active infection while a titer <8 signifies biological false‐positive reactions, therefore using this indicator, the Hexagon assay did not miss any cases of active infection while one case was missed by the SD Bioline assay 11.

The recent interest shown by the South African Department of Health in the use of syphilis rapid tests has come at a time when there has been much debate on the use of reverse algorithm for syphilis diagnosis 12, 13. Contrary to the tradition algorithm where a nontreponemal test is used for screening followed by a confirmatory treponemal test, in the reverse algorithm, the treponemal test precedes the performance of nontreponemal test. While the latter may increase the detection of syphilis, particularly early and late latent infections, the detection of previously treated infections would also increase leading to unnecessary treatment and followup of patients. On the other hand, the well‐established traditional algorithm may miss some cases of syphilis while it is able to differentiate between active infection and past infection. Nevertheless, the choice of diagnostic algorithm depends on the clinical needs and laboratory resources available in the country. At the same time, there are new dual rapid tests underway that combine both treponemal and nontreponemal tests into one cassette, which can potentially simplify the diagnosis of syphilis 14.

The limitations of the study include the use of serum samples that could have a higher sensitivity compared to the whole blood and that the study was conducted in a laboratory rather than in the field where these tests would be used. There was also no available clinical information for better interpretation of the results. Nonetheless, the two syphilis rapid tests were found to have a high negative and positive concordance when compared to the TPHA in serum samples in the laboratory. Further field studies are still needed before any consideration for routine use in syphilis screening.

CONFLICT OF INTEREST

The syphilis kits for the study were provided by the suppliers.