Challenges of RPR interpretation in syphilis screening in Uganda: variability in non-treponemal results between different laboratories

Abstract

Background

Syphilis is a cause of morbidity and mortality and is of particular concern in pregnancy in low income countries because of risks associated with maternal-fetal transmission. Ugandan national guidelines recommend a non-treponemal Rapid Plasma Reagin (RPR) followed by treponemal testing for diagnosis of syphilis. The RPR test confirms a reactive specific treponemal test, or confirms serological ‘cure’ with a 4-fold dilutional decrease; RPR is beset with technical and biological limitations making accurate diagnosis and appropriate treatment problematic. The aim of this analysis was to compare performance of RPR-testing in different laboratories.

Methods

Stored, freeze-thawed sera from 215 participants were additionally tested for RPR and dilutional titer in two different reference laboratories. Discrepant results were tested at a third reference lab which served as a tie-breaker. Equivalence in RPR titer was defined as within ≤2 fold. All patients with a reactive rapid test were treated as per Ugandan national guidelines.

Results

Of 215 sera, 97 (45.1%) were RPR reactive in clinic laboratory A, 81 (37.7%) and 65 (30.2%) were RPR reactive in labs B and C respectively. All reported positive in the lab C were positive in lab B. Discrepant results were tested in lab D. Chi square test was highly significant (p=<0.001) for difference between each dyad of labs (A&B, A&C and B&C) RPR results. There were significant differences between RPR titers by paired t-test and Wilcox rank test (p=<0.001); with up to a 3-fold difference between labs. Two one-sided test approach demonstrated non-equivalence. Agreement between labs B–D, and C–D: 48/49 (98.0%) and 35/49 (71.4%) respectively (p=<0.001). Labs B and D showed no significant difference and had equivalent RPR titers. Labs C and D had different titers (p=<0.001) and were not equivalent.

Conclusions

We found significant inter- laboratory discrepant RPR results. A 3-fold difference in results is likely to be clinically significant and could result in under- or over-treatment. These data demonstrate a key limitation of the RPR test and underline the urgent need for a more reproducible quantitative test than the current RPR for diagnosing and determining cure of syphilis.

Background

The World Health Organization (WHO) estimated that in 2012 there were 5.6 million cases of syphilis, in 2015 the median case rate of syphilis per 100,000 adults was 25.1 (range 0.1–1664). In Uganda that rate was reported as 372.8 in 2014¹. There were an estimated 350,000 annual global adverse pregnancy outcomes due to Mother To Child Transmission of syphilis¹. Consequently, the diagnosis, treatment and prevention of syphilis are global health priorities.

Syphilis is a major global infectious disease with huge individual and public health consequences² including congenital syphilis and increased potential for sexual transmission of HIV³, Ugandan national guidelines specifically highlight the additional risk for HIV infection posed by genital ulcer disease⁴; such “epidemiological synergy” increases the transmission potential for HIV⁵. Without screening for and treatment of syphilis it is estimated between 53–82% of infected women will have poor pregnancy outcomes including neonatal death, and congenital syphilis⁶; this is 4 times the rate in syphilis-uninfected women.

T. pallidum (TP) has remained fully sensitive to penicillin therapy, therefore, failures to control the spread of infection, both sexually and mother-to-child⁷, largely represent failures in public health and health systems approaches. Like other Sub-Saharan African (SSA) countries Uganda has high rates of syphilis with correspondingly high prevalence of antenatal syphilis. While many nations in SSA have seen decreases in the antenatal syphilis prevalence, Uganda remains a high-prevalence country⁸.

In a 2015 study of 43 SSA countries, it was estimated that Uganda had a syphilis prevalence of 3.0%, (based on 2004 data)⁹, with estimated adverse pregnancy outcomes of 10,670 (5982–19,633) attributable to missed cases of active syphilis.

Ugandan 2016 National guidelines recommend Rapid Plasma Reagin (RPR) testing followed by TP Hem- Agglutination (TPHA) to confirm a positive non-treponemal test⁴. The WHO highlight the benefits of treponemal rapid diagnostic tests (RDT)¹⁰ e.g. treponemal lateral flow to screen for syphilis, and this approach is increasingly used in Antenatal (ANC) and Sexually Transmitted Disease (STD) clinic settings. In this context the treponemal RDT is confirmed using the RPR non-treponemal test. RPR is used in the diagnosis and assessment of treatment response in syphilis infection in health settings globally, and is currently the best available test for assessing serological response to treatment as well as indicating reinfection and/or treatment failure. It is based on antigens that contain cardiolipin, lecithin and cholesterol which cause visible flocculation of positive sera containing IgM and/or IgG antibodies to lipoidal antigens that are putatively released from bacteria. However, since these antigens are also normal components of human cells it is not possible to determine if the RPR measures products released by TP subspecies pallidum, the causative agent of syphilis, or damaged host cells or both¹¹. Seroconversion from RPR-negative to –positive generally occurs within 3 weeks of exposure to TP but may be delayed up to 6 weeks¹². The advantages of the RPR is that it is relatively inexpensive and easy to perform, and can provide titers (by dilution) to aid initial diagnosis of disease activity and against which to monitor treatment response. There are several disadvantages particularly in low and middle income country (LMIC) settings specifically: the tests require an electricity supply, refrigeration for antigen storage, a light source, constant humidity¹³, temperature monitoring, and trained personnel¹³. Laboratory variation¹⁴ and false positive reactions are relatively common in the context of pregnancy, aging, autoimmune conditions and other infectious disease e.g. HIV¹⁵. RPR may vary by one serial dilution on repeated testing, so titer changes of less than two serial dilutions (a fourfold change) are, according to Hook and Marra, rarely clinically “meaningful”¹⁶. The RPR is prone to false positive results¹⁵; for this reason it is recommended that a specific treponemal test (e.g. TP Particle- or Hem- Agglutination TPPA/TPHA) is used in an algorithmic approach with RPR to avoid false positive ¹² and minimize false negative results.

An RPR product insert highlights difficulties in initial interpretation; there are only 2 categories: reactive and nonreactive (rather than allowing for inferences made about titers), it is only with dilution that a titer is generated. However, it is occasionally necessary to repeat the test on a different platform or to report as “indeterminate” pending further evaluation. It also cautions about the susceptibility of the antigen to bright sunlight and freeze-thaw cyles¹⁷. There are reports of wide variations between and within laboratories; a 2009 paper by Gupta et al. examined 26 microbiology laboratories testing 138 sera panels and demonstrated inter- and intra-laboratory variation of 58.7% and 53.1% respectively by RPR when compared to the reference laboratory¹⁸. In the described large external quality control assurance (EQA) study the authors identified 7 key errors ranging from improper interpretation of RPR results to delayed testing and laboratory temperatures¹⁸.

The analysis presented in this paper compared the performance of RPR both against a treponemal test; and across different laboratories in Kampala, Uganda and in Baltimore, Maryland, USA.

Methods

A full description of the study from which the sera were drawn can be found in Nakku-Joloba et al¹⁹. Briefly, the study was undertaken between February 2012 and June 2013. The population were adults identified in ANC and STD clinics at Mulago National Tertiary Referral Hospital, Kampala, Uganda. The parent study offered inclusion and enrollment of a minimum of 100 participants with negative followed by 100 participants with positive syphilis serological tests (Figure 1). Potential participants were offered information about the study by trained research assistants and written informed consent was obtained.

Figure 1.

Flow chart of samples and testing.

Participants were screened using the Ugandan national standard of care algorithm for syphilis which consisted of non-treponemal RPR testing (Carbon, Cypress Diagnostics, Langdorp Belgium) followed by treponemal test (TPAb, ABON™ Syphilis Ultra Rapid Test) when the RPR was reactive. The treponemal test uses a rapid chromatographic immunoassay platform for the qualitative detection of antibodies (IgG and IgM) to TP. Blood was tested at the local on-site ANC/STD clinic (A) for RPR reactivity and sera were batched for repeat RPR testing with titers at laboratories B, C, and D (Figure 1). Other than the local ANC/STD on-site labs, the other labs included; an International STD reference lab (in the USA) and 2 College of American Pathologist (CAP) Certified clinical laboratories (in Uganda). Technicians who perform RPR testing undergo a period of 6 months initial training in serological techniques, are then assessed using CAP EQA RPR samples followed by annual assessment. Labs B and D initially diluted sera up to a maximum dilution of 1:16 therefore, there may have been higher RPR titers within the samples. One laboratory served as a ‘tie-breaker’ following discrepant results in initial RPR results and when RPR titers were >2 fold different between the other labs (Figure 1).

Analysis

The primary outcome for this analysis was the difference in RPR positivity between labs A, B and C; when discrepant results arose, samples were tested at lab D. A chi square statistic was used to test for differences in the primary outcome. Secondary outcomes were agreement in RPR titer, allowing for ≤2 fold variation, between the sets of tests; the difference and equivalence between labs B and C results with those from reference lab D.

Due to the nature of the data, a log² transformation was conducted to normalize the data; when RPR = “Negative”, Titer score = 0 therefore a very small pseudo-number (0.0001) was added to these values. Chi square test was used to evaluate whether scores from these three clinics were statistically different. This allowed us to test if the difference in transformed scores was different from zero. Secondary endpoints were further analyzed using the two one-sided test approach (TOST), which allows testing for equivalence, as distinct from difference.

The t-test looks at whether the confidence interval (CI) include 0, and if it does not, the t-test states that the two clinics give different scores. However, the equivalence test looks at whether the CI goes beyond the acceptable range ((log2(2) = 1, therefore, is [−1, 1]).

Results

Of participants, 144 (66.9%) were women and their median age was 26 years (IQR 22, 32). In a separate analysis, 110 of 215 sera were determined to be RPR/TPAb, ABON™/TPHA positive and 105 RPR/TPHA negative as part of the parent study (Figure 1); these tests were conducted at the Makerere University Medical Microbiology Laboratory¹⁹ (data not shown).

Primary outcome

Of 215 sera, 97 (45.1%), 81 (37.7%) and 65 (30.2%) were RPR reactive in labs A, B and C respectively. All reported RPR positives in lab C were positive in labs A and B (Table 1). Chi square test was highly significant (χ^2= 150.35, df = 1, p=<0.001) for difference. Chi square test was highly significant (p=<0.001) for difference between each dyad of labs (A&B, A&C and B&C) RPR results (table 1).

Table 1.

Comparison of lab A, B and C results.

B\A	positive	negative	Total
Positive	76	5	81
Negative	21	113	134
Total	97	118	215
C\A	positive	negative	Total
Positive	62	3	65
Negative	35	115	150
Total	97	118	215
C\B	positive	negative	Total
Positive	65	0	65
Negative	16	134	150
Total	81	134	215

A chi square test was conducted to evaluate the differences in RPR positivity: A and B gave very different RPR scores (χ2= 121.4, df = 1, p-value < 2.2e–16), as did A and C (χ2= 92.2, df = 1, p-value < 2.2e–16) and B and C (χ2= 150.35, df = 1, p-value < 2.2e–16).

Of samples, with discrepant results 49 were re-run in lab D, lab D was selected as a ‘tie-breaker’ in order to adjudicate discrepant results after comparisons between labs A, B and C. Discrepancies were classed as lack of RPR positivity between labs A, B and C or a 2-fold or greater difference in RPR titer between labs B and C (Table 2).

Table 2.

Comparison of discrepant lab A, B, C compared to lab D results.

A\D	positive	negative	Total
Positive	47	0	47
Negative	1	1	2
Total	48	1	49
B\D	positive	negative	Total
Positive	48	1	49
Negative	0	0	0
Total	48	1	49
C\D	positive	negative	Total
Positive	34	0	34
Negative	14	1	15
Total	48	1	49

Agreement between labs A/D, B/D, and C/D was 47/49 (95.9%); 48/49 (98.0%) and 34/49 (69.4%) respectively. Lab C was statistically different from the others (p<=0.001), while labs A, B and D were not.

Secondary outcomes

There were significant differences between RPR titers by paired t-test and Wilcox rank test (p=<0.001) between labs B/C, and C/D; lab B RPR being 3.2 times higher, on average, than lab C. Using the TOST approach we demonstrated that the labs were not equivalent (Figure 2).

Figure 2.

Comparing difference and equivalence of lab RPR titer scores – panels i, ii & iii.

If we can accept that as long as the difference is within two-fold change, the equivalence tests states the two clinics (B and C) did not give equivalent scores (Figure 2i). Labs B and D gave comparable titer scores: paired t-test and Wilcox rank test showed that there were no significant differences between these two results. Tests for equivalence indicated that the 2 gave equivalent titers (Figure 2ii). Labs C and D reported significantly different RPR titers; paired t test and Wilcox rank test p=<0.001. Further, the 2 clinics were not equivalent (Figure 2iii).

Discussion

Syphilis diagnosis and treatment are urgent and ongoing priorities for improving maternal and child outcomes and those with STD in SSA. The discrepancies in RPR results highlights the need for ongoing regular EQA, staff training and quality standards as well a need to better understand underpinning host immune responses to syphilis. The 3.2 fold differences in RPR titers between lab B and C is of potential clinical relevance particularly since higher titers increase the risk of congenital syphilis and poor fetal/infant outcomes⁶ as well as more serious syphilis sequelae²⁰; in practice this might change an RPR profile from 1:2 to >1:16. Such inconsistency in titers is a key limitation of the RPR test since it may result in either over-treatment of adequately treated past infection or, more problematically, under-treatment of active disease or treatment failures. Of the positive treponemal-specific TPHA between 65–97/110 (59.1% – 88.2%) were non-treponemal test (RPR) reactive depending on the laboratory. This may indicate: i) poor performance of either test, ii) false positive treponemal results, iii) false negative non-treponemal tests, iv) late/inactive syphilis with negative RPR, v) prior infection, treated or untreated, (with RPR titer reversion) and vi) training and quality assurance issues. While this analysis is not able to distinguish the cause of these discrepancies the data highlight the need for combined treponemal and non-treponemal testing in the ANC/STD clinic setting.

There has been a call to automate non-treponemal testing in order to overcome some of the issues seen in RPR variability and labor costs. Some argue that the problem of objectivity and reproducibility are inherent to the manual method of RPR that might be overcome using automation²¹ but others describe inconsistent results with automated RPR assessment²² particularly as a way of monitoring treatment response; comparative large scale trials are lacking²³. Other suggestions include the use of paired RPR testing of acute and convalescent samples in the context of repeated infection and for follow up²⁴. This may overcome some of the intra-laboratory variation but lacks utility in the diagnosis of new infections. It may also be cost prohibitive in resource limited settings such as Uganda.

Possible explanations for the discrepancies in RPR data in general include the use of different RPR card tests (BD Macro-Vue™ RPR and Carbon, Cypress Diagnostics) in different laboratories, environment (temperature and humidity), personnel, endemic treponemal disease, HIV-status, laboratory variation and the effects of freeze-thawing of serum; storage and transportation. As there is only one lab (C) that had statistically significant RPR titer difference to the others, it could be argued that this is an outlier. However, Lab B, C and D are all CAP certified/accredited and undergo external proficiency testing as well as on-going daily internal quality assurance procedures. This should minimize the effects of environmental conditions and laboratory variation, however this study suggests that other factors are having an impact on these results. As highlighted by the RPR manufacturers, the reading of these tests has a degree of subjectivity. The labs in the capital city, Kampala, all have good access to water, electricity, highly qualified personnel and good lab infrastructure. If the RPR is underperforming in these settings, it is highly unlikely to give reproducible results in other less well-resourced labs outside of Kampala.

Limitations of the current study include a lack of dark field microscopy or TP PCR data which raises the possibility of missed early infections before serological positivity. The parent study¹⁹ did not set out to compare laboratory- or test- performance so we do not have data on all samples collected. The sample size is relatively small, the study population lacks descriptive characteristics such as HIV prevalence or assessment of non-syphilis treponemal disease which may have resulted in false positive results. Stored sera underwent freeze/thaw cycles which may have affected results, and not all samples had contemporaneous treponemal testing. As these results are from patients attending ANC and STD clinics in Kampala they are not generalizable to other populations, therefore larger studies in various populations are warranted to ascertain performance of RPR tests in different settings. It would also be interesting to characterize the phenotype of patients with an isolated positive RPR and ascertain follow-up to assess maintenance, evolution or regression of syphilis serology, development of connective tissue disease or other infection, and rates of syphilis infection in their sexual contacts. Future work could involve repeat, simultaneous testing of all samples with treponemal/non-treponemal tests in a reference laboratory.

To conclude, these data demonstrate significant differences in RPR results (reactive vs. non-reactive) between all 3 labs measuring RPR and also demonstrate significant difference in laboratories where RPR titer was measured. Further work is required to better understand these differences and ensure the most accurate and reproducible results in the future. They also serve to remind the global health community of the limitations of the RPR as a test for syphilis diagnosis. This is especially important in LMIC where there is a high burden of syphilis-associated morbidity, in an environment with minimal access to equipment and laboratory support. Important future developments could include a better, more reliable and affordable dual Point-Of-Care Tests (POCT) for syphilis diagnosis than those currently available which can have reduced sensitivity compared to serological tests. This would have the double advantage of fewer false negatives in early syphilis and reduced overtreatment of previously treated infection. A recent meta-analysis of dual path platform POCT assessing both treponemal and non-treponemal antibodies for syphilis is encouraging with good sensitivity and specificity²⁵, however data are urgently needed in ANC/STD settings in Africa and in HIV-positive populations generally. Whilst the diagnosis issue may be resolved with a dual treponemal/non-treponemal syphilis POCT, the evaluation of treatment response and possible re-infection (including need for re-treatment during pregnancy) is a challenging one, and RPR titers are routinely used globally to determine if re-treatment is needed. The data generated from this analysis suggest that the results need to be interpreted with utmost caution in a Ugandan setting. In recent years, driven by the WHO, there has been a renewed interest in syphilis diagnostics and POCT technology, this should bode well for the future development of more sensitive, reliable and affordable near patient syphilis tests in LMIC²⁶.