Reverse syphilis screening algorithm fails to demonstrate cost effectiveness in persons living with HIV

Abstract

Although the reverse syphilis screening algorithm is more efficient than the traditional algorithm, it may lead to exorbitant costs for health systems serving persons living with HIV needing annual syphilis screening. Alternatively, the traditional screening algorithm is cost saving in many scenarios.

Introduction

Optimal medical care of persons living with HIV (PLWH) involves comprehensive preventive and screening services, which are costly to health systems.^1,2 Syphilis screening is recommended at least annually for PLWH, and more frequently for those at high risk for sexually transmitted infections.^3,4 Due to laboratory efficiency, the reverse syphilis screening algorithm is widely used for syphilis detection even in the absence of comparative and cost effectiveness data justifying its use.^5,6 The traditional syphilis screening algorithm relies on an initial non-treponemal test such as the rapid plasma reagin (RPR) or the Venereal Disease Research Lab (VDRL), whereas the reverse algorithm incorporates a treponemal test, the enzyme immunoassay (EIA), as the first step. The EIA may be positive in those without syphilis (e.g., false positive) and in those who have a history of syphilis but no active infection (e.g., previously-treated individuals).

Appropriate syphilis screening is closely monitored in clinics that rely on Ryan White Care Act “safety net” funding to support the care of PLWH, and failure to comply results in financial penalties.¹ This policy and the use of electronic clinical reminders have led to high rates of syphilis screening in our clinic and others.^7–9 Because our population has a high prevalence of previously-treated syphilis, many patients have a positive EIA on each annual screen, which requires subsequent confirmation with an RPR and is costly to the health system. This common scenario has led us to question the cost effectiveness of the reverse syphilis screening algorithm relative to the traditional algorithm in this scenario: PLWH, engaged in care, receiving at least annual syphilis screening. The objective was to compare the cost effectiveness of the reverse and traditional syphilis screening algorithms in PLWH engaged in care. We hypothesized that the reverse algorithm may be more costly due to positive EIA results in many PLWH with previously treated infection and/or false positives.

Methods

A decision tree analysis compared the cost effectiveness of annual syphilis screening for PLWH engaged in care at the 1917 HIV/AIDS Clinic affiliated with the University of Alabama at Birmingham (UAB) over a one-year time horizon from the health system perspective. This time frame was chosen as the standard of care is annual screening. We evaluated (1) the reverse algorithm relying on initial EIA results followed by a confirmatory RPR +/− Treponema pallidum particle agglutination assay (TPPA) as indicated by EIA and RPR results and (2) the traditional algorithm relying on an initial RPR test followed by a confirmatory TPPA if indicated.¹⁰ We assumed a syphilis prevalence of 3.4% consistent with a North American clinic serving PLWH (supplementary Table 1).¹¹ Sensitivities and specificities of tests were taken directly from UAB laboratory vendors (EIA and TPPA information obtained via M. Diaz-Datka, personal communication, 22 July 2016) and publications (RPR) and were used, together with prevalence, to estimate the frequency of syphilis diagnoses.¹⁰ Costs were assigned using Centers for Medicare and Medicaid Services fees for the EIA, RPR, and TPPA and the average wholesale price of penicillin G benzathine (per 2.4 million unit intramuscular dose).^12,13 We assumed that patients received annual testing consistent with guidelines and clinical practice^7,8 and incident cases were primary, secondary, or early latent infections, responding to one dose of penicillin. We assumed that care of untreated syphilis (false-negative results) would cost $60.¹⁴ Incremental cost effectiveness ratio (ICER) was calculated for base case parameters as the difference in costs for the two algorithms, divided by the difference in effectiveness in terms of quality-adjusted life years (QALYs).

One-way sensitivity analyses were performed using variables with the greatest uncertainty and variability in the literature (supplementary Table 1).^11,14–17 To account for uncertainty in the stage of syphilis at the time of diagnosis, RPR, TPPA, and EIA test characteristics and costs of untreated syphilis were varied to account for primary, secondary, and early latent syphilis.^9,14,15,17 We varied the health utility of having HIV and concomitant syphilis based on published reports.¹⁵ Due to limited data on test costs, we used ranges of 50 to 150% of base case parameters. A probabilistic sensitivity analysis was performed to determine the cumulative effect of multiple uncertainties on strategy preference. All parameters were varied within a triangular distribution and effectiveness was recalculated over 1000 iterations of the model. We assumed a health system perspective for reasons outlined above and due to a largely uninsured patient population. We assumed a willingness to pay of $100,000/ QALY.¹⁸ We used TreeAge Pro 2016 (TreeAge Software, Williamstown, MA, USA) for primary analysis and SIP Math to aid in probabilistic sensitivity analyses. All costs are in 2017 US dollars.

Baseline results

The ICER of the reverse relative to traditional algorithm is $300,817/QALY. There were more false-negative results using the traditional (0.8%) relative to reverse algorithm (0%). There were more patients incorrectly identified as having syphilis infection using the reverse (2.5%) relative to traditional algorithms (0.1%), all of whom had previously treated syphilis.

One-way sensitivity analysis

The analysis was sensitive to changes in prevalence of prior and current syphilis infection, RPR sensitivity and cost of untreated syphilis (supplementary Table 2). The reverse algorithm was dominated (more costly, less effective) when prevalence of current infection was extremely low (e.g., approximately zero). The ICER of the reverse algorithm was most costly when prevalence of current infection was low (i.e., <5%) but declined to $65,064 when prevalence reached 21%.

The ICER of the reverse relative to the traditional algorithm ranged from $186,048 to $894,914 when the prevalence of prior syphilis infection was increased from 0 to 21%, respectively. When the sensitivity of RPR was increased from 77 to 100%, the ICER of the reverse algorithm also increased from $232,433 to $602,436. As the annual cost of untreated syphilis increased from $0 to $600, the ICER of the reverse algorithm decreased from $306,817 to $246,817.

Multiple univariate sensitivity analyses are summarized by a tornado diagram that demonstrates the relative impact of uncertainty in individual variables on the ICER (see supplementary Table 1 and supplementary Figure 1). Included in this diagram are results from univariate sensitivity analyses of all parameters. According to this analysis, the broad range of possible prevalence of prior syphilis infection had the largest effect on the ICER.

Probabilistic sensitivity analysis

When all parameters were varied simultaneously in 1000 second-order Monte Carlo simulations at a willingness-to-pay threshold of $100,000/QALY, the traditional screening algorithm was the optimal approach in 59.8% of iterations, while the reverse algorithm was favored in 40.2% (Figure 1). As the willingness-to-pay threshold increased to $150,000, the traditional approach was favored in 55.5% of iterations while the reverse was favored in 44.5%.

Figure 1.

Probabilistic sensitivity analysis comparing cost effectiveness of the traditional and reverse syphilis screening algorithms for annual syphilis screening in persons living with HIV at a willingness to pay of $100,000.

Discussion

In this analysis, we analyzed the cost effectiveness of two syphilis screening algorithms used for annual screening in PLWH engaged in care and found that the traditional algorithm is less costly and similar in effectiveness to the reverse screening algorithm. These results were robust when accounting for variations in several parameters including prior and current syphilis prevalence, test characteristics, and cost scenarios. The incremental cost of using the reverse algorithm is $300,817 more per QALY when compared to the older traditional algorithm. This additional cost is significant for our health system, which serves 3304 PLWH and has high annual syphilis screening rates as reported previously based on data obtained in our clinic from 2013 to 2015.⁷

The reverse syphilis screening algorithm became popular in the last five years due to its ease of use and reliance on the highly-sensitive EIA test. This algorithm has been widely adopted for the screening of PLWH in the absence of comparative or cost effectiveness research in PLWH.⁵ Our results demonstrate that the reverse algorithm is more costly for the annual screening of PLWH in settings with both low and high syphilis prevalence. Furthermore, additional costs for the reverse relative to the traditional algorithm reach almost $900,000/QALY when the prevalence of prior infections approaches 21%, a scenario that is plausible for high-risk PLWH.¹⁶ As RPR test sensitivity increases, as is seen in secondary and early latent infections,¹⁵ the incremental cost of the reverse algorithm may exceed $600,000. The reverse algorithm may lead to significant costs in PLWH and others with a high prevalence of syphilis, high syphilis EIA seroprevalence due to prior syphilis infections, and many secondary and early latent infections diagnosed at screening.^14,19

By its nature, this analysis required certain assumptions, and these limitations are worth noting. First, our analysis demonstrates cost effectiveness is highly sensitive to the prevalence of prior syphilis infection. Our analysis assumed a prior syphilis prevalence of 3.4% (which was increased to 21% in sensitivity analysis),^11,16 but, in reality, this number may be higher. In one study of herpes simplex virus 2 (HSV2) seropositive individuals receiving care in our HIV clinic, 26% reported a history of syphilis.²⁰ While univariate sensitivity analysis suggests that the traditional screening algorithm always results in greater effectiveness per QALY, this effect is largely diminished when the prevalence of prior infection is low (i.e., in low-risk populations), a scenario which is unlikely in HIV infected populations. Second, there are limited data on the cost of untreated syphilis, an important component of strategy selection. The natural history of syphilis is long and indolent, and although patients with late complications such as neurosyphilis or aortitis would likely incur significant costs, this screening model applies only to those without symptoms (i.e., screening rather than diagnostic), engaged in care, in a clinic with high rates of syphilis screening. The range of costs for untreated syphilis, therefore, excludes the costs of late, advanced syphilis. Costs associated with laboratory staff time and overhead were not available at our clinic or the literature; thus, we have assumed costs based on fees. Test characteristics vary by vendor, syphilis prevalence, and the stage of syphilis. We assumed excellent test characteristics for the EIA based on data provided by our vendor; however, there are reports that the EIA is actually less sensitive than the RPR and VDRL in high-risk cohorts, including PLWH.^21–23 In scenarios when EIA sensitivity or specificity is below 77 or 94%, respectively, the above model does not apply, as these values were outside the bounds of the sensitivity analyses. We did not incorporate risk of transmission in the case of false-negative tests. As annual syphilis screening rates in our clinic and others are high, we assumed that a missed diagnosis would be detected within 12 months at subsequent annual screening.⁸ Finally, the optimal frequency of syphilis screening in high-risk populations, such as men who have sex with men, is not well defined.⁴ Our model assumes annual screening only based on current clinical practice.^7,9

Today’s economic climate requires that providers and health systems opt for cost conscious diagnostic and treatment services, but some services are still quickly adopted even in the absence of cost effectiveness research justifying their use. Although the reverse syphilis algorithm is more efficient and sensitive than the traditional algorithm,⁶ it may lead to exorbitant costs for health systems serving uninsured and underinsured PLWH needing annual syphilis screening.¹ Alternatively, the traditional screening algorithm is cost saving under a range of scenarios.

Syphilis screening is essential in PLWH to prevent further syphilis and HIV transmission and syphilis-related morbidity. Providers, administrators, and health systems should be aware that added sensitivity comes at a cost when selecting the revised syphilis screening algorithm, especially for PLWH in need of frequent screening.