Assessment of the Cost-Effectiveness of a Brief Video Intervention for Sexually Transmitted Disease Prevention

Abstract

Background:

Cost-effective, scalable interventions are needed to address high rates of sexually transmitted diseases (STDs) in the United States. Safe in the City, a 23-minute video intervention designed for STD clinic waiting rooms, effectively reduced new infections among STD clinic clients. A cost-effectiveness analysis of this type of intervention could inform whether it should be replicated.

Methods:

The cost-effectiveness of a brief video intervention was calculated under a baseline scenario in which this type of intervention was expanded to a larger patient population. Alternative scenarios included expanding the intervention over a longer period or to more clinics, including HIV prevention benefits, and operating the intervention part time. Program costs, net costs per STD case averted, and the discounted net cost of the intervention were calculated from a health sector perspective across the scenarios. Monte Carlo simulations were used to calculate 95% confidence intervals surrounding the cost-effectiveness measures.

Results:

The net cost per case averted was $75 in the baseline scenario. The net cost of the intervention was $108,015, and most of the alternative scenarios found that the intervention was cost saving compared with usual care.

Conclusions:

Single session, video-based interventions can be highly cost effective when implemented at scale. Updated video-based interventions that account for the changing STD landscape in the United States could play an important role in addressing the recent increases in infections.

Sexually transmitted diseases (STDs) remain a considerable public health concern, with approximately 20 million annual incident infections and a lifetime estimated direct medical cost of $15.6 billion.^1,2 Although effective behavioral interventions exist to reduce the risk of acquiring STDs, many are complex (e.g., require multiple sessions and are too time-consuming) or expensive in high-volume clinical settings. Brief interventions that can be integrated easily into routine practice are desirable. Patients visiting STD clinics often spend more than 20 minutes in the waiting room,³ thereby providing a unique opportunity to provide health messages via interventions, such as video, at a low cost.

Safe in the City (SITC) was a 23-minute theory-based video intervention that displayed young couples negotiating proper condom use. Posters in the waiting room directed attention to the video and reinforced key messages. The effectiveness of the video was assessed among patients visiting a participating STD clinic in Denver, Long Beach, or San Francisco, from December 2003 to August 2005. A controlled trial design was used where clinics alternated the intervention and control condition in 4-week intervals (i.e., one 4-week period of the intervention, followed by one 4-week period of the control) for a period of 20 months. Participants in the control group received the standard waiting room experience without videos or posters, but condoms and educational pamphlets on STD prevention were made available to patients in both the treatment and control groups. The trial demonstrated that a short video with targeted prevention messages in a waiting room setting effectively reduced new infections after the initial clinic visit.⁴ The intervention was later disseminated by the Centers for Disease Control and Prevention’s Diffusion of Effectiveness Behavioral Interventions project and was made available for adoption by publicly funded STD clinics across the United States.⁵ Our previous findings estimated the program costs of the SITC intervention from the original trial.⁶ Because most costs of the video production were incurred upfront,⁶ expanding this type of intervention to more sites nationwide could prove to be cost-effective. In this analysis, we use evidence from SITC to estimate the program costs and benefits of developing a video-based intervention and implementing it in more clinics with a larger patient population.

MATERIALS AND METHODS

Cost-effectiveness was examined from a health sector perspective. We calculated direct program costs and averted lifetime direct medical expenditures; however, we omitted indirect costs and benefits of the program. Calculated intervention costs and effects were in addition to the costs/effects of usual STD clinic care with no video intervention. Data on intervention effectiveness were drawn from the original trial of SITC, details of which are described elsewhere.⁴ Incident STDs in patients were measured using clinic diagnoses or reported cases for chlamydia, gonorrhea, syphilis, trichomoniasis, and HIV. During a mean of 14.8 months of observation, 2418 new laboratory-confirmed STDs were diagnosed for 2042 patients. Across all infection types, the intervention was found to reduce the number of new infections in patients attending any of the 3 STD clinics by 9% overall among those receiving versus not receiving the intervention.⁴

We then applied the intervention effect observed in the trial to a larger patient population represented in 40 STD clinics across 12 geographic areas in the United States from 2010 to 2011. These clinics, which make up the Sexually Transmitted Disease Surveillance Network (SSuN), receive support from the Centers for Disease Control and Prevention to collect enhanced surveillance data. Sexually Transmitted Disease Surveillance Network sites recorded 608,536 clinic visits from 363,607 unique patients and diagnosed 212,765 infections from 2010 to 2011.⁷ In our baseline scenario, we assumed that each patient received the video intervention during his/her index visit and that infections decreased by 9% during subsequent visits. The previous study found that 40% of the clinic visits in the SSuN data were repeat visits. To estimate the number of cases diagnosed during these visits, we assumed the same decrease in positivity from index to repeat visit as was observed in Warner et al.⁴: 16% of patients tested positive for at least one infection during the index visit and 6% tested positive during repeat visits (in the control group). Thus, we assumed that positivity from repeat visits in the SSuN data was 38% of positivity from index visits. Given the proportion of visits in the SSuN data that were repeat visits, we applied the intervention effect to 20% of the 80,261 cases of chlamydia, gonorrhea, syphilis, and trichomoniasis. Cases classified as late syphilis were omitted, as they were most likely acquired before the index visit. Although they were included in the original trial, HIV cases were excluded from most scenarios because of the small number of diagnoses during the intervention period and complexities in assigning benefit values to averted cases. However, as a robustness check, we calculated cost-effectiveness with HIV also included in the analysis.

Next, we used lifetime medical cost data,² adjusted to 2017 dollars using the medical care portion of the consumer price index, to associate averted cases with avoided direct medical expenditures. Costs were aggregated over infection type, by sex, to calculate the overall costs averted from the intervention.

To derive program costs, we first adjusted for inflation the program cost estimates from previous findings into 2017 dollars.⁶ We assumed web-based distribution of video content for all clinics, as opposed to DVDs, which were used in the initial dissemination. Video production costs were fixed, but overall program costs varied by the number of clinics and the length (in years) of the intervention. The costs of televisions, their installation, and video operation labor varied by the number of clinics running the intervention. The cost of video web-hosting and video operation labor varied by the length of the intervention.

Total program costs were calculated by adding fixed program costs to variable costs. For example, to estimate the total cost when operating the intervention in 40 clinics over the span of 1 year, we carried out the following calculation:

$$\text{Total Program Cost}=\text{Fixed Costs}+40*\left(\text{TV Installation Cost}\right)+40*1*\left(\text{Video Operation Cost}+\text{TV Depreciation Cost}+12*\text{Monthly Web Hosting Cost}\right)$$

Cost-Effectiveness

We estimated costs and benefits of implementing the video intervention in the 40 SSuN clinics for 1 year. Pathela et al.⁷ did not report on the timing of index and repeat visits, so we assumed that all index visits occur during the first year, and we therefore calculate program costs as if it operated continuously for 1 year. We calculated 2 measures of cost-effectiveness: the net cost of the intervention and the net cost per case averted. Cases averted were estimated by multiplying the estimated number of repeat visit cases of chlamydia, gonorrhea, syphilis, and trichomoniasis in the SSuN data by the same 9% intervention effect size. Averted costs were estimated using data on lifetime health costs per case and the estimated number of cases averted. The net cost of the program was calculated as the difference between the program cost and averted costs. Negative net costs indicate that the intervention was cost saving. Dividing net costs by estimated cases averted yielded the cost per case averted.

Probabilistic Sensitivity Analysis

We performed a Monte Carlo simulation analysis to account for uncertainty in the model parameters and to derive 95% confidence intervals surrounding our cost-effectiveness estimates. For most model inputs, we took 10,000 random draws from Beta-PERT distributions defined by the parameter means and ranges in Table 1.⁹ For the intervention effect parameters, draws were taken from a normal distribution fit using the mean and 95% confidence interval.⁴ For example, the normal distribution for the overall effect size used a mean of 0.09 and an SD of 0.04 [(mean − lower bound of range)/1.96], so that 95% of the draws from the distribution lay within the 95% confidence interval. The values from each draw were then used to calculate the cost-effectiveness measures. The lowest and highest 2.5% of these calculated measures form the lower and upper confidence interval bounds, respectively.

TABLE 1.

Base Estimates and Ranges for Health Outcome and Cost Parameters Used in the Cost-Effectiveness Analysis

Parameters	Base Estimate	Range	Reference
Health outcomes
Intervention effect 1 (all patients)	0.09	0.01–0.16	Warner et al.4
Intervention effect 2 (male patients only)	0.13	0.04–0.22
SSuN repeat visit cases
Gonorrhea (GC)	4526		Pathela et al.7*
Chlamydia (CT)	8314
Syphilis	1234
Trichomoniasis	1977
HIV	466
Averted health costs per infection, $
Male GC	97	49–146	Owusu-Edusei et al.2
Female GC	433	217–650
Male CT	37	18–55
Female CT	446	223–668
Syphilis	868	435–1,302
Trichomoniasis	27	13–40
HIV	56,727†	11,345–464,753	Schackman et al.8
Fixed program costs, $
Video production, labor	187,937	140,952–234,921	Gift et al.6
Video production, other	186,872	140,154–233,590
Web-based distribution, labor	1901	1426–2377
Variable program costs, $
Video operation labor (per clinic per year)	1022	766–1277	Gift et al.6
Web hosting (per month)	200	150–250
TV installation (per clinic)	257	193–321
TV depreciation (per clinic per year)	62	32–375	Study estimate

Seventy-six percent of gonorrhea cases and 60% of chlamydia cases in the SSuN data were among men. Costs were inflated to 2017 dollars. The intervention effect is the reduction in STI incidence in patients receiving the intervention versus those not receiving the intervention in the original study; the range for the intervention effect size came from the reported 95% confidence intervals,⁴ where the 9% effect corresponds to the hazard ratio of 0.91. Ranges for lifetime health costs were derived from cited articles. The lower bound for TV depreciation assumed a 25% lower TV price and 7-year depreciation. The upper bound for TV depreciation assumed a 25% higher TV price and full depreciation. All other program cost ranges were calculated as ±25% of the base estimate. In the Monte Carlo simulation analysis, uncertainty in the intervention effect parameters was modeled using a normal distribution, and a Beta-PERT distribution was used for all other model inputs.

^*Repeat visit cases were approximated based on Pathela et al.⁷ as described in text.

^†For HIV, the baseline cost estimate assumed a 5-year delay in acquisition. The lower bound was calculated for a 1-year delay in acquisition (1/5 the cost of a 5-year delay), and the upper bound assumed lifelong prevention.

Alternative Scenarios

The assumptions used for our baseline estimates may have significant impacts on the cost-effectiveness results. To better understand the potential impacts of the video intervention, we assessed cost-effectiveness under 5 alternative scenarios. Two of these scenarios expanded the reach of the intervention. The first assumed the intervention was implemented in twice as many (80) STD clinics for the same length of time. This scenario implicitly assumed that the patient population in the 40 additional clinics was identical to those in the baseline intervention, so expanding the intervention led to a linear increase in cases and costs averted. The second scenario doubled the length of time the intervention operated, from 1 to 2 years. Again, we assumed that cases, and therefore costs, averted increased linearly with the length of the intervention. Although these assumptions may be violated for reasons discussed in the limitations section, this exercise allowed us to estimate the relative returns to scale of expanding the intervention across clinics and time.

One potentially large benefit of the intervention omitted from the baseline analysis was the prevention of cases of HIV. The original SITC study found differences in diagnoses of HIV in the treatment (4 cases) and control groups (10 cases), but the overall number of cases was small, and the difference was not statistically significant (P = 0.12). Recent efforts to increase viral suppression among those living with HIV and use pre-exposure prophylaxis to prevent HIV acquisition.¹⁰ Wide adoption of these medications might decrease the effectiveness of video interventions for preventing HIV because they can prevent transmission in the absence of condom use. Furthermore, because HIV is not curable like the other 4 STDs included in our analysis, the effect of the intervention might be more appropriately measured in terms of delayed HIV acquisition. As a robustness check, we estimated the cost-effectiveness of the 9% intervention effect with an assumed 5-year delay of HIV for 9% of repeat cases, based on an estimated benefit of delayed acquisition.⁸ The benefits were calculated as the difference in discounted lifetime medical spending between HIV-infected and HIV-uninfected individuals who are at high risk of infection. The estimated costs averted from delaying acquisition of HIV by 5 years was $56,727 in 2017 dollars. Lower and upper bounds of this cost were derived assuming a 1-year delay and lifelong prevention, respectively (Table 1). Increased use of costly pre-exposure prophylaxis by high-risk uninfected individuals has decreased the difference in medical expenditures between these 2 groups, which therefore lowers the estimated benefit of HIV prevention or delayed acquisition.

Our next alternative scenario estimated the cost-effectiveness of the video intervention, assuming that it only had an effect on male cases. HIV was again omitted from this scenario. Warner et al.⁴ found a larger effect in male (13%) than in female patients (not significantly different from zero). The patient volume data from the SSuN study do not provide the number of cases by sex for syphilis and trichomoniasis, so overall cases were allocated using the proportion of male and female patients in the SSuN data and male-to-female ratios of rates (syphilis) or prevalence (trichomoniasis).^11,12 Playing the video continuously for long periods may be distracting for clinic workers.⁵ As a final robustness check, we estimated the cost-effectiveness of alternative scenarios in which the intervention runs only 50% of the time for 2 years. In this scenario, video operation labor costs and estimated cases averted per year were halved.

RESULTS

A cumulative 80,261 chlamydia, gonorrhea, syphilis, and trichomoniasis infections were diagnosed in SSuN clinics from 2010 to 2011. Table 1 presents the estimated number of cases diagnosed during repeat visits by disease type, averted health cost inputs, and components of the program cost.

Table 2 presents results from the baseline and alternative scenarios. In the baseline scenario, the video intervention operated continuously in 40 clinics for 1 year. Total program costs were $432,742, and an estimated 1445 cases were averted. Costs saved from the averted cases amounted to $324,728, generating a net cost of $108,015 for the baseline scenario. However, because of uncertainty surrounding the intervention effect size and the lifetime medical costs associated with each infection, the 95% confidence intervals on averted costs, and therefore net cost, are wide. Specifically, we found that the 95% confidence interval for net cost ranges from −$194,894 to $403,751. The negative lower bound here indicates that the program is potentially cost saving. The net cost per case averted in the baseline scenario is $75 (95% confidence interval, −$91 to $1328).

TABLE 2.

Cost-Effectiveness of the Video Intervention: Baseline and Alternative Scenarios

	Program Cost, $	Cases Averted	Costs Averted, $	Net Cost, $
Baseline scenario
40 clinics, 1 y	432,742 (389,227–480,300)	1445 (157–2712)	324,728 (35,089–626,536)	108,015 (−194,894 to 403,751)
Alternative scenarios
80 clinics, 1 y	486,375 (442,261–538,457)	2889 (313–5425)	649,455 (70,178–1,253,071)	−163,080 (−761,200 to 422,553)
40 clinics, 2 y	479,983 (432,833–528,040)	2889 (313–5425)	649,455 (70,178–1,253,071)	−169,473 (−770,708 to 412,706)
HIV included, 40 clinics, 1 y	432,742 (389,227–480,300)	1487 (162–2791)	2,703,858 (314,388–10,209,134)	−2,271,116 (−9,780,644 to 132,523)
50% run, 40 clinics, 2 y	439,118 (394,429–485,810)	1445 (157–2712)	324,728 (35,089–626,536)	114,391 (−189,385 to 408,752)
13% effect on men only, 40 clinics, 1 y	432,742 (389,227–480,300)	1287 (400–2171)	190,784 (57,258–341,585)	241,958 (−86,177 to 386,960)

Costs and outcomes in each scenario are compared with having no intervention (usual care). The means for each output were calculated using the base estimates from Table 1. Ninety-five percent confidence intervals, representing the lower and upper 2.5 percentiles of the simulated data, are in parentheses. Negative net costs indicate that the scenario is cost saving. A 50% run indicates that the intervention was only operating 50% of the time over the intervention period. Results in the last row used a 13% effect size applied only to men and assumed no female effect, whereas results in the other rows applied the 9% effect size to all patients.

The uncertainty in the baseline analysis is further illustrated in Figure 1. The figure shows combinations of costs averted and program costs from each of 1000 random draws from the Monte Carlo simulation. The simulated range in costs averted is much wider than the range in the program costs, largely due to relatively high uncertainty in the effect size estimate. The dotted line shows where costs averted were equal to program costs. Above this line, costs averted are higher than program costs, and the net cost is therefore negative (cost saving). Most points lie below this region, but the few points above the dotted line illustrate why the 95% confidence intervals have negative lower bounds.

Figure 1.

Scatterplot of simulated program costs and costs averted for baseline scenario. Each dot represents 1 of 1000 random draws from the simulated data. Costs averted and program costs are with respect to having no intervention (usual care). The dotted line shows the threshold where costs averted = program costs. Values above the dotted line have a negative net cost (program costs − costs averted < 0). These values are cost saving and therefore dominate usual care. However, values below the threshold may still be considered cost-effective.

The first 2 alternative scenarios expanded the reach of the intervention by doubling the number of clinics (80 clinics for 1 year) or doubling the length of the intervention (40 clinics for 2 years). Incremental program costs (costs in addition to the baseline scenario) were very similar between these scenarios at approximately $54,000 and $47,000, respectively. The incremental costs of expanding the program, either in length or in the number of clinics, are relatively small compared with the upfront program costs. Because we assumed that costs averted grew linearly in both of these scenarios, the incremental cases and costs averted were equivalent, so the program costs drove the small differences in incremental net cost for these 2 scenarios. The negative net costs for these 2 scenarios demonstrate that the video intervention is likely to be cost saving as it is expanded.

Table 2 presents results from 3 additional scenarios using some alternative assumptions. First, benefits from averted HIV costs were considered. Because the costs of HIVare high, its inclusion dramatically increased the estimated costs averted, even when assuming delayed, instead of permanently prevented, HIVacquisition. The costs averted from running the intervention in 40 clinics for 1 year are over $2.3 million greater than in the baseline scenario, and the net costs are large and negative because program costs are unchanged. The next scenario assumes that the intervention only runs 50% of the time over the 2-year period. Program and net costs do not change substantially when using this assumption. The final scenario uses the 13% effect size found for male patients in the original study and assumes no effect for female patients. In this scenario, program costs are unchanged, but we no longer calculate costs averted from female cases. The net cost in this scenario, assuming a 40 clinic run continuously for 1 year, increases by approximately $134,000.

DISCUSSION

This analysis demonstrates that brief video-based interventions can be cost-effective tools to prevent new infections in STD clinic settings. Our findings provide further support for the development of such brief interventions, especially in settings where provider time is limited or patients are not motivated to attend longer, more comprehensive STD prevention programs. The estimated net cost is positive in the baseline scenario; however, net cost was negative (cost saving) in 3 of the alternative scenarios, and the 95% confidence intervals contained zero for all net cost estimates. The wide confidence intervals are primarily due to the large range of the estimated effect sizes. For example, the lower bound on the 9% effect was only 1%. With a small intervention effect, even extending the length of the program or the number of clinics treated does not lead to positive net benefits because the variable program costs outweigh the small additional benefits. For example, in the 40 clinic, 2-year scenario, an effect size of approximately 7% would balance program costs and benefits, leading to zero net costs. In the HIV scenario, however, the intervention would be cost saving even at 1.5% effectiveness.

Estimated program costs did not vary substantially between the baseline and alternative scenarios. This reflects the fact that the added cost of expanding the program to additional clinics or extending the length of the intervention is negligible relative to the fixed costs of the program, that is, the video production costs. Therefore, video interventions are more cost-effective if they are broadly adopted and implemented. Interestingly, the variable costs of expanding the number of clinics and extending the length of the intervention are very similar.

The SSuN data included 363,607 index clinic visits, so the baseline program cost per visit was just $1.19. For comparison, this cost was slightly higher than previous condom distribution interventions, which have been estimated to cost $0.17 to $0.26 per condom distributed, in 2017 dollars.^13,14 However the costs and benefits of these intervention are not directly comparable because condoms may still have been used in the absence of the distribution programs, multiple condoms may be distributed to a single person, and the SITC intervention had the added benefit of teaching and promoting proper condom usage. The low cost per patient visit demonstrates that video interventions can inexpensively reach patients and quickly be scaled up. Similar evidence on behavioral interventions broadly focused on STD and HIV prevention is scarce, but Project RESPECT, an evaluation of individually focused counseling interventions, found that behavioral counseling was effective at increasing condom use and reducing STDs.¹⁵ The incremental societal cost per STD case prevented of 2 counseling sessions, compared with usual care, was found to be $132 (adjusted to 2017 dollars).^16,17 This is considerably higher than the $75 net cost we find for the baseline intervention, although the 2-session intervention was still considered to be cost-effective compared with usual care. Interventions such as RESPECT also require substantially more clinic staff effort than an intervention such as SITC.

Our analysis attempted not to overestimate the benefits of the intervention. The baseline results in Table 2 excluded benefits from HIV reductions, but the estimated net benefits of the intervention increase dramatically when HIV costs are considered. Lifetime medical costs of gonorrhea and chlamydia are higher in female than in male patients because of their associated sequelae.² As a sensitivity analysis, we estimate the cost-effectiveness of the video intervention when assuming an effect on male patients only.

Our analysis is subject to several limitations. First, the analysis was conducted from a health sector perspective instead of a societal perspective. This omits potential nonmonetary benefits of the intervention, such as quality of life improvements from averted infections. Omitted intervention costs, such as patient transportation and time spent in the waiting rooms, were minimal because patients would already spend time in the clinics in the absence of the intervention. Late syphilis cases were omitted from the analysis, but cases classified as “unknown latent” and “unspecified/other” were included. Some of these cases may not have been acquired between the index and follow-up visits. Furthermore, the potential benefits of prevented HIV cases are not included in the baseline estimates. When HIV is included in the model, averted direct medical costs of delayed acquisition are calculated, but other indirect benefits may also be important. For example, delaying infection by a year also prevents secondary transmission of HIV during that period.¹⁸ Similarly, bacterial STDs may be transmitted to partners and also may increase risk of HIV transmission,¹⁹ but these downstream benefits of STD prevention are not considered in our analysis.

Next, the costs of producing a video have likely changed since the original intervention produced nearly 2 decades ago, in ways not captured by the inflation adjustment. Furthermore, the effectiveness of a video intervention may also be different today, as patients are increasingly focused on their smartphone devices, a modality that also lends itself to video interventions. However, a recent study in HIV clinics suggests that waiting room video interventions may be effective at improving treatment initiation and achieving viral suppression in patients.²⁰ Clinics that adopt the SITC intervention may also adapt its implementation,⁵ potentially altering its effectiveness.

The effectiveness of the program is limited to the original study population and not necessarily generalizable to other samples. For example, fewer cases will be averted among a population with lower positivity. In addition, the number of cases averted may not increase linearly as the intervention is expanded. For example, increasing the number of clinics may include clinics with lower patient positivity and fewer cases. Increasing the length of the intervention may have diminishing effectiveness as the same population is repeatedly exposed to the intervention, or it may lead to finding more eligible cases as the window for observing repeat visits is extended. Finally, alternative assumptions could have been made regarding the uncertainty surrounding the input parameters. The sensitivity analysis further assumed the covariance between parameters to be zero, which affects the estimated 95% confidence intervals.

Under a variety of scenarios, developing and expanding video interventions with effectiveness similar to that of the SITC intervention to the SSuN patient population proved to be cost-effective. Updated video interventions that account for the changing STD landscape in the United States could play an important role in addressing the recent increases in STDs.