The program cost of a brief video intervention shown in STD clinic waiting rooms

Abstract

Background:

Patients in STD clinic waiting rooms represent a potential audience for delivering health messages via video-based interventions. A controlled trial at three sites found that patients exposed to one intervention, Safe in the City, had a significantly lower incidence of STDs compared with patients in the control condition. An evaluation of the intervention’s cost could help determine whether such interventions are programmatically viable.

Materials and Methods:

The cost of producing the Safe in the City intervention was estimated using study records, including logs, calendars, and contract invoices. Production costs were divided by the 1650 digital video kits initially fabricated to get an estimated cost per digital video. Clinic costs for showing the video in waiting rooms included staff time costs for equipment operation and hardware depreciation and were estimated for the 21-month study observation period retrospectively.

Results:

The intervention cost an estimated $416,966 to develop, equaling $253 per digital video disc produced. Per-site costs to show the video intervention were estimated to be $2699 during the randomized trial.

Conclusions:

The cost of producing and implementing Safe in the City intervention suggests that similar interventions could potentially be produced and made available to end users at a price that would both cover production costs and be low enough that the end users could afford them.

Summary:

A cost analysis of a video intervention found that production costs ($253 per DVD kit) plus clinic operating costs ($2699) equaled an estimated $0.46 per clinic patient.

Background

Patients visiting sexually transmitted disease (STD) clinics typically have waiting times of 20 minutes or more before being seen by a clinician (1–3). The time spent in the waiting room provides an opportunity to deliver positive health messages via video intervention. Estimating the cost and impact of developing and delivering such interventions can help determine whether the interventions are a cost-effective use of resources.

Safe in the City is a 23-minute video that features three vignettes depicting couples in situations that model decision making about safer sex practices. Between vignettes, an animated condom character demonstrates proper condom usage and depicts the different types of condoms that are available (4). In a multi-site randomized controlled trial, the intervention was shown to reduce the incidence of STDs in patients who first visited the clinics when the intervention was playing versus those who first visited the same clinics when the video was not playing (5). Clinics systematically alternated the intervention (during which the video was shown in the waiting room on continuous loop) versus the control condition (which was the standard waiting room experience) in four-week intervals (i.e., one four-week period of the intervention followed by one four-week period of the control condition throughout the study period). Incident STDs in patients in the study were measured using clinic diagnoses or reported cases for chlamydia, gonorrhea, syphilis, HIV, and trichomoniasis that were reported during the study follow-up period (5).

To assess the feasibility of developing, producing, distributing, and using interventions such as Safe in the City, we estimated the costs of developing and delivering the video intervention.

Materials and Methods

Costs associated with the intervention were categorized into production and operating costs. Production costs were those incurred during the creation, development and distribution of the intervention. Operating costs were those that would be incurred by end users as they showed the video in their facilities. Total costs per clinic were defined as the sum of the production costs (divided by the initial number of digital video kits fabricated) plus the operating costs. The intervention was released on digital video disc (DVD) Development of the intervention began in 2002. The time frame for the trial was 21 months, from December, 2003 to August, 2005 (5). Because the intervention was alternated with the control condition, the intervention was in use for half of the 21 months at each site. All costs were estimated from the program perspective and were estimated after the conclusion of the intervention. Costs related to patient time were excluded from the analysis, because the patient wait time was not affected by the intervention. Costs related to subsequent clinic visits, including testing for or treatment of incident STDs and their sequelae, were also excluded.

Production of the intervention included 12 focus groups composed of individuals representative of the target clinic population. Input from focus group participants contributed to script development and modifications to the initial version of the video. The video itself was produced under a contract. Throughout the process, there were conference calls and planning meetings concerning various aspects of the project. Post-production of the video, and fabrication of digital video and poster kits were also accomplished via contract. The dollar value of contract invoices was used to estimate the cost of services performed under contract. Incentive payments to focus group participants ($25 each) were included as a production cost. For all activities, time spent by funded staff on production activities was estimated retrospectively from electronic and paper study logs and project calendars and assessed as a production cost using representative wages for the occupational categories of the staff, including fringe benefits (6;7). Wage costs for staff were assumed to be 69.7% of total compensation costs (6). A complete breakdown of staff costs is available from the corresponding author upon request. Given that the intervention was a research activity in addition to an intervention development activity, time spent on wholly research-related activities was excluded. Intervention development and production costs were apportioned among the 1650 digital video kits fabricated. Each site using the intervention would only need one kit.

Clinics and other sites that wanted to employ the intervention also incurred costs of operation. Operating costs that were assessed included depreciation costs for hardware (a DVD player and a flat panel television, including installation) and staff costs (10 minutes per day during intervention months) to operate the equipment when showing the video. This amount of time was assumed to be adequate to cover any minor maintenance needs on the equipment that might have been required and that could have been performed by clinic staff (e.g., light cleaning or restarting the DVD player if it stopped during the day). Although staff operating the audio/visual (A/V) equipment may have varied by site and within each site over the duration of the study, we assessed the Bureau of Labor Statistics costs for Healthcare Support Workers, All Other (31–9099) for Local Government, Excluding Schools and Hospitals (999300) (7). This category of employee received average wages of $23.68 per hour in 2014 dollars. Audio/visual (A/V) equipment costs were determined by first adjusting current-dollar prices to December, 2003 levels using the Consumer Price Index for televisions, then the 2003-level A/V costs were adjusted to 2014 dollars as described below (8). This two-phase process was necessary because television prices have fallen in recent years while the overall price level has risen—thus, A/V costs in 2014 were increased to reflect higher A/V equipment costs that prevailed in 2003, then increased again to reflect the increase in the general price level between 2003 and 2014. All A/V hardware was depreciated over a 5-year time frame, though this was varied in sensitivity analysis (9).

The original number of unique patients seen in the 3 study clinics combined during the time frame of the study was 38,635 (19,073 intervention and 19,562 control) (5). Some patients visited the clinic more than once during the study period. These numbers were used for purposes of calculating intervention cost per clinic patient.

As a sensitivity analysis, all cost components in the table were varied randomly over their ranges simultaneously using a Monte Carlo simulation process. A triangular distribution was assumed for each cost component, using the baseline as the modal value (6). The simulation was repeated 10,000 times. Average values, along with 95% uncertainty intervals (derived by taking the 2.5th and 97.5th percentiles of the range of values) were calculated. For the Monte Carlo analysis, we also examined the potential impact of increasing digital video fabrication to 5000 kits. We assumed that the preproduction cost per kit cost when fabricating 5000 kits was the same as the cost when fabricating 1650 kits. The lower bound of the digital video fabrication cost was based on a model of Internet-based (Web) distribution and included 40 hours of Web developer labor plus $200 per month for Web hosting over 21 months (7;8). To estimate the costs clinics would incur if showing the video as part of normal daily operations, we calculated the annualized cost of daily use, rather than alternating intervention / control use, as was done during the study. Annualized costs were calculated by multiplying the daily costs that would have been incurred during the 21-month study period by 0.57 (12/21).

Costs were adjusted to 2014 dollars using the Consumer Price Index for All Urban Consumers (CPI-U) except as noted (9).

Results

The estimated baseline production cost of the intervention was $416,966, including fabrication of 1650 digital video kits, for an average cost of $253 per digital video kit (Table). Operating costs for each clinic were estimated to be $2699 over the 21 months of the study. The total cost for each clinic, including the digital video kit and operating costs, was $2952.

Table:

Safe in the City Intervention Costs

Description	Cost*	Sensitivity analysis range†	Source‡
Production costs
Focus group costs for participants	$5790	$4343 - $7238	Primary data
Video production and post-production (including filming, travel, editing)	$356,198	$267,149 - $445,248	Primary data, (6;7)
Fabrication costs for 1650 DVD kits	$54,977	$6036 - $68,722§	Primary data
Total production costs	$416,966
Operating costs¶
A/V hardware (including installation)	$1836	$1449║ - $3857║	(9;13)
Daily operating costs (staff time to operate equipment)	$863	$647 - $1079	Study estimate
Total operating costs (including daily costs and A/V hardware)	$2952	$2265 - $5252

Notes

^*All costs are in 2014 US dollars.

^†Unless otherwise noted, the sensitivity analysis range represents +/− 25% from the baseline cost

^‡’Primary data’ refers to data from the study; ‘study estimate’ refers to staff time estimates unsupported by data

^§The lower bound assumed Web-only distribution; the cost shown is for 40 hours of Web developer time plus $200 per month over 21 months for Web hosting and maintenance(6;7)

^¶Operating costs shown are per clinic for the 21-month time frame of the study: December, 2003 – August, 2005, alternating the intervention and control conditions (5)

^║The baseline, low, and high range for A/V hardware assumed 5-year, 7-year, and 100% depreciation, respectively (9).

During the intervention months of trial when the video was in operation, 19,073 patients visited the 3 clinics; the per-clinic intervention and digital video costs equaled $0.46 per patient.

In sensitivity analysis, setting all costs at their lowest and highest values alternately yielded operating costs of $2097 - $4936 over the study period.

Assuming that 1650 digital video kits were fabricated, the Monte Carlo analysis yielded an average production cost of $245 per digital video kit (95% uncertainty interval $176 - $307), with a total clinic cost over the 21-month study period of $3492 (95% uncertainty interval $2399 - $5100). When modeling the fabrication of 5000 digital video kits, the average production cost was $98 per kit (95% uncertainty interval $65 - $126); the total clinic cost during the study period would have been $3339 (range $2276 - $4944).

The cost per STD clinic patient over the study period when estimated via Monte Carlo analysis was an average $0.55 (95% uncertainty interval, $0.37 - $0.81). A Monte Carlo analysis of the expected annualized cost per clinic of the intervention assuming daily use yielded an average annualized total cost of $2404 (95% uncertainty interval $1679 - $3450), equaling an estimated $0.34 per patient (95% uncertainty interval, $0.23 - $0.48).

Discussion

The Safe in the City intervention cost approximately $417,000 to produce. Clinics incurred an estimated additional $2952 over the course of the study to implement the intervention. Had the clinics been delivering the intervention continuously during the trial, they each would have incurred additional costs of $3563 over 21 months, or $2036 per year. Given the relatively high patient volume in the clinics, cost for the digital video kit and for intervention delivery was relatively low on a per-patient basis. The cost to clinics was estimated to be $0.46 per patient when alternating intervention and control months, and would have been $0.34 had the video been shown daily during the study period. This cost is comparable to other low-cost primary prevention interventions such as condom distribution programs, which have been estimated to be about $0.16 - $0.25 per condom distributed in 2014 dollars (11;12). While condoms are single-use, the reduction in STD incidence over time in the Safe in the City study suggests that the impact of the intervention may have lasted for some time (5).

Although STD incidence was reduced in intervention patients compared to control patients, the intervention itself would probably not have been considered a cost-effective use of resources if the entire production cost were absorbed by the 3 clinics alone: the cost per clinic under this scenario would have been $141,688.

The relatively low production cost per DVD kit fabricated shows that there is a potential business model to produce video-based health education messages for settings such as clinic waiting rooms. The fixed production costs could be recaptured by digital video kit sales or Web distribution as long as enough end users would be available to purchase the intervention. The price that would be required to self-fund intervention development is low enough that health departments, non-profit organizations, and other similar entities could potentially find such video-based interventions to be worthwhile investments. Whether development was funded by end users or by grants, these calculations provide an estimate of what the cost per end user would be. Internet distribution and streaming has become more common and would be substantially less costly than DVD fabrication, potentially extending the reach of the messages. However, digital video kit fabrication costs only accounted for 13% of the production costs of the intervention, suggesting that most of the funding required to develop Safe in the City would still be needed to develop additional video interventions. No-cost distribution (as is currently the case with the Safe in the City intervention) would obviously require a funding mechanism other than end user purchase.

We calculated the intervention cost per STD clinic patient to illustrate the relatively modest cost of the intervention, but the results should be interpreted cautiously for two reasons. First, the intervention cost (production and operation) is largely independent of the number of persons exposed to the intervention—it takes the same amount of time to show a video to 10 people throughout the day as it does to show it to 200. Second, the number of incident STDs that could potentially be averted in persons exposed to the intervention is a function of average patient risk for infection and prevalence of STDs in the population, which will vary in different settings. A community-based organization that acquired the intervention to show twice a year to groups of 20 people with relatively little STD risk would not be expected to achieve the same health benefits (in terms of incident STDs averted) as a high-volume STD clinic showing the video in a loop every day. Even among similar sites, the operational costs associated with the intervention could differ based on individual sites’ practices. Third, the intervention impact could change by population exposed, region of the country, and other factors not assessed in the original study. It is also possible that the intervention could have achieved additional beneficial outcomes beyond incident STDs through normalizing STD screening, condom use and other safer sex behaviors, as well as providing examples of sexual health communication between partners. Additional messages could be produced targeting other areas of sexual health that would be relevant for patients and others at risk for STDs.

Although the proportion of STD cases diagnosed in STD clinics has been declining in recent years, they remain an important point of access to patients at risk for STDs (14). Delivering STD prevention messages during time that patients spend waiting to see a clinician could potentially have a beneficial impact. In any setting, the cost of the intervention, rather than the cost per unit of outcome achieved, is likely to be the most important driver of intervention adoption.

This analysis is subject to limitations; one is that the costs were estimated retrospectively, which introduces the potential for error. Another limitation is that the costs were estimated over the 21-month time frame of the intervention study period; in actual practice the life cycle of the intervention might be different (either shorter or longer, but any change would affect the cost calculations, as it would change the rate at which new interventions would need to be acquired). Differential costs that might have been experienced by patients in each condition after the initial visit were not included. It is possible that the video intervention led to different rates in subsequent visits for STD-related services between intervention and control condition patients. The number of STD tests received by patients after the initial study visit is unknown. We did not include any costs associated with STDs during the follow-up period (incurred or averted). Therefore, this cost analysis is potentially incomplete, even when considered from the perspective of the local health department program. Finally, implicit in the discussion of intervention feasibility is an assumption that additional health messages could be produced on video for a comparable cost; this may or may not be the case. Changes in technology available to health departments, which offers the potential to develop customized messaging, may augment or detract from the impact of interventions delivered in STD clinic waiting rooms or other settings.

The cost analysis of the Safe in the City intervention shows that there is a potential for interventions with positive health messages to be developed in a manner that could be self-supporting and not reliant on funding other than that which could be paid by end users of the interventions. Data on the extent of adoption and use of the Safe in the City intervention are not available, but distribution of the original digital video kits and downloads available from the project Web site [http://www.cdc.gov/std/safe-in-the-city/default.htm] suggest that a potential market exists for such interventions. The calculations shown here suggest that such interventions could be developed and released approximately every 2 years without increasing end-user costs above those estimated in this analysis. Whether that would be frequent enough to provide timely, relevant, and continuously effective messages for use in STD clinic waiting rooms and other settings is a question that is—for now—unexplored.