An Interactive Modeling Tool for Projecting the Health and Direct Medical Cost Impact of Changes in the Sexually Transmitted Diseases Prevention Program Budgets

Abstract

Context:

Estimating the return on investment for public health services, tailored to the state level, is critical for demonstrating their value and making resource allocation decisions. However, many health departments have limited staff capacity and expertise to conduct economic analyses in-house.

Program:

We developed a user-friendly, interactive Excel-based spreadsheet model that health departments can use to estimate the impact of increases or decreases in sexually transmitted infection (STI) prevention funding on the incidence and direct medical costs of chlamydia, gonorrhea, syphilis, and STI-attributable HIV infections. Users tailor results to their jurisdictions by entering the size of their population served; the number of annual STI diagnoses; their prior annual funding amount; and their anticipated new funding amount. The interface was developed using human-centered design principles, including focus groups with 15 model users to collect feedback on an earlier model version and a usability study on the prototype with 6 model users to finalize the interface.

Implementation:

The STI Prevention Allocation Consequences Estimator (“SPACE Monkey 2.0”) model will be publicly available as a free downloadable tool.

Evaluation:

In the usability testing of the prototype, participants provided overall positive feedback. They appreciated the clear interpretations, outcomes expressed as direct medical costs, functionalities to interact with the output and copy charts into external applications, visualization designs, and accessible information about the model’s assumptions and limitations. Participants provided positive responses to a 10-item usability evaluation survey regarding their experiences with the prototype.

Discussion:

Modeling tools that synthesize literature-based estimates and are developed with human-centered design principles have the potential to make evidence-based estimates of budget changes widely accessible to health departments.

Rates of sexually transmitted infections (STIs) have increased markedly, with disproportionate impacts on certain racial, ethnic, and other minority groups.¹ Annual new sexually acquired infections contribute to $15.9 billion in discounted lifetime direct medical costs,² and cases of primary and secondary syphilis, the most infectious stages, have increased 781% between 2001 and 2021.³ Several analyses have provided evidence that increases in public health funding for STI programs are associated with subsequent reductions in reported STI rates.^4-7

Past studies have documented challenges in providing STI services, given declining public health funding.^8,9 In this context, it is useful to enable public health staff to demonstrate their program’s return on investment for internal and external communications, respond to inquiries from decision-makers regarding the potential impact of changes in prevention funding allocations, and make evidence-informed decisions about allocating limited resources. However, health departments frequently lack the staff capacity and expertise to conduct economic analyses in-house.

We describe the development of a user-friendly, interactive Excel-based spreadsheet model that health departments and other partners interested in the effects of STI funding on outcomes can use to estimate the impact of changes in their state’s STI prevention funding on the incidence and direct medical costs of chlamydia, gonorrhea, syphilis, and STI-attributable HIV infections. The model is based on scientific literature including the impact of federal funding on state-level STI rates,⁴ the probability of an STI-attributable HIV infection per STI infection,¹⁰ and the lifetime direct medical costs per HIV and STI infection.^11-13 We used a human-centered design process—a development approach that incorporates users throughout the project to ensure the product is tailored to their needs^14-16—to understand the health department organizational context, elicit requirements of STI program staff, design an interface and model structure that would meet their needs and technical skills, and evaluate the prototype model’s usability.

Methods

Overview

The Centers for Disease Control and Prevention (CDC) previously developed the STI Prevention Allocation Consequences Estimator model (“SPACE Monkey 1.0”) to help health departments estimate the effect of changes in their STI prevention budgets on direct medical costs and infections averted.^17,18 (Authors T.L.G. and H.W.C. were lead authors on SPACE Monkey 1.0.) Since SPACE Monkey 1.0’s release in 2017, there has been updated scientific literature on the relationships between state-level STI prevention funding and reported chlamydia and gonorrhea diagnosis rates⁴ and the lifetime medical cost of STIs and HIV infection.^10-13 In addition to updating the scientific evidence underlying the model parameters, we improved the model’s usability to enhance use by health departments. Although SPACE Monkey 1.0 was shared with 5 users (representatives from 4 jurisdictions and 1 national organization) for early feedback prior to release, we identified an opportunity to engage more users throughout the model development process.

We revised the model in 3 steps. First, we conducted focus groups with 15 national, state, and local public health professionals regarding their experiences with SPACE Monkey 1.0. Second, we developed SPACE Monkey 2.0, incorporating updated literature-based parameters (the impact of STI prevention funding changes on reported infections, the probability of an STI-attributable HIV infection per STI infection, and the lifetime costs per HIV and STI infection), a new model structure to generate confidence intervals, and other enhancements addressing focus group feedback. Third, we conducted usability testing on the SPACE Monkey 2.0 prototype with 6 state and local public health professionals. Their feedback was integrated into the final model design.

The University at Albany Office of Regulatory Research and Compliance and human subjects officials at the CDC determined that this project did not meet the regulatory definition of human subjects research.

Focus groups on SPACE Monkey 1.0

Prior to model development, user feedback was solicited on SPACE Monkey 1.0 and another STI cost model not described here. The 15 participants comprised STI program experts from 8 state health departments, 5 local health departments, and 2 national organizations. Although the model is most suitable for federally funded jurisdictions, we included local and national representatives because the other STI cost model developed in parallel is appropriate for both state and local jurisdictions; furthermore, we envisioned that local and national audiences would find the model useful for demonstrating the value of STI prevention funding more generally. A convenience sample of participants from state and local STI programs was recruited through an e-mail to the CDC’s Strengthening STD Prevention and Control for Health Departments (STD PCHD) program Listserv, which included STD PCHD principal investigators, program directors, program managers, and surveillance coordinators. We did not impose eligibility criteria beyond working in a health department because we hoped to recruit users with varying technical skills, past model experiences, and perspectives. State and local participants were from diverse regions and settings including large and small health departments in California (1 state, 1 local), Florida (1 state, local), Idaho (1 state), Maryland (1 local), Minnesota (1 state), Missouri (1 local), Ohio (1 state), Oregon (1 local), Pennsylvania (1 state), Rhode Island (1 state), and Vermont (1 state). In addition, we included 1 representative each from the Association of State and Territorial Health Officials and the National Association of County and City Health Officials to elicit broader perspectives on how the model could be used by different jurisdictions.

For state and local participants, focus groups comprised 2 or 3 participants, with separate focus groups for state and local respondents (3 state focus groups and 2 local focus groups). We intentionally organized small groups, rather than a typical focus group size, to improve participants’ comfort with sharing ideas, a relevant consideration, given the COVID-19–related “Zoom fatigue” at the time of data collection (April and May 2022). Where possible, we organized focus groups by jurisdictions’ Census population size. The 2 national representatives were interviewed individually due to scheduling difficulties. Data collection occurred virtually by authors B.A. and E.G.M. and lasted up to 90 minutes. We took detailed notes, rather than recording the focus sessions, to help participants provide candid feedback. Prior to the session, participants completed a set of tasks with the SPACE Monkey 1.0 model to ensure a common experience interacting with the model. The guided exercise (see Supplemental Digital Content Appendix 1, available at http://links.lww.com/JPHMP/B281) directed participants to enter model inputs and review their results. Participants answered questions regarding their user experience, the relevance of results to their information needs, the model assumptions, how they could use the model in their own work, and other desired model enhancements.

The focus groups covered the following topics: participants’ prior familiarity with and use of SPACE Monkey 1.0, usability problems encountered during their guided exercise and desired usability enhancements, whether the model sufficiently captures the STI program environment, how participants envisioned using the model, and other suggestions (see Supplemental Digital Content Appendix 1, available at http://links.lww.com/JPHMP/B281).

Data from participants’ guided exercises and focus group discussions were synthesized by B.A. and E.G.M. for themes, such as specific usability problems, desired usability enhancements, and required updates to the model’s underlying assumptions.

SPACE Monkey 2.0 development

After the focus groups, we developed a new Excel-based model that incorporated the general approach from SPACE Monkey 1.0, with numerous modifications to incorporate updated scientific literature and feedback from the public health professionals. These are described in the “Results” section.

Usability testing on SPACE Monkey 2.0

After revising the model to reflect updated scientific literature and user feedback, we conducted usability testing with 6 state and local public health professionals who represented model users. All participants were STI program subject matter experts. We recruited participants with varying Excel skills, prior familiarity with the model, and geographic location (Georgia, Ohio, Pennsylvania, Rhode Island, South Carolina, and Vermont). We invited 3 participants from the focus groups who expressed an interest in pilot-testing the prototype and 3 additional individuals in our professional network who were interested in the model but had not previously interacted with it. In describing their experience with Excel, 4 participants self-identified as having a basic familiarity with Excel, 1 reporting using Excel regularly for work tasks (eg, developing budget spreadsheets or basic data analysis), and 1 reporting completing advanced work in Excel (eg, creating pivot tables, using array formulas, or creating advanced data visualizations).

The usability testing occurred as one-on-one virtual meetings with B.A. (additionally, E.G.M. attended 2 sessions), lasting 60 to 90 minutes. Participants received a usability exercise (see Supplemental Digital Content Appendix 1, available at http://links.lww.com/JPHMP/B281) and the prototype model via e-mail. During the session, participants completed a set of tasks including exploring the spreadsheet model, entering inputs, generating output, and exporting a chart into another application. While working on the tasks, they shared their screens to allow us to see how they were navigating the spreadsheets and were prompted to verbalize their thoughts using a think-aloud approach.^19,20(p365) The usability sessions ended with an open discussion of participants’ experiences and suggestions. Participants additionally completed a short 10-question online survey regarding their user experience, derived from a common usability evaluation instrument.²¹

Meeting notes and observations from the usability testing sessions were synthesized by B.A. for themes. Usability problems (eg, bugs and cosmetic issues) were addressed iteratively after each usability session, with subsequent usability study participants working with updated model versions. After the usability testing, the full study team discussed findings and made additional cosmetic and minor usability enhancements.

RESULTS

User experiences with SPACE Monkey 1.0 and desired enhancements

Most focus group participants reported awareness of the model, although there was limited prior use. The most common anticipated use case was generating findings for advocacy and education. Examples of potential model use were demonstrating the economic value of STI programs to executive leadership, providing justification for STI program revenue in budget proposals to state legislators, making a business case for continued investment in the disease intervention specialist (DIS) workforce, and motivating internal and external audiences about “the impact of our work.” Participants at all levels (national, state, and local) expressed enthusiasm about using the model.

Although the SPACE Monkey 1.0 model was perceived as easy to use, participants had suggestions for improving the user interface and underlying model assumptions. Major usability recommendations included making it easier to compare results of different analyses, including confidence intervals on the same page as the main results, adding a functionality to print input and output screens for meeting handouts, adding visualizations, simplifying the interface to a single screen rather than multiple tabs of user-defined inputs, and revising the interpretation guide for improved clarity. Participants desired clearer information about model assumptions and the calculations to understand the spreadsheet and describe it to external audiences. Ideally, such information would be included in the model rather than a separate user guide or scientific manuscript. Most participants preferred retaining the format as a downloadable Excel-based application rather than revising the model to be an interactive Web-based tool. Because it is a common software, users can save versions with their inputs for documentation, it can be used offline, and it would not elicit concerns about the privacy and storage of data entered in an online platform.

Participants expressed some concerns about SPACE Monkey 1.0’s underlying conceptual model, which incorporated 2 methods to estimate program impact: a “historical formula approach” and a “DIS approach.” The “historical formula approach” used findings from a study that assessed the relationship between state-level gonorrhea case rates and federal STI funding allocations from 1981 to 1998.⁵ The “DIS approach” used findings from a study that measured the association between DIS partner notification and gonorrhea case rates in New York State excluding New York City from 1992 to 2002.²² Participants noted several critiques: (1) the studies used to inform both approaches were published in the mid-2000s and perceived to be outdated; (2) the “DIS approach” had less face validity due to interstate variation in DIS staff structures, salaries, and activities; and (3) it was difficult for many state-level public health professionals to quantify the number of DISs and their salaries because many DISs are not state employees and their job responsibilities vary.

SPACE Monkey 2.0 model structure

For the updated SPACE Monkey 2.0 model, to address focus group participants’ concerns about the validity of the “DIS approach” and the difficulty of quantifying DIS staff and salaries, we relied exclusively on the “historical formula approach.” In response to participants’ concerns about outdated data, the updated “historical formula approach” was informed by a recent analysis of the impact of STI prevention funding on reported chlamydia and gonorrhea rates through 2016.⁴ (SPACE Monkey 1.0’s historical allocations approach cited an older analysis of data through 1998, published in 2005.⁵) The study underlying our model has been incorporated into other modeling analyses of the impact of STI prevention funding.²³

The updated SPACE Monkey 2.0 model calculates the impact of a permanent, one-time change in their STI prevention budget over 10 years using the following steps and base case values from the Table. The 10-year time frame reflects the number of years over which the number of infections averted (or additional infections) is calculated. The direct medical costs saved (or additional medical costs) are the discounted, lifetime direct medical costs of these infections, regardless of when their costs are incurred.

TABLE.

SPACE Monkey 2.0 Model Parameters

Parameter	Base Case Value	Low-Impact Scenarioa	High-impact Scenarioa
Chlamydia: Estimated number of incident infections, nationally 201824,b	3 983 000	3 401 000	4 719 000
Reported number of chlamydia cases, 201825,b	1 758 668	Not varied	Notvaried
Gonorrhea: Estimated number of incident infections, nationally 201824	1 568 000	1 236 000	2 107 000
Reported number of gonorrhea cases, nationally 201825	583 405	Not varied	Notvaried
P&S Syphilis: Estimated number of incident infections, nationally 201824,b,c	146 000	108 000	197 000
Reported number of P&S syphilis cases, nationally 201825,c	35 063	Not varied	Not varied
Probability of STI-attributable HIV infection, per chlamydial infection10	.00022	.000022	.000418
Probability of STI-attributable HIV infection, per gonococcal infection10	.00022	.000022	.000418
Probability of STI-attributable HIV infection, per syphilitic infection10	.00462	.000462	.008778
Average lifetime direct medical cost per chlamydial infection11,d,e	$190	$100	$340
Average lifetime direct medical cost per gonococcal infection11,d,e	$190	$80	$380
Average lifetime direct medical cost per syphilitic infection12,d	$1 300	$800	$2 070
Average lifetime direct medical cost per HIV infection13,d,f	$461 000	$358 000	$537 000
Impact of funding on incidence rate, chlamydia4,g,h	−0.179	0	−0.348
Impact of funding on incidence rate, gonorrhea4,g	−0.313	−0.081	−0.486
Impact of funding on incidence rate, syphilis4,g,i	−0.313	−0.081	−0.486

Abbreviations: P&S syphilis, primary and secondary syphilis; STI, sexually transmitted infection.

^aFor inputs, values correspond to the reported 95% confidence intervals from the original source unless otherwise noted. We use the terms “low-impact scenario” and “high-impact scenario” rather than lower and upper bound because the 3 parameters for the impact of funding on the 3 bacterial STIs are reversed, with lower numerical values representing a greater improvement.

^bThe reference study provided interquartile ranges (25th and 75th percentiles); the ranges we applied reflect unpublished approximations of the 95% confidence intervals (2.5th and 97.5th percentiles) obtained from the authors.

^cWe used P&S syphilis to be consistent with the user input for the number of P&S syphilis cases. This parameter is used as a denominator for calculating the jurisdiction’s share of nationally reported cases, which is then assumed to be the same as the jurisdiction’s share of estimated national syphilis incidence.²⁴

^dCosts were converted to 2022 US dollars using the medical care component of the consumer price index. Costs reflect direct medical costs only, from the health system perspective, which includes all direct medical costs without regard to who incurs these costs. The lifetime direct medical cost estimates include the possibility of incurring direct medical costs of sequelae (eg, pelvic inflammatory disease in women due to chlamydial or gonococcal infection; see the source publications referenced earlier for details). Other types of costs besides direct medical costs (eg, productivity losses due to morbidity and mortality, the value of a statistical life lost, and intangible costs of pain and suffering) were not included.

^eThe authors reported costs separately for men and women. We created a weighted average for the full population by applying the number of estimated incident infections nationally.²⁴

^fThe base value is the mean of the authors’ reported base value. The ranges are the mean results from the authors’ least and most favorable scenarios.

^gSee Supplemental Digital Content Appendix 2 (available at http://links.lww.com/JPHMP/B282) for the calculations from Williams et al⁴ to develop the model parameters. The values were inflation-adjusted using the general all-item consumer price index.

^hIn the source study, the cumulative effect of STI prevention funding was significant at the P < .10 level for chlamydia and at the P < .05 level for gonorrhea. Accordingly, the 95% confidence interval for the cumulative impact of funding on chlamydia overlapped 0. For the low-impact scenario for chlamydia, we set the funding impact value to 0, thereby assuming that in the worst-case scenario, increased STI prevention funding would have no effect rather than to worsen outcomes.

ⁱSyphilis was not included in the analysis by Williams et al.⁴ We assumed the effect of funding on gonorrhea could be used as approximation for the effect of funding on syphilis.

1. Users enter information on their jurisdiction’s number of reported cases, population size, prior funding amount, and anticipated future funding (as a one-time change).

2. The jurisdiction’s reported and unreported chlamydia, gonorrhea, and syphilis infections are estimated by multiplying the user-defined reported cases by an adjustment factor (ratio of the estimated incident infections nationally²⁴ divided by reported infections nationally²⁵).

3. The anticipated rate of chlamydia, gonorrhea, and syphilis infections under the new funding amount is calculated on the basis of extrapolated results from the Williams et al⁴ historical formula analysis to estimate the impact of a change in the jurisdiction’s STI prevention federal budget. (See Supplemental Digital Content Appendix 2, available at http://links.lww.com/JPHMP/B282, for details on the extrapolation.) The historical formula analysis⁴ estimated a cumulative impact of prevention funding on reported STI rates over a 3-year period. In applying these findings to the SPACE Monkey 2.0 model, we use a simplification that the impact (based on the “cumulative effect” coefficient from Williams et al⁴) is the same for years 3 through 10. In our application of the Williams et al⁴ analysis, the impact of a budget change does not peak until the third year, when the “cumulative effect” is achieved. We assume the impacts in year 1 and year 2 after the budget change are one-third and two-thirds of the “cumulative effect” coefficient, respectively.

4. The number of chlamydia, gonorrhea, and syphilis infections before and after the change in prevention funding (from steps 2 and 3) is compared to calculate the anticipated number of averted or additional infections due to the annual funding change.

5. The anticipated number of averted or additional STI-attributable HIV infections is calculated by multiplying the anticipated changes in the number of STI infections (from step 4) by published estimates of the probability of an STI-attributable HIV infection per STI infection.¹⁰

6. The number of averted or additional infections due to the funding change (from steps 4 and 5) is multiplied by published estimates of lifetime direct medical costs per infection of STIs^11,12 and HIV infection¹³ to estimate total costs saved by averted infections (for funding increases) or additional costs incurred because of increased infections (for funding decreases).

Confidence intervals were generated on the basis of the “high-impact scenario” and “low-impact scenario” values for parameters in the Table. The estimated impact of STI funding changes is more pronounced when applying the “high-impact scenario” funding parameter values and less pronounced when applying the “low-impact scenario” parameter values. See Supplemental Digital Content Appendix 2 (available at http://links.lww.com/JPHMP/B282) for additional details including confidence interval calculations and comparison of results between the SPACE Monkey 1.0 and 2.0 models.

SPACE Monkey 2.0 user interface

Feedback from the focus groups was incorporated into the final model design. A colorful landing page provides information on the model’s purpose and troubleshooting tips for common downloading errors (Figure 1). We retained the SPACE Monkey 1.0 logo for consistent branding with the original tool.

FIGURE 1. SPACE Monkey 2.0 Landing Page^a.

Abbreviation: STI, sexually transmitted infection.

^aUpon opening the Excel model, users see this greeting page with an overview of the model and troubleshooting instructions for common problems with the download.

After hitting the “Click to Begin!” button, users are directed to the main user interface (Figure 2). To address participants’ desires for a simpler interface, we included all required inputs and outputs on the same page. This allows users to enter their inputs on the left (“Program inputs”) and immediately see the changes in the main output tables. The output tables include dynamic text interpretations that automatically adjust as the user input values change. The output tables have embedded conditional coding so that outcomes of funding increases are labeled as “infections averted” and “costs saved” and outcomes of funding decreases are labeled as “additional infections” and “additional costs.” A button at the bottom of the page allows users to print the screen as a handout. For this main page, we decided to present a 3-year impact as the default because it was the most consistent with the original historical formula analysis that estimated impact⁴ and a common time horizon for presenting budget analyses to policymakers. All cells on the main page of the Excel spreadsheet are locked except for the user inputs to avoid accidental manipulations of the pivot table and automated text interpretation.

FIGURE 2. Main Page for User Inputs and Key Findings^a.

^aThe program inputs are based on a hypothetical jurisdiction, for illustration purposes. The “i” icons prompt users to click on the relevant cells for a pop-up box describing the inputs in more detail. The interpretation text is dynamic, with the numbers changing based on the user-defined scenario. If users model a scenario of an annual funding decrease, the display automatically changes to show findings in terms of “additional infections” and “additional costs,” and the interpretation text automatically adjusts to discuss a “decrease” in annual funding. Users can click on the “Print” button to create a PDF or hard copy printout of the page. The “Advanced Options” button takes users to a different page where they can view additional tables and charts. The colorful arrows on the right provide snippets of instructions to catch the users’ eye and make explanations easier to follow.

The “Advanced Options” button directs users to the subsequent screen with more detailed output and charts (Figure 3). Consistent with the prior screens, we used the Viridis color palette,²⁶ which is visually appealing, colorblindness compliant, and can be printed in grayscale. On this screen, users can adjust the time frame (from 1 to 10 years). For each infection, there is a table and chart displaying infections averted and direct medical costs saved (for budget increases) or additional infections and additional direct medical costs (for budget decreases). The pivot tables and pivot charts have a dynamic feature based on Excel’s Slicer tool that allow users to quickly change the number of years included in their display. This sheet is unlocked so that users can copy and paste tables and charts to different applications.

FIGURE 3. Advanced Options Page^a.

Abbreviation: STI, sexually transmitted infection

^aUsers can scroll down to see additional charts for gonorrhea, syphilis, STI-attributable HIV infections, and total STI infections excluding HIV infection. Users can modify the time frame for the number of years over which the number of infections averted (or additional infections) is calculated by entering a different number from 1 to 10 in the box on the top left. Users can change the years highlighted in the filter tool (bottom left) to change the data points displayed on the table and chart. All tables and charts can be copied and pasted into an external application such as PowerPoint.

We also incorporated users’ requests for clear explanations of the model. A “Model Overview” tab (not shown) provides users with a nontechnical description of the model including a short summary that users can adapt, an overview of the calculations, key assumptions and limitations, and references. A “Model Parameters” tab provides all values, citations, and other notes for each parameter listed in the Table.

Usability evaluation of SPACE Monkey 2.0

Overall, participants provided positive feedback during the usability testing. They found the model to be easy to navigate and perceived that it was suitable for their intended use. They expressed appreciation for multiple functionalities that we added to the updated model: dynamic charts and tables that could be manipulated and copied into external applications, detailed yearly output broken down by infection, clear text for the model overview and interpretation, and having confidence intervals.

Several usability problems emerged during the evaluation, including model bugs, issues with enabling macros, accidental deletion of charts in the “Advanced Options” page, occasional confusion on interpretation, and not reviewing the instructions. To address these issues, we resolved the bugs, added instructions for enabling macros, added cautionary notes in the “Advanced Options” page to warn users about deleting charts, added “i” icons (“information”) to prompt users to click on the boxes for pop-up explanations of the program inputs (see Figure 2, top left section), revised the instructions to be in bright yellow arrows with smaller snippets of text to catch model users’ attention, added new charts and tables, and edited instructions.

Participants’ responses to the usability survey (Figure 4) mirrored their positive comments during the usability evaluation sessions. Participants had general agreement with statements such as “I would like to use this system frequently” and “I thought the system was easy to use” and general disagreement with statements such as “I thought there was too much inconsistency in this system” and “I found the system very cumbersome to use.”

FIGURE 4. Usability Testing Survey Results^a.

^aN = 6 participants.

Discussion

Through our human-centered design process, our revised SPACE Monkey 2.0 model incorporates updated scientific literature and has a customized interface and functionalities to meet end users’ information needs and technical skills. We plan to post the model to a public Web site for free download and implement a comprehensive dissemination campaign to promote the model. This campaign will include videos and digital presentations to train users and promote use, webinars to diverse audiences in collaboration with professional organizations, and targeted outreach to end users who we hope can become champions to encourage use among their peers.

SPACE Monkey 2.0 has several limitations. First, it cannot be used for outcomes other than chlamydia, gonorrhea, syphilis, or STI-attributable HIV infections. Second, it does not calculate the impact of funding on congenital syphilis because those infections were not in the scope of the statistical analysis⁴ that was the basis for the model. Third, the model includes only direct medical costs (eg, productivity and other types of costs are excluded) and the estimated costs saved or averted are not disaggregated by payer (eg, costs saved by a county health department vs health insurance). Fourth, the model is most suitable for small to medium relative changes in annual program funding (less than $2 per capita) to federally funded jurisdictions. Fifth, the historical allocations study on which SPACE Monkey 2.0 was based⁴ provided gonorrhea- and chlamydia-specific estimates of the impact of STI funding but did not provide syphilis-specific estimates. We assumed that the relative impact of changes in funding would be the same for syphilis as for gonorrhea. Sixth, the underlying data series from the historical allocations study⁴ ends in 2016; the model would need to be updated in the future as new data become available. In addition, although the underlying historical allocations study⁴ controlled for a range of state-level characteristics, SPACE Monkey 2.0 does not allow users to include jurisdiction-specific characteristics such as sociodemographics or the number of providers per capita. Finally, it does not account for differential performance of STI prevention programs.

Overall, users were satisfied with the finalmodel design and its suitability for their information needs and expressed interest in using it in practice. Future efforts to create accessible modeling tools for practitioners could be useful for enabling health departments and other STI prevention practitioners to incorporate economic analyses into their communications and decision-making.

Implications for Policy & Practice.

• Estimating the return on investment for public health services is critical for demonstrating their value and making resource allocation decisions.

• A challenge to developing such estimates is that many health departments have limited staff capacity and expertise to conduct economic analyses in-house.

• The SPACE Monkey 2.0 spreadsheet model allows users to estimate the impact of increases or decreases in STI prevention funding to federally funded jurisdictions on the incidence and direct medical costs of chlamydia, gonorrhea, syphilis, and STI-attributable HIV infections.

• Modeling tools have the potential to make evidence-based estimates of budget changes widely accessible to health departments and other partners interested in the effects of STI funding on outcomes.

• Developing modeling tools with human-centered design principles and involving users throughout the design process can enhance use.