Economic Evaluation Design within the HEAL Prevention Cooperative

Abstract

The rapid rise in opioid misuse, disorder, and opioid-involved deaths among older adolescents and young adults is an urgent public health problem. Prevention is a vital part of the nation’s response to the opioid crisis, yet preventive interventions for those at risk for opioid misuse and opioid use disorder are scarce. In 2019, the National Institutes of Health (NIH) launched the Preventing Opioid Use Disorder in Older Adolescents and Young Adults cooperative as part of its broader Helping to End Addiction Long-term (HEAL) Initiative (https://heal.nih.gov/). The HEAL Prevention Cooperative (HPC) includes ten research projects funded with the goal of developing effective prevention interventions across various settings (e.g., community, health care, juvenile justice, school) for older adolescent and young adults at risk for opioid misuse and opioid use disorder (OUD). An important component of the HPC is the inclusion of an economic evaluation by nine of these research projects that will provide information on the costs, cost-effectiveness, and sustainability of these interventions. The HPC economic evaluation is integrated into each research project’s overall design with start-up costs and ongoing delivery costs collected prospectively using an activity-based costing approach. The primary objectives of the economic evaluation are to estimate the intervention implementation costs to providers, estimate the cost-effectiveness of each intervention for reducing opioid misuse initiation and escalation among youth, and use simulation modeling to estimate the budget impact of broader implementation of the interventions within the various settings over multiple years. The HPC offers an extraordinary opportunity to generate economic evidence for substance use prevention programming, providing policy makers and providers with critical information on the investments needed to start-up prevention interventions, as well as the cost-effectiveness of these interventions relative to alternatives. These data will help demonstrate the valuable role that prevention can play in combating the opioid crisis.

Introduction

The rapid rise in opioid misuse, disorder, and opioid-involved deaths among older adolescents and young adults is an urgent public health problem. Between 2001 and 2016, opioid-related deaths in the USA more than quadrupled, from 9489 to 42,245 per year, with those ages 15 to 34 exhibiting the greatest increases in burden from opioid-involved death (Gomes et al., 2018). Rates of opioid use disorder among those ages 18 to 34 also increased significantly in this timeframe, especially in the 25 to 34 age range (Martins et al., 2017). Epidemiological studies point to a pattern of increasing opioid misuse prevalence from early to late adolescence and young adulthood (Bonar et al., 2020; Hu et al., 2017; Martins et al., 2015, 2017), similar to patterns observed for other substances, with younger individuals who misuse at heightened risk for opioid use disorder and heroin use (Cerda et al., 2015; Schepis & Hakes, 2017, Schepis et al., 2020). Furthermore, opioid misuse by adolescents and young adults has been associated with co-occurring substance use, major depression, depressive symptoms, and suicidality (Barnett et al., 2019; Boyd et al., 2014; Catalano et al., 2011; Edlund et al., 2015 Groenewald et al., 2020; Zullig & Divin, 2012). Collectively, studies indicate that a vital part of the nation’s response to the opioid crisis must include effective preventive interventions that strengthen individuals’ responses to adversity and stress and reduce risk for opioid misuse initiation and escalation in young people (Compton et al., 2019; Volkow et al., 2019).

Although several preventive interventions have been shown to reduce substance use initiation among adolescents and young adults (Substance Abuse and Mental Health Services Administration, 2016), only two have demonstrated secondary impacts on young adult prescription opioid misuse (Crowley et al., 2014; Spoth et al., 2013), and evidence-based preventive interventions for those at higher risk for opioid initiation, escalation, and disorder do not exist.

To overcome the significant gap in effective interventions targeting opioid misuse initiation, escalation, and the negative consequences in older adolescents and young adults, the National Institutes of Health (NIH) launched the Preventing Opioid Use Disorder in Older Adolescents and Young Adults Cooperative in 2019 (RFA DA-19–035) as part of its broader Helping to End Addiction Long-term (HEAL) Initiative (https://heal.nih.gov/). The specific objectives of the HEAL Prevention Cooperative (HPC) are to develop strategies to identify, reach, and engage older adolescent and young adult populations at risk for opioid misuse and opioid use disorder (OUD) in prevention interventions and services; test the effectiveness of these prevention strategies and interventions; develop and test strategies to facilitate implementation and sustainability of these prevention interventions and strategies for accessing and engaging at-risk older adolescents and young adults across various settings (e.g., health care, justice and other systems); and conduct an economic evaluation of the programmatic costs and cost-effectiveness of the interventions and strategies.

The HPC includes 10 research projects and one coordinating center, and funding is administered by the National Institute on Drug Abuse (NIDA). Each project includes an evaluation of a specific intervention approach for youth between the ages of 15 and 30. Funded projects incorporate a variety of prevention strategies (e.g., community-based primary prevention with Native American adolescents, selective prevention with homeless youth and young adults, or with youth at higher risk for opioid misuse) that are being implemented in different organizational settings (e.g., healthcare, juvenile justice, school), and target older adolescents (ages 15–19), young adults (ages 20–30), or both. Nine of the 10 funded projects include an economic evaluation of the intervention, and these projects are summarized in Table 1. NIH also funded a coordinating center to support the funded projects and help synthesize research conducted under the cooperative.

Table 1.

Summary of HPC research projects with an economic evaluation

Research project lead organization	Principal investigator	Intervention summary	Intervention target population	Target population age range (years)	Intervention setting
Emory University/Cherokee Nation (CN)	Emory: Kelli Komro/ CN: Terrence Kominsky and Juli Skinner	Universal school-based screening with motivational interviewing, brief intervention, and teacher training combined with community-level organizing and media strategy	Native American and White youth in rural areas in the CN in Oklahoma	15–20	School and community
Massachusetts General Hospital/Harvard University	Timothy Wilens, Amy Yule	Assessment of pharmacotherapy, psychosocial treatment, and/or the combination of these two treatments on substance use (including opioid initiation) and related outcomes	Youth receiving treatment for mental health disorder or comorbid mental health and non-opioid substance use disorder (SUD)	16–30	Hospital and behavioral health and SUD clinics
The Ohio State University	Natasha Slesnick, Kelly Kelleher	Housing + opioid and related risk-prevention services vs. opioid and related risk-prevention services alone	Homeless youth who do not have opioid use disorder (OUD)	18–24	Drop-in centers, shelters, broader community
Oregon Social Learning Center	Lisa Saldana	Integrated preventive intervention including evidence-based substance abuse and mental health treatment, parent skills training, and intensive case management	Young parents involved with or at-risk for involvement with child welfare and/or self-sufficiency services	16–30	Child welfare and/or self-sufficiency referrals to community clinic
RAND/University of California, Los Angeles (UCLA)	Elizabeth D’Amico, Daniel Dickerson	Group-based motivational interviewing to address opioid, alcohol, and cannabis use through discussion of social networks and engagement in traditional practices, combined with monthly community wellness circles	English-speaking American Indian/Alaska Native emerging adults living in urban areas who do not have opioid dependence at study baseline	18–25	Community
Seattle’s Children Hospital (SCH)/University of Washington (UW)	SCH: Kym Ahrens/UW: Kevin Haggerty	Assertive community care/assertive continuing care-based OUD prevention interventions of various intensity levels; more intensive arm also includes trauma-focused intervention	Youth re-entering community after justice involvement; includes youth with and without prior opioid use	15–25	Juvenile justice
Texas Christian University	Danica Knight	Adapted evidence-based intervention called the Trust-Based Relational Intervention as a prevention intervention. Includes group sessions while youth are in custody and coaching visits post-release	Youth transitioning to their communities after a period of detainment in a secure treatment or correctional facility	15–25	Juvenile justice
University of Michigan	Maureen Walton, Erin Bonar	Emergency Department (ED)-based video-delivered single session with a health coach and post-ED web-based messaging with a health coach in a portal-like platform for 30 days using motivational interviewing strategies	Youth who present to the ED and report past 12-month prescription opioid use with accompanying risk factor (e.g., screen positive for current depression, past-year suicide attempt/past 2-week ideation, past 3-month binge drinking, cannabis, other illicit drug use, or prescription drug misuse) or past 12-month opioid misuse (prescription or illicit)	16–30	ED/healthcare
Yale University	Lynn Fiellin	Video game intervention to prevent initiation of opioid misuse delivered in school-based health centers	Youth aged 16–19 who are enrolled in a school-based health center, report not having engaged in prior opioid misuse, and be at greater risk due to: 1) past 30-day use of cigarettes, e-cigarettes, Juul, alcohol, marijuana (including synthetics), amphetamine, cocaine, benzodiazepines, ecstasy, bath salts, or any other misuse of non-opioid prescription drugs or, use of non-opioid prescription drugs or use of non-opioid illicit drugs, OR 2) screen positive for symptoms of depression, OR 3) screen positive for symptoms of anxiety	16–19	School-based health centers

The HPC offers a significant opportunity to systematically develop scientific knowledge about the costs and cost-effectiveness of a range of interventions united by a common goal of preventing opioid misuse and escalation in older adolescents and young adults. Each project includes its own cost and cost-effectiveness analyses designed to yield policy-relevant economic evidence that complements evidence about implementation feasibility and impact. In addition, subsets of related projects (e.g., similar settings, similar target populations) will engage in cross-research project analyses of costs and cost-effectiveness. Finally, the HPC also will be performing a cross-project budget impact analysis (BIA). The BIA will utilize the findings from the cost and cost-effectiveness analysis (CEA) to estimate the financial or budgetary impact of adopting a given intervention(s).

Estimating start-up and ongoing costs of delivering the interventions is important to understanding the resources that are needed to ensure effective implementation and sustainability. CEA allow us to jointly examine the intervention delivery costs and outcomes achieved, thereby providing information on the relative value of alternative approaches to achieving a program or policy objective like reducing the negative consequences of the opioid crisis. BIA will allow us to examine the affordability and cost savings of one or more interventions as they are scaled up over multiple years.

Though commonly used to compare varying health and medical interventions, including treatments for opioid misuse and dependence (Acharya et al., 2020; Banerjee & Wright, 2020; Busch et al., 2017), economic evaluations of prevention interventions for substance use and other behavioral health problems in young people are relatively rare (for exceptions, see, for example, Caulkins et al., 1999; Corso et al., 2015; Hollands et al., 2014; Ingels et al., 2013; Jensen et al., 2005). To our knowledge, only one economic study has compared alternative interventions to prevent opioid misuse in adolescents (Crowley et al., 2014). It showed that two universal prevention approaches, life skills training and strengthening families 10–14 (Spoth et al., 2013), were cost-effective from a societal perspective to prevent negative consequences associated with adolescent and young adult opioid misuse. These two interventions were conducted in a similar setting (schools), leaving gaps in the evidence as to how they would perform in different settings.

Although the HPC interventions are promising, economic analyses will be critical to assist decision makers in selecting the most effective strategies that meet the needs of the populations served. The HPC’s economic analyses are aimed at meeting this need, but careful planning and coordination across the projects are essential if ensuing comparisons are to be valid.

Achieving quality and methodological comparability across intervention studies is generally difficult, given the siloed way that trials are typically carried out and the length of time between trial implementation and the dissemination of findings. The HPC is structured to overcome these limitations in important ways, including economic evaluation among funded research projects, the presence of one or more health economists on research project teams, and the regular convening of the HPC economic teams (referred to as the Health Economics Work Group [HEWG]) beginning in year 1 of the 5-year funding period. Regular meetings and rich discussion among the HEWG members have led to the development of an HPC protocol for the conduct of the cost data collection and economic analyses, which is guided by the recommendations of the second panel on cost-effectiveness in health and medicine wherever possible (Neumann et al., 2016; Sanders et al., 2016). Along the way, conceptual and methodological questions have surfaced, not surprisingly given the variety of intervention approaches, implementation settings, and populations served (e.g., community- and school-based primary prevention intervention serving adolescents in a rural setting, individually delivered intervention serving adolescents and young adults in juvenile justice settings, community-based intervention targeting homeless youth and young adults who are at higher risk for opioid initiation and misuse, and school-based health centers providing health/behavioral care to students and families). The resulting protocol shows how these challenges have been dealt with, so that a comparable methodology can be employed across these research projects.

Taken together, the three types of economic evaluation being undertaken by the HPC will provide program implementers, funders, and policy makers with economic data to help inform programmatic decisions. The overarching goal of these analyses is to increase the utility of these economic findings for decision makers tasked with determining effective, efficient approaches to addressing the opioid crisis in their jurisdictions.

Methods

Nine of the 10 research projects funded under the HPC have an economic evaluation component and are included in the cross-project analyses, although the scale and scope of economic evaluations varies widely across studies.

The HEWG was convened to design the HPC’s economic evaluation so that reliable and appropriate comparisons can be made across research projects. Through this work group, research projects have adopted a common approach to assessing costs, agreed on which economic measures to use for the analyses, aligned measures such that the cost estimates across research projects can be meaningfully combined despite differences in data sources, and agreed to using the same effectiveness measures.

Cost Data Collection and Analysis

The research projects will assess costs over two distinct periods: start-up of the intervention and intervention delivery. Costs of start-up (activities described below) are typically incurred after the intervention has been developed and before the first participant receives services at a particular site or location. Because these costs are fixed (sometimes referred to as sunk costs), are incurred only once during the initial start-up of the intervention, and are not incurred in the ongoing management or delivery of the interventions, start-up costs will not be included in the CEA. However, these costs will provide useful information about initial resource needs and investments to decision makers interested in building capacity and otherwise preparing to deliver the intervention.

Intervention delivery costs are incurred from the point of the first participant receiving the intervention onward and these will be used to estimate incremental intervention costs for the CEA. Our approach excludes the costs of research (e.g., the administration of participant surveys used solely for research) as these would not be relevant for describing future implementation costs in a non-research, real-world scenario.

Activity-Based Costing

Cost data collection and analysis are guided by the principles of activity-based costing, whereby the researcher decomposes relevant costs into pre-defined activities and measures the quantity and unit price of each resource (e.g., labor, supplies, office space) used for each activity performed. The HEWG identified a common set of broad activity categories around which to organize collection of resource use data for both start-up and ongoing delivery of the interventions. Specifically, intervention start-up comprises five mutually exclusive activity categories: planning meetings (e.g., meetings to engage community and site stakeholders to implement the program); initial staff hiring and acquisition of minor equipment and supplies (e.g., purchase of licensed software); initial staff training for program implementation; development and/or revisions to policies and procedures to accommodate organizational workflow and processes (e.g., producing a workflow manual for implementation in a particular site); and management (e.g., weekly team meetings).

Intervention delivery comprises seven mutually exclusive and comprehensive activity categories: three of the seven categories are directly linkable to an individual participant—pre-contact activities (e.g., scheduling of intervention sessions, identification of eligible participants), direct intervention delivery, and post-contact activities (e.g., record-keeping))—and three categories are not linkable to an individual participant and will be allocated across all or some study participants—clinical supervision, ongoing staff training, and program management and site engagement. The final category, which is also not linkable to an individual participant, is community-level intervention activities. This category is important to appropriately capture the resources used for intervention activities that target a larger community (as opposed to a single participant in one organization, for example). The value of resources used in this category will be allocated across the entire population served. (See Table 2 in the Appendix for more detailed descriptions of the cost categories.)

Some cost categories (e.g., staff training) can exist in both start-up and ongoing delivery phases, and they are classified according to the timing of the resource use. For example, we classified any staff training prior to the start of the intervention delivery as a startup cost. This initial training is a critical component to ensure effective implementation. However, training is also part of normal business operations once an intervention is launched because an intervention site may need to train new staff or provide additional training to existing staff. For our estimation, we classify any training of staff after the start-up period as an ongoing delivery cost.

Each research project will collect data on the intensity of resource use within each of these activities, as well as the unit costs of resources such as an hour of staff time. Labor costs are expected to be the primary driver of start-up and intervention costs. For the cross-research project analyses, the HEWG will use national wage estimates from O*Net (US Department of Labor, 2021) and the Bureau of Labor Statistics (U.S. Bureau of Labor Statistics, 2021). To account for benefits, the HEWG will use the rate of 29.5% reported by the Bureau of Labor Statistics (US Bureau of Labor Statistics, 2022). These benefits include employer insurance costs, paid leave, and legally required benefits (i.e., Social Security, Medicare, unemployment insurance and worker’s compensation). Research project economists identify the list of staff, titles/occupations, and associated wages for their own research project analyses, and then the HEWG works collaboratively to map project-specific staff positions to national wage estimates for the cross-research project analyses. The use of national wages will remove the influence of local and regional wage variation on cost differences observed across projects.

The cost of starting-up an intervention will be calculated at the site level for each research project. These costs will be standardized across projects (e.g., by using national wage estimates from O*NET). Start-up costs represent the sum of the costs of each of the five activities over the start-up period for that site. The total cost of intervention start-up for the research project is then the sum of these site-level costs.

The cost of intervention delivery will be calculated at the level of the individual participant. The per-person cost of each of the seven activities is the sum of the per-person costs for each resource (labor and nonlabor) used for that activity. Labor costs for direct intervention delivery, for example, are the amount of direct intervention delivery time for each staff type multiplied by the wage (including the benefits rate) for that staff type, summed over all staff types. Total intervention delivery cost is then the sum across intervention participants of the person-specific costs across all seven activities.

Cost-Effectiveness Analysis

Intervention delivery costs will support a full CEA to compare HPC interventions in terms of both effectiveness and cost. Given budget constraints faced by policy makers and providers, CEA can be a useful tool in decision making, as it provides information on how much an intervention may cost per unit of desired outcome, compared with an alternative intervention. This inherently incremental analysis provides evidence of economic value (i.e., the relative costs to achieve desired outcomes across interventions and research projects).

Our CEA approach is guided in general by the principles of the Second Panel on Cost-Effectiveness in Health and Medicine (Neumann et al., 2016; Sanders et al., 2016). One exception, however, concerns the analytic perspective. The second panel recommends a minimum of two analytic perspectives for a CEA: the health care sector and the societal. The goals of the overall cooperative are broader than identifying which interventions are cost-effective. Rather, the focus of the cross-project analysis is to support real-world implementation and scale-up of individual or combinations of cost-effective interventions. Because of these goals and because decision makers at provider sites would be responsible for real-world implementation, the cross-project health economic study adopted the provider perspective for the core analysis, rather than the broader health care sector and societal perspectives. This approach recognizes the variation in available data across the research projects—which would have impeded the uniform application of a societal perspective—and provides a common key perspective that allows for appropriate comparisons. Some research projects may, however, incorporate broader perspectives (e.g., health sector, juvenile justice, participants/families, societal) in their individual economic analyses.

The primary outcomes for our CEA will be derived from the effectiveness measures used by the research projects to assess opioid initiation and escalation at the level of the participant (Ridenour et al., 2022). For legally (e.g., prescription opioids) and illegally manufactured opioids (e.g., heroin), we will examine initiation as the percentage of participants who report misuse for the first time (i.e., having not previously misused in their lifetime) as measured during the assessment periods. In addition, we will examine escalation of misuse using a count measure for days of use in the past 30 days across the assessment periods.

Because of the heterogeneity in target populations and approaches across the research projects, we do not plan to compare each intervention relative to all others. Rather, we plan to group interventions such that appropriate and meaningful comparisons can be made. For example, one comparison could be among interventions implemented within juvenile justice settings, while another could be among those implemented within school settings. These groupings will be developed in collaboration with each research project.

For each grouping and outcome, we will rank interventions in order of increasing mean cost per participant (C), regardless of the statistical significance of the effectiveness estimates. For a given intervention i, cost-effectiveness will be evaluated in relation to the next most costly intervention j using the incremental cost-effectiveness ratio (ICER_ij), which will be calculated as the difference in mean cost divided by the difference in mean effectiveness (E), (C_j − C_i)/(E_j − E_i). ICERs will be computed for each intervention relative to the next most costly option (Neumann et al., 2016).

The next step is to determine the relative cost-effectiveness of each intervention. An intervention can be excluded from consideration either through strict dominance (i.e., another intervention is less expensive and more effective than the eliminated intervention) or extended dominance (i.e., has a greater ICER than a more costly intervention; Drummond et al., 2015). The remaining non-dominated interventions comprise the cost-effectiveness frontier; from these interventions, a decision maker (e.g., provider or policy maker) chooses the cost-effective option. Economic theory suggests that the optimal intervention will be the one with the greatest ICER that is less than or equal to the decision maker’s willingness to pay (WTP) for an additional unit of the outcome (Drummond et al., 2015). Non-economic factors may also be important to the decision maker and society as a whole. Economic analyses, such as CEA, can help inform decisions regarding intervention choices, but these other factors should also be considered. Furthermore, CEA is limited to the included interventions. If other interventions become available, then new comparisons would be needed.

To reflect sampling variability in our cost-effectiveness analysis, we will use Monte Carlo, nonparametric bootstrapping (Dunlap et al., 2019), or parametric methods (Murphy et al., 2019) to characterize joint parameter uncertainty around our ICER estimates (e.g., adjusted standard errors, confidence intervals). We also will calculate cost-effectiveness acceptability curves (CEACs) to examine the probability that a given intervention is cost-effective, compared with the alternatives within the cost-effectiveness frontier. CEACs incorporate the joint variability of the cost and outcome estimates and show the probability that an intervention is the cost-effective choice, as a function of the decision maker’s WTP for that outcome over a range of WTP values (e.g., $10,000 to $50,000 WTP per opioid initiation avoided; Fenwick et al., 2001, 2006). Finally, we will examine the impact that key parameters (e.g., wage rates) have on our findings in sensitivity analyses. These analyses will allow us to explore alternative scenarios for these parameters (e.g., varying wages for intervention delivery staff) and their effect on cost-effectiveness results.

Budget Impact Analysis

For the BIA, we will develop a model that will incorporate data from the individual research projects (i.e., intervention costs, condition-related costs, intervention effectiveness, condition-related health impacts, characteristics of the target population, current practices, reach (number of people served), and generalizability of the intervention) and from published literature (i.e., epidemiology and nature history to characterize the eligible population over time; downstream effects of OUD such as social costs associated with health care use, productivity losses, and involvement with criminal justice system). Using these data, the BIA will estimate the short-term impact (no more than 5 years) that implementing the intervention to the desired scale will have on a funding decision maker’s budget in terms of cost expenditures (the unit cost of an intervention multiplied by the number of people to be served by the intervention) and cost offsets or cost savings (e.g., reduction in social costs). Because all funders must make decisions about how to allocate budgets, understanding the affordability of a new intervention is critical. The BIA is important to provide that information and complements the cost-effectiveness analyses (Mauskopf et al., 2007).

Discussion

Prevention, especially among older adolescents and young adults, is a critical part of the solution for the opioid epidemic. The HPC is positioned to significantly accelerate the nation’s prevention response to this epidemic by promoting the development and testing of interventions and strategies to prevent initiation and escalation of opioid misuse. Specifically, the HPC will jointly examine the effectiveness and costs of multiple prevention interventions being implemented across health care, justice, school, and other settings targeting at-risk older adolescents and young adults.

The HPC includes 10 research projects that are developing individual- and community-level interventions aimed at diverse populations across different settings—all with the goal of preventing opioid misuse initiation and escalation among older adolescents and young adults. Of these, nine projects are engaged in the HPC economic analyses and collecting detailed cost data that will support the cost, cost-effectiveness, and budget impact analyses. The scale and diversity of these projects will provide valuable economic data that will inform the substance use prevention field more broadly. The HPC cross-research project economic study will provide important estimates of the resources needed (and associated costs) to start up and maintain ongoing delivery of each intervention. Start-up costs are not commonly estimated in prevention intervention research. Yet, understanding start-up costs for interventions is extremely important, as costs are often cited as a reason why new interventions are not adopted. Funders and implementers need to know not only the ongoing costs of providing the intervention but also the initial investments that may be required to launch the intervention. Start-up costs will be available from 8 of the 10 research projects, providing a valuable resource to decision makers. The HPC cross-research project analyses will also provide data on the cost-effectiveness and budget impact of these interventions for avoiding cases of opioid initiation and escalation, so that policy makers and providers who struggle with limited budgets can make informed decisions concerning resource allocations across alternative prevention options.

The HPC has the opportunity to advance methods for conducting comparative economic evaluations of preventive interventions (Crowley et al., 2018). Through supporting nine simultaneous trials with a common goal and funding a coordinating center and the HEWG, NIH facilitated the development of common methods for ascertaining costs and outcomes, ultimately allowing for methodologically meaningful economic comparisons. NIDA Scientific Officers participated in the HEWG and the Measures and Data committees. Although the substantial diversity in populations (homeless young adults, youth in juvenile justice systems), settings (school-based, emergency departments, hospital-based and outpatient clinics, juvenile justice), and intervention approach (community-level and individual-level strategies) will provide a wealth of knowledge about effective preventive interventions with older adolescents and young adults, this diversity also poses a challenge for creating true intervention alternatives for which comparisons of relative costs and impacts make sense. Moreover, an intervention that is cost-effective in one grouping or with respect to one opioid misuse outcome may not be cost-effective when a different grouping or outcome is considered. Transparency around the rationale for various comparisons, the interventions being compared, and conclusions reached will be essential if economic evaluation findings are to be of value to decisionmakers (Steuerle et al., 2016). The HEWG decisions can provide a roadmap for the conduct and reporting of CEAs of interventions that share a common goal but attempt to achieve it in different ways.

With its diversity across interventions and settings, it is not surprising that the HPC’s economic evaluation has a few limitations. First, the HEWG determined early on that a single, standardized data collection instrument for costs would not work because the research projects are relying on different combinations of data sources and strategies, including use of surveys and administrative data. Rather, the research projects will follow a similar data collection approach (i.e., activity-based costing), and data measures will be harmonized to ensure that cost estimates are comparable across the projects. Second, while a societal perspective is often cited as the preferred perspective for cost analyses, most of the HPC research projects are not collecting the broad data necessary to perform analyses at this level. The HEWG determined that a provider-level perspective would be appropriate to use across the research projects. It will allow us to accommodate the variation in data collection across projects, while still providing critical information to policy makers and prevention providers. Some research projects will be performing cost analyses that include broader perspectives, such as health care sector or juvenile justice or a limited societal perspective with participant time costs included, and these projects may be part of a sub-analysis. In addition, most of the HPC research projects are not collecting the necessary data to estimate quality-adjusted life years (QALYs), a preferred measure for economic studies of healthcare interventions (Wichmann et al., 2017). We therefore focused on the substance use outcomes that are of interest to all research projects and feasible given the data that are being collected. The outcome measures chosen are common measures within the substance use research field (Bjornestad et al., 2020).

Finally, we would be remiss if we did not note the impact of the COVID-19 pandemic on the development and implementation of the HPC interventions. Most of the research projects had to adjust their interventions away from in-person activities and toward all virtual or a combination of virtual and in-person activities. There also have been delays in start-up activities and ongoing delivery as implementation sites and their staff have had to navigate COVID-19 challenges. The planned economic analyses will be focused on estimating costs of the interventions as implemented, but we also plan to examine the potential impact on costs if the interventions had been implemented as originally planned (pre-COVID).

Despite these challenges, the HPC offers an extraordinary opportunity to generate economic evidence for substance use prevention programming, providing policy makers and providers with critical information on the investments needed to start-up prevention interventions, as well as the overall cost-effectiveness and affordability of these interventions relative to alternatives. Overall, these data should help demonstrate the valuable role that prevention can play in combating the opioid crisis.

Appendix

Table 2.

Cost category descriptions

Cost category	Description
Start-up activities
Planning meetings	Engagement with stakeholders outside the core Research Projects (RP) team to get buy-in to and arrange implementation of the program. The time can be preparation, travel, and actual engagement. The engagement can be virtual or in-person. Examples include introducing the program discussing the program to obtain buy-in from state agencies, discussing logistics with a clinic, and identifying and addressing challenges to getting the program up and running.
Training	Instruction to implement the program. The time can include modifying an existing protocol to implement it at a site, arranging the logistics to schedule the training, and delivery of the training. These initial trainings occur prior to the intervention delivery launch.
Acquisition and hiring	Purchase of goods, arrangement of contracts for services, and employment of implementation staff. Examples include buying licensed software and hiring a community outreach worker.
Development of policies and procedures	The creation of documentation and agreements that govern the process of the intervention operations. Examples include drafting and securing a memorandum of understanding with a county agency and tailoring, writing. and producing a workflow manual for program implementation in a site.
Management	Organization and coordination of tasks throughout the course of start-up. Examples include weekly core intervention team meetings and emails with intervention staff concerning program operations.
Ongoing intervention delivery activities
Pre-contact activities	Activities that are in preparation for and support direct delivery of the intervention. Examples of pre-contact activities may include scheduling of intervention sessions, identification of eligible participants, review of records or notes from previous intervention sessions.
Direct intervention delivery	Activities performed in the direct delivery of the intervention, such as delivery of individual or group intervention sessions or delivery of a wellness gathering.
Post-contact activities	Activities that occur following intervention delivery in support of the intervention such as record-keeping.
Clinical supervision	Activities that provide supervisory professional support to individual interventionists such as meeting with a supervisor to discuss intervention participants and review work. Supervision does not include time spent on research activities.
Training	Staff training activities to support ongoing intervention delivery.
Program management and site engagement	Non-research activities required to ensure the ongoing administrative operation and coordination of the intervention program (e.g., completing required reporting (such as time sheets), buying supplies, and staff meetings. It also includes activities needed to ensure site engagement for continued program implementation such as discussing logistics within a site and identifying and addressing challenges that may be experienced during the ongoing delivery phase.
Community-level intervention activities	Intervention activities that target a larger community (as opposed to a single participant in one organization). Examples include community organizing, wellness gatherings, and media campaigns.

Funding

This research was supported by the National Institutes of Health through the NIH HEAL Initiative as part of the HEAL Prevention Initiative. The authors gratefully acknowledge the collaborative contributions of the National Institute on Drug Abuse (NIDA) and support from the following awards: Emory University and the Cherokee Nation (UH3DA050234; MPIs Kelli Komro, Terrence Kominsky, Juli Skinner); Massachusetts General Hospital (UH3DA050252; MPIs Timothy Wilens, Amy Yule); the Ohio State University (UH3DA050174; MPIs Natasha Slesnick, Kelly Kelleher); Oregon Social Learning Center (UH3DA050193 PI Lisa Saldana); RAND Corporation (UH3DA050235; MPIs Elizabeth D’Amico, Daniel Dickerson); RTI International (U24DA050182; MPIs Phillip Graham, Ty Ridenour); Seattle Children’s Hospital (UH3DA050189; MPIs Kym Ahrens, Kevin Haggerty); Texas Christian University (UH3DA050250; PI Danica Knight); University of Michigan (UH3DA050173; MPIs Maureen Walton, Erin Bonar); and Yale University (UH3DA050251; PI Lynn Fiellin).

Declarations

Ethics Approval

All studies that are described or referenced herein have been reviewed and approved by the Investigators’ respective Institutional Review Boards. All procedures with human subjects were performed in accordance with the ethical standards of the institute and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Consent to Participate

N/A.

Conflict of Interests

The authors declare no competing interests.

Disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or its NIH HEAL Initiative.