Budget impact analysis for implementation decision making, planning, and financing

Abstract

Shelley et al. (in Accelerating integration of tobacco use treatment in the context of lung cancer screening: relevance and application of implementation science to achieving policy and practice. Transl Behav Med 2022;12:1076–1083) laid out how implementation science frameworks and methods can advance the delivery of tobacco use treatment services during lung cancer screening services, which until recently was mandated by the Centers for Medicare and Medicaid Services. Their discussion provides an important overview of the full process of implementation and highlights the vast number of decisions that must be made when planning for implementation of an evidence-based practice such as tobacco use treatment: what specific tobacco use treatment services to deliver, when to deliver those services within the lung cancer screening process, and what implementation strategies to use. The costs of implementation play a major role in decision making and are a key implementation determinant discussed in major implementation frameworks. When making decisions about what and how to implement, budget impact analyses (BIAs) can play an important role in informing decision making by helping practitioners understand the overall affordability of a given implementation effort. BIAs can also inform the development of financing strategies to support the ongoing sustainment of tobacco use treatment service provision. More attention is needed by the research community to produce high-quality, user-friendly, and flexible BIAs to inform implementation decision making in health system and community settings. The application of BIA can help ensure that the considerable time and effort spent to develop and evaluate evidence-based programs has the best chance to inform implementation practice.

Budget impact analysis forms an essential bridge connecting evidence-based practices to real-world implementation efforts.

Implications.

Practice: Using budget impact analysis (BIA) can help practitioners make key implementation decisions by providing information on the affordability of different implementation options, such as what tobacco use treatment services to deliver and what implementation strategies to use.

Policy: Funders and policy makers should encourage the inclusion of BIAs in economic evaluations of implementation efforts.

Research: Researchers should develop and conduct high-quality, user-friendly, and flexible BIAs alongside cost-effectiveness analyses to illuminate potential disconnects between context-specific affordability and overall value.

Background

Shelley et al. recently advocated for the use of implementation science frameworks and methods to advance the delivery of tobacco use treatment services during lung cancer screening [1]. Their work offers a comprehensive overview of the entire process of implementation, including establishing an implementation team and objectives, conducting needs assessments, defining core components of the tobacco use treatment intervention, selecting and using implementation strategies, and developing a detailed implementation plan. They also discuss considerations for continuous quality improvement and sustainability of tobacco use treatment services after the active implementation period has ended. A robust discussion of implementation science frameworks, implementation strategy considerations, and implementation outcome measures are integrated throughout. While their discussion is grounded in the context of tobacco treatment and lung cancer screening, the lessons are broadly applicable to programs that address health behaviors. In the present article, we respond and build on their work to discuss the importance of budget impact analysis (BIA) in supporting the implementation of such programs. We too ground our discussion in the context of tobacco treatment, where appropriate, for illustration.

Decisions, costs, and implementation

Shelley et al.’s discussion highlights a vast array of decisions that must be made when planning for implementation of an evidence-based practice (EBP) such as tobacco use treatment. After the initial decision to implement tobacco use treatment services during lung cancer screening, implementors must make decisions about, for example, what specific tobacco use treatment services to deliver, when and where to deliver them, and by whom, what implementation strategies to use, and how services will be sustained.

Costs play a major role in these decisions and are a key implementation determinant discussed in major frameworks such as the Consolidated Framework for Implementation Research [2] and the Exploration, Preparation, Implementation, and Sustainment framework [3]. Two major components of costs are implementation and EBP costs [4]. Implementation costs are those that are directly related to getting the EBP into routine practice—e.g. by training personnel in the program, developing and adapting workflows to deliver the EBP, programming new registry or decision support tools in an electronic health record, or adapting program materials [4]. These costs are distinct from the direct costs of EBPs—such as required equipment or material purchases or the personnel costs of delivering an EBP—although in practice it is often difficult to distinguish between the two [4, 5]. Both implementation costs and direct EBP costs play a role in implementation decisions and feed into whether an implementing organization can adopt and implement an EBP. A growing body of literature is focused on understanding the costs of implementation efforts, above and beyond direct EBP costs [4–15].

Cost estimates are often put in context alongside effectiveness estimates through the use of cost-effectiveness analyses (CEAs); CEAs help determine if implementing an EBP is good value for money (i.e. the ratio of costs to effects is within an acceptable threshold) [16, 17]. Work in this area has, for example, evaluated the cost-effectiveness of smoking cessation for cancer patients [18] and the cost-effectiveness of implementing tobacco treatment programs among Cancer Center Cessation Initiative sites [19]. Cost-effectiveness results help decision makers think through implementation questions focused on whether novel implementation strategies or EBPs provide an adequate value for their cost. That is, they answer the question, “Should we do it?”

However, even if something is good value for money and “should” be pursued, it must also be affordable within an organization’s available budget. To assess affordability and better inform implementation decisions, BIA can play an important and complementary role to CEA, particularly in situations where the evidence for a program’s effectiveness and value are strong but implementation is rare [5, 10, 16, 17, 20]. BIA provides crucial information for implementation planners who need to understand what it would take to get an EBP such as tobacco use treatment services into place and sustain them over time in the organization’s particular context. Thus, BIA forms an important “bridge” that links cost-effective EBPs to real-world implementation and can help decision makers think through a key implementation questions asking “Can we do it?”

BIA in the Context of Implementation

BIA is the process of estimating the expenditures a specific implementing group or organization is likely to incur when implementing an EBP, given context-specific information [17]. Prior work has discussed the importance and application of BIA to policymaking, priority setting, and reimbursement decisions (e.g. insurance coverage of a pharmaceutical) [16, 17, 21]. Here, we focus on the potential of BIA to inform implementation decisions about complex health interventions [5]. Specific to Shelley et al.’s work, BIA can help inform decision making about the implementation of tobacco use treatment services during lung cancer screening and could be especially useful because of the decentralized nature of lung cancer screening in the USA. Differing economic and resource-related barriers to implementation are faced by facilities that deliver lung cancer screening, which range from academic medical centers to community-based organizations [1]. Despite the potential of BIA to inform implementation, BIA methods have seldom been applied to tobacco control interventions in clinical settings [22].

BIA for implementation decision making and planning

Making decisions about what EBP to implement and planning the implementation itself is a difficult process complicated by factors such as uncertainty and the availability of many decision options [23]. In tobacco use treatment, these include a range of methods for identifying patients who use tobacco and connecting them with treatment, tobacco use treatment services (EBPs, such as referrals to quit lines, in-person counseling, and various pharmacotherapies), financing options (funding treatment through insurance vs. health system resources), potential provision of adjunct services designed to bolster the effectiveness of tobacco use treatment (e.g. assistance with health-related social needs) [24, 25], and decisions about where to situate tobacco use treatment in the timeline and workflow of lung cancer screening [26].

Creating an implementation plan and choosing appropriate formal implementation strategies also adds complexity [23, 27]. Implementation strategies are methods and techniques used to improve adoption, implementation, and sustainment of EBPs, and there are over 70 different strategies that practitioners can select from that apply to clinical and community-based settings [28, 29]. Appropriate implementation strategies are necessarily context specific with direct consequences for implementation costs such as staffing, training, and space, as well as other physical resources [27].

Implementation decision making and planning typically involves choosing an EBP and/or implementation strategy(ies) [23]. BIA can support this phase by outlining clear, transparent estimates of the resources required for different combinations of EBPs and implementation strategies [23]. This can help provide structure to understand the tradeoffs inherent in different scenarios, such as one scenario including a costly EBP with an inexpensive implementation strategy versus a less expensive EBP with a more costly multi-pronged implementation strategy. Furthermore, BIA can illuminate when resources are needed, potentially weighing short-term upfront costs against longer-term sustained costs.

BIA for implementation financing

Shelley et al. also draw attention to the importance of securing adequate funding for sustainment of tobacco use treatment services [1]. BIA can also support this by informing the importance and development of financing strategies [12]. Financing strategies may include obtaining contracts or grants or shifting internal funds between programs to support implementation and sustainment [12]. For some EBPs, funds may be obtained through pay-for-performance contracts or billing for services provided [12]. For example, Medicare requires tobacco counseling for current tobacco users as part of lung cancer screening, and covers both counseling and prescription tobacco use treatments [30]. Where those arrangements do not exist, organizations may seek to negotiate new payment mechanisms that provide financial support. To inform which financing strategies might be pursued, decision makers need a good understanding of what implementation would cost in their context. For example, BIA results can inform the scope of funds requested in a grant application by a community-based organization, provide an estimate of costs for healthcare clinic administrators to use in funding allocation, or determine a target performance level needed for program affordability.

Methodologic and Analytic Considerations for BIA

Creating BIAs can be done in conjunction with other economic evaluation methods like CEA, or by using a prior CEA or cost identification study as the basis for further BIA analyses [5, 16, 17]. Because BIA is inherently forward looking (estimating likely costs in a new context), the goal of a BIA is to create a flexible model that allows users to change inputs and obtain context-specific estimates of budget impacts [17]. This means that BIA, as we discuss it here, is a simulation exercise [31], where our best available knowledge on implementation processes and implementation costs (often from research-based evaluations) are used to construct a representation of what implementation would look like in a new context. Then, context-specific variables can be changed to see potential costs, incorporating context-specific choices such as patient mixes, disease epidemiology, personnel choices, wage rates, payment contracts, or the availability of/need for essential infrastructure [5, 17].

Concretely, we operationalize this guidance by first focusing on understanding the costs of a prior or ongoing implementation effort in a specific context (or contexts) [4, 5]. In much implementation science work, a method called activity-based costing (also called micro-costing) is used to understand the specific actions and resources that are required to implement and sustain an EBP, allowing for information on the implementation processes to drive cost estimates [4, 5, 32]. Estimating costs may be a goal in and of itself, but it may also form the basis for further analysis [22]. Often, including in most economic evaluations of tobacco use treatment services, such cost estimates are used in subsequent CEAs, which weigh costs versus realized health effects in the context where the study was done [22].

When conducting a BIA, cost estimates can be used as the scaffolding for estimating implementation costs in future contexts (i.e. replication costs), from the budget-holders’ perspective [33]. The extent to which such scaffolding informs a BIA can be variable and depends on specific goals, available time and resources, and questions of generalizability. At its simplest, reported results from one context can be assumed to apply to a new context, with rough adjustments or caveats. For example, one might note that wages are lower in the new context, so overall costs might be lower. Furthermore, if the hours for specific cost elements are reported, wages from the new context can be applied to generate a new context-specific cost estimate [5].

As a next step in complexity, researchers can build a standalone BIA simulation model. Here, it is especially important to consider how the structure of prior implementation efforts generalize to future replication efforts, often a difficult task [31]. Ideally, analysts will have data on which aspects of implementation are core processes that need to be reproduced with fidelity and which are appropriate for context-specific adaptation, but that may not be the case [15]. Parallel qualitative analyses can provide insight into how the implementation efforts would replicate and scale, a particularly important consideration for BIAs generated via research-based implementation evaluations [5, 6, 33]. Engaging with implementation team members and utilizing qualitative/mixed-methods analyses can help identify elements of context that impact cost estimates [5–7].

Regardless of the complexity of BIA modeling efforts, it is important to acknowledge that BIA estimates are extrapolations that rest on assumptions about implementation processes and cost estimates from prior work. Therefore, as with all economic evaluation and decision analytic modeling, uncertainty should be thoughtfully considered and assessed. Prior work has advised BIA developers to be attentive to uncertainty both in the data used (i.e. parameter uncertainty) and assumptions about the implementation processes being modeled (i.e. structural uncertainty) [17]. Wherever possible, flexibility should be built into BIA models to allow users to conduct sensitivity and scenario analyses to evaluate the impact of uncertainty on potential budget impacts [17, 31].

While BIA simulation models may be stand-alone efforts for singular decisions, they may be more widely applied through broader-scale dissemination efforts, informing decision making, planning, and financing in further contexts. Given that BIA is necessarily specific to each context, these models need to be flexible enough to allow users to easily customize input data to be relevant to their context and perform sensitivity, uncertainty, and scenario analyses [17]. Other best practices include presenting user-friendly output, making downloadable summaries of results available, and clearly describing the methodological details of BIA calculations [17]. These kinds of analyses can be done with spreadsheets [17, 33], and creating user interfaces is possible in common spreadsheet programs like Microsoft Excel. The Shiny package in R software is another option that can be used to develop interactive tools that make BIA calculations accessible to users (e.g. https://r-hta-in-lmics.github.io/workshops/). R Shiny is an attractive option for this kind of research as it allows for relatively simple development of user-friendly, interactive, flexible, and customizable calculators, which can be easily disseminated via the website. Tools developed with R Shiny are also set up to be particularly useful for BIA analyses because it is simple for users to change input data and see how outputs change. The process of developing such tools also offers the opportunity to incorporate principles of user-centered design (e.g. defining target users, conducting co-creation sessions, or usability testing) to ensure that final products have the greatest potential to be used in practice [34, 35].

These methodological considerations mean that a team science approach [36, 37] is important for researchers who want to incorporate BIA into their research, particularly if a stand-alone BIA model will be built to simulate potential costs. The team should include someone with expertise in BIA and economic evaluation for implementation science, as well as simulation modeling. This could be an individual with expertise in decision science or health economics. The team should also include someone who has a deep understanding of the EBP and associated implementation processes (e.g. someone who has implemented tobacco treatment programs in the context of lung cancer screening), to ensure that the core components of the EBP and implementation strategies are included in all analyses. Third, the team should include potential users of the tool (e.g. decision makers, such as cancer center or clinic leadership) who understand how the BIA output will be used in context, which will help ensure that the BIA is built to best inform decision making. Fourth, engagement with individuals who will implement and sustain the EBP in context (e.g. oncology or radiology providers and office staff) can help ensure that elements of the BIA reflect their actual workflows and processes.

If a BIA model will be built for wider dissemination, additional individuals can help ensure the success of the final tool. An individual with knowledge of, or expertise in, user-centered design can help lead the project’s design to ensure that the final tool reflects the needs of end users and is user-friendly. Professional programming support can also be useful, especially if there are complicated workflows and programming logic to incorporate.

Limitations of BIA

BIA has limitations that should be acknowledged. Through its focus on the budget impacts of implementation, BIA necessarily has a narrower focus than other forms of economic evaluation and has the potential to encourage short-sightedness by decision makers because of contextual or capital constraints (e.g. large upfront costs), especially if the potential effects of implementation on downstream healthcare costs are not included [5, 31]. As a result, decision makers may implement strategies that are feasible given their constraints, but which do not provide the best value (e.g. most favorable CEA estimates) for them and/or society [31]. Fortunately, taken together with CEA, BIA can help illuminate disconnects between context-specific affordability and overall value, providing the opportunity to identify EBPs, implementation strategies, financing strategies, or policies that will align these short- and longer-term outlooks [5, 16]. In addition, budget impact is only one aspect of the decision-making process, and other objectives, such as the expected health benefits, impacts on equity, and feasibility, are also important factors that decision makers consider [5, 23].

Conclusions

Costs are an important factor in implementing EBPs. Where appropriate, our discussion is grounded in the integration of tobacco use treatment services into lung cancer screening, but BIA methods can be applied to nearly any implementation activity in the field of translational and behavioral medicine. Regardless of substantive area, costs contribute to implementation decision making and planning and the development of sustainable financing for implementation efforts. BIA can play an important role in these efforts by helping decision makers understand more than just whether an EBP is good value for money; BIA can help inform whether the implementation effort will be affordable in their context. More attention is needed by the research community to produce high-quality, user-friendly, and flexible BIAs to inform implementation decision making in health system and community settings. This will ensure that the considerable time and effort spent developing EBPs and assessing their costs and cost-effectiveness has the best chance to inform implementation practice.

Conflict of Interest Statement

None declared.

Funding

N.R.S. was supported by T32CA057711 and K99CA277135, and D.E.L. was supported by R01CA218123. This article was partially funded through a Patient-Centered Outcomes Research Institute (PCORI) Dissemination and Implementation Award (DI-2017C3-9005). The views in this article are solely the responsibility of the authors and do not necessarily represent the views of the PCORI and its Board of Governors or Methodology Committee.

Ethical Approval

This article does not contain any studies with human participants performed by any of the authors.

Informed Consent

This study does not involve human participants and informed consent was therefore not required.

Welfare of Animals

This article does not contain any studies with animals performed by any of the authors.

Transparency Statements

Not required of commentaries.