Abstract
To support implementation of important public health policies, policymakers need information about implementation costs over time and across stakeholder groups. We assessed implementation costs of two federal sugar-sweetened beverage (SSB) policies of current policy interest and with evidence to support their effects: excise taxes and health warning labels. Our analysis encompassed the entire policy life cycle using the Exploration, Preparation, Implementation, and Sustainment framework. We identified implementation actions using key informant interviews and developed quantitative estimates of implementation costs using published literature and government documents. Results show that implementation costs vary over time and among stakeholders. Explicitly integrating implementation science theory and using mixed methods improved the comprehensiveness of our results. Although this work is specific to federal SSB policies, the process can inform how we understand the costs of many public health policies, providing crucial information for public health policy making.
Keywords: public policy, policy, cost and cost analysis, implementation science, sugar-sweetened beverages, public health
Introduction
Public health policy plays a major role in setting the foundations for public health, and is a major component of essential public health services in the United States [1–3]. In deciding among policy options, assuming all cannot or should not be implemented, or planning for the implementation of a policy, the anticipated costs of a given course of action are critical for decision makers to consider [4–6]. Previous work has also identified a need to understand how the costs of action accrue across stakeholder groups (i.e., the need to include multiple analytic perspectives in analyses) [6–10]. Often the stakeholder responsible for implementation may not see the benefits, or benefits may be far into the future; thus, buy-in for action may be difficult [6–8].
Reflecting the importance of costs in decision making, researchers in implementation science have drawn attention to a major need to understand implementation costs [7,11]. The field has primarily focused efforts to improve the adoption and sustainment of evidence-based programs in clinical, practice, community, and system levels [2]. In recent years the field has begun to increase its focus on policy implementation [12–17]; there is a particular need for methods and measures to understand health policy implementation outcomes and determinants, including policy implementation costs [15].
Assessing the costs of policy implementation is difficult. Policies can be driven by various branches of government and have different implementation processes, timelines, and cost components to consider, such as drafting legislation or a regulation or creating an executive order. Policies also differ in how they aim to improve public health [13]. Policies can be interventions themselves, such as the United States (US) policy requiring graphic health warning labels on cigarettes [18]. Policies may also pave the way for other programs or interventions, such as the policy that allows Medicare -- a publicly funded health insurance program for the elderly and disabled -- to reimburse for the Diabetes Prevention Program [19], or tax policies to fund behavioral health services [20,21]. In all cases, implementation is a process [22], meaning that analysts must consider costs across the entire policy life cycle – including actions taken prior to enactment as well as those to sustain it over time like advocacy, lobbying, litigation [20,23–26].
Managing high rates of sugar-sweetened beverage (SSB) consumption through policy action represents an area of major public health interest across the US [27–29], given the association of intake of these beverages with issues of public health importance including type II diabetes, heart disease, and tooth decay [30–32]. In this article, we explored implementation costs of SSB policies in the United States using mixed methods and stakeholder engagement, and using an implementation science framework to guide our analysis.
Methods
Prior literature
We illustrate how prior groups have approached understanding the costs of SSB policies in Table 1, using eight US-focused economic evaluation articles drawn from a scoping review conducted by members of our team [33]. Most articles relied on some form of societal or governmental perspective, with only one article explicitly focusing on how the costs would accrue to groups involved [34]. All articles incorporated some consideration of healthcare or medical costs associated with downstream health impacts of SSB policies, as well as implementation costs.
Table 1.
Components of prior economic evaluations of SSB policies
| Article | SSB Policies | Perspective(s) | Costs assessed |
|---|---|---|---|
| Basu et al. 2013 | Ban on SSB purchases with SNAP dollars; volumetric tax | Government | Tax revenue; deadweight loss; healthcare (MEPS) |
| Gortmaker et al. 2015 | Volumetric tax | Modified societal perspective | Implementation (micro-costing); tax revenue; healthcare (MEPS) (Uses values from Long 2015) |
| Long et al. 2015 | Volumetric tax | Societal | Implementation (micro-costing); tax revenue; healthcare (MEPS) |
| Long et al. 2019 | Ban on SSB purchases with SNAP dollars; volumetric tax | Societal (assumed) | Implementation (micro-costing); healthcare (prior literature and Medicaid spending data) |
| Wilde et al. 2019 | Volumetric tax | Healthcare; societal; 9 stakeholder groups (six insurance-based consumer categories; government; private sector; beverage industry) | Implementation (percentage of revenue); revenue; deadweight loss; healthcare (MEPS and CMS) |
| Basu et al. 2020 | Workplace ban on SSB sales | Employer; healthcare | Healthcare (MEPS); productivity |
| Du et al. 2020 | Volumetric tax | Government; societal | Implementation (percentage of revenue); tax revenue; healthcare (MEPS and SEER) |
| Lee et al. 2020 | Volumetric, sugar, and tiered sugar taxes | Healthcare; government; societal | Implementation (percentage of revenue); tax revenue; healthcare (prior literature); reformulation |
Notes: SSB = Sugary-sweetened beverage; SNAP = Supplemental Nutrition Assistance Program; MEPS = Medical Expenditure Panel Survey; CMS = Centers for Medicare and Medicaid Services; SEER = Surveillance, Epidemiology, and End Results.
Methods for understanding the potential healthcare costs are fairly consistent across articles, but the methods for understanding implementation costs differ markedly and there is minimal guidance in them despite a clear tradeoff between approaches [34]. For tax evaluations, some work assumed implementation costs to be 2% of tax revenue generated [34–36]. This typically represents the most conservative approach to estimating costs (yielding the highest implementation cost), although often implementation costs will be small relative to other costs included in economic evaluations (such as revenue, healthcare) [34]. A ‘percentage of revenue’ approach, however, only works if the policies under study generate revenue. For policies such as health warning labels or restrictions on portion sizes, this method cannot be applied, and researchers typically must use a micro-costing approach.
Time-driven activity-based costing (TD-ABC)
Micro-costing requires direct measurement of all components of costs, and encompasses specific methods such as time and motion analyses or time-driven activity-based costing (TD-ABC) [7,37,38]. Prior SSB evaluations typically identified cost components using information from state-level implementation experiences or other areas of policy (such as alcohol or tobacco taxes) [39–41]. Studies generally provided an overview of the costs included but focused primarily on costs incurred after enactment of a policy. Thus, although these studies clearly explicate costs incurred, such studies might not include the complete picture of time- and stakeholder-dependent implementation costs.
Here, we apply TD-ABC and implementation science theory to understand the costs to implement two federal SSB policies: excise taxes and health warning labels. Policy at the federal level is important to examine because potential costs and benefits are likely higher than at state and local levels. Evidence is mounting to support effectiveness of both of these policies [42–45] and more US state legislatures have also proposed to adopt them [46–48]. Such decisions by states often indicate likelihood of federal action, as was the case for restaurant calorie labeling policies [49]. Many US cities such as Berkeley, CA, Boulder, CO, and Philadelphia, PA have also imposed excise taxes [28] and members of the US Congress have proposed them for the federal level [50]. Thus, these policies are realistic options for researchers to consider proactively, so that research is available when policymakers need it.
We chose TD-ABC because it improves the specificity for assessing costs, and better information helps to guide decision making [6,37]. We also build on work at the nexus of implementation science and economic evaluation that has championed the use of stakeholder-engaged and mixed-methods approaches to appropriately and thoroughly assess implementation costs [11,38,51]. Because we focused on implementation costs, we did not analyze other costs potentially relevant to decision making, such as downstream healthcare costs averted.
Framework
We followed the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework, which describes actions undertaken in four general phases of implementation (Figure 1) [22,52]. Implementors first decide whether to implement a policy (Exploration), and plan how to implement it (Preparation). Then they put the policy in place (Implementation) and maintain it over time (Sustainment). Guided by this framework, we used an exploratory sequential mixed-methods approach [53], where qualitative key informant interviews informed our quantitative cost analysis (Figure 2).
Figure 1.

EPIS framework overlaid onto policy process. EPIS=Exploration, Preparation, Implementation, and Sustainment. Underlying policy process figure from Golden and Moreland-Russell 2016 [52] and EPIS framework from Moullin et al., 2019 [85].
Figure 2.

Overview of analytic approach used to assess policy implementation costs. Methods for each step of the process are shown in bolded font. E=Exploration phase, P=Preparation phase, I=Implementation phase, and S=Sustainment phase. Team talk by Musmellow and cost by Nhor from the Noun Project.
Qualitative key informant interviews
We engaged key informants in interviews to elicit and scope time- and stakeholder-specific activities undertaken to implement SSB policies, always in alignment with the EPIS framework. We used an ‘activity matrix’ to organize this information, placing stakeholders’ information in rows and EPIS phases of implementation in columns. We included four stakeholder groups that would be involved with or impacted by an SSB tax: the US federal government, public health advocates, the beverage industry (both manufacturers and distributors), and sugary drink consumers. Our team’s expertise informed composition of an initial activity matrix along with descriptions in published literature [24,25,34,35,41,54–56].
We refined the activity matrix using input from key stakeholders. We sought to refine the boundaries of the cost analysis and to identify all stakeholder actions within those boundaries, even if the costs would be difficult to assess formally, or if the expected costs would likely be small. We conducted 45-minute semi-structured interviews via Zoom with nine key informants in the fall of 2020 (NRS conducted interviews with guidance from LF, KHL, and MGH). We recruited key informants purposively based on our team’s connections and using a listserv for SSB researchers and advocates. We recorded interviews for reference after obtaining a verbal consent, and the research team took notes during the interviews. All individuals had a working knowledge of SSB excise tax or warning label policies or understood similarities between tobacco and SSB policies. All understood federal policymaking procedures. Key informants included individuals with expertise in federal rulemaking and policy implementation (tobacco, n=1), public health law (tobacco and SSB policies, n=2), the beverage industry (n=1), and public health and healthy food and drink policy advocacy (n=5). Advocates represented five distinct non-profit groups and individuals could contribute information about areas of advocacy such as building strategic partnerships or effectively advocating for policy action.
In the first two interviews, we asked informants to review the draft activity matrix and provide feedback on the actions included, with particular attention to the actions taken by the stakeholder group(s) with which they were most familiar. This proved to be difficult, however, and in subsequent interviews we asked informants to describe the steps that they thought would be required to get an SSB policy (tax or warning label, or both) implemented and sustained. Using notes and recordings from each interview, we iteratively incorporated feedback from key informants into the activity matrix to improve it (see Supplementary Material Parts 1 and 2 for details about our key informant interviews, including the interview guide).
Quantitative cost assessment
We then developed estimates of the potential cost of each action (resources required to accomplish that activity, such as personnel time or equipment [37]). We inflated all costs to 2022 values [57]. Because we were interested in both societal and stakeholder perspectives, we included costs that would often be considered ‘transfers’ focusing solely on a societal perspective. For example, if a company owed $500,000 in tax, then the government would gain $500,000, but at the societal level no money would be gained or lost [34]. These costs are, however, a net loss for the company and a net gain for the government, thus we included them.
We also used probabilistic Monte Carlo analyses to generate 95% uncertainty intervals for point estimates. See Supplementary Material Parts 3–11 for full detail on how we arrived at each point estimate, embedded assumptions, probabilistic components of our calculations, additional notes, and code to replicate the results.
Results
Qualitative key informant interviews
Key informants provided important insights that helped us to refine the activity matrix that specifies actions taken by each stakeholder group for each EPIS phase (Table 2). For a full description of how key informants provided input across stakeholder groups and for specific actions (see Supplementary Material Parts 3–9). In some cases, key informants helped identify actions that were not relevant. For example, we had included legal assistance from public health legal organizations in our original activity matrix but removed it because key informants with public health legal expertise noted that such national public health legal organizations typically focus on local or state legal assistance. Key informants noted the US federal government would likely have sufficient expertise and resources to defend against industry lawsuits contesting tax policy (included in activity matrix under US government), and thus public health organizations would not contribute substantially.
Table 2.
Refined activity matrix showing likely actions required to implement a federal sugary drink tax or sugary drink warning label policy in the United States
| Stakeholder | Phase 1: Exploration | Phase 2: Preparation | Phase 3: Implementation | Phase 4: Sustainment |
|---|---|---|---|---|
| Public Health / sugary drink policy advocates | Coalition building and advocacy efforts | Mass media public outreach campaign to accompany policy rollout | ||
| Beverage Industry / sugary drink policy opposition | Anti-regulation advocacy | Lobbying against specific policies | Litigation to repeal legislation or hold up rulemaking (Tax) One-time startup costs for compliance (WL) Label redesign | Ongoing litigation Reformulation (Tax) Compliance management; payments (WL) Rotating labels, as/if required |
| US federal legislatureb | Decide on the need for sugary drink-focused policies | Enact legislation: write, introduce, debate, and vote | ||
| US Internal Revenue Serviceb (Tax policy) | (Tax) Infrastructure setup | (Tax) Ongoing tax collection; compliance education; tax revenue | ||
| US Food and Drug Administration (warning label policy) | (WL) Develop warning label requirements | (WL) Ongoing compliance monitoring | ||
| US Government (specifics unknown) | Litigation defense | Litigation defense (Tax) Distributing tax dollars | ||
| US Population | (WL) Participate in FDA studies |
Notes: WL=Warning Label policy, US=United States.
Advocacy is about an issue in general, lobbying is in reference to a specific piece of legislation.
Tobacco and alcohol excise taxes are not collected by the IRS, so there is no guarantee that sugary drink taxes would be. We have assumed the IRS for now.
Several key informants noted the importance of equitable distribution of SSB tax dollars, likely during the Sustainment phase [54,58]. How the federal government would carry this out is not clear. A recent federal SSB tax proposal directed tax revenues to the National School Breakfast Program [50], similar to the way Philadelphia supports pre-Kindergarten using a sweetened drinks tax [59]. Local jurisdictions, such as Seattle, WA, use Tax Advisory Boards to manage and distribute revenue [54]. We categorize this action as part of the federal government’s responsibilities, although neither the specifically responsible federal party nor the actions required to distribute revenue are clear.
Quantitative cost assessment
We summarize costs by stakeholder and EPIS phase in Table 3. We developed the point and interval estimates for each activity using a variety of data sources, including peer-reviewed literature, government documents, and reports. We identified these data sources through searching the literature and government websites, and with input from key informants. The elements of cost assessments varied by action (see the last column of Table 3 for a short description of the data source(s) used to cost each action). We provide a detailed explanation of how we estimated each activity cost in Supplementary Material Parts 3–9.
Table 3.
Point and interval estimates of implementation costs
| Stakeholder | EPIS Phase | Policy | Action | Timing | Estimate | Data Source(s) |
|---|---|---|---|---|---|---|
| Public Health | Exploration | Both | Coalition building | yearly until legislation | $3,420,000 ($2,140,000, $5,490,000) | Estimated using a proportion of special project expenditures by the Center for Science in the Public Interest, an example agency who might lead coalition building efforts |
| Implementation | Both | Mass media public outreach campaign to accompany policy rollout | one time | $1,050,000 ($710,000, $1,760,000) | Estimated using a proportion of annual expenditures on public education by the Center for Science in the Public Interest | |
|
| ||||||
|
Beverage Industry |
Exploration | Both | Anti-regulation advocacy | yearly | $0 | Assumed no additional advocacy costs |
| Preparation | Both | Lobbying | one time | $37,000,000 ($32,500,000, $45,300,000) | Estimated using lobbying expenditure reports from the Center for Science in the Public Interest | |
| Implementation | Both | Litigation to repeal legislation or hold up rulemaking | one time | - | ||
| Implementation | Label | Label redesign | one time | $81,100,000 ($74,900,000, $101,000,000) | Estimated cost to relabel various SSB categories (e.g., fruit drinks, sweetened carbonated beverages) using a labeling cost model developed by the Research Triangle Institute, supplemented with an estimate of unique beverages per category via Nielsen Homescan data | |
| Implementation | Tax | Startup costs for tax accounting and compliance | one time | $47,900,000 ($45,300,000, $54,200,000) | Assumed equivalent to federal government startup costs | |
| Sustainment | Both | Yearly litigation | yearly | - | ||
| Sustainment | Both | Reformulation | yearly | $0 | Assumed no reformulation costs | |
| Sustainment | Tax | Yearly internal accounting and compliance management | yearly | $23,900,000 ($22,700,000, $27,100,000) | Assumed equivalent to federal government ongoing compliance and monitoring costs | |
| Sustainment | Tax | Yearly tax payments | yearly | $15,000,000,000 | Gortmaker 2015 [39] | |
|
| ||||||
| Federal Legislature | Exploration | Both | Decide to pursue sugary drink policy | one time | $0 | Assumed no additional costs incurred |
| Preparation | Both | Enact legislation (write, introduce to committee, debate, vote) | one time | $6,900,000 ($5,980,000, $11,400,000) | Estimated using process via Wilson 2012 [87], legislator time use information from Congressional Management Foundation, legislative outlays, and number of acts passed per year | |
|
| ||||||
| Internal Revenue Service | Implementation | Tax | Develop infrastructure for tax collection and monitoring | one time | $47,900,000 ($45,300,000, $54,200,000) | Estimated using expenditure report on alcohol and tobacco tax collection costs from the Alcohol and Tobacco Tax and Trade Bureau, assumed startup costs were double the yearly compliance and monitoring costs |
| Sustainment | Tax | Yearly compliance monitoring and tax collection | yearly | $23,900,000 ($22,700,000, $27,100,000) | ||
| Sustainment | Tax | Yearly educational programs to improve tax compliance | yearly | $1,590,000 ($1,480,000, $1,790,000) | Estimated using expenditure report on educational outreach activities from the Alcohol and Tobacco Tax and Trade Bureau | |
| Sustainment | Tax | Yearly tax revenue | yearly | +$15,000,000,000 | Gortmaker 2015 [39] | |
|
| ||||||
| Food and Drug Administration | Implementation | Label | Develop warning label requirements | one time | $11,800,000 ($10,500,000, $14,500,000) | Prior Food and Drug Administration requisitions to develop vending machine labeling requirements |
| Sustainment | Label | Yearly compliance monitoring | yearly | $5,920,000 ($5,250,000, $7,240,000) | Half of startup costs to develop warning label requirements | |
|
| ||||||
| US Government (specifics unknown) | Implementation | Both | Defend against litigation | one-time | - | |
| Sustainment | Both | Defend against litigation | yearly | - | ||
| Sustainment | Tax | Distribute tax dollars | yearly | - | ||
|
| ||||||
| US Population | Implementation | Label | Participate in warning label study | one time | $25,600 ($22,300, $31,800) | Food and Drug Administration estimates of costs to conduct study of tobacco graphic warning labels |
Notes: Cells are left blank if we were not able to estimate a cost for that activity while cells with $0 indicate an assumption of no cost for that activity. Further detail on data sources and calculations are given in Appendix sections 3–9.
As one example, we expect that the US Food and Drug Administration (FDA) would have to develop a final warning label and associated regulatory requirements. The activities to accomplish this would include reviewing published literature on labels, conducting independent studies of labeling effectiveness, and writing formal rules. We used estimates of the FDA requisitions from 2012 to develop and administer restaurant menu and vending machine labeling regulations: ~$8 million 2012 dollars, and 7 full time equivalent employees [60,61]. This assumes a similar scope of work to label all SSBs in the US as for relabeling restaurant and vending machines.
Across all phases and stakeholders, the greatest cost of ~$15 billion dollars per year is for the beverage tax, a loss for the industry and gain for the federal government agency (the Internal Revenue Service) and therefore zero at the societal level. Warning labels are more costly for the beverage industry to implement initially: ~$81 million dollars to relabel using methods developed by the Research Triangle Institute [62] versus ~$48 million dollars to set up tax infrastructure using observed expenditures to collect and monitor alcohol and tobacco excise taxes [63]. We would not expect large ongoing costs for industry under a warning label policy, assuming any labeling updates can be coordinated with routine relabeling, although industry would incur substantial annual accounting and compliance costs under a tax policy. Over the long term, therefore, a tax policy would be more expensive for the beverage industry.
We encountered a structural uncertainty as to how to cost ongoing warning label compliance monitoring by the government. In our primary analysis, we assumed that startup costs to develop warning label requirements would be twice those of ongoing compliance monitoring costs, using assumptions that we and others employ when estimating startup costs for tax infrastructure [40,41]. However, other researchers used different estimates and methods, for example, assuming $500,000 in annual monitoring costs in an economic evaluation of added sugar labels [60], using a specified number of full time compliance monitors in a study including restaurant and menu relabeling policies [39], or using 25% of industry relabeling costs in a study of processed meat health warning labels [56]. These methods produced markedly different results that ranged from $500,000 to ~$20 million (Table 4); all are lower than our estimate of ~$24 million as ongoing costs to collect and monitor tax compliance.
Table 4.
Sensitivity analysis of compliance and monitoring costs of warning labels
| Description | Cost |
|---|---|
| Primary Analysis: half of the startup costs for labels (similar to tax policy logic), where first year startup costs include ~$8 million in requisitions plus 7 FTEs | $5,920,000 |
| Huang et al: $500,000 yearly based on internal FDA documents | $500,000 |
| 50 FTEs: based on Gortmaker 2015 use of FTEs for ongoing compliance monitoring. Use 50 state FTEs (1/state) at $85,000 and 2 FDA-level FTEs at $100,000 to benchmark person costs to monitor compliance. | $5,830,000 |
| 100 FTEs: based on Gortmaker 2015 use of FTEs for ongoing compliance monitoring. Use 100 state FTEs (2/state) at $85,000 and 2 FDA-level FTEs at $100,000 to benchmark person costs to monitor compliance. | $11,400,000 |
| Kim et al: 25% of industry relabeling costs to estimate government ongoing compliance monitoring (industry labeling costs helpful to benchmark the scope required) | $20,300,000 |
Discussion and recommendations
We used TD-ABC to assess the costs to implement an SSB excise tax or health warning label policy, employing mixed methods and implementation science principles. Because a nuanced understanding of time- and stakeholder-dependent implementation costs is so critical in public health decision making, we believe results from TD-ABC approaches offer the best guidance for decision making because it improves the specificity of costs assessed [6,37]. It also provides the foundation for cost cost-effectiveness and budget impact analyses [64–66]. However, using TD-ABC can be difficult as it involves making assumptions about likely actions. It can also be time-consuming as it requires researchers to set boundaries and decide what actions to include through the implementation process, and to quantify the costs of each.
Using an established implementation science framework along with stakeholder engagement increased our confidence that we had accounted for all relevant actions. This is particularly important for complex policy implementation efforts that involve a variety of stakeholders and unfold over time. Other implementation science frameworks provide alternative ways to conceptualize the stages of implementation [67] or determinants of successful implementation, and thus what sorts of costs should be considered [68].
However, frameworks do not drive decisions about what specific costs analysts should consider. For that, context specific knowledge and stakeholder engagement are key [38]. More work is needed to examine how best to engage stakeholders in the costing process. Initial interviews proved difficult where we asked key informants to provide feedback on our draft activity matrix. Systems science methods, like system support mapping, might be more suitable because they offer a more structured approach to outlining stakeholder responsibilities and required resources [38,69]. Another systems science method, causal loop diagramming, focuses on diagramming how different elements are related and influence each other [70,71]. Thinking through these dynamics using group model building activities [72] could help ensure that all relevant costs are included through a better understanding of system boundaries.
We focus here on the costs for implementing and sustaining each policy. Thus, we do not include additional costs commonly included in policy evaluation such as downstream healthcare costs averted. Nor have we included a comprehensive assessment of other critically important factors that could influence policy decision making that may differ by stakeholder [73–75]. Complete policy analyses should always acknowledge that although costs may be an important outcome to consider, others may be equally -- if not more -- important. For example, the U.S. Centers for Disease Control and Prevention’s Policy Analytical Framework guides analysts to consider public health impact (mechanism of action, magnitude of effects, impacts on health equity), feasibility (political and operational), and economic and budgetary impacts of policies (cost-effectiveness, distributional effects, and budgetary impact) when making decisions about what policies to prioritize and pursue [76]. A full analysis of each of these factors is beyond the scope of our analysis, as we aimed to provide research information to inform policy analysts’ consideration of the economic and budgetary impacts of SSB excise taxes and warning labels.
We also focused on potential costs of each policy if implemented independently, rather than in tandem. Implementing both policies at once would likely result in greater total benefits via targeting both prices and intentions to purchase, though to our knowledge no evidence on dual implementation exists. However, political capital and will to do so is often limited. Our results can help inform such decisions because we highlight unique actions required for each of the policies as well as areas of potential overlap in actions required for implementation, suggesting possible efficiencies.
Our analyses focuses on general stakeholder groups (e.g., the beverage industry, public health). This limits our analysis because we do not specify how more specific members of each stakeholder group would incur costs – for example, the beverage industry group in our analysis includes large manufacturers (such as The Coca-Cola Company, typically not the taxed entity, but one likely to lobby the government) and distributors, typically responsible for paying excise taxes. Future analyses to inform policy at lower ‘levels’ such as local government may benefit from additional specificity. At all levels of government, we recommend that policy makers inform key stakeholder groups and key policy players whether local, state, or federal, about policies of interest. Key policy players include those responsible for making the decision (for example, the US Congress) and those groups impacted by a decision (beverage industry). While not every stakeholder is responsible for decision making, all stakeholders will be affected by decisions; without understanding those effects, decision makers run the risk of choosing a less than optimal action, or one that may fail [10].
Our results contain a great deal of uncertainty. Qualitatively, key informants often noted that all policies are different, and this created difficulties for specifying what actions might or might not happen. Quantitatively, we often had to make major assumptions in estimating cost for actions because values were nonexistent. As in the case of litigation around policies, we sometimes could not estimate costs at all. Because of our inability to estimate some costs as well as difficulties defining specific timing of costs (beyond general EPIS phases), we also did not aggregate costs into societal nor stakeholder-specific totals and did not discount future costs.
We believe that our mixed-methods approach helped ensure that the outcomes of our work were useful, even with such substantial uncertainty: we were able to understand what actions could be expected. This helps decision makers to anticipate potential costs and cost-drivers for more informed decision making (we acknowledge that costs are not and should not be the only driver of decisions). Understanding where pushback could arise because of costs (for example, industry relabeling) is important. Stakeholders may waste resources advocating for or pursuing policies that may encounter substantial pushback, including litigation. Where policies may be important to pursue even with expectations of substantial pushback, advocates should be prepared to respond and sustain the policy (for example, by committing resources to defend against litigation).
We also identified major limitations in data availability for these types of analyses; this helps us to define future research and data needs. Teams with greater resources to draw on could conduct primary data collection to better quantify the cost of different actions. They could collect new data or work with government partners to improve quantitative estimates. Even if researchers collect additional data, they should still expect uncertainty. And all should acknowledge uncertainty in cost estimates and clearly incorporate, communicate, and visualize them. We used probabilistic Monte Carlo simulation to generate a range of potential cost estimates and worked to present our results in a way that communicates the levels of uncertainty. Researchers can glean insights from research in data visualization and uncertainty communication [77–79] and research translation [80–84] to consider how best to communicate and visualize uncertainty in a useful way for policymakers.
Conclusions
Our work provides new evidence on the potential costs of implementing and sustaining SSB policies at the federal level by outlining clear, specific information about what would be required of stakeholders throughout the implementation process. We explicitly integrated health economics and implementation science and used mixed methods, which helped us to identify stakeholder- and time-dependent costs throughout the implementation process and improved our confidence in the comprehensiveness of our results. Although our work is specific to the costs of SSB policies, the process we lay out can be used to estimate the costs of policies targeting other public health issues, whether local, state, or national, to provide crucial information for public health policy decision making.
Supplementary Material
Key messages:
Sugary drink policies require resources across the policy life cycle to ensure that policies are implemented with fidelity and sustained over time.
Mixed methods can help ensure that assessments of sugary drink policy costs are comprehensive and clear, improving their use for decision makers.
Applying similar approaches to understanding implementation costs of other public health policies can help decision makers recognize the full scope of what it would take to implement policies, helping to inform public health decision making in the US and globally.
Footnotes
Supplementary Material: Supplementary material (Parts 1-11) includes details about the key informant interviews, how we arrived at each point estimate and embedded assumptions, probabilistic components of our calculations, additional notes, and code to replicate the results, and is provided here <TYPESETTERS ADD LINK>.
Conflict of Interest Statement
On behalf of all authors, the corresponding author states that there is no conflict of interest.
References
- 1.Pollack Porter KM, Rutkow L, McGinty EE. The Importance of Policy Change for Addressing Public Health Problems. Public Health Rep. 2018;133(1_suppl):9S–14S. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Emmons KM, Chambers D, Abazeed A. Embracing policy implementation science to ensure translation of evidence to cancer control policy. Transl Behav Med. 2021. Nov 30;11(11):1972–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Centers for Disease Control and Prevention. CDC - 10 Essential Public Health Services - Public Health Infrastructure Center [Internet], 2020. [cited 2023 Jun 21]. Available from: https://www.cdc.gov/publichealthgateway/publichealthservices/essentialhealthservices.html
- 4.Teutsch SM, Glied S, Roy K. Strengthening the Use of Economics in Informing U.S. Public Health Policy. Am J Prev Med. 2016;50(5):S1–3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Rabarison KM, Bish CL, Massoudi MS, Giles WH. Economic Evaluation Enhances Public Health Decision Making. Front Public Health. 2015. Jun;3(164). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Crowley DM, Dodge KA, Barnett WS, Corso P, Duffy S, Graham P, et al. Standards of Evidence for Conducting and Reporting Economic Evaluations in Prevention Science. Prev Sci Off J Soc Prev Res. 2018;19(3):366–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Gold HT, McDermott C, Hoomans T, Wagner TH. Cost data in implementation science: categories and approaches to costing. Implement Sci. 2022. Jan 28;17(1):11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Dopp AR, Kerns SEU, Panattoni L, Ringel JS, Eisenberg D, Powell BJ, et al. Translating economic evaluations into financing strategies for implementing evidence-based practices. Implement Sci. 2021. Jun 29;16(1):66. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Sanders GD, Neumann PJ, Basu A, Brock DW, Feeny D, Krahn M, et al. Recommendations for Conduct, Methodological Practices, and Reporting of Cost-effectiveness Analyses: Second Panel on Cost-Effectiveness in Health and Medicine. Jama. 2016/09/14 ed. 2016. Sep 13;316(10):1093–103. [DOI] [PubMed] [Google Scholar]
- 10.Eisman AB, Quanbeck A, Bounthavong M, Panattoni L, Glasgow RE. Implementation science issues in understanding, collecting, and using cost estimates: a multi-stakeholder perspective. Implement Sci. 2021. Aug 3;16(1):75. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Eisman AB, Kilbourne AM, Dopp AR, Saldana L, Eisenberg D. Economic evaluation in implementation science: Making the business case for implementation strategies. Psychiatry Res. 2020. Jan 1 ;283:112433. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Crable EL, Lengnick-Hall R, Stadnick NA, Moullin JC, Aarons GA. Where is “policy” in dissemination and implementation science? Recommendations to advance theories, models, and frameworks: EPIS as a case example. Implement Sci. 2022. Dec 12;17(1):80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Oh A, Abazeed A, Chambers DA. Policy Implementation Science to Advance Population Health: The Potential for Learning Health Policy Systems. Front Public Health [Internet], 2021. [cited 2023 Feb 14];9. Available from: 10.3389/fpubh.2021.681602 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Emmons KM, Chambers DA. Policy Implementation Science - An Unexplored Strategy to Address Social Determinants of Health. Ethn Dis. 2021;31(1):133–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Allen P, Pilar M, Walsh-Bailey C, Hooley C, Mazzucca S, Lewis CC, et al. Quantitative measures of health policy implementation determinants and outcomes: a systematic review. Implement Sci. 2020. Jun 19;15(1):47. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Bullock HL, Lavis JN, Wilson MG, Mulvale G, Miatello A. Understanding the implementation of evidence-informed policies and practices from a policy perspective: a critical interpretive synthesis. Implement Sci IS. 2021. Feb 15;16(1):18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.McGinty EE, Seewald NJ, Bandara S, Cerdá M, Daumit GL, Eisenberg MD, et al. Scaling Interventions to Manage Chronic Disease: Innovative Methods at the Intersection of Health Policy Research and Implementation Science. Prev Sci [Internet], 2022. Sep 1 [cited 2022 Sep 1]; Available from: 10.1007/s11121-022-01427-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Center for Tobacco Products. Cigarette Labeling and Health Warning Requirements. FDA [Internet], 2021. Jul 9 [cited 2021 Aug 3]; Available from: https://www.fda.gov/tobacco-products/labeling-and-warning-statements-tobacco-products/cigarette-labeling-and-health-warning-requirements
- 19.Centers for Medicare and Medicaid Services. Medicare Program; Revisions to Payment Policies Under the Physician Fee Schedule and Other Revisions to Part B for CY 2018; Medicare Shared Savings Program Requirements; and Medicare Diabetes Prevention Program [Internet], 2017. [cited 2021 Aug 3]. Available from: https://www.federalregister.gov/documents/2017/11/15/2017-23953/medicare-program-revisions-to-payment-policies-under-the-physician-fee-schedule-and-other-revisions [PubMed]
- 20.Purtle J, Stadnick NA. Earmarked Taxes as a Policy Strategy to Increase Funding for Behavioral Health Services. Psychiatr Serv Wash DC. 2020. Jan 1;71(1):100–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Purtle J, Brinson K, Stadnick NA. Earmarking Excise Taxes on Recreational Cannabis for Investments in Mental Health: An Underused Financing Strategy. JAMA Health Forum. 2022. Apr 1;3(4):e220292. [DOI] [PubMed] [Google Scholar]
- 22.Moullin JC, Dickson KS, Stadnick NA, Becan JE, Wiley T, Phillips J, et al. Exploration, Preparation, Implementation, Sustainment (EPIS) framework. In: Handbook on Implementation Science. Edward Elgar Publishing; 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Chriqui JF, Sansone CN, Powell LM. The Sweetened Beverage Tax in Cook County, Illinois: Lessons From a Failed Effort. Am J Public Health. 2020;110(7):1009–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Chriqui JF, Pipito AA, Asada Y, Powell LM. Lessons learned from the adoption and implementation of sweetened beverage taxes in the United States: a narrative review. 2021. [Google Scholar]
- 25.Falbe J, Grummon AH, Rojas N, Ryan-Ibarra S, Silver LD, Madsen KA. Implementation of the First US Sugar-Sweetened Beverage Tax in Berkeley, CA, 2015–2019. Am J Public Health. 2020. Jul 16;110(9):1429–37. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Rhoads N, Martin S, Zimmerman FJ. Passing a Healthy Homes Initiative: Using Modeling to Inform Evidence-Based Policy Decision Making in Kansas City, Missouri. J Public Health Manag Pract. 2021. Dec;27(6):539–45. [DOI] [PubMed] [Google Scholar]
- 27.Krieger J, Bleich SN, Scarmo S, Ng SW. Sugar-Sweetened Beverage Reduction Policies: Progress and Promise. Annu Rev Public Health. 2021;42(1):null. [DOI] [PubMed] [Google Scholar]
- 28.The Global Food Research Program. Sugary Drink Policy Maps [Internet], 2022. [cited 2022 Aug 17]. Available from: https://www.globalfoodresearchprogram.org/resources/maps/
- 29.Pomeranz JL. Advanced policy options to regulate sugar-sweetened beverages to support public health. J Public Health Policy. 2012;33(1):75–88. [DOI] [PubMed] [Google Scholar]
- 30.Li H, Zou Y, Ding G. Dietary factors associated with dental erosion: a meta-analysis. PloS One. 2012;7(8):e42626. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Malik VS, Hu FB. Sugar-Sweetened Beverages and Cardiometabolic Health: An Update of the Evidence. Nutrients. 2019. Aug 8;11(8):E1840. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Valenzuela MJ, Waterhouse B, Aggarwal VR, Bloor K, Doran T. Effect of sugar-sweetened beverages on oral health: a systematic review and meta-analysis. Eur J Public Health. 2021. Feb 1;31(1):122–9. [DOI] [PubMed] [Google Scholar]
- 33.Smith NR, Grummon AH, Ng SW, Wright ST, Frerichs L. Simulation models of sugary drink policies: A scoping review. PloS One. 2022;17(10):e0275270. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Wilde P, Huang Y, Sy S, Abrahams-Gessel S, Jardim TV, Paarlberg R, et al. Cost-Effectiveness of a US National Sugar-Sweetened Beverage Tax With a Multistakeholder Approach: Who Pays and Who Benefits. Am J Public Health. 2019;109(2):276–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Du M, Griecci CF, Kim DD, Cudhea F, Ruan M, Eom H, et al. Cost-Effectiveness of a National Sugar-Sweetened Beverage Tax to Reduce Cancer Burden and Disparities in the United States. JNCI Cancer Spectr. 2020; [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Lee Y, Mozaffarian D, Sy S, Liu J, Wilde PE, Marklund M, et al. Health Impact and Cost-Effectiveness of Volume, Tiered, and Absolute Sugar Content SugarSweetened Beverage Tax Policies in the United States: A Microsimulation Study. Circulation. 2020; [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Cidav Z, Mandell D, Pyne J, Beidas R, Curran G, Marcus S. A pragmatic method for costing implementation strategies using time-driven activity-based costing. Implement Sci. 2020. May 5;15(1):28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.O’Leary MC, Hassmiller Lich K, Frerichs L, Leeman J, Reuland DS, Wheeler SB. Extending analytic methods for economic evaluation in implementation science. Implement Sci. 2022. Apr 15;17(1):27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Gortmaker SL, Wang YC, Long MW, Giles CM, Ward ZJ, Barrett JL, et al. Three Interventions That Reduce Childhood Obesity Are Projected To Save More Than They Cost To Implement. Health Aff Millwood. 2015/11/04 ed. 2015. Nov;34(11):1932–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Long MW, Polacsek M, Bruno P, Giles CM, Ward ZJ, Cradock AL, et al. Cost-Effectiveness Analysis and Stakeholder Evaluation of 2 Obesity Prevention Policies in Maine, US. J Nutr Educ Behav. 2019;51(10):1177–87. [DOI] [PubMed] [Google Scholar]
- 41.Long MW, Gortmaker SL, Ward ZJ, Resch SC, Moodie ML, Sacks G, et al. Cost Effectiveness of a Sugar-Sweetened Beverage Excise Tax in the U.S. Am J Prev Med. 2015. Jul;49(1):112–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Grummon AH, Hall MG. Sugary drink warnings: A meta-analysis of experimental studies. PLOS Med. 2020;17(5):e1003120. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Andreyeva T, Marple K, Marinello S, Moore TE, Powell LM. Outcomes Following Taxation of Sugar-Sweetened Beverages: A Systematic Review and Meta-analysis. JAMA Netw Open. 2022. Jun 1;5(6):e2215276. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Petimar J, Gibson LA, Yan J, Bleich SN, Mitra N, Trego ML, et al. Sustained Impact of the Philadelphia Beverage Tax on Beverage Prices and Sales Over 2 Years. Am J Prev Med. 2022. Jun 1;62(6):921–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Edmondson EK, Roberto CA, Gregory EF, Mitra N, Virudachalam S. Association of a sweetened beverage tax with soda consumption in high school students. JAMA Pediatr. 2021;175(12):1261–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Monning B. Sugar-sweetened beverages: safety warnings [Internet]. SB-347 California, USA; 2019. Available from: http://leginfo.legislature.ca.gov/faces/billStatusClient.xhtml?bill_id=201920200SB347
- 47.Bloom R. California Community Health Fund [Internet]. AB-138 California, USA; 2019. Available from: http://leginfo.legislature.ca.gov/faces/billStatusClient.xhtml?bill_id=201920200AB138
- 48.Robinson Carlyle, Conway Dhingra, Pedersen Saldaña. Funding public health services and health equity initiatives through a statewide sweetened beverage tax. [Internet]. SB 5371 2021. Available from: https://apps.leg.wa.gov/billsummary/?BillNumber=5371&Year=2021&Initiative=false
- 49.Block JP. The calorie-labeling Saga—Federal Preemption and delayed implementation of public health law. N Engl J Med. 2018;379(2):103–5. [DOI] [PubMed] [Google Scholar]
- 50.DeLauro RL. SWEET Act [Internet]. H.R.2772 Apr 22, 2021. Available from: https://www.congress.gov/bill/117th-congress/house-bill/2772
- 51.Dopp AR, Mundey P, Beasley LO, Silovsky JF, Eisenberg D. Mixed-method approaches to strengthen economic evaluations in implementation research. Implement Sci. 2019. Jan 11;14(1):2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Golden SD, Moreland-Russell S. Public policy explained. In: Prevention, policy, and public health. Oxford University Press; New York, NY; 2016. p. 17–39. [Google Scholar]
- 53.Creswell JW. A concise introduction to mixed methods research. SAGE publications; 2014. [Google Scholar]
- 54.Asada Y, Pipito AA, Chriqui JF, Taher S, Powell LM. Oakland’s Sugar-Sweetened Beverage Tax: Honoring the “Spirit” of the Ordinance Toward Equitable Implementation. Health Equity. 2021/03/09 ed. 2021;5(1):35–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Basto-Abreu A, Barrientos-Gutiérrez T, Vidaña-Pérez D, Colchero MA, Hernández-F. M, Hernández-Ávila M, et al. Cost-Effectiveness Of The Sugar-Sweetened Beverage Excise Tax In Mexico. Health Aff (Millwood). 2019Nov 1;38(11):1824–31. [DOI] [PubMed] [Google Scholar]
- 56.Kim DD, Wilde PE, Michaud DS, Liu J, Lizewski L, Onopa J, et al. Cost Effectiveness of Nutrition Policies on Processed Meat: Implications for Cancer Burden in the U.S. Am J Prev Med. 2019. Nov 1;57(5):e143–52. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.US Bureau of Labor Statistics. CPI Inflation Calculator [Internet], Available from: https://www.bls.gov/data/inflation_calculator.htm
- 58.Krieger J, Magee K, Hennings T, Schoof J, Madsen KA. How sugar-sweetened beverage tax revenues are being used in the United States. Prev Med Rep. 2021. Sep 1;23:101388. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Clarke Henon. Amending Title 19 of The Philadelphia Code, entitled “Finance, Taxes and Collections,” by adding a new Chapter 19-4100, entitled “Sugar-Sweetened Beverage Tax,” under certain terms and conditions. [Internet], 160176 Mar 16, 2016. Available from: https://phila.legistar.com/LegislationDetail.aspx?ID=2595907&GUID=36060B21-D7EE-4D50-93E7-8D2109D47ED1&FullText=1
- 60.Huang Y, Kypridemos C, Liu J, Lee Y, Pearson-Stuttard J, Collins B, et al. Cost-Effectiveness of the US Food and Drug Administration Added Sugar Labeling Policy for Improving Diet and Health. Circulation. 2019. Jun 4;139(23):2613–24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Food and Drug Administration. Justification of Estimates for Appropriations Committees. 2012. p. 29. [Google Scholar]
- 62.Muth M, Bradley S, Brophy J, Capogrossi K, Coglaiti M, Karns S. 2014 FDA labeling cost model. US Food Drug Adm [Internet], 2015; Available from: https://www.rti.org/publication/2014-fda-labeling-cost-model
- 63.Alcohol and Tobacco Tax and Trade Bureau. Annual Report Fiscal Year 2021 [Internet], 2022. Available from: https://www.ttb.gov/about-ttb/plans-and-reports-annual-reports
- 64.Bilinski A, Neumann P, Cohen J, Thorat T, McDaniel K, Salomon JA. When cost-effective interventions are unaffordable: Integrating cost-effectiveness and budget impact in priority setting for global health programs. PLOS Med. 2017. Oct 2;14(10):e1002397. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Bilinski A, MacKay E, Salomon JA, Pandya A. Affordability and Value in Decision Rules for Cost-Effectiveness: A Survey of Health Economists. Value Health. 2022. Jul 1;25(7):1141–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Sullivan SD, Mauskopf JA, Augustovski F, Jaime Caro J, Lee KM, Minchin M, et al. Budget Impact Analysis—Principles of Good Practice: Report of the ISPOR 2012 Budget Impact Analysis Good Practice II Task Force. Value Health. 2014. Jan 1;17(1):5–14. [DOI] [PubMed] [Google Scholar]
- 67.Saldana L, Ritzwoller DP, Campbell M, Block EP. Using economic evaluations in implementation science to increase transparency in costs and outcomes for organizational decision-makers. Implement Sci Commun. 2022. Apr 11;3(1):40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Damschroder LJ, Reardon CM, Opra Widerquist MA, Lowery J. Conceptualizing outcomes for use with the Consolidated Framework for Implementation Research (CFIR): the CFIR Outcomes Addendum. Implement Sci. 2022. Jan 22;17(1):7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69.Calancie L, Margolis L, Chall SA, Mullenix A, Chaudhry A, Hassmiller Lich K. System Support Mapping: A Novel Systems Thinking Tool Applied to Assess the Needs of Maternal and Child Health Title V Professionals and Their Partners. J Public Health Manag Pract JPHMP. 2020. Aug;26(4):E42–53. [DOI] [PubMed] [Google Scholar]
- 70.Mills SD, Golden SD, O’Leary MC, Logan P, Lich KH. Using systems science to advance health equity in tobacco control: a causal loop diagram of smoking. Tob Control [Internet], 2021. Sep 17 [cited 2022 Jun 7]; Available from: http://tobaccocontrol.bmj.com/content/early/2021/09/16/tobaccocontrol-2021-056695 [DOI] [PMC free article] [PubMed]
- 71.Lich KH, Ginexi EM, Osgood ND, Mabry PL. A call to address complexity in prevention science research. Prev Sci. 2013;14(3):279–89. [DOI] [PubMed] [Google Scholar]
- 72.Hovmand PS. Group model building and community-based system dynamics process. In: Community Based System Dynamics. New York, NY: Springer; 2014. p. 17–30. [Google Scholar]
- 73.Brownson RC, Fielding JE, Maylahn CM. Evidence-Based Public Health: A Fundamental Concept for Public Health Practice. Annu Rev Public Health. 2009. Apr 1;30(1):175–201. [DOI] [PubMed] [Google Scholar]
- 74.Lobstein T, Neveux M, Landon J. Costs, equity and acceptability of three policies to prevent obesity: A narrative review to support policy development. Obes Sci Pract. 2020. Oct;6(5):562–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 75.Smith NR, Knocke KE, Hassmiller Lich K. Using decision analysis to support implementation planning in research and practice. Implement Sci Commun. 2022. Jul 30;3(1):83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 76.Office of the Associate Director for Policy and Strategy. CDC’s Policy Analytical Framework [Internet], 2019. [cited 2021 Sep 24]. Available from: https://www.cdc.gov/policy/analysis/process/analysis.html
- 77.Padilla LM, Creem-Regehr SH, Hegarty M, Stefanucci JK. Decision making with visualizations: a cognitive framework across disciplines. Cogn Res Princ Implic. 2018;3(1):1–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 78.Franconeri SL, Padilla LM, Shah P, Zacks JM, Hullman J. The Science of Visual Data Communication: What Works. Psychol Sci Public Interest. 2021. Dec 1;22(3):110–61. [DOI] [PubMed] [Google Scholar]
- 79.Trevena LJ, Bonner C, Okan Y, Peters E, Gaissmaier W, Han PKJ, et al. Current Challenges When Using Numbers in Patient Decision Aids: Advanced Concepts. Med Decis Making. 2021. Oct 1;41(7):834–47. [DOI] [PubMed] [Google Scholar]
- 80.Niederdeppe J, Roh S, Dreisbach C. How Narrative Focus and a Statistical Map Shape Health Policy Support Among State Legislators. Health Commun. 2016;31(2):242–55. [DOI] [PubMed] [Google Scholar]
- 81.Purtle J. Disseminating Evidence to Policymakers: Accounting for Audience Heterogeneity. In: Weber MS, Yanovitzky I, editors. Networks, Knowledge Brokers, and the Public Policymaking Process [Internet], Cham: Springer International Publishing; 2021. [cited 2021 Nov 8]. p. 27–48. Available from: 10.1007/978-3-030-78755-4_2 [DOI] [Google Scholar]
- 82.Smith NR, Mazzucca S, Hall MG, Hassmiller Lich K, Brownson RC, Frerichs L. Opportunities to improve policy dissemination by tailoring communication materials to the research priorities of legislators. Implement Sci Commun. 2022. Mar 4;3:24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 83.Brownson RC, Dodson EA, Kerner JF, Moreland-Russell S. Framing research for state policymakers who place a priority on cancer. Cancer Causes Control CCC. 2016. Aug;27(8):1035–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 84.Purtle J, Nelson KL, Gebrekristos L, Lê-Scherban F, Gollust SE. Partisan differences in the effects of economic evidence and local data on legislator engagement with dissemination materials about behavioral health: a dissemination trial. Implement Sci. 2022. Jun 22;17(1):38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 85.Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA. Systematic review of the exploration, preparation, implementation, sustainment (EPIS) framework. Implement Sci. 2019;14(1):1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 86.Basu S, Seligman H, Bhattacharya J. Nutritional policy changes in the supplemental nutrition assistance program: a microsimulation and cost-effectiveness analysis. Med Decis Mak. 2013/07/03 ed. 2013. Oct;33(7):937–48. [DOI] [PubMed] [Google Scholar]
- 87.Wilson N, Nghiem N, Foster R, Cobiac L, Blakely T. Estimating the cost of new public health legislation. Bull World Health Organ. 2012;90:532–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
