Integrating Financing Variables into Behavioral Health Policy Implementation Research

High rates of overdose, suicide, and mental health crises in the United States have promoted a flurry of public policies aimed at improving behavioral health outcomes. Precise measurement of all relevant variables is critical to evaluating the effects of these policies. Recent articles have provided guidance to improve measurement in health policy research—such as by accounting for nuances in policy design,¹ clusters of similar policies within jurisdictions,² and variations in policy exposure intensity across groups.³ An acknowledged,^4,5 but less discussed, area relates to measuring policy implementation.

Long studied in the field of public administration research, policy implementation can be defined as the process through which government agencies engage—or do not engage—in activities to support the execution of a statute or regulation.⁶ In health policy research, policy implementation variables can determine the degree to which policies “on the books” impact population-level health outcomes.⁵ The specific implementation variables that have the most influence on a policy’s success vary according to the specific policy and outcome of interest. Financing, however, is almost always necessary—even if not sufficient—for government agencies to meaningfully engage in activities that foster successful policy implementation.

In this Viewpoint we offer guidance to advance measurement of financing variables in evaluations of behavioral health policies. We focus on public policies, but the guidance also largely applies to the evaluation of private sector policies. Here we offer a four-step process to advance work at this intersection of public administration, implementation science, and health policy research.

1. Conceptualize the Role of Financing in Policy Implementation Success

The first step is to identify key implementation variables that might influence the relationship between a policy and an outcome, and then to consider how financing could affect these variables. Causal pathway diagrams, directed acyclic graphs, and other visual presentations can be useful at this step. Table 1 provides examples of policy implementation financing variables and possible applications in behavioral health policy research.

Table 1.

Example Domains of Policy Implementation Financing Variables and Potential Applications in Behavioral Health Policy Research

Domain of Policy Implementation Financing Variable	State Behavioral Health Policy Example	Research Question/ Hypothesis Example
Funding for goods: Funding to purchase tangible goods essential for policy success	Funds for bulk purchases of naloxone following passage of state naloxone access laws	Research question: Does the amount budgeted for naloxone per state resident mediate the relationship between state passage of a naloxone access law and the overdose death rate?
Funding for staffing: Funding to hire providers that offer services related to a policy and are typically not reimbursable through existing sources, such as health insurance	State funding earmarked for hiring counselors at 988 call centers following creation of the 988 Suicide and Crisis Lifeline by federal law	Research question: Does state adoption of a 988 telecom fee impact 988 call center staffing ratios, call wait times, call abandonment rates, and moderate the association between 988’s launch and state suicide death rates?
Funding for marketing: Funding for communication campaigns to increase awareness about, and prompt behavior changes related to, a policy	Funds for communication campaigns to cultivate knowledge about the existence and logistics of state Extreme Risk Protection Order (ERPO) laws as they relate to suicide risk	Hypothesis: The effect of state ERPO laws on state firearm suicide death rates is moderated by the existence of state funding earmarked for communication campaigns about ERPO laws
Funding for enforcement: Funding to monitor and enforce policy compliance	Funds for dedicated staff to monitor and enforce compliance with state behavioral health parity laws	Hypothesis: the effect of comprehensive state behavioral health parity laws on state rates of mental health and substance treatment utilization among a state’s commercially insured population is mediated by the per capita amount of state funding earmarked for parity compliance monitoring and enforcement

2. Acquire and Interpret Relevant Financing Data

The second step is to acquire and interpret data related to the financing variables identified. Enacted state budgets, which are readily available on state websites, are a good source of information about funds earmarked by the legislature for policy implementation activities. However, the level of detail at which budgets are itemized varies across states. Information about funds earmarked for policy implementation can also be included in non-budget bills (e.g., the bill that created the policy that is being evaluated). It should be noted, however, that budget data represent a plan for future spending while expenditure data reflect actual audited spending. Budget and expenditure figures might not be interchangeable, and expenditure data are often better indicators of financial resources to support policy implementation because they measure financial resources actually spent. Most state and local governments also make their Annual Comprehensive Financial Reports available on their websites.

The U.S. Substance Abuse and Mental Health Services Administration’s Uniform Reporting System and NRI’s datasets provide detailed data about state behavioral health agencies’ annual expenditures. Resources such as the U.S. Census’ Annual Survey of State and Local Government Finances and Census of Governments and the National Association of State Budget Officers also centralize executive branch agencies’ expenditure data. State and local Freedom of Information Act requests can also yield relevant expenditure data.

3. Operationalize Financing Variables

The third step is to operationalize policy implementation financing variables. One approach is to create a continuous measure of annual per capita expenditures using relevant state or local expenditure (numerator) and population size (denominator) data. To enhance precision, the denominator could be limited to the population that would be most affected by the policy implementation activities (e.g., youth ages 12–18 in the case of school mental health sick day policies).³ An alternative approach is to create a dichotomous variable that indicates whether there is record of any funds budgeted or spent on the policy implementation activity of interest in a given year.

An important consideration when linking financing variables to outcomes is that states’ fiscal years are not uniform, nor do they align with calendar years. For example, 19 states enact a biennial budget every two years, the fiscal year starts on July 1 in 46 states, and on different date in four states. Some county fiscal years also differ from their state (e.g., NY). When creating financing variables, it is often appropriate to annualize biennial budgets and expenditure data so that they align with annual outcome data.

4. Model Financing Variables

The final step is modeling how implementation financing variables contribute to a policy’s effect on outcomes of interest. Implementation financing variables can be treated as mediators and causal mediation analysis can be used to estimate the amount of a policy’s effect that is dependent on financing for different policy implementation activities.⁷ In such analyses, financing for policy implementation is conceptualized as being on the causal pathway between the policy and the outcome. In this scenario, the policy is presumed to affect the mediator. The mediation approach is therefore most appropriate when financing is part of the “package” of provisions within the policy itself (e.g., a law creates a new program and funds that program).

Implementation financing variables can also be treated as moderating variables in policy evaluations. In these instances, financing for policy implementation is conceptualized as influencing the strength of the relationship between the policy and the outcome (i.e., assessing the significance of a policy*implementation financing interaction term in a model). The moderation approach is appropriate in instances when funding for policy implementation is from different sources across different time periods.

Conclusion

Integrating financing variables into behavioral health policy research can help elucidate how and the conditions under which a policy works. Further, such research can generate information about the true fiscal resources needed for a policy to achieve intended outcomes; information that is important to public policymakers.⁵ We contend that the process outlined above can advance understanding of the ways in which money matters to the successful implementation and impact of behavioral health policies.