Perspectives on Financing Strategies for Evidence-Based Treatment Implementation in Youth Mental Health Systems

Abstract

Background:

Evidence-based treatments (EBTs) are critical to effectively address mental health problems among children and adolescents, but costly for mental health service agencies to implement and sustain. Financing strategies help agencies overcome cost-related barriers by obtaining financial resources to support EBT implementation and/or sustainment.

Aims:

We sought to (1) understand how youth mental health system decision-makers involved with EBT implementation and sustainment view key features (e.g., relevance, feasibility) that inform financing strategy selection and (2) compare service agency, funding agency, and intermediary representative perspectives.

Method:

Two surveys were disseminated to 48 representatives across U.S. youth mental health service agencies, funding agencies, and intermediaries who were participating in a larger study of financing strategies. Quantitative and qualitative data were gathered on 23 financing strategies through quantitative ratings and open-ended responses. Data were analyzed using descriptive statistics and rapid content analysis.

Results:

The financing strategies rated as most relevant include braided funding streams, contracts for EBTs, credentialing/rostering providers, fee-for-service reimbursement (regular and increased), and grant funding. All other strategies were unfamiliar to 1/3 to 1/2 of participants. The six strategies were rated between somewhat and quite available, feasible, and effective for EBT sustainment. For sustaining different EBT components (e.g., delivery, materials), the mix of financing strategies was rated as somewhat adequate. Qualitative analysis revealed challenges with strategies being non-recurring or unavailable in representatives’ regions. Ratings were largely similar across participant roles, though funding agency representatives were the most familiar with financing strategies.

Discussion:

Despite the breadth of innovative financing strategies, expert representatives within the youth mental health services ecosystem had limited knowledge of most options. Experts relied on strategies that were familiar but often did not adequately support EBT implementation or sustainment. These findings underscore more fundamental issues with under-resourced mental health systems in the U.S.; financing strategies can help agencies navigate EBT use but must be accompanied by larger-scale system reforms. Limitations include difficulties generalizing results due to using a small sample familiar with EBTs, high agreement as a potential function of snowball recruiting, and limited responses to the open-ended survey questions.

Implications for Health Care Provision and Use:

Although EBTs have been found to effectively address mental health problems in children and adolescents, available strategies for financing their implementation and sustainment in mental health systems are insufficient. This constraint prevents many children and adolescents from receiving high-quality services.

Implications for Health Policies:

Financing strategies alone cannot solve systematic issues that prevent youth mental health service agencies from providing EBTs. Policy changes may be required, such as increased financial investment from the U.S. government into mental health services to support basic infrastructure (e.g., facility operations, measuring outcomes).

Implications for Further Research:

Future work should examine expert perspectives on EBT financing strategies in different contexts (e.g., substance use services), gathering targeted feedback on financing strategies that are less well known, and exploring topics such as strategic planning, funding stability, and collaborative decision-making as they relate to EBT implementation and sustainment.

The number of children and adolescents with mental health problems in the United States has been growing, underscoring the need for effective treatments. Recent estimates suggest that one in five children has a diagnosable mental health condition (1–3) and some diagnoses have shown increases of up to 96% in the past few decades (4). Mental health conditions have become one of the leading health problems among children and adolescents (5) and are associated with low educational achievement, substance abuse, poor health outcomes, increased comorbidities, low socio-economic status, and premature death (5–8). Furthermore, increased mental health diagnoses escalate healthcare system costs, as youth with a mental illness diagnosis have greater medical needs in comparison to youth with no mental illness diagnosis (4, 6). The economic burden of youth mental health conditions has been reported at $250 billion annually, including family and system healthcare costs as well as broader national economic losses (6–9). To enhance mental health, well-being, and societal outcomes, youth require effective mental health treatment whose implementation carries its own financial complications.

Challenges with Implementation of EBTs

Evidence-based treatments (EBTs) are available that have been found to be effective for addressing youth mental health needs (10). EBTs refer to interventions that have been rigorously studied and produced improved outcomes (compared to no treatment or, ideally, usual services) for specific mental health problems and populations (11). Unfortunately, there has been less focus directed towards dissemination and implementation of EBTs once they have been developed, meaning that EBTs are not offered routinely in usual care settings (12, 13). Currently, many youth may not receive services or are offered services with limited or unknown effectiveness (14), particularly in marginalized or underserved populations (15). U.S. mental health service systems need to invest in implementing youth-focused EBTs widely and consistently to address the growing health and societal impacts of youth mental health conditions (16).

One key reason that service agencies may not implement or sustain EBTs is the limited financial resources available. Indeed, a common barrier cited by service agencies is access to funding (5). EBTs can be particularly difficult to implement and sustain because they typically include additional elements beyond service delivery costs (e.g., monitoring treatment outcomes and fidelity, training and consultation, materials or equipment). Without associated funding, it is common for agencies to report that these elements of the EBT may not be implemented or are implemented to a lesser degree, which in turn may lower the efficacy of EBTs (10). Agencies may also opt to select treatment options that are not evidence-based due to lower costs, and therefore fail to offer the most effective treatment to address youth mental health needs. Further, when financing strategies outside of the typical channels are available, they are commonly insufficient to address EBT implementation costs (17).

Financing Strategies as a Potential Solution

One potential solution for service agencies to overcome funding-related barriers is to leverage and integrate different financing strategies for implementation or sustainment. Implementation researchers have started to define and describe financing strategies, as in a recent scoping review (17) that identified 23 potential financing strategies which can be used to support EBT implementation and sustainment. Examples of strategies include fee-for-service reimbursement, where agencies receive reimbursement for the provided EBT by including the service in an insurance fee-for-service list (18–20); braided funding streams, which involves coordinating across different funding sources to cover the cost of providing the EBT (21–23); and grant funding, which refers to funding awarded from the government or private agencies to deliver certain EBTs (24–26). The studies found in the scoping review focused on describing the financing strategies observed for a given EBT in specific service agency and funding environment contexts.

Unfortunately, there was a distinct lack of research on the process of using financing strategies–such as how agencies selected which strategies to use or the impacts of different strategies–for EBT implementation in youth mental health systems. Furthermore, very little existing research on financing strategies addressed strategies used in the sustainment phase (after initial implementation supports are withdrawn), which poses a major challenge for long-term and large-scale impact of youth mental health EBTs. An important next step is understanding how youth mental health system decision-makers view key features (e.g., relevance, feasibility) that inform their selection of financing strategies for implementation and sustainment. In the aforementioned scoping review (17), the authors suggested that it would be critical to find other sources of evidence about financing strategies and their effectiveness beyond what is currently in the research literature. This study addresses this research need by garnering the perspectives of key groups of experts who are involved with EBT implementation and sustainment within the larger youth mental health services ecosystem.

Present Study

As part of a larger project examining the tailored selection of EBT financing strategies, we surveyed representatives from youth mental health service agencies and their funding agency partners (i.e., government, insurance, and foundation funders) and intermediary partners (i.e., EBT experts who provide training, consultation, and other support). We sought to gather the perspective of those who provide the service, as well as those involved in the financing system and the service delivery system of EBT implementation (27), on the 23 financing strategies identified in the previously published scoping review (17). Utilizing quantitative and qualitative data gathered across two surveys, we sought to answer the following research questions:

Research Question 1:

How do youth mental health system representatives view various financing strategies as contributing to their EBT implementation and sustainment efforts?

Research Question 2:

What are the similarities and differences in perspective between individuals in the service agency, funding agency, and intermediary roles?

Method

Participants

Surveys were administered to 48 youth mental health system representatives from across the United States who were participating in a larger research project, which focused on developing guidance for tailored selection of financing strategies in youth mental health services. Most participants self-identified their gender as female (85.4%) and race/ethnicity as White (68.8%), followed by multi-racial (12.5%), American Indian or Alaska Native (10.4%), Latino/a (6.3%), and Black (2.1%). For highest level of education, participants had attained a(n) associate degree (6.3%), bachelor’s degree (14.6%), master’s degree (52.1%), or doctoral degree (27.1%). Finally, participants represented three key roles involved in delivering and funding EBTs in US youth mental health services: EBT intermediary representatives, service agency representatives, and funding agency representatives. Next, we describe the recruitment process for these groups.

Twelve EBT intermediary representatives were recruited first. They had expertise in the high-fidelity implementation and sustainment of EBTs, specifically Parent-Child Interaction Therapy (PCIT), Trauma-Focused Cognitive Behavioral Therapy (TF-CBT), or both. Using snowball sampling (28), intermediaries then nominated youth mental health service agencies with whom they had worked previously to implement PCIT or TF-CBT to participate in the project. To be eligible, agencies were required to have fully implemented the EBT of focus with at least one clinician. Service agencies applied to join the project and nominated service agency and funding agency representatives who had been involved in the agencies EBT implementation and sustainment efforts; 10 agencies were selected to participate in the project.

From those agencies, we enrolled 24 service agency representatives who had experience with the financial aspects of implementing and sustaining the EBTs at their service agency. Representatives were typically in an agency leadership role (e.g., CEO, Chief Financial Officer, Vice President), a clinical administration role (e.g., clinical director, program supervisor), and/or a financial administration role (e.g., grants administration, development officer). We also enrolled 12 funding agency representatives from funders that had provided support for EBTs at the service agency. Funders were representative of a diverse range of organizations, such as state and tribal agencies, private foundations, and managed care.

A protocol paper (16) provides further details of the recruitment plan and sample. That protocol also describes other activities participants engaged in for the larger research project.

Procedures

The current study focuses on data from two electronic surveys that we sent to all participants (N = 48) four months apart via invitational email. If a participant did not respond, they were reminded by the project manager and/or principal investigator through email and phone calls. Although reminders were sent, participation in project data collection activities (e.g., the surveys) was optional per the project consent form.

Participants provided electronic informed consent when they enrolled in the overall project. In addition, we included consent language at the beginning of each survey. All procedures were reviewed by the RAND Corporation Institutional Review Board and determined to not constitute human subjects research (Protocol #2020-N0607); nevertheless, we followed all ethical principles for the protection of human research participants to minimize any risk of harm. For example, in cases where the project manager or principal investigator was engaged in coaching activities with a service agency (see 16), survey invitations and reminders were sent to the service agency representatives by another team member and never discussed as part of coaching, to avoid undue influence.

The first survey was fielded from 9/15/21 to 11/1/21 and participants received a $50 e-gift card for their participation. The second survey was fielded from 3/14/22 to 6/6/22 and participants were provided with a $60 e-gift card for their participation; we increased project incentives over time to maximize retention in the overall project.

Survey Design and Administration

We used SelectSurvey, a secure web-based platform, to administer both surveys. The surveys were only open to invited project participants, and each participant was assigned a unique, anonymous identification number to identify their data in the survey; this also prevented multiple submissions from the same participant. We did not collect names, although it was possible to link unique identification numbers to participant names using a linking file (stored separately from the data). Project files were stored electronically in the SelectSurvey platform and on project servers, all of which had access restricted to authorized users via secure encryption and password protection.

Survey items were not randomized. There were some adaptive survey questions, which displayed differently depending on the participant’s role (intermediary, service agency, or funding agency) or responses to previous questions, but every question analyzed for the present study were displayed identically to all participants. Question responses were required, but all items included a non-response option (e.g., “choose not to respond”). Participants were able to review their responses before submitting. Several team members pilot-tested each survey for usability and functionality issues prior to fielding.

Both surveys included some items relevant to the research questions for this study (i.e., focused on discrete financing strategies) and items that related to other project aims. Our descriptions of the surveys next focus on items relevant to our research questions. All survey questions analyzed in this study and related materials (e.g., invitation e-mails, consent language) are available in an online appendix.

Survey 1: Financing Strategy Relevance and Definitions

In the first survey, participants reviewed a list of the 23 financing strategies identified and defined in the scoping review (17). Participants rated each strategy on its relevance to youth mental health services using a 7-point Likert scale (−3 = highly irrelevant; 3 = highly relevant) after reviewing the strategy name and definition (e.g., “Increased fee-for-service reimbursement: Increase insurance reimbursement rates to providers for the evidence-based treatment, relative to other services, in order to offset increased costs to providers for delivering that practice.”). Participants could also select “Not familiar enough with strategy to answer.” Each rating was followed by open-ended questions requesting feedback on the strategy’s definition and comments about similar (overlapping) strategies. After rating all 23 strategies, participants were asked to add suggestions for financing strategies not covered in the survey.

Overall, Survey 1 included 111 items across 13 pages. The questions about financing strategies described above were displayed on a single page and numbered 71 items total (3 items each for 23 strategies, plus two additional items). Other survey questions, not analyzed in this study, gathered information about key steps and processes for selection of financing strategies and (service agency representatives only) ratings of service agency characteristics.

Survey 2: In-Depth Ratings of Financing Strategies

For the second survey, we identified six strategies with the highest relevance ratings on Survey 1: grant funding, contracts, braided funding, increased fee-for-service reimbursement, standard fee-for-service reimbursement, and credentialing/rostering providers. Participants rated these six strategies on 5-point Likert scales reflecting each strategy’s availability in settings they were familiar with (0 = not available at all; 4 = completely available), feasibility of obtaining funding through the strategy for service agencies (0 = not feasible at all; 4 = completely feasible), and effectiveness of the strategy for sustaining evidence-based youth mental health treatments when used (0 = not effective at all; 4 = completely effective).

Survey respondents then rated the adequacy of all typical financing strategies (or combinations of strategies) to collectively sustain various components of youth mental health EBTs, also on a 5-point Likert scale (0 = not at all adequate; 4 = completely adequate). The specific components rated were delivery of EBTs; inclusion of caregivers, family, and other social supports; materials and equipment; clinician training, supervision, and/or consultation in EBTs; care coordination and multi-disciplinary team activities; and general program support. An open-ended question asked for general comments on the financing strategies typically used for youth EBTs. Finally, participants rated how important availability, feasibility, effectiveness, and sustainment adequacy (for each component) are when selecting a new financing strategy (0 = not important at all; 4 = critically important), and again an open-ended question asked participants to share thoughts about other important considerations for financing strategy selection.

Survey 2 consisted of 78 items across 10 pages. There were 35 questions about financing strategies, displayed across two pages. Other survey items focused on feedback for project training and activities, and so were not analyzed for this study.

Note that we shifted from a 7-point Likert scale to a 5-point Likert scale in Survey 2 because participants were not using the lower half of the scale in survey one. Data characteristics do not largely change when comparing 5-point to 7-point scale formats (29).

Data Analytic Procedures

We utilized a mixed-method approach, specifically a QUAN + Qual triangulation design (30), to combine and compare the quantitative data (standardized scales) and qualitative data (open-ended survey responses). Within this triangulation approach, qualitative data were leveraged to expand on and provide additional context for the primarily quantitative ratings (31). This analysis allowed us to gain higher-level insights regarding youth mental health system representatives’ assessments of financing strategies and factors driving those assessments, which would not be feasible if we used either approach alone.

Survey Response Rates

Of the 48 individuals invited to the surveys, the number who completed the questions on financing strategies were 35 (73%) in Survey 1 and 32 (67%) in Survey 2. Twenty-four individuals participated in both surveys, and the numbers of unique respondents in Surveys 1 and 2 were 11 and 8, respectively; overall, 43 participants (90%) responded to at least one survey. A chi-square test of independence confirmed there was no significant association between survey participation and group (intermediary, service agency representative, funding agency representative), X² (4, N = 46) = 6.21, p = .18, race and ethnicity, X² (8, N = 46) = 14.07, p = .08, or education, X² (6, N = 46) = 2.50, p = .87. There was a significant association between gender and survey participation, X² (2, N = 46) = 6.88, p = .03. We were able to monitor survey start and end times in SelectSurvey and did not detect any atypical timestamps that suggested we should discard data. We analyzed surveys with missing responses, as participants were permitted to skip questions.

Quantitative Analysis

We calculated descriptive statistics (means, standard deviations, counts for each response option) for each of the Likert scale ratings: relevance to youth mental health services (all 23 financing strategies); strategy availability, feasibility, effectiveness, and sustainment adequacy (six highest-relevance strategies); and importance of strategy availability, feasibility, effectiveness, and sustainment adequacy when selecting new strategies. These summaries included statistics for the overall sample and separate statistics for each participant role: intermediary, service agency representative, or funding agency representative. Given the small sample, particularly when differentiating by participant role, we did not examine inferential statistics or perform corrections (e.g., weighting, propensity scores) to the statistics. We also conducted a series of t-tests to compare responses from participants who completed survey 1 only vs. both surveys and responses from participants who completed survey 2 only vs. both surveys.

Qualitative Analysis

Qualitative analysis was completed by the first author, an industrial and organizational psychology graduate student, with input from a core research team consisting of a clinical psychologist and implementation researcher, a social worker with legal training, and a health economist. Open-ended responses that provided feedback on individual financing strategies were reviewed and summarized, but not further analyzed because the amount of feedback per-strategy was limited. For the other open-ended questions (suggestions for additional financing strategies and sources of strategies, comments on typical financing strategies and considerations for strategy selection), we used rapid content analysis methods to understand the experiences of key experts and how the groups viewed financing strategies when seeking to implement and sustain EBTs to inform future implementation activities.

For each open-ended survey question, all qualitative data were transferred into an Excel spreadsheet for analysis. The first author conducted a preliminary read-through of all responses for each question to become familiar with the data. Then for each response, using an inductive approach, key words or phrases that encapsulated the meaning of the response as it related to the question were pulled out and summarized. Summarized key words and phrases were reviewed comprehensively to determine if commonalities existed. Commonalities were collapsed into categories. All steps of this analysis were reviewed by multiple individuals of the research team and suggestions or edits were made to ensure no important elements of the responses were missed. Finally, the full team came together to reach consensus and approve the final summary of findings and discuss any noticed patterns or connections across the data. The qualitative data were examined comprehensively and then by group to provide richer context and detailed nuance to their experiences, which were less clear when examining the quantitative data alone.

Results

Survey 1

Participant input on the 23 financing strategies was used by the research team to edit and refine the list of strategies from the scoping review (17). No strategies were added, removed, or merged, but names and definitions were updated to 15 (65%) strategies with surface-level changes to clarify their purpose and form. Table 1 presents the updated list of strategies and definitions.

Table 1.

Updated List of Financing Strategies and Definitions

Financing Strategy	Definition
Fee-for-service reimbursement	Include the evidence-based treatment in an insurance provider or management organization’s fee-for-service list/formulary so that providers can receive reimbursement for providing that practice (see also: increased fee-for-service reimbursement)
Increased fee-for-service reimbursement rate	Increase insurance provider or management organization’s reimbursement rates to providers for the evidence-based treatment, relative to other services, in order to offset increased costs to providers for delivering that practice (also called an “enhanced rate”) (see also: standard fee-for-service reimbursement)
Removed/altered billing limits	Allow insurance payments to providers for services that are disallowed under typical billing limits (additional sessions, services for patients with designated diagnoses, etc.) when delivering the evidence-based treatment
Technical support for billing	Provide assistance (from insurance or a third party) to providers for preparing and successfully submitting claims for delivery of the evidence-based treatment
Pay-for-success (PFS) financing	Establish agreements in which private or nonprofit investors prospectively provide funding for providers to deliver the evidence-based treatment and in which a government entity provides a payout to the investors if preestablished outcomes or quality metrics are achieved in a designated period
Global budget allocations	When allocating government general funds, include funding in the overall annual budget of a state, county, or municipal agency to cover the costs of having the agency’s providers deliver the evidence-based treatment
Line-item budget earmarks	When allocating government general funds, include designated (protected or earmarked) funding in the annual budget of a state, county, or municipal agency to cover the costs of having the agency’s providers deliver an evidence-based treatment
Braided funding streams	Coordinate multiple funding sources across two or more government agencies to cover the costs of an agency’s delivering the evidence-based treatment, such that each individual funding source remains accounted for separately; generally established through annual budgets (see also: blended funding)
Contracts for EBTs	Award funding contracts from state, county, or municipal agencies to provider organizations that agree to deliver the evidence-based treatment
Inclusion in block grants	Allow payment to providers for delivering the evidence-based treatment using block grants, which provide a fixed amount of money to state, county, or municipal agencies to pay for a designated set of health services
Shifting funds between programs	Reallocate funds from other programs and practices, within or across state, county, or municipal agencies, to cover the costs of delivering the evidence-based treatment; often funds currently dedicated to services whose use is expected to decline due to the evidence-based treatment
Grant funding	Award grant funding from government or private funders to provider organizations that propose to deliver the evidence-based treatment
Fundraising and investor donations	Collect donations from private or nonprofit investors, including fundraising and philanthropy, that will be used by provider organizations to cover the costs of delivering the evidence-based treatment
Cost offset	State, county, or municipal funders directly pay for infrastructure (such as a dedicated intermediary organization) that supports the delivery of evidence-based treatments by providing training, technical assistance, and other nonmonetary resources to providers within a given jurisdiction
Credentialing/rostering providers	Establish between insurance provider or management organizations and training organizations agreements for evidence-based treatments in which designated providers are allowed to receive payment for the evidence-based treatment (often at an increased rate)
Blended funding streams	Combine multiple funding sources across two or more government agencies to cover the costs of an agency’s delivering the evidence-based treatment, such that all funding is accounted for together and no longer separable by source; generally established through legislative action (see also: braided funding)
Dedicated taxes	Collect a state, county, or municipal tax, then allocate the revenue from that tax to provider organizations that deliver the evidence-based treatment
Capitated or patient-based payments	Provide a set, prospective insurance payment to providers that is expected to cover all expenses for a given patient’s care (including increased costs of delivering the evidence-based treatment) in a designated period (see also: bundled or episode-based payments)
Pay-for-performance (P4P)	Provide a financial bonus (on top of other insurance payments) to provider organizations for achieving preestablished outcomes or quality metrics in a designated period, with bonuses sufficient to cover increased costs to providers associated with delivering the evidence-based treatment
Value-based purchasing	Provide insurance reimbursement to providers only when they achieve preestablished outcomes or quality metrics for a designated period, with reimbursement sufficient to cover increased costs to providers associated with delivering the evidence-based treatment
Government bonds	Issue state, county, or municipal bonds (securities) for purchase by private or nonprofit investors, then allocate the revenue from those bonds to provider organizations that deliver the evidence-based treatment
Vouchers for EBTs	Provide vouchers of a predetermined value that patients can redeem to receive the evidence-based treatment from providers, who are then repaid by the insurance or government payor that issued the voucher
Bundled or episode-based payments	Provide a set insurance payment to a group of providers that is expected to cover all expenses for a given diagnosis or episode of care (including increased costs of delivering and coordinating the evidence-based treatments across providers); can be prospective or retrospective payments (see also: capitated or patient-based payments)

Note. The original list of financing strategies and definitions was published in a scoping review (17); surface-level changes made to incorporate Survey 1 participant feedback are denoted with bold text. EBT = evidence-based treatment.

The original list of strategies and definitions was published under the terms of the Creative Commons Attribution 4.0 License

(https://creativecommons.org/licenses/by/4.0/) which permits any use, reproduction, and distribution of the work without further permission provided the original work is attributed.

Table 2 presents relevance ratings for the six financing strategies that were rated as most relevant to youth mental health services; we found that participants’ use of the “not familiar” option varied widely across strategies, so the table also indicates the percentage of respondents familiar (i.e., who provided a rating). Strategies rated as most relevant – with mean relevance ratings from 2.42 to 2.78 – included braided funding streams, contracts for EBTs, credentialing/rostering providers, fee-for-service reimbursement, grant funding, and increased fee-for-service reimbursement. All strategies with high ratings were also familiar to most respondents (M = 82%, range: 71–94% familiar). Supplemental Table 1 presents relevance and familiarity information for all 23 strategies. All other strategies were unfamiliar to about one-third to one-half of participants and were typically rated lower, although relative ratings varied by group. T-tests revealed no significant differences in relevance or familiarity ratings between participants that completed survey 1 only and those that did survey 1 and survey 2.

Table 2.

Relevance and Familiarity Ratings for Financing Strategies Rated Most Relevant to Youth Mental Health Services, by Participant Role

	All Participants (n = 35)		Funding Agency Representatives (n = 9)		Intermediaries (n = 10)		Youth Mental Health Service Agency Representatives (n = 16)
Strategy	Relevance M (SD)	% Familiar	Relevance M (SD)	% Familiar	Relevance M (SD)	% Familiar	Relevance M (SD)	% Familiar
Braided Funding Streams	2.70 (0.47)	71%	2.83 (0.41)	75%	2.71 (0.49)	70%	2.60 (0.52)	69%
Contracts for EBTs	2.73 (0.52)	91%	2.71 (0.49)	87%	2.78 (0.44)	90%	2.71 (0.61)	94%
Credentialing/Rostering Providers	2.42 (1.10)	82%	2.50 (0.84)	75%	2.44 (0.73)	90%	2.36 (1.50)	81%
Fee-For-Service Reimbursement	2.48 (1.03)	91%	2.13 (1.57)	89%	2.78 (0.44)	90%	2.50 (1.02)	94%
Grant Funding	2.78 (0.42)	94%	2.78 (0.35)	100%	2.78 (0.44)	90%	2.79 (0.43)	94%
Increased Fee-For-Service Reimbursement	2.61 (0.74)	83%	2.38 (0.98)	89%	2.63 (0.74)	80%	2.75 (0.62)	81%

Note. EBT = evidence-based treatment. Relevance ratings are on −3 to 3 scale representing relevance to youth mental health services, and % familiar refers to the percentage of participants who provided a rating and did not select the option “Not familiar enough with strategy to answer.” Ratings for the remaining 17 financing strategies rated are presented in Supplemental Table 1.

Survey 2

The six financing strategies with highest relevance for youth mental health services (braided funding streams, contracts for EBTs, credentialing/rostering providers, fee-for-service reimbursement, grant funding, and increased fee-for-service reimbursement) were consistently rated between somewhat and quite available, feasible, and effective. Across participants, the mean ratings for all six strategies ranged from 1.04 to 2.71 for availability, 1.91 to 2.67 for feasibility, and 2.26 to 2.66 for effectiveness. Table 3 depicts average ratings across all groups for availability, feasibility, and effectiveness, as well as the percent of respondents familiar with the strategy (M = 82%, range: 66–100% familiar).

Table 3.

Availability, Feasibility, and Effectiveness Ratings for Financing Strategies Rated Most Relevant to Youth Mental Health Services

Strategy	Availability M (SD)	Feasibility M (SD)	Effectiveness M (SD)	% Familiar (n = 32)
Braided Funding Streams	2.08 (0.91)	2.16 (0.80)	2.26 (0.81)	75%
Contracts for EBTs	2.13 (0.81)	2.45 (0.99)	2.55 (0.78)	91%
Credentialing/Rostering Providers	1.76 (1.48)	2.48 (1.29)	2.57 (1.08)	66%
Fee-For-Service Reimbursement	2.71 (1.21)	2.67 (1.04)	2.41 (0.97)	84%
Grant Funding	2.50 (0.72)	2.47 (0.92)	2.66 (1.18)	100%
Increased Fee-For-Service Reimbursement	1.04 (0.96)	1.91 (1.24)	2.36 (1.29)	78%

Note. n = 32. Ratings are on 0 to 4 scale. EBT = evidence-based treatment. % familiar refers to the percentage of participants who provided a rating and did not select the option “Not familiar enough with strategy to answer.”

Table 4 summarizes ratings of how adequately typical financing strategies sustain various components of youth mental health EBTs (care coordination/MDT, delivery, family/caregiver inclusions, general program support, materials/equipment, and training/supervision/consultation). Experts reported these EBT aspects are somewhat adequately sustained by typical financing strategies, with mean adequacy for sustaining scores ranging from 1.47 to 2.52.

Table 4.

Ratings for Adequacy of Financing Strategies Sustaining Components of EBT and Degree of Importance of EBT Components when Selecting Financing Strategy

Financing Strategy Characteristic	Adequacy for Sustaining M (SD)	Importance in Strategy Selection M (SD)
Availability	-	3.38 (0.79)
Feasibility	-	3.35 (0.75)
Effectiveness	-	3.31 (0.78)
EBT Component
Service Delivery	2.52 (0.81)	2.91 (0.86)
Caregiver Inclusion	2.17 (0.95)	2.72 (0.81)
Materials	1.77 (1.10)	2.78 (0.87)
Training	1.72 (1.10)	3.00 (1.02)
Care Coordination	1.47 (1.11)	2.34 (0.87)
General Program Support	1.77 (1.15)	2.59 (0.91)

Note. n = 32. Ratings are on 0 to 4 scale. EBT = evidence-based treatment.

Table 4 also summarizes ratings of how important different characteristics of financing strategies were for selecting which strategy to pursue to fund and sustain EBTs. Strategy characteristics include availability, feasibility, effectiveness, ability to sustain delivery, caregiver inclusion, materials, training, care coordination, and program support. Mean importance ratings for these characteristics ranged from 2.34 to 3.38.

Across all Survey 2 items, groups largely agreed in their ratings of financing strategies. Some ratings did differ by group (e.g., service agencies provided a higher rating for financing strategies sustaining caregiver inclusion or care coordination than did funding representatives or intermediaries), but we focused on results for the full sample because the sample was not large enough to test which of those differences between sub-samples were statistically meaningful. Supplemental Tables 2-4 present breakdowns of ratings by role type for, respectively: financing strategy availability, feasibility, and effectiveness; adequacy for sustainment; and importance for sustainment. Although t-tests revealed significant differences in ratings of grant availability, contract effectiveness, and braided feasibility between participants that completed survey 2 only and those that completed both surveys, it should be noted that a few significant findings should be expected due to random chance when conducting a large number of tests, as was the case here. We did not discern any underlying pattern in response rates that would contribute to these findings. Additionally, t-tests cannot account for the use of snowball sampling, or the nesting of observations within organizations, which may have biased the results towards finding significant differences.

Content Analysis

Open-ended responses from representatives provided additional context for the agencies experience around the availability, feasibility, and effectiveness of various financing strategies to fund and sustain EBTs. Experts reported challenges with strategies being unfamiliar, non-recurring, or unavailable in their state or via insurance agencies. In terms of financing strategy feasibility, some representatives suggested getting funding was particularly difficult when trying to start a new EBT, whereas others reported being to acquire funding for EBT startup costs but not ongoing costs. One service agency representative noted, “It’s challenging to find funding strategies that can support all components of maintaining evidence-based treatments. Some that assist with training are short-lived and don’t necessarily provide general support.” Some service agency representatives and intermediaries suggested that it may be beneficial to have an expert on staff dedicated to finding opportunities and acquiring funds based on the agency’s needs to alleviate these obstacles. Groups broadly spoke more to availability and feasibility, but some funders noted that effective sustainment may go beyond EBT-specific financing strategies, such as through investing in recruitment or making efforts to retain employees.

Although many experts reported issues with financing strategies, it should be noted that some felt that the common financing strategies are adequate for EBT sustainment, such as when one funder reported:

The financing strategies listed above are extremely adequate for startup costs of these types of programs. There also seems to be a good mix of strategies that can ensure sustainability for provider payment and follow-up training requirements for the programs/type of therapy.

Others reported that adequate sustainment is possible through using a blend of diverse strategies, rather than relying on one traditionally used strategy. In addition, some noted that agency size and location will influence expenses and support needed, meaning effective strategies for one agency may be ineffective for others.

Across the groups, it was reported that there were unmet needs that should be considered during the financing strategy selection process. More specifically, service agency representatives reported needing support for equipment, materials, and training; funding agency representatives for caregiver involvement and assessment; and intermediaries for clinician support and non-billable time.

Discussion

Understanding the role of financing strategies in implementing and sustaining youth mental health EBTs (such as PCIT and TF-CBT) is necessary to inform tactics to reduce personal, community, and economic impacts of mental health problems within the resource-constrained U.S. healthcare system. This study advanced prior research that described financing strategies (e.g., 17) to explore, across two surveys, how (a) U.S. youth mental health system representatives view various financing strategies as contributing to EBT implementation and sustainment efforts and (b) perspectives may differ by representatives’ roles and expertise. This information can help inform efforts to assist service agencies in tailored selection of financing strategies, as called for in recent literature (16–17). However, understanding the implications of our findings requires careful consideration of the complex health service ecosystem within which EBTs are implemented (see 27). Therefore, we organize our discussion to detail implications for service agencies, their direct partners for EBT implementation (e.g., intermediaries and funding agencies), and broader policy contexts (e.g., legislation).

Within service agencies, representatives reported having limited knowledge of most financing strategies, echoing the finding from the scoping review (17) that many financing strategies have not been used in practice to fund EBTs in mental health or related service systems. Innovative and (at least theoretically) beneficial financing strategies may emerge in research literature, but youth mental health system representatives appear to be pragmatically prioritizing what they are already familiar with (and what is routinely available) over innovative, but less certain, options. Indeed, of the strategies rated highest in familiarity and relevance (braided funding streams, contracts for EBTs, credentialing/rostering providers, fee-for-service reimbursement, grant funding, and increased fee-for-service reimbursement), there is prior research evidence that all have been used to fund the implementation of EBTs (17). It is vital that researchers do not underestimate the importance of these routine financing strategies–despite longstanding criticisms that such financing does not incentivize effective or high-value services (32, 33), mental health service agencies continue to rely on them.

That said, the strategies considered most relevant for youth mental health services were rated as only somewhat available, feasible, and effective and only somewhat able to sustain components of youth mental health EBTs (care coordination/MDT, delivery, family/caregiver inclusions, general program support, materials/equipment, and training/supervision/consultation). Qualitative evidence supports these ratings as expert representatives noted the current financial climate in mental health systems is insufficient for EBT implementation and sustainment, especially considering the ongoing added costs (e.g., fidelity and assessment). These results align with prior findings that inadequate financial support acts as a common barrier to EBT implementation and sustainment (34) and that shifts in policy may be necessary to overcome cost barriers to ensure EBT use (35). Furthermore, our qualitative findings suggest that service agencies may desire or require a staff member that is knowledgeable about EBT financing strategies to help identify and manage new opportunities.

The challenges evident in using EBT financing strategies also apply to service agencies’ direct partners, as survey ratings were broadly similar across representatives in service agency, funding agency, and intermediary roles. Although there were slight nuances found through the content analysis in terms of what each group valued as being important for EBT funding and sustainment (e.g., funding for equipment vs. assessment vs. nonbillable time), there seemed to be agreement that the discrete strategies reviewed here are not able to solve larger systematic issues that prevent youth mental health service agencies from providing EBTs. The consistency in ratings suggests that financial challenges are broadly recognized and communicated about across roles in youth mental health services, and barriers lie in developing viable solutions rather than understanding the problem. This is consistent with research demonstrating that adoption of alternative payment models requires considerable support and coordination within systems; two illustrative examples are the challenges using billing codes that became available in 2018 for the evidence-based Collaborative Care model (36) and the extensive guidance recently published by the American Psychological Association on billing Group Caregiver Behavior Management Training Services (37). However, it is also possible that our recruitment strategy (which relied on snowball sampling within existing networks) resulted in a sample of individuals with higher-than-usual agreement across roles. There was consistency across survey participation and participant group, race and ethnicity, and education. Although a significant association was found between survey participation and gender, with count expectancies suggesting that less males completed survey 2, this finding is consistent with research finding a gender bias in online survey response behavior with higher responses rates from females (38).

These surveys were part of a larger effort to develop a strategic planning tool that youth mental health service agencies could use to guide selection of financing strategies to implement and sustain EBTs (16). More specifically, that tool–the Fiscal Mapping Process–was designed to assist agencies in determining resources needed to sustain EBTs, defining funding objectives, brainstorming financing strategies, aligning strategies and objectives within a Fiscal Map, and monitoring progress. The present findings (i.e., 23 financing strategies and their relevance ratings) were already incorporated into the tool to help service agencies brainstorm financing strategies. This information may be useful as youth mental health service agencies and their partners in implementation (e.g., funding agencies, intermediaries) consider which strategies to pursue in their efforts of EBT funding and sustainment. While pilot testing the tool, service agencies worked with a coach to assist them with Fiscal Mapping tool use; although coaching may not always be necessary to use the tool, should coaching be an option, the findings here may help inform coaches about how familiar service agencies (and their partners) may be with the strategies, and encourage greater coordination and information-sharing among these roles. The lack of familiarity from survey respondents with many of the strategies also speaks to the need for coaches to have an adequate understanding of the various strategies, since they may need to provide further clarification to service agencies, intermediaries, and funding agencies.

At the broader policy level, the U.S. continues to invest far too few resources into mental health and substance use services, representing only 6% of healthcare spending (39)–not nearly enough to support basic infrastructure such as facility operations, measuring quality and outcomes, or workforce training and retention (40). A recent survey of community mental health clinicians (41) documented high levels of education loan debt, over one-third of clinicians working in precarious contractor positions (without guaranteed income or benefits), and 51% working a second job. Under such conditions, it makes sense that our survey participants did not strongly differentiate among priorities for funding or consistently rate any strategy as fully meeting their needs. Maintaining viable services in general, let alone services that prioritize use of EBTs, will be incredibly challenging until policymakers improve the financial climate. Key policy priorities for strengthening public behavioral health service systems include equitable pay, loan forgiveness and stipends, investment in professional capacity-building, ensuring a diverse workforce, and improving access through integrated care (42).

It is critical to note that this study took place under a particular context–U.S. youth mental health services within a subset of states–and may not fully generalize to other service systems, such as adult mental health or substance use services or systems in other countries. It may be useful to collect similar data about use of financing strategies in specific contexts, as there may be differences in which strategies are prioritized and how they are rated by system representatives. For example, the prior scoping review (17) found that only 53% of financing strategies with evidence for use in mental health services also had evidence for use in substance use services, whereas the overlap between mental health and child welfare was 76%. Representatives from substance use service systems may identify different (or fewer) high-priority strategies, whereas child welfare representatives may identify more similar strategies but differ in their views on factors like feasibility or effectiveness. A strength of the current study was that we gathered input from U.S. youth mental health system representatives who had experience with financing EBTs and contributed varied perspectives. However, given the consistency of ratings we obtained across measures, there may be more efficient ways to gather this information for other contexts. For example, one approach would be to ask service agency representatives to review the full list of financing strategies (see Table 1), then have representatives from other roles (e.g., funding representatives) provide more in-depth ratings and qualitative responses on the strategies that service agencies viewed as most relevant and/or familiar.

It is important to acknowledge several limitations to our findings. First, all participants were involved in ongoing EBT implementation and sustainment efforts; this was necessary to ensure they could participate in all project activities, but it means our findings may not generalize to system representatives unfamiliar with EBTs. Service agency representatives sought to sustain an ongoing EBT, not incorporate a new EBT into their service system. Second, as mentioned previously, we might have observed greater differences in ratings by participant role if we had recruited a random or representative, rather than snowball, sample of participants. Third, qualitative data was collected through open-ended questions in the surveys, meaning that we could not investigate participant comments in depth through follow-up or clarifying questions. Future work should use interviews and focus groups to explore topics such as strategic planning, funding stability, and collaborative decision-making as they relate to EBT implementation and sustainment. Fourth, due to our small sample size, the statistical power for comparisons between participant roles was limited to basic descriptive statistics. This made it difficult to draw conclusions about whether the few observed differences between groups were statistically robust or due to chance. Finally, participants were asked to provide feedback on 23 financing strategies, but individuals were often unfamiliar with some of the financing strategies asked about. More targeted research may be necessary to obtain well-grounded feedback on less common financing strategies (e.g., pay-for-success).

In summary, our surveys provided in-depth insights into how representatives from U.S. youth mental health systems view the available strategies for financing EBT implementation and sustainment. Continued refinement of these strategies is clearly necessary, as participants had consensus across roles–service agency, intermediary, and funding agency representatives–around need for improvement on every rated dimension (availability, feasibility, effectiveness, adequacy for sustainment). That said, while improvement of financing strategies is important, it is also a highly technical solution to more fundamental issues of chronic under-funding for behavioral health services in the U.S. Broader policy challenges around limited funding for state and local services, stigma and de-prioritization of mental health and substance use by policymakers and the public, and workforce conditions that lead to high burnout and turnover must be addressed in tandem with financing strategy development before large-scale impacts on EBT implementation and sustainment can be expected.