Evidence-based Treatment for Substance Use Disorders in Community Mental Health Centers: the ACCESS Program

Abstract

A significant gap remains in the availability and accessibility of evidence-based treatments (EBTs) in community substance use disorder (SUD) treatment. This study describes a 2-year statewide training initiative that sought to address this gap by training community-based therapists in motivational enhancement/cognitive behavioral therapy (MET/CBT). Therapists (N = 93) participated in a 2-day MET/CBT workshop followed by bi-weekly clinical consultation, fidelity monitoring, guided readings, and online resources. Therapists completed pre-training and follow-up assessments measuring knowledge, attitudes, confidence, and implementation barriers. Most therapists attended 10 or more consultation calls. Submission of session recordings for feedback was the least utilized training element. Therapists reported increased confidence in their ability to implement MET/CBT for SUD and demonstrated improvement in MI and CBT knowledge. Therapists reported several implementation barriers, including lack of time and opportunity to treat patients with MET/CBT. Recommendations for future training initiatives and addressing the barriers identified in this study are discussed.

Introduction

According to the National Survey on Drug Use and Health for 2019, 20.4 million individuals aged 12 and older were diagnosed with a substance use disorder (SUD), yet only 4.2 million received substance use treatment.¹ The negative outcomes of SUDs are well documented and have implications for an individual’s health,^2,3 interpersonal relationships,⁴ and finances.^5,6 Furthermore, the societal impacts of SUDs are substantial, with annual SUD-related health care costs exceeding $13 billion, and annual total costs estimated to exceed $700 billion in the USA.^7,8 Given the wide range of negative outcomes related to SUDs, access to and engagement in effective treatment is critical.

Although several evidence-based treatments (EBTs) for SUD exist, relatively few individuals receive them. It has been estimated that at most 25% of providers provide evidence-based services for individuals with SUD or co-occurring psychiatric disorders.⁹ Barriers to EBT adoption include lack of awareness of treatment modalities, lack of training or education about SUDs or EBTs, lack of experience, patient misconceptions about the nature and effectiveness of treatment options, and stigma.^10–12 Brief, didactic training has been shown to improve clinician understanding and confidence in treating SUD, as well as increase education of the disease model of addiction.¹¹ Nevertheless, even when providers are willing to provide SUD treatments, there is a need for dissemination and implementation initiatives that are directly aimed at training clinicians on how to provide EBTs so that patients with SUDs can more easily access high quality services.

The most common approach to training therapists practicing in community settings (e.g., community mental health centers, non-academic settings) in delivery of EBTs for behavioral health disorders has been stand-alone workshops, but these workshops tend to be ineffective in creating sustainable practice change without additional or ongoing support (e.g., technical assistance, clinical consultation, and fidelity monitoring).^13,14 For example, in a systematic review of dissemination and implementation strategies for the implementation of contingency management, an EBT for SUD, organizational input and ongoing support (e.g., supervision, consultation) were highlighted as vital to the adoption and sustainment of contingency management.¹⁵ A nationwide implementation of contingency management in the Department of Veterans Affairs, which provided in-person training and coaching calls to support ongoing implementation, found almost three-quarters of the programs adopted the contingency management protocol and also found positive patient outcomes (e.g., attendance rate, negative urine screens).^16,17 Additionally, statewide EBT (specifically CBT) initiatives in both Washington¹⁸ and Louisiana¹⁹ that utilized a combination of in-person trainings followed by multiple sessions of case-based consultation with expert CBT practitioners found positive outcomes, including reduction in patient symptom severity, increased self-reported provider CBT skill/knowledge, and greater inclusion of CBT components in later case presentations.

Dissemination and implementation initiatives that provide ongoing training to therapists may increase the availability of EBTs for SUD in community settings, but implementation efforts focused on EBTs for SUD, in particular, have been understudied to date.⁹ Moreover, as states consider investing heavily in EBT dissemination and implementation efforts, descriptions of how such efforts have unfolded in existing programs are crucial for informing future initiatives. The present study describes the structure and scope of a 2-year statewide quality improvement initiative, the Amplifying Community Capacity for Evidence-Supported Services (ACCESS) program, aimed at addressing the significant need for increased availability of EBTs for SUDs in community settings. ACCESS was launched to expand the workforce of therapists who could deliver MET/CBT—an empirically supported SUD treatment that can be implemented flexibly across a range of settings^20–25—to adults and adolescents with SUDs by offering therapists a multi-component training consisting of a workshop, bi-weekly clinical consultation, guided readings, and online resources. Following the first training cohort, pre-implementation changes were made in the second year of the initiative to address barriers therapists had encountered in the first year. The present study sought to (1) describe the development, structure, and scope of the ACCESS program; (2) examine therapist engagement in the initiative; (3) evaluate therapist training outcomes, including therapist knowledge gain, attitudes, and self-reported confidence; and (4) summarize implementation barriers encountered by therapists during the program.

Methods

Design

This study was based on data collected as part of the ACCESS Program, which was a quality improvement project carried out in partnership between state government and university partners at an academic medical center. One aim of ACCESS is to expand availability of EBTs for SUDs in adults and adolescents through therapist training. MET/CBT was selected to be implemented based on its strong support in the research literature for addressing a broad range of substance types in diverse clinical populations.^20–25 MET/CBT is delivered as an individual-focused intervention that integrates motivational interviewing, behavioral and emotion regulation skill building, and cognitive strategies to reduce substance use.²⁶ The project was conducted over 2 years and had two training cohorts. The project included several training components consistent with other statewide EBT initiatives^18,19 and the training literature,¹⁴ including a 2-day initial training workshop in the MET/CBT model; ongoing group consultation with MET/CBT clinical experts; fidelity monitoring of MET/CBT clinical implementation; and intermittent evaluation of MI and CBT knowledge, attitudes regarding EBTs, and general program feedback. Use of these data for this secondary analysis was approved by the [university masked] Institutional Review Board (#1,912,496,951).

Setting and participants

In Year 1, community mental health center (CMHC) clinicians were recruited to participate through a request for applications that was sent to all CMHCs in the state. CMHC-based therapists were targeted in Year 1, because CMHCs represent an established, statewide, publicly available network of behavioral health service providers that offer SUD clinical programming. Regarding typical SUD detection and treatment at CMHCs, most CMHCs include questions about substance use in their intake evaluations with some, but not all, using standardized screening instruments for detecting substance use, while SUD treatment is highly variable across CMHCs. Informational sessions were held to describe the scope and goals of the training initiative, as well as expectations of implementation sites. The application window was open for 2 months. Participant inclusion criteria included being employed by a CMHC, providing therapy to adult and/or adolescent patients with SUD, and being fluent in English due to materials being available only in English at the time of the training. Expectations of CMHC participants included attending the 2-day in-person training, completing training evaluations, attending bi-weekly consultation calls for 8 months, and providing monthly recordings (at least one per month) of MET/CBT therapy sessions for fidelity monitoring. Benefits included the therapist receiving free continuing education credits for the 2-day training and bi-weekly consultation calls, and free consultation calls with national MET/CBT experts. Twenty-four CMHCs were approached, and nine agreed to have therapists participate in the training.

Year 2 had identical training components and many of the same procedures as Year 1. Changes were made during the pre-implementation or pre-training phase of the project to address barriers identified and feedback provided by therapists in Year 1. First, participant inclusion criteria were the same as Year 1 except that registration was opened beyond CMHCs, so therapists did not have to be employed by a CMHC. Second, additional details around the training expectations were provided to therapists via an optional, informational webinar. Third, a signed agreement was required prior to enrollment. This agreement included a commitment to the following: attending live training, completing evaluation measures, participating in bi-weekly consultation, taking a minimum of two SUD patients, and providing recordings of MET/CBT sessions with SUD patients. Fourth, extra troubleshooting around technical barriers and recording therapy sessions was provided. In Year 1, therapists were asked to provide MET/CBT session recordings, via secure file upload, in any way that was compliant with their organization’s IT policies. In Year 2, each participant was provided an account with Lyssn, an AI-based platform that provides HIPAA-compliant recording and secure cloud storage,^27,28 for recording and submitting MET/CBT sessions.

Training components

The initial, live MET/CBT training workshop was conducted over 2 days, in-person in January 2020 in Year 1 and virtually in February 2021 in Year 2 due to the COVID-19 pandemic. Of note, a subset of participants (n = 8) in Year 2 attended an abridged (~ 3 h) live virtual training in May 2021 augmented with access to the full recorded training. An interdisciplinary team of expert SUD clinicians (psychologists, psychiatrists, social workers) conducted the training using interactive lectures, role plays, case discussions, and demonstrations. The MET/CBT training included topics related to background on and integrated delivery of MI, goal setting, personal feedback, functional analysis, CBT skills, SUD screening and assessment, and case selection. The training also addressed logistics for subsequent training activities, including ongoing consultation and fidelity monitoring.

Following the initial training, participating therapists were assigned to consultation groups that met bi-weekly across 36 (Year 1) or 32 (Year 2) weeks. Each group was facilitated by a national MET/CBT expert with experience delivering and providing training and supervision in MET/CBT for adolescents and/or adults with SUDs. Group assignment was based on fit of schedule between therapists and consultants. Both cohorts had the same six consultants leading the consultation groups; in Year 1, therapists were assigned to one of nine consultation groups (three consultants had two groups) and to one of seven consultation groups in Year 2 (one consultant had two groups). Groups ranged in size from 3 to 10 participants in Year 1 and 3 to 5 participants in Year 2. Consultation sessions involved case presentations and discussions, guided readings from provided texts on MI, CBT, and MET/CBT, and problem-solving around implementation challenges. Sessions were conducted using Zoom Health videoconferencing software, and participants were encouraged to turn their cameras on during session both to promote a sense of community and to help consultants evaluate participants’ level of engagement in sessions.

Measures

Baseline (pre-training) data were collected as part of the training registration process. In the first year, data were also collected after the 2^nd day of the in-person workshop and at 6- and 9-month follow-ups. In the second year, 3-, 6-, and 8-month follow-up assessments were collected.^a Completion of each evaluation was incentivized with the option to opt-in for a $20 US digital gift card upon completion.

Therapist characteristics

Therapists indicated their professional credentials, role at CMHC, years of active professional practice in current profession, and years of practice treating SUDs, as well as an indication (yes/no) of prior training in MI, CBT, and SUD treatment.

Motivational interviewing and CBT knowledge test

The MI knowledge test consisted of 21 multiple choice questions and measures therapists’ knowledge and understanding of basic facts about motivational interviewing.²⁹ In Year 2, therapists were also administered a shortened version of the CBT for Addictions Exam, which included 18 of the 50 multiple choice questions.^b An MI percentage score was computed by grading the 21 MI test multiple choice questions as correct/incorrect, and a CBT percentage score was also computed for the 18 CBT exam multiple choice questions. Therapists completed the MI knowledge test at baseline, 6- and 9-months follow-up in Year 1 and the MI and CBT knowledge tests at baseline, 3-month, 6-month, and 8-month follow-up in Year 2.

Evidence-based practice attitude scale (EPBAS)

The EBPAS is a 15-item scale that assesses therapists’ attitudes towards evidence-based practices (EBPs), and is comprised of four subscales (Requirements, Appeal, Openness, Divergence).³⁰ The EBPAS has demonstrated good reliability and validity.^30–31 Cronbach’s alpha values were 0.81 for appeal, 0.63 for divergence, 0.75 for openness, 0.92 for requirement, and 0.78 for overall EBPAS, similar to those originally reported.³⁰ In Year 1, the EBPAS was administered at the post-workshop and 6-month assessment points, and in Year 2, at baseline and 6-month assessment points.

Confidence

Therapists rated their overall confidence in their ability to implement MET/CBT for SUDs using an 11-point Likert scale from “not at all” to “extremely” and how this confidence rating compared to prior to the training on a scale ranging from 1 (“much less confident”) to 5 (“much more confident”). Additionally, therapists indicated how much they agreed that as a result of the training initiative they were better able to implement six MET/CBT components from “strongly disagree” to “strongly agree.”

Training helpfulness

Therapists indicated the number of cases they presented during consultation calls and the level of helpfulness of each training element on a 11-point scale from “not at all” to “extremely.”

Implementation barriers

Therapists were presented with a list of barriers (e.g., resources, policy issues) and asked to check which barriers they encountered to the implementation of MET/CBT in their practice at the 9-month follow-up in year 1 and 8-month follow-up in Year 2.

Statistical analyses

To examine training engagement, the authors reported descriptive data on participant workshop and consultation attendance, submission of session recordings, number of cases presented during consultation calls, and therapist-reported helpfulness of the various training elements. Mann–Whitney U tests and Fischer’s exact test were performed to evaluate if there were any differences in training engagement between cohorts. In addition, t-test, ANOVA, and correlations were conducted to explore predictors of consultation call attendance rate, quantified as number of sessions attended by total number of sessions scheduled; predictors investigated included prior training or experience, years of experience, degree, role in agency, and pre-training knowledge score.

For evaluating change in knowledge over time, a linear mixed model was performed to account for within-subject correlations, and pairwise comparisons between knowledge scores across assessments were adjusted using Bonferroni correction. For EBPAS, a Mann–Whitney U test was performed comparing Year 1 training groups’ scores at post-workshop and 6-month time points (as scale scores were not linked between participants in Year 1 due to being administered through a different web-based survey than the knowledge test), and a Wilcoxon signed rank test was conducted comparing Year 2 participant’s scores at pre-training and 6-month time points. Finally, descriptive data were reported for therapist-rated confidence in their ability to implement MET/CBT for SUDs in their practice and barriers to implementation therapists indicated encountering.

Missing data and attrition

In Year 1, 48 (73.8%) therapists completed the baseline assessment; 62 (95.4%) completed the post-workshop; 38 (58.5%) and 40 (61.5%) completed the MI knowledge test and EBPAS, respectively, at the 6-month follow-up; and 47 (72.3%) completed the MI knowledge test and 38 (58.5%) completed the program feedback survey at 9-month follow-up. Comparison was made between follow-up assessment completers and non-completers on pre-training variables. Therapists’ agency affiliation (i.e., agency at which a therapist works) was significantly related to having completed the 6-month assessment (knowledge test: χ² (8, N = 65) = 19.64, p = 0.012; EBPAS: χ² (8, N = 65) = 19.75, p = 0.011) and 9-month assessment (knowledge test: χ² (8, N = 65) = 24.36, p = 0.002; program feedback survey: χ² (8, N = 65) = 19.64, p = 0.01).

In Year 2, 25 (92.6%) therapists completed the baseline assessment; 23 (85.2%) completed the 3-month follow-up; 21 (77.8%) completed the MI and CBT knowledge tests and 18 (64.3%) completed the EBPAS at the 6-month follow-up; and 19 (67.9%) completed the MI and CBT knowledge tests; and 18 (66.7%) completed the program feedback survey at the 8-month follow-up assessment. No significant differences on pre-training variables were found between completers and non-completers of the 3- and 6-month knowledge tests. Therapists who completed the 6-month EBPAS indicated a lower likelihood of adopting EBTs in response to requirements to do so at pre-training (i.e., EBPAS requirements subscale) than non-completers (U = 27.50, Z =  − 2.20, p = 0.028), suggesting that the training initiative engaged therapists who at baseline were more ambivalent about required EBP activities. In addition, completers of the 8-month knowledge test had significantly higher pre-training MI knowledge scores (t(23) = 2.50, p = 0.02) and greater openness to new practices (EBPAS Openness subscale; U = 30.00, Z =  − 2.03, p = 0.042) than non-completers. A significant difference was also found for the program feedback survey at 8 months, with completers having greater openness to new practices at pre-training than non-completers (U = 30.00, Z =  − 2.03, p = 0.042).

Results

Participants

Ninety-three therapists participated in the training initiative across the two cohorts (Year 1 n = 65, Year 2 n = 28). Therapists in both cohorts who provided demographic (n = 52) and clinical practice information (n = 70) were predominantly female (n = 39, 75.0%), white (n = 45, 86.5%), between 30 and 39 years old (n = 20, 38.5%), primarily master’s level (n = 47, 67.1%), had been in the field for ≤ 5 years (n = 22, 31.4%) or 6–10 years (n = 22, 31.4%), and had ≤ 5 years of experience providing SUD treatment (n = 45, 64.3%). Around three-quarters or more of therapists reported prior experience with MI (n = 65, 92.9%), CBT (n = 52, 74.3%), and SUD training (n = 55, 78.6%). Year 1 therapists were employed at 9 different CMHCs (M = 7.2 therapists per CMHC, range = 5 to 12), and Year 2 therapists were employed at 15 different agencies (M = 2 therapists per agency, range = 1 to 6). Descriptive data at baseline are provided in Table 1. Cohort 1 therapists were more likely to report having prior CBT training at baseline than cohort 2 therapists (χ² (1, N = 69) = 4.29, p = 0.04), but no other pre-training differences were found between the cohorts.

Table 1.

Therapist characteristics

	Year 1 (n = 65)a		Year 2 (n = 28)b
	#	%	#	%
Race
Asian/Pacific Islander	0	0	1	3.8
Black/African American	1	3.8	3	11.5
Multiracial	2	7.7	0	0
White	23	88.5	22	84.6
Gender
Female	19	73.1	20	76.9
Male	7	26.9	6	23.1
Age
20–29	4	15.4	4	15.4
30–39	12	46.2	8	30.8
40–49	4	15.4	9	34.6
50–59	2	7.7	4	15.4
60–69	3	11.5	0	0
70–79	1	3.8	1	3.8
Credentials
BA/BS/BSW	2	4.5	-	-
LCAC	2	4.5	2	7.7
LCSW	14	31.8	9	34.6
LMFT	2	4.5	-	-
LSW	5	11.4	3	11.5
MA/MS/MSW	5	11.4	5	19.2
PhD	-	-	2	7.7
Other	14	31.8	5	19.2
Role in agency
Therapist	27	61.4	24	92.3
Supervisor	17	38.6	2	7.7
Years in field
0–5	14	31.8	8	30.8
6–10	15	34.1	7	26.9
11–15	7	15.9	5	19.2
16–20	4	9.1	2	7.7
21–25	1	2.3	2	7.7
26 +	3	6.8	2	7.7
Years SUD treatment
0–5	29	65.9	16	61.5
6–10	10	22.7	5	19.2
11–15	2	4.5	3	11.5
16–20	2	4.5	1	3.8
21–25	-	-	1	3.8
26 +	3	2.3	-	-
Prior MI training
Yes	41	93.2	24	92.3
No	3	6.8	2	7.7
Prior CBT training
Yes	36	81.8	16	61.5
No	7	15.9	10	38.5
N/A	1	2.3	-	-
Prior SUD training
Yes	37	84.1	18	69.2
No	7	15.9	8	30.8

^aMissing therapist demographics for n = 39 and characteristics for n = 21 in Year 1, ^bmissing therapist demographics for n = 2 and characteristics for n = 2 in Year 2

Training engagement

Year 1

In year 1, 65 therapists attended the two-day workshop. Seventeen or 18 consultation sessions (depending on the consultation group) were offered over the 8 months. On average, therapists attended 11.14 (SD = 5.23, range = 0 to 18) consultation calls; 60 (92.31%) therapists attended one or more sessions, and 48 (73.85%) attended 10 or more sessions. Therapist consultation call attendance rate varied significantly based on agency affiliation (F(8, 56) = 7.01, p < 0.001), but was not related to background characteristics, including prior training or experience, years of experience, degree, role in agency, and MI knowledge pre-test scores (p’s > 0.25). Therapists reported on average presenting on approximately two cases (M = 2.2, SD = 1.4, range = 0–6) throughout the consultation sessions. Therapist compliance with submission of MET/CBT session recordings was very low (n = 2), with only two therapists (3.08%) submitting one session recording.

Year 2

In Year 2, 28 therapists attended the workshop. Sixteen to 18 consultation sessions (depending on the group) were offered over 8 months. Therapists attended 10.68 (SD = 4.53, range = 0 to 17) consultation sessions on average, with 26 (92.9%) therapists attending at least one session and 21 (75%) attending 10 or more sessions. Therapists reported on average presenting on approximately two cases (M = 2.44, SD = 1.1, range = 0–4) throughout the consultation sessions. There were no significant differences between cohorts on number of consultation calls attended or cases presented (p’s > 0.25). Significantly more therapists submitted MET/CBT session recordings than in year 1 (p = 0.02, Fischer’s exact test), with 36 recordings submitted by therapists (n = 5, 17.9%). Consultation call attendance rate was not related to therapist prior training, years of experience, degree, role in agency, or MI and CBT knowledge pre-test scores (p’s > 0.10). Therapist agency affiliation was not examined due to many agencies being represented with very few therapists per agency.

Training helpfulness

Therapists in both cohorts rated the helpfulness of the workshop, online resources, and books as between “somewhat” and “very much” helpful in learning and implementing MET/CBT in their practice. Therapists in Year 2 rated the helpfulness of the consultation calls higher than therapists in year 1 (U = 178, Z =  − 2.54, p = 0.01; year 1 “somewhat”/ “very much”; year 2 “very much”/ “extremely”). Receiving personalized feedback on a recorded session was rated as “a little bit” helpful by one therapist in Year 1, whereas four therapists in Year 2 who participated in this training element rated it as “very much”/ “extremely” helpful on average.

Training outcomes

Attitudes

In Year 1, Mann–Whitney U tests indicated no significant changes in attitudes at post-workshop compared to 6-months on any EBPAS subscale (appeal, divergence, openness, requirements) or overall (p’s > 0.31, Table 2), with therapists maintaining fairly positive attitudes towards EBTs at both time points. As therapists’ responses were able to be linked across assessment points in Year 2, paired-samples Wilcoxon signed rank tests were conducted to examine within-therapist change in attitudes. There were no significant changes between pre-training to 6 months on any EBPAS subscale or overall scores (p’s > 0.33).

Table 2.

Analyses examining changes in evidence-based practice attitudes

	Year 1				Year 2
Variables	Post-workshop (Mdn)	6 months (Mdn)	U	p	Pre-training (Mdn)	6 months (Mdn)	Z	p
EBPAS Total	48.00	49.00	1169.5	0.71	49.50	49.00	− 0.52	0.60
Requirements	3.00	3.00	1202.0	0.53	2.67	3.00	− 0.60	0.55
Appeal	3.25	3.50	1223.0	0.44	3.63	3.50	− 0.18	0.86
Openness	3.00	3.00	1114.5	0.97	3.38	3.13	− 0.98	0.33
Divergence	0.50	0.75	1253.0	0.31	0.50	0.63	− 0.55	0.58

EBPAS Evidence-based Practice Attitudes Scale

Knowledge

Compared to baseline scores (M = 67.9%, SD = 21.7%), MI knowledge scores were significantly higher at 6-months (M = 80.3%, SD = 8.5%, p < 0.01) and 9 months (M = 76.9%, SD = 10.1%, p = 0.003) in Year 1, with no difference between 6- and 9-month assessments (p = 0.82). Unlike Year 1, therapists’ MI knowledge test scores did not significantly change from baseline (M = 73.7%, SD = 11.6%) to 3 months (M = 76.2%, SD = 10.8%, p = 1.00), 6 months (M = 77.8%, SD = 6.8%, p = 0.98), or 8 months (M = 76.2%, SD = 10.3% p = 1.00) in Year 2. However, therapists’ CBT knowledge test scores significantly increased from baseline (M = 68.9%, SD = 21.2%) to 3 months (M = 78.0%, SD = 13.9%, p = 0.03) and persisted at 6 months (M = 82.0%, SD = 13.9%, p = 0.005), and 8 months (M = 82.5%, SD = 12.7%, p = 0.005).

Confidence

In both years, therapists on average reported feeling confident in their ability to implement MET/CBT for SUDs in their practice at the end of the training initiative (Year 1: M = 7.49/10, SD = 1.37, range = 5–10; Year 2: M = 7.61/10, SD = 1.34, range = 5–10). A large proportion of participating therapists (Year 1: 84.2%, Year 2: 83.3%) indicated feeling much more or somewhat more confident compared to before the training initiative, with a small portion of therapists (Year 1: 15.8%, Year 2: 16.7%) reporting feeling about the same confidence or somewhat or much less confident at the follow-up assessment. Regarding certain MET/CBT components, most therapists agreed or strongly agreed that they were better able to describe the rationale for integrating MI and CBT for SUD treatment (Year 1: 92.1%, Year 2: 94.4%), implementing MI strategies (Year 1: 86.8%, Year 2: 100%), performing a functional analysis of substance use in MI consistent style (Year 1: 86.8%, Year 2: 94.4%), identifying CBT goals and strategies for SUDs (Year 1: 84.2%, Year 2: 100%), and implementing CBT activities using an MI style and skills (Year 1: 86.8%, Year 2: 100%). However, only around half of therapists in Year 1 (55.3%) agreed they were better able to conduct outcomes and fidelity monitoring for MI/CBT for SUDs, while in Year 2, most therapists (77.8%) agreed they were better able to do so.

Barriers to implementation

In Year 1, therapists most frequently reported encountering the following barriers (see Table 3 for full list) to implementation of MET/CBT: lack of opportunity (44.7%), patient compliance (44.7%), and other barriers (36.8%). Importantly, the most cited “other” barrier among Year 1 participants was the COVID-19 pandemic, as Year 1 took place during 2020. Year 1 therapists reported an average of 2 barriers (SD = 1.21, range = 1 to 5), while Year 2 therapists reported an average of 1.5 barriers (SD = 0.99, range = 0 to 3), though this was not a significant difference. Therapists in Year 2 most frequently reported lack of time to access/counsel patients (38.9%) and patient compliance (38.9%), followed by policy issues within the agency (22.2%) and lack of opportunity (22.2%) as barriers to implementation of MET/CBT.

Table 3.

Barriers encountered during implementation of MET/CBT

Barriers	Year 1 N (%)a	Year 2 N(%)b
Economic resources	4 (10.5%)	1 (5.6%)
Equipment resources	6 (15.8%)	1 (5.6%)
Administrative Support	8 (21.1%)	2 (11.1%)
Lack of opportunity (patient referrals)	17 (44.7%)	4 (22.2%)
Lack of time to access/counsel patients	7 (18.4%)	7 (38.9%)
Policy issues within agency	2 (5.3%)	4 (22.2%)
Insurance/reimbursement	1 (2.6%)	0 (0%)
Patient compliance	17 (44.7%)	7 (38.9%)
Other	14 (36.8%)	1 (5.6%)

^aYear 1 n = 38, ^bYear 2 n = 18

Discussion

This study describes a statewide dissemination and implementation initiative to increase the availability of evidence-based SUD treatment, namely MET/CBT, in [state masked]. Two cohorts of therapists were offered a multi-component training consisting of a 2-day workshop on MET/CBT, bi-weekly group consultation with MET/CBT clinical experts, fidelity monitoring of MET/CBT clinical implementation, and guided readings and online resources. Overall, therapists found the training elements to be “somewhat” to “extremely” helpful and engaged in or used most of the training elements. For example, around three-quarters of therapists in both cohorts attended 10 or more consultation calls. This rate is comparable to or higher than consultation call attendance rates found in other statewide training initiatives.^18,19,33

However, while providing monthly recordings of MET/CBT therapy sessions for fidelity monitoring was an expectation of the training initiative, therapist compliance with this training element was very low in Year 1, with only 2 out of 65 therapists submitting a recording. Notably, even with the addition of extra troubleshooting around recording therapy sessions and providing therapists with a Lyssn account in Year 2, this number, though significantly increased, was still low, as only 5 out of 28 therapists submitted recordings. This finding is reflected in therapists’ assessment of their ability to conduct outcome and fidelity monitoring for MI/CBT for SUDs. Only 56% of therapists in Year 1, and 78% in Year 2 were confident in their ability to conduct outcome and fidelity monitoring. Direct observation of sessions is considered the gold standard for fidelity monitoring,³³ but numerous barriers may hinder the recording of therapy sessions, including agency policies, cost and resources, and perceived confidentiality and HIPAA challenges.^34,35 Given the barriers to obtaining as well as coding session recordings, analogue methods to assessing fidelity, such as therapist self-report, behavioral role-play assessments, text-based conversational agent, and chart-stimulated recall, may provide more practical alternatives for assessing fidelity in statewide implementation initiatives.^36–38

Study results demonstrated improved therapist knowledge and confidence post-training. Therapists’ knowledge increased from pre- to post-training, with these knowledge gains maintained at follow-up assessments. In Year 1, therapists MI knowledge increased pre- to post-training. Interestingly, in Year 2, therapists’ CBT knowledge increased pre- to post-training, but MI knowledge did not. Additionally, therapists in both cohorts reported feeling confident in their ability to implement MET/CBT for SUDs in their practice and better able to implement several MET/CBT components at the end of the training initiative. A therapist’s confidence in their ability to implement EBTs may play a role in EBT adoption and sustainment in practice, as therapists who are less confident in their ability to implement MET/CBT may struggle with initial implementation. Difficulties with implementation may result in less confident therapists precipitously stopping their use of MET/CBT or perhaps, not implementing the full protocol or the more challenging components of MET/CBT.^39,40 Future research should examine the role of therapist confidence in the implementation and sustainment of EBTs.

Therapists’ EBT attitudes, however, did not change during the training initiative. This lack of change may be due to a ceiling effect, with therapists’ attitudes already being highly favorable towards EBTs. Indeed, EBPAS scores in this study were more favorable towards EBTs than those in the original study³⁰ and national norms³¹ as well as those in studies on SUD treatment providers.^41,42 Consequently, therapists who chose to participate in the training initiative may have already had more favorable attitudes and been on board with EBTs.

Several barriers to training engagement and MET/CBT implementation were identified in this study. Organizational-related barriers to MET/CBT implementation were frequently cited by therapists, including policy issues within the agency and lack of time. Furthermore, in analyses exploring factors associated with consultation call attendance, therapist’s agency affiliation was found to be significantly related to attendance, with therapists in some agencies having lower attendance rates compared to other agencies. Thus, for therapists to fully engage in training activities and implement MET/CBT, it will be important to have a therapist’s agency on board to facilitate training engagement and EBT implementation. Even though several pre-implementation changes were incorporated between year 1 and 2 of the training initiative (e.g., introducing formal agreements, troubleshooting submitting session recordings), the only difference in training engagement was in the number of therapists who submitted session recordings for fidelity monitoring. These findings highlight that pre-implementation or parallel work with agency leaders, to get them bought in and invested in training and implementation initiatives, may increase the success of these efforts. For instance, in a study implementing CBT into residential treatment facilities, strategies for addressing identified barriers were collaboratively selected with the treatment facilities during the pre-implementation phase, and subsequent implementation of these strategies eliminated the identified barriers prior to moving into the implementation phase, increasing the probability for successful implementation.^43,44

Another barrier noted by therapists was patient compliance and lack of opportunity to implement MET/CBT into their practice (i.e., no patient referrals). Therapists did not necessarily have time protected or specifically allocated for SUD patients on their caseload, suggesting the need for participating therapists (and agencies) to have a plan for ensuring therapists have the capacity for appropriate training cases. It is critical for therapists to have enough training cases with which to practice implementing MET/CBT and on which to receive consultation and feedback in order to improve their skill and confidence.⁴⁵ Moreover, having training cases to begin implementing MET/CBT immediately, with minimal lag between training and actual implementation of the new practice, may increase clinician competence and intentions to sustain use of the new practice.⁴⁶ Future implementation efforts may want to focus time during training on connecting and collaborating with community referral sources, so these sources know to refer SUD cases to these therapists and agencies, especially if the treatment being learned represents a major expansion of the scope of services offered by a therapist or agency. One promising implementation model that may address this need is the Community Based Learning Collaborative (CBLC), which is an adaptation to the Learning Collaborative model that conjointly trains senior leaders, clinicians, and service brokers who identify and refer families to services (e.g., case managers).⁴⁷ Findings from a statewide initiative that conducted five CBCLs on trauma-focused CBT showed increased interprofessional collaboration in the community and decreased community barriers to youth and families accessing trauma-focused EBTs following the CBCL.⁴⁸ Therefore, working directly with service brokers in future trainings may increase the likelihood that participating therapists will have appropriate training cases and be able to sustain their MET/CBT practice beyond the training period.

Limitations

There were several limitations to the current study. First, while organizational factors were noted as likely influencing training engagement and implementation, no organizational measures (e.g., implementation climate, culture, and leadership) were collected in this study. Future work should include organizational measures as well as community-level measures (e.g., interprofessional collaboration) to determine their influence on training and implementation outcomes. In addition, the current study did not collect patient or MET/CBT use or fidelity outcomes. Attempts were made to assess MET/CBT fidelity through submission of session recordings, but as noted above, very few therapists complied with this training component. Lastly, follow-up assessment response rates were low, and several differences were found between follow-up assessment completers and non-completers. Low survey response rates are not uncommon, however, and those reported here are similar to those in previous studies.^18,49

Implications for Behavioral Health

Ongoing training in EBTs for SUDs may help address the limited workforce trained to deliver EBTs for SUDs in community settings. The current study describes the scope and outcomes of a statewide implementation initiative that sought to increase the availability of and number of community-based therapists trained in MET/CBT for SUDs. Therapists participated in most of the training elements (e.g., workshop, consultation calls), but few participated in the fidelity monitoring component. Given this finding, analogue methods to assessing fidelity may provide a more practicable, feasible alternative for future iniaitives. Therapists indicated feeling more confident in their ability to implement MET/CBT and significantly increased their MI and CBT knowledge pre- to post-training. Nevertheless, several barriers to training engagement and implementation were identified by therapists. Based on these results, future implementation efforts should engage organizational leadership and community referral sources in order to increase the success of the initiative by addressing organizational-level barriers and improving the referral stream of appropriate training cases.

Funding

Funded by the National Institute on Drug Abuse (NIDA) by the National Institutes of Health (NIH; UG1DA050070).

Data Availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Declarations

Conflict of interest

The authors declare that they have no conflict of interest.