A Preliminary Investigation of Provider Attitudes Toward a Transdiagnostic Treatment: Outcomes from Training Workshops with the Unified Protocol

Abstract

Evidence-based psychological treatments (EBPTs) for common mental health conditions are efficacious but remain underutilized in clinical service settings. Novel transdiagnostic and modular approaches that treat several disorders simultaneously promise to address common barriers to the dissemination and implementation of traditional EBPTs. Despite the promise that transdiagnostic treatments hold, the claims that these interventions can be more easily disseminated and implemented have not been widely tested. The present study examined whether a transdiagnostic treatment, the Unified Protocol (UP), addresses some barriers to dissemination and implementation for clinicians. Exploratory aims of the current study were to examine the effects of a UP introductory training workshop on clinician attitudes and behaviors by: (1) evaluating UP knowledge and treatment delivery, (2) determining relationships between clinician characteristics and their knowledge acquisition, satisfaction with UP, and UP penetration, and (3) exploring clinicians’ perceptions of the UP’s characteristics utilizing mixed methods. Workshop participants showed a good understanding of UP treatment concepts following training, and over a third of survey respondents reported use of the intervention 6-months after training. Positive attitudes toward EBPTs and fewer years of clinical practice were associated with greater satisfaction with the UP. Clinicians held positive views of the UP’s flexibility and relative advantage over standard EBPTs but held negative views toward the manual’s design and packaging. Overall, our findings suggest that clinicians may view transdiagnostic treatments such as the UP favorably and may consider them appealing over standard EBPTs. However, barriers associated with traditional EBPTs may extend to transdiagnostic treatments like the UP.

Introduction

Evidence-based psychological treatments (EBPTs) are considered the gold standard of care for common mental health conditions, including anxiety, depressive, obsessive–compulsive, and traumatic stress disorders (Spring & Neville, 2014). Despite robust empirical support for these treatments, EBPTs remain underutilized in clinical service settings (Karlin & Cross, 2014; Kazdin, 2016; McHugh & Barlow, 2010). Transferring EBPTs from research settings to community mental health clinics and non-specialist practitioners is a complex process (Addis, 2002; Karlin & Cross, 2014), which involves strategies for the adoption, implementation, and sustainment of EBPTs (Nilsen & Bernhardsson, 2019). Adoption is typically defined as the decision to uptake a new treatment while implementation is conceptalized the strategies for integrating novel interventions into clinical and community settings (Rogers, 2003; Schoenwald, Mchugh, & Barlow, 2012). One of the most common implementation strategies for community clinicians in clinical service settings is EBPT training aimed at improving knowledge and encouraging providers to adopt and employ novel interventions (Barwick et al., 2012; Taylor & Abramowitz, 2013). EBPT trainings have been shown to help clinicians learn new material better than on their own (Beidas & Kendall, 2010). However, workshops alone are not likely a sufficient strategy to lead to high rates of implementation, also referred to as intervention penetration (Proctor et al., 2011), or for providers to implement EBPTs with fidelity (i.e., as intended by treatment developers; Addis, 2002; Beidas & Kendall, 2010; Herschell, Kolko, Baumann, & Davis, 2010).

Several factors may impact whether clinicians adopt and apply new interventions, including intervention characteristics and adopter characteristics. The first factor, intervention characteristics, involves clinicians’ perceptions of an intervention (i.e., how compatible providers consider a treatment for a patient population or their setting). Treatments that are seen as complex and inflexible are less likely to be adopted and implemented by clinicians (Damschroder et al., 2009; Harned, Dimeff, Woodcock, & Contreras, 2013). The second factor, adopter characteristics, includes clinician attitudes toward EBPTs (Aarons, Hurlburt, & Horwitz, 2011; Stirman, Gutner, Langdon, & Graham, 2016). Clinicians with more open attitudes toward EBPTs are more likely to apply techniques as intended, or with fidelity (Aarons, 2005; Beidas et al., 2015). Unfortunately, Deacon et al. (2013) have demonstrated many clinicians may hold negative views toward elements of EBPTs, such as exposure therapy, a well-established intervention for anxiety disorders (Farrell, Deacon, Kemp, Dixon, & Sy, 2013). In addition, although adopter characteristics frequently interact with factors at other levels (e.g., organizational support for EBPTs; Aarons, Sommerfeld, & Walrath-Greene, 2009; Beidas et al., 2015), evidence suggests provider attitudes may independently influence the degree to which EBPTs are incorporated into routine clinical practice (Beidas et al., 2015; Harned et al., 2013). For example, Beidas et al. (2015) found that when clinicians in the public sector were required to deliver EBPTs, those with negative attitudes toward and lower knowledge of EBPTs were highly likely to use psychodynamic or eclectic therapy techniques and veer away from treatment fidelity. Decreased adherence to EBPTs may lead to worse outcomes for patients in community clinics (Marques et al., 2019).

Clinicians’ attributes, another adopter characteristic, may also impact the adoption and implementation of novel interventions. Attributes include education level, theoretical orientation, and years of clinical experience. For example, clinicians’ training (i.e., high level of education) has been associated with positive attitudes toward and higher application of EBPTs (Aarons, 2004). Similarly, theoretical orientation such as cognitive-behavioral has been linked to greater use of EBPTs compared to other orientations (Nelson & Steele, 2007; Stewart, Chambless, & Baron, 2012). Years of clinical experience have shown mixed results in relation to implementation outcomes, with some studies showing an association between more years and lower use (Aarons, 2004; Aarons et al., 2010; Beidas, Edmunds, et al., 2014; Garner, Hunter, Godley, & Godley, 2012; Martino, Canning-Ball, Carroll, & Rounsaville, 2011) and others showing an association between more years and greater use of EBPTs during implementation efforts (Carpenter et al., 2012). In sum, understanding adopter characteristics and intervention characteristics for EBPTs may be critical for their application in routine clinical settings.

Transdiagnostic Treatments

Over the past decade, transdiagnostic and modular treatments have received increasing empirical support for their efficacy and efficiency (Andersen, Toner, Bland, & McMillan, 2016; Barlow et al., 2017; Newby, Twomey, Li, & Andrews, 2016; Norton & Paulus, 2017; Pearl & Norton, 2017). These treatments, like the Unified Protocol for the Transdiagnostic Treatment of Emotional Disorders (Barlow et al., 2018), target several disorders at once and may address some barriers to adoption and implementation of traditional EBPTs (Harvey & Gumport, 2015; McHugh, Murray, & Barlow, 2009; Stirman et al., 2016). For example, providers may save time and resources by training in one intervention that addresses a range of diagnoses, rather than learning numerous EBPT protocols—each for a single disorder (McHugh et al., 2009). Similarly, clinicians may view transdiagnostic treatments as relevant to highly comorbid patient presentations that are common in routine care, perhaps increasing their willingness to adopt and implement them (Harvey & Gumport, 2015; McHugh et al., 2009).

In addition, modular treatments represent a personalized approach as skills are drawn from a bank of available intervention strategies to match patients’ unique constellation of clinical problems (Chorpita & Daleiden, 2009; Murray et al., 2014). The UP is also one such treatment that consists of eight modules, five of which are considered core, including exposures. Evidence suggests that UP modules are efficacious on their own (Sauer-Zavala et al., 2017) although the treatment manual does not provide instruction on how to deliver the UP in a modular manner. Modular treatments may be viewed by practitioners as more flexible than standard EBPTs, in large part because these protocols focus on only necessary components for specific presenting problems instead of rigid and prescriptive content and timelines for care (Chorpita & Daleiden, 2009; Murray et al., 2014). Thus, providers may have positive attitudes and may apply modular treatments with greater fidelity than standard EBPTs that they consider superfluous to their patients’ pressing needs (McHugh et al., 2009).

Despite the promising claims made by treatment developers for improving dissemination and implementation through transdiagnostic and modular care, the research on whether these aims are achieved is very limited. For example, only a handful of recent studies have empirically reported on transdiagnostic treatments in relation to implementation factors. One such study reported on clinicians who were trained to deliver the UP in an eating disorders clinic. Providers perceived the UP as “logical” and likely to help their patients. Findings from this study also suggested that providers with better attitudes toward the UP were those from CBT backgrounds and those who already viewed EBPTs and exposure therapy favorably (Thompson-Brenner et al., 2018). Unfortunately, a weakness of this study is that the authors did not compare clinician attitudes to outcomes of fidelity or penetration in delivering the UP (Thompson-Brenner, Boswell, Espel-Huynh, Brooks, & Lowe, 2019). Another study on pre-implementation perceptions toward the UP across stakeholder groups in a Veterans’ Affairs (VA) setting found favorable views for UP’s fit with patients’ needs, treatment compatibility with the setting, efficiency, and cost-effectiveness, and alignment of the intervention with organizational values and priorities (Gutner, Canale, Vento, & Stirman, 2019). A weakness of this study is that impressions toward the UP were not tied to implementation outcomes.

While evidence suggests EBPTs generally are not effectively implemented through training workshops alone, transdiagnostic treatments may differ (Gros, Szafranski, & Shead, 2017). A recent effort to implement a transdiagnostic treatment via workshops resulted in moderate intervention penetration within a VA medical setting (Gros et al., 2017). Specifically, this study examined the dissemination and implementation of transdiagnostic behavioral therapy (TBT) through a voluntary 4-hour training with 28 providers who rated themselves pre- and post- workshop on knowledge and confidence in delivering TBT. Responses indicated improved understanding and confidence from the workshop alone. Additionally, implementation results showed self-reported use of TBT for 50% of clinicians who responded 6-months following the training. Providers also indicated positive perceptions of the effectiveness of TBT and data from their patients suggested meaningful changes in treatment outcomes (Gros et al., 2017). This preliminary finding suggests transdiagnostic treatments may indeed address some barriers to dissemination and implementation. However, there were several limits to the generalizability of this study’s findings, which included a small sample size, providers with previous experience in delivering EBPTs, a VA setting that differs from routine clinical care, and lack of information on implementation determinants (e.g., intervention characteristics, adopter characteristics). Thus, further research is needed to determine if trainings alone are a sufficient implementation strategy to positively impact implementation outcomes for innovative transdiagnostic treatments, including fidelity to the original protocols (Beidas & Kendall, 2010; Herschell et al., 2010) and intervention penetration across clinical service settings in the community (Addis, 2002).

Present Study

Given the limited research on transferring transdiagnostic treatments to routine clinical care, the current study sought to explore community clinician attitudes, attributes, perceptions, and behaviors following an introductory workshop on the UP (Barlow et al., 2018). Several workshops on the UP are held each year at a specialty research clinic, and attendees consist of community clinicians who pay to attend and receive continuing education credit. These workshops are delivered as standard introductory trainings on the UP and offer an opportunity for study of real-world trainings attended by community clinicians.

In the present study, we intend to replicate and expand upon Gros et al.’s (2017) preliminary finding that transdiagnostic interventions can be learned like other EBPTs via workshops, and may be adopted at a moderate intervention penetration rate. We plan to expand on the prior study by including information on implementation determinants such as intervention characteristics and adopter characteristics. As such, we will examine UP trainings as both a dissemination strategy (i.e., effectiveness in learning treatment content) and an implementation strategy (i.e., effectiveness for intervention penetration). Our study aims to identify adopter characteristics (i.e., clinician demographic variables, attitudes) and intervention characteristics of the UP and to examine intervention penetration as an implementation outcome. Our aims are exploratory in nature given the limited literature on these variables in relation to transdiagnostic treatments. Specifically, we intend to: (1) evaluate clinicians’ knowledge of basic UP concepts following a 1-day introductory workshop both directly following the workshop and 6-months later and to evaluate UP intervention penetration 6-months later (i.e., uptake of UP by workshop attendees into their clinical practices), (2) examine relationships between adopter characteristics (e.g., years practiced, orientation) and knowledge uptake, satisfaction with the UP, and UP intervention penetration, and (3) characterize clinicians’ attitudes and views towards the UP intervention (e.g., UP’s compatibility with their clinical setting, UP’s flexibility, UP’s relative advantage) using a mixed-methods approach with quantitative data drawn from a pool of workshop attendees and qualitative data collected from a smaller subset of these individuals.

Method

Participants

Study participants consisted of community clinicians that completed an in-person UP training. All attendees were sent online e-mail surveys pre-, post- and 6-months following each workshop. Of note, our final sample included only participants who completed pre- and post, or pre- and 6-month follow-up surveys. Participants that completed only one timepoint or did not complete pre-workshop surveys were not included in the final sample.

Study Assessment and Procedures

This mixed-method study utilized a concurrent triangulation approach to the collection and analysis of qualitative and quantitative data (Hanson, Creswell, Clark, Petska, & Creswell, 2005). In this design, both types of data were collected at the same time (as opposed to sequentially) and were also analyzed concurrently with data integration taking place during the data interpretation stage of research. Qualitative data were included to supplement quantitative data given the exploratory nature of the study and the limited available information on implementation determinants for transdiagnostic treatments.

Quantitative data were collected at pre-, post-, and 6 months following participants’ workshop attendance via an online survey platform (Qualtrics). Before the workshop, all registrants received an email invitation and a link to the survey that included an opportunity to indicate consent. Following completion of the workshop and 6 months after, another survey was emailed to all workshop participants, regardless of whether they completed the pre-workshop questionnaires, including a question to indicate consent and complete unanswered pre-workshop items (e.g., demographic information) or allow demographic information to be collected from their registration form.

For qualitative data, study staff also approached clinicians at the workshop, or by phone following workshops, and asked them to participate in interviews about their perceptions of the UP. For interview data, we utilized purposive sampling, which involves selecting participants who add to the richness of sought information, instead of probabilitydriven sampling in quantitative methods that aims for a sample that is representative of a population (Patton, 2015). Given our purposive sampling and our goal of diverse characteristics in our participants, initially all workshop participants were approached to participate in the interviews. However, after the first workshop, graduate students were excluded from recruitment for interviews to narrow in and include viewpoints of clinical providers from routine care settings. This type of segmentation is common in purposive sampling (Hennink, Hutter, & Bailey, 2020). No graduate students from Boston University were included in any of our data collection, including interviews. Workshop participants who were approached for participation in interviews were given an information sheet about the study, and interviewees provided verbal consent. All procedures were approved by the Boston University Institutional Review Board.

Pre‑workshop Questionnaire and Registration Form

The pre-workshop survey inquired about demographic characteristics (e.g., age, gender identification, race/ethnicity), as well as clinicians’ professional background, such as therapeutic orientation, disorders typically treated, caseload, and typical number of sessions per case. Also, we asked clinicians to fill in the blank for “years of experience conducting psychotherapy (individual or group)”; we did not specify whether participants should exclude training years (i.e., the time they were in graduate school) from their responses. Survey responders could select more than one orientation and disorder they treated. For caseload, clinicians selected from the following categorical choices “1–4 patients,” “5–10,” “10–20,” “20–30” and “30+.” Similarly, clinicians selected from pre-defined categories (e.g., “1–4 sessions,” “5–10”) to identify the average duration or number of sessions for which they see patients.

Additionally, as part of the pre-workshop questionnaire, two validated questionnaires commonly used in implementation research were administered. The Evidence Based Practice Attitude Scale (EBPAS; Aarons, 2004) is a 15-item scale that assesses clinicians’ attitudes toward evidence-based intervention strategies. Participants rate their agreement with statements about these strategies on a scale of 0 (not at all) to 4 (to a very great extent). Higher total scores (range 0–60) indicate a more positive global attitude towards the adoption of evidence-based treatments. Overall, the EBPAS displays good internal consistency, reliability, and support for face and content validity (Aarons, 2004).

The Therapist Beliefs about Exposures Scale (TBES; Deacon et al., 2013) was included in line with prior research (Thompson-Brenner et al., 2018) to assess attitudes toward exposure, one of the five core modules of the UP. We also utilized the TBES given common negative views toward exposures. TBES contains 21-items to assess therapists’ beliefs about the use of exposure therapy techniques in psychological treatment. Participants rated their agreement with statements on a scale from 0 (disagree strongly) to 5 (agree strongly). Higher scores on the TBES indicate beliefs that exposures are harmful or aversive. The TBES has demonstrated good 6-month test–retest reliability and excellent internal consistency (Deacon et al., 2013).

Post‑workshop Questionnaires

Directly following the workshop, participants responded to items examining their satisfaction with the UP; individual items are displayed in Table 2. Each item ranged from 0 to 4 with higher scores indicating greater satisfaction. Additionally, participants completed a knowledge acquisition quiz to assess their understanding of UP concepts. The UP Knowledge Acquisition Quiz consists of 11 multiple choice questions such as “What are the core deficit(s) targeted by the Unified Protocol?” A correct answer counts as 1 point, and scores on this quiz range from 0 to 11 with higher scores indicating greater understanding. This quiz has not been validated, and no data exist on its psychometric properties.

Table 2.

UP satisfaction data at post and 6-month follow-up

Item	Post-workshop N = 57 M (SD)	6-month follow-up N = 21 M (SD)	T-value df = 32
UP is a good addition to repertoire	3.37 (0.79)	3.19 (0.98)	0.85
UP will lead to better outcomes	3.04 (0.91)	2.71 (0.96)	1.40
UP is straightforward to use	3.11 (0.72)	2.81 (0.98)	1.48
UP skills and concepts make sense to me	3.60 (0.62)	3.48 (0.93)	0.66
UP skills and concepts will make sense to patients	3.07 (0.78)	3.00 (1.00)	0.33
UP modules are in a logical order	3.23 (0.76)	3.00 (1.00)	0.83

Note. All satisfaction items were scored 0–4, with higher scores representing greater satisfaction. All t-tests were non-significant

6‑Month Follow‑Up

The 6-month follow-up questionnaire included the same items as the post-workshop questionnaire (i.e., knowledge acquisition quiz, satisfaction with UP) with the addition of self-report items that primarily assessed the extent to which therapists used the UP in the 6 months following workshop attendance (i.e., strictly followed the manual, included minor changes to the protocol, reordered or skipped modules, occasionally used UP exercises, or did not use the UP).

Clinician Interviews

Clinicians (N = 57) who attended workshops in January, May, and September of 2016 were approached to participate in interviews during their workshop sign-in. Graduate students were included only during the January 2016 workshop and were subsequently excluded to avoid overrepresentation of this group in our sample. No Boston University graduate students were interviewed. Of the eligible attendees (N = 57) approached at sign-in, 44 participants agreed to be contacted by phone for individual interviews by study staff (A.A.A and J.W.T.) about their perceptions of the UP. Their views were sought given the limited data on providers’ perceptions of this novel intervention. Unfortunately, most participants were unable to be contacted (i.e., did not return two phone calls, or missed scheduled appointments and did not respond to efforts to reschedule), and for this logistical reason, 8 interviews were completed. We did not pursue further interviews from additional workshops because we determined that we reached meaning saturation (Hennink et al., 2020). Interviews occurred between February and October 2016 and ranged from 17.63 to 59.78 min (M = 38.90, SD = 14.62).

The qualitative interview was developed based on the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009) and consisted of semistructured questions that were similar across participants. CFIR was specifically developed to assess and guide implementation efforts in transferring EBPTs from research settings to routine care and was derived by extracting key constructs from other evidence-based implementation theories. Though CFIR includes a number of constructs, we primarily focused on intervention characteristics for the purposes of this study given perceptions of an intervention can impact implementation outcomes. According to the framework, the key characteristics of an intervention include but are not limited to the intervention’s adaptability, complexity, design quality and packaging, evidence strength, and relative advantage compared to other interventions. For example, interventions with low adaptability that are perceived as a poor fit will be resisted by clinicians tasked with implementing them (Damschroder et al., 2009). Questions from qualitative interviews asked about the UP’s adaptability, complexity, evidence strength, and design quality.

Training Workshop

Training in the UP treatment consisted of a 6-hour workshop, in line with practices for standard introductory workshops for continuing education, and was led by study coauthors (A.A.A, C.C.R., and S.S.Z.). The content of the workshop first focused on the empirical evidence supporting the UP and principles of the treatment, such as cultivating mindful acceptance of emotions. In addition, the trainers reviewed each of the eight treatment modules with overhead slides and provided de-identified audio examples from UP therapy cases. Experiential activities (e.g., mindfulness exercises, interoceptive exposures) from the UP manual were demonstrated during trainings. The principles underlying the protocol were revisited as trainers reviewed modules, case examples from audio, and experiential exercises.

Coding Process and Data Analysis

Audio recordings from qualitative interviews were transcribed by an undergraduate research assistant and checked by a second assistant. Transcribed data were imported utilizing NVivo Version 11.4. Prior to data coding, two of the investigators (A.A.A. and J.W.T) discussed and agreed on classification of material into codebook themes (e.g., treatment’s adaptability, complexity, evidence strength, and design quality) based on the CFIR framework. Investigators then independently analyzed interviews using a process of top-down, a priori, or directed theoretical thematic analysis (Braun & Clarke, 2006). Data were analyzed with rapid coding technique, meaning only relevant themes and text identifying perceptions about intervention’s characteristics were coded as part of this study (McMullen et al., 2011). Following independent coding, discrepancies between coders were resolved through discussion. Themes from these data are identified and characterized in the result section below and triangulated with quantitative results where possible.

Results

Sample Characteristics

Workshop attendees (N = 132) were sent surveys prior to workshops, directly following and 6-months following workshops that occurred between October 2014 and May 2016. As indicated previously, our final sample included only participants who completed pre- and post-, or pre- and 6-month follow-up surveys. Forty-eight participants completed the pre-workshop survey and at least one other time-point (response rate of 36.4%).

Table 1 displays the demographic characteristics for participants at each survey timepoint (pre-, post-, and 6 months following each workshop). Most participants identified as female and White across survey timepoints (see Table 1). Average years of clinical experience for participants ranged from 11.41 to 14.94 (SD = 9.79–10.93) across time points (see Table 1). Therapists could select one or more theoretical orientation(s), and the majority (n = 30, 62.5%) identified as cognitive-behavioral. A number of survey responders were trainees (n = 25 for pre-; n = 24 for post-; and n = 11 for 6 months following workshops). Participants could also select more than one diagnosis they typically treat, and across timepoints most indicated they usually treat depression and anxiety. Additionally, when asked about their typical caseload, therapists most often selected the categorical answer choice “11–20 patients.” Across all time points the majority of therapists answered “10–20 sessions” when asked how long their typical cases last. Demographic data are presented using all available responses from workshop attendees (see Table 1). Not all participants completed all questions at each timepoint, therefore the number of responses for each item varied.

Table 1.

Participant characteristics at each timepoint

	Pre-workshop M (SD); N	Post-workshop M (SD); N	6 months follow UP M (SD); N
Age	39.85 (13.79); 40	40.42 (13.91); 38	33.23 (10.91); 13
Years experience	14.41 (10.82); 48	14.94 (10.93); 45	11.41 (9.79); 17
Years treating mood/anxiety disorders	12.54 (10.08); 46	13.02 (10.19); 43	10.00 (10.37); 16
Trainees n (%)	25 (52.0)	24 (53.3)	11 (52.4)
Female n (%)	40 (60.6)	36 (59.0)	16 (76.2)
Male n (%)	12 (18.2)	12 (19.7)	4 (19.0)
Race/ethnicity
White n (%)	44 (66.7)	44 (72.1)	14 (66.7)
Black n (%)	2 (3.0)	2 (3.30)	1 (4.8)
Asian n (%)	3 (4.5)	1 (1.6)	3 (14.3)
Latinx n (%)	1 (1.5)	0	1 (4.8)
Other n (%)	2 (3.0)	2 (3.3)	0
Therapeutic orientation
CBT (n)	30	26	14
Eclectic (n)	23	22	7
Disorder(s) typically treated
Depression (n)	48	44	19
Anxiety (n)	48	44	19
Eating (n)	12	12	3
Psychotic (n)	8	7	2
Personality (n)	24	22	8
Bipolar (n)	17	15	3
Substance use (n)	11	10	5
Other (n)	6	5	2
No clinical diagnosis (n)	2	2	1
Caseload
1–4 (n)	9	8	5
5–10 (n)	10	9	5
11–20 (n)	14	13	6
21–30 (n)	9	9	1
30+ (n)	7	7	1
Typical number of sessions provided
1–4 (n)	1	1	0
5–10 (n)	3	3	1
10–20 (n)	22	20	7
20–40 (n)	17	16	6
40+ (n)	6	6	4

Note. Therapists could identify with more than one theoretical orientation. Participants who are listed as CBT only endorsed cognitive and behavioral. Those listed as eclectic either selected other orientations in addition to cognitive and/or behavior or only selected another category. Eclectic category of orientations included psychodynamic, humanistic, supportive, or other. Participants could also select more than one diagnosis they typically treat. Other category of diagnoses listed were ADHD, disruptive behavior, conduct, adjustment, grief, and hoarding

At the pre-workshop timepoint we examined scores for EBPAS and TBES. Shapiro–Wilk tests indicated data from the EBPAS and TBES were normally distributed (p > .05). On the EBPAS, which was given as part of the pre-workshop questionnaires, participants indicated positive attitudes towards evidence-based practices (n = 23; M = 45.30, SD = 8.10, out of 60 total score). These scores are similar to the averages reported by Aarons (2004). Moreover, participants indicated they did not believe exposures are harmful on the TBES (M = 23.87, SD = 11.25), and these scores indicate better views than the validation sample reported by Deacon et al. (2013; M = 32.8, SD = 18.9).

Eight individuals participated in additional qualitative assessment. On average, participants who completed qualitative interviews were 33.75 years old (SD = 10.53) and had 4.07 years of clinical experience (SD = 2.98). Most of the sample identified as female (n = 7), White (n = 4), and non-Latinx (n = 8). Additionally, two participants identified as African American, one identified as other, and one preferred not to answer. Educational backgrounds varied in this sample and included two participants with a Ph.D. in clinical psychology, one medical doctor, one with a Master’s degree in clinical psychology, one with a Master’s degree in social work, and three with a Bachelor’s degree (and currently in clinical psychology Ph.D. programs).

Aim 1: UP Knowledge Acquisition and Intervention Penetration

As noted above, the first goal of the present study was to explore whether participating in a workshop for the UP transdiagnostic treatment resulted in an adequate understanding of UP concepts, as well as uptake of UP in clinicians’ practices. Knowledge acquisition questionnaires, completed as part of both the post-workshop and 6-month follow-up surveys, indicated participants had a moderate understanding of the UP concepts. Directly following the workshop, 47 participants completed this assessment and their scores indicated good knowledge of the UP (M = 8.68, SD = 1.51, out of 11). Shapiro–Wilk tests indicated this measure was not normally distributed (p = .003). The 21 participants who completed this measure at 6-month follow-up had similar scores (M = 8.81, SD = 1.57, out of 11), suggesting those who completed the follow-up survey maintained a similar level of knowledge. Shapiro–Wilk tests indicated this measure was normally distributed (p > .05).

At 6-month follow-up, 21 participants provided data on their current use of the UP. One participant (4.8%) indicated they follow all modules in the manual in the order specified and four (19.0%) indicated they deliver the protocol only making minor changes (i.e., they are about 80–90% adherent). Two (9.5%) endorsed using the UP as a guide but reordering or skipping modules, eight (38.1%) occasionally incorporated UP exercises into non-UP treatment, and six (28.6%) had not used it at all. Overall, 71.4% of participants who responded to 6-month follow-up used the UP in some capacity, with 33.3% of them reporting using the UP as specified, with minor changes, or in a modular fashion. In addition, we calculated intervention penetration based on the number of individuals trained (N = 132), or intent-to-train. Rates were much lower when considering this more conservative estimate with 5.3% (i.e., seven participants) reporting delivering the UP to their patients, 6.1% (i.e., eight survey responders) stating they occasionally incorporate UP components, and 4.5% (i.e., six participants) reporting they had not used the UP in the prior 6 months.

Aim 2: Clinician Characteristics and UP Knowledge Acquisition, Satisfaction, and Intervention Penetration

The second goal of the study was to explore relationships between clinician characteristics (e.g., orientation, total years of clinical experience) and knowledge acquisition as well as satisfaction with and use of the UP. Exploratory analyses were conducted to examine predictors of knowledge acquisition at post-workshop and 6-month follow-up. Kendall-tau correlations were used because this measure was not normally distributed at post-workshop and there was a small sample size. Results indicated pre-workshop EBPAS scores were not significantly associated with UP knowledge at post-workshop (r = .28, p = .10) and this relationship could not be assessed at 6-month follow-up due to small sample size. Pre-workshop TBES scores were not significantly associated with UP knowledge at post-workshop (r = − .10, p = .56) and could not be assessed at 6-month follow-up due to the small sample size. Similarly, years of experience (r = − .09, p = .51; r = − .07, p = .73 at post and 6-month follow-up, respectively) and years of experience treating depression and anxiety (r = − .07, p = .62; r = .13, p = .51 at post and 6-month follow-up, respectively) were not associated with UP knowledge. An independent samples t-test indicated therapists’ orientation was not significantly associated with UP knowledge at either post (t(32) = .08, p = .94) or 6-month follow-up (t(19) − .19, p = .85).

Descriptive statistics for the quantitative satisfaction data at post- and 6-month follow-up are displayed in Table 2. Of note, these responses were not normally distributed (Shapiro–Wilk tests < .05 for all items at both timepoints). There were no significant differences between satisfaction scores at the two time-points. Kendall-tau correlations were used to examine the relationship between therapist characteristics (EBPAS, TBES, years of experience, and years of experience treating depression/anxiety) with satisfaction ratings at post and 6-month follow-up. Results indicated pre-workshop EBPAS score was significantly associated with beliefs the UP is a good addition to the therapist’s repertoire (r = .24, p = .02), the UP will lead to better outcomes (r = .47, p = .01), the UP will be straightforward to use (r = .38, p = .03), the UP will make sense to patients (r = .44, p = .01), and the UP modules are in a logical order (r = .42, p = .02) at the post-workshop survey. Pre-workshop TBES score was negatively associated with “the UP is straightforward to use” (r = − .38, p = .03), “UP concepts and skills will make sense to patients” (r = − .41, p = .02), and “the UP modules are in a logical order” (r = − .45, p = .02) at post-workshop. These results suggest therapists who view exposures as potentially helpful indicated greater satisfaction with the UP on the mentioned items. Finally, years of experience and years of experience treating anxiety/depression were both negatively associated with “UP is a good addition to my repertoire” (r = − .39, p = .002; r = − .37, p = .003, respectively), indicating therapists with less experience tended to rate this item more highly. There were no significant correlations between therapist characteristics and satisfaction at 6-month follow-up.

To examine UP use at 6-month follow-up, the variable was dichotomized into participants who indicated they use the UP as written, make minor changes to the protocol, or use the protocol as a guide versus those who indicated they occasionally incorporate a UP exercise into non-UP treatment or did not use it at all. A series of logistic regressions examining a test of the full model against an intercept only model indicated years of experience (β = .04, p = .51) and years of experience treating depression and anxiety (β = .05 p = .38) were not predictive of UP use. EBPAS and TBES scores were not able to be used as predictors because only four participants completed these measures pre-workshop and also completed the 6-month follow-up survey. All predictors were associated with a non-significant Wald criterion indicating they did not predict the likelihood of using the UP above and beyond the intercept.

Aim 3: UP Intervention Characteristics

The third aim of the study was to use mixed-methods of qualitative and quantitative data to identify UP intervention characteristics, which may serve as barriers or facilitators toward its implementation in clinical settings. Below, we review each qualitative coding theme, including quotes that are representative of them. When applicable, we also integrated quantitative data from surveys to provide additional information for triangulation with qualitative themes. Overall, we note that we reached code saturation with our interviews where no new codes were identified at the eighth interview (Hennink, Kaiser, & Marconi, 2016).

Adaptability

In line with the CFIR framework, we coded adaptability as the extent to which participants expressed the UP could be adapted, tailored, or refined to meet relevant clinical needs (Damschroder et al., 2009). The UP was mostly seen as adaptable, flexible and relevant to providers’ clinical practice, including to group treatment. Participants tended to perceive that the UP may be used to meet many patient and clinician needs. An illustrative quotation that summarizes clinicians’ views is as follows:

I think it’s pretty flexible. I don’t have a problem adapting it. I think sometimes just reading through it, it can seem like the modules themselves are a little bit rigid, but my understanding is it actually can be more flexible as needed in a real clinical population.

At the same time as affirming the UP’s flexibility, a subset of clinicians indicated concerns about maintaining fidelity when faced with patient needs not covered in the UP. For example, one participant noted the need to “veer off” from the UP to add safety planning and another noted a limitation from using one approach, saying, “I think that the problem with having only one source of therapy approach is that sometimes not all patients benefit equally from it, having a range of evidence-based strategies is always helpful.”

Complexity

We also coded for perceived difficulty of implementing the UP, which includes clinicians’ views on complexity of the intervention and steps or resources needed to apply it (Damschroder et al., 2009). Overall, most clinicians thought the UP appeared to be less complex than single diagnosis EBPTs. A common theme included ease of use in their practice so far, or the belief it would be easy to use should they choose to implement it. Illustrative quotations included “I think it’s pretty easy so far,” “I think it is pretty approachable for patients,” and “The theory makes sense. I feel that’s a no brainer.” At the same time, another theme related to complexity included that UP treatment concepts were at a high level for patients presenting to community clinics, who have low educational attainment and “chaotic lives.” For example, one clinician stated, “I work with a very sick population on the hierarchy of needs. I tend to be dealing with the ones that are more salient, right, like safety, housing, shelter. I am usually dealing with more pressing needs.” Similarly, another participant indicated the UP tends to work only for certain patients, “I’ve been lucky to have some very psychologically minded, kind of high-functioning patients who take really well to CBT approach, and what the UP brings, so I think it’s been good [for] them.” As a result, while the UP was seen as simple to learn and apply for many patients, the intervention was seen as complex for vulnerable patients presenting to community clinics.

As illustrated in Table 2, the satisfaction data from surveys at both post-workshop and 6-month follow-up indicated participants found the UP straightforward to use, the UP skills and concepts make sense to the clinician and the UP skills and concepts will make sense to patients. These results are in part consistent with the qualitative results above. However, survey findings did not capture concerns of UP’s complexity for vulnerable patients that may seek care at high rates at community clinics.

Design Quality and Packaging

We also sought participants’ perceptions about the way in which the UP intervention is presented and assembled (Damschroder et al., 2009). A common theme was dissatisfaction with a prescriptive and lengthy manual. Specifically, clinicians expressed wishing for clear information in the intervention materials for how to “skip around” or provide modules out of order. Despite seeing the UP treatment as adaptable and flexible, they did not see this reflected in the intervention materials. An illustrative example from one of the clinicians is as follows:

People might not like something so prescribed … maybe they don’t want to spend two sessions on this and that … I’m not saying that’s what the UP is. I’m saying that might be someone’s perception based on if there’s a workbook and follow it.

Here, the quantitative and qualitative data appear to differ as data from the satisfaction questions on the surveys (see Table 2) suggested high average scores on the item assessing the logical order of UP modules.

Empirical Evidence

Participants’ perceptions were coded with regard to evidence quality and validity for the UP on outcomes of clinical interest (Damschroder et al., 2009). The theme that emerged consisted of views of strong empirical evidence for the UP, with one participant capturing the sentiment by stating, “there’s evidence to suggest it works and anticipate it to be pretty effective therapy when you use it … the evidence is compelling.” At the same time, some expressed the need for more empirical support in varied clinical settings outside of research trials, with one participant saying, “I don’t believe in its effectiveness in like a real-world implementation.”

Relative Advantage

Individuals’ perceptions of implementing the UP over an equivalent EBPT intervention were evaluated (Damschroder et al., 2009). A theme emerged of UP’s perceived relative advantage, including UP’s ability to effectively target comorbidity in treatment. For example, one participant noted, “I think it would be a lot more efficient learning something that could address comorbid disorders.” Similarly, another stated that if one is not a CBT therapist but “has to do CBT,” the UP is ideal because it is so comprehensive in addressing several disorders at once. Some participants also expressed it would increase efficiency in training in one protocol that targets several disorders, with one participant explaining:

…from a student perspective especially, when you’re just starting out in therapy, it can feel very overwhelming to know that there are so many different approaches to treating different things and there’s a comfort in having one protocol to build your treatment approach around.

These qualitative responses are consistent with quantitative data at both post-workshop and 6-month follow-up (see Table 2) wherein participants endorsed, on average, that the UP is a good addition to their repertoire and that it will lead to improved patient outcomes.

Discussion

Our exploratory aims for the present study were to (1) evaluate if clinicians learned and applied a transdiagnostic treatment, the UP, following an introductory workshop, (2) examine the relationship between clinician variables and clinicians’ perceptions and use of the UP, and (3) incorporate in-depth qualitative interviews and quantitative data from surveys to explore clinicians’ perceptions of the UP (i.e., intervention characteristics such as UP’s compatibility with their clinical setting, UP’s flexibility, UP’s relative advantage) for their clinical service settings. First, findings from the quantitative surveys suggest providers understood UP concepts and retained this knowledge 6-months later. As for applying the UP, our findings are in line with prior research indicating dissemination through workshops is not sufficient for high treatment penetration (Beidas & Kendall, 2010; Herschell et al., 2010). A portion of workshop participants reported using the UP with regularity (over a third of the survey responders or 5.3% of the intent-to-train sample), the remainder of the survey responders reported only using the UP occasionally or never. Our findings were somewhat similar to those of Gros et al. (2017), who reported 50% of clinicians who responded adopted TBT. These results should also be interpreted with the understanding that clinicians’ self-reported application of EBPTs is more optimistic than objective evaluations of their application of treatments, their fidelity in delivery, and their patients’ outcomes (Beidas & Kendall, 2010; Herschell et al., 2010).

Second, an examination of clinician characteristics (e.g., years of experience), knowledge uptake, satisfaction, and UP use yielded findings that are mostly aligned with the literature in this area. Generally, clinicians’ positive attitudes toward EBPTs and fewer years of clinical experience were related to higher satisfaction with the UP following the workshop. Prior experience is a characteristic that has yielded mixed associations in relation to EBPT use (Aarons et al., 2010; Beidas, Edmunds, et al., 2014; Carpenter et al., 2012; Garner et al., 2012; Martino et al., 2011). Unfortunately, due to the small size of the 6-month follow-up sample, clinician characteristics could not be evaluated in relation to treatment penetration. Future research should further explore the relationship between adopter characteristics and other variables impacting implementation outcomes, such as treatment penetration with larger sample sizes.

Third, mixed-methods results from surveys and interviews suggest that several aspects of the UP were universally valued. Community clinicians saw the UP as flexible and advantageous over traditional EBPTs. Specifically, both interviewees and survey participants saw an advantage to the UP both for enhancing their own clinical skills and their patients’ outcomes; in particular, interviewees noted the UP would address comorbidity efficiently. Also, community clinicians thought that the UP has strong empirical support. These findings were similar to two prior studies on UP intervention characteristics (Gutner et al., 2019; Thompson-Brenner et al., 2018). At the same time, some nuances appeared in the qualitative data that was not apparent in the quantitative surveys. Clinicians identified concerns about the UP’s complexity, design and packaging, and empirical support for effectiveness in community settings. Concerns included that the UP may be complex and have lower effectiveness for community patients and that the manual lacks instructions on omitting or administering modules out of order. These preliminary findings and those from a study by Gutner et al. (2019), which found that therapeutic orientation, limited training availability, and competing EBPTs may be barriers for UP adoption in a VA setting, all hint that at least some traditional barriers to EBPTs may also be relevant to UP and thus potentially to other similar transdiagnostic and modular manuals.

Limitations

These findings should be understood within their limitations. This study utilized a convenience-sample of workshop attendees from in-house trainings who paid to attend and provided self-report data and/or one-time interviews on their personal characteristics and impressions of the UP treatment. It is likely that workshop participants were already interested in the UP and are not representative of community clinicians more broadly. For example, many clinicians in our sample identified their orientation as cognitive-behavioral and their training as clinical psychologists. Many respondents were also trainees (though averages did not differ when trainees were removed from the analyses). Similarly, study participants may have held favorable attitudes towards EBPTs compared to community clinicians. In particular, views on exposure therapy for our survey sample were better than a heterogeneous sample of community clinicians (Deacon et al., 2013), and as a result, clinicians may have been more likely to hold positive attitudes and perceptions of the UP and more easily inclined to uptake the UP than most community providers. At the same time, given our offer for continuing education credit, a wider net of clinicians likely attended workshops and participated in our study than only those fervently interested in the UP. Also, scores on the EBPAS, which measure attitudes toward EBPTs fell in an average range that is comparable to other samples of community clinicians (Aarons, 2004).

Another major limitation of the current study is the exploratory nature of our analyses due to insufficient prior research in this area. Given the limited power for our analyses and the number of analyses that we ran, there is an increased chance for a Type 1 error. Also, the sample sizes for the 6-month quantitative survey represents a limitation. It is possible only clinicians who felt strongly about the protocol (positively or negatively) responded to the survey or requests to participate in the interviews. However, data from both samples indicate a range of responses which makes this possibility unlikely. As a result of the small sample sizes, the generalizability of the quantitative data may be limited, and further studies are needed in this area to replicate and expand on our findings. Similarly, the number of interviews could pose a limitation. However, saturation of information is more important in qualitative data than sample size (Hennink et al., 2016). Several studies that engaged in simulations using in-depth interviews, as we have done, showed that code saturation in qualitative data can be reached between 7 and 12 interviews (Guest, Bunce, & Johnson, 2006; Hennink et al., 2016; Namey, Guest, McKenna, & Chen, 2016). Despite these limitations, the current study is one of the first studies on the UP and other transdiagnostic and modular treatments to explore clinician characteristics and views related to workshop training—a notable strength.

Several other limitations should be noted when considering our results. Objective data on clinical applications of the UP with patients is not available, and thus we cannot conclude whether workshop training alone was sufficient in improving clinicians’ skills or their ability to balance fidelity and flexibility. In addition, we lack data on patient outcomes to determine if they showed symptom and daily functioning improvements as a result of receiving the UP. Future research can introduce more rigorous research methods for implementing the UP in community clinics in order to draw more conclusive results.

Additionally, the measures used to assess satisfaction with the UP and knowledge from UP workshops were created for the purposes of this study and are not validated. We did not administer a baseline knowledge test and cannot conclusively determine that the workshop alone led to improvements in knowledge. This limits our ability to draw strong conclusions about clinicians’ satisfaction and knowledge (Beidas & Kendall, 2010). Lastly, the present study only focused on clinicians, but barriers exist at the system, organizational, and client levels and can interact with one another in complex ways. It is possible the UP and other transdiagnostic treatments are better suited to address barriers at these levels than for clinicians, and more research is needed to examine this possibility.

Future Directions

Transdiagnostic treatments are thought to be relevant and flexible for patients’ presenting concerns and thus to lead to high fidelity and improved treatment penetration over traditional EBPTs (McHugh et al., 2009). In order for this hypothesis and others to be explored empirically, future research would benefit from examining the relationships between adopter characteristics, intervention characteristics and transdiagnostic treatments. For example, if providers see transdiagnostic and modular treatments similarly to other single-disorder EBPTs, the extent to which these novel treatments address traditional barriers to EBPTs is likely limited. We saw some positive attitudes toward the UP, but more research is needed in this area. Also, the role of clinicians’ orientation may be a relevant adopter characteristic for clinicians who subscribe to set schools of psychotherapy (e.g., psychodynamic, humanistic, integrative/holistic) and apply a similar approach to most of their patients. These clinicians may potentially see little appeal in treatments designed to address multiple disorders. Given the high percentage of CBT-affiliated clinicians in our study, future research would benefit from exploring views of relative advantage for transdiagnostic treatments for other orientations.

Furthermore, barriers to the implementation of EBPTs can exist across various system levels, including patients, clinicians, clinics, organizations, and even mental health systems (Addis, 2002; Cooper & Bailey-Straebler, 2015; Harvey & Gumport, 2015). For example, at the organizational level, a commonly cited barrier to implementation is the high cost of the infrastructure needed to train, supervise, and monitor clinicians to competently deliver a given EBPT with fidelity (Harvey & Gumport, 2015; Stirman et al., 2016). The UP and other transdiagnostic treatments are thought to be cost-saving for both training and mental-health care settings by instructing clinicians on one protocol and by reducing the duration of a course of treatment with one intervention for multiple disorders. For example, preliminary findings suggest that the UP may improve patient outcomes when implemented in inpatient settings for comorbid disorders (see, Bentley, Sauer-Zavala, Stevens, & Washburn, 2018; Thompson-Brenner et al., 2019). Future research should build on this premise and conduct economic cost-spending analyses (e.g., such as those conducted by Olmstead, Carroll, Canning-Ball, & Martino, 2011).

Overall, findings from this study and others like it suggest in order to understand how to adapt and apply an EBPT, clinicians may require a high level of understanding of theory of clinical practice (Roth & Pilling, 2007). This principle may also apply to transdiagnostic treatments. Even though the UP is comprised of stand-alone modules (Sauer-Zavala et al., 2017), community clinicians may need clear instructions within treatment manuals like the UP on how to flexibly apply the protocol, such as decision rules about administering modules and explicit instructions for adapting treatment principles to patients’ presenting problems. Workshops alone may not be enough to provide a high-level understanding of treatment mechanisms of action and how to tailor care (Rousseau & Gunia, 2016). More specialized tools and delineated guides may be needed in non-specialty clinics to identify and assess presenting problems (e.g., functional analyses) and to adapt treatment elements to presenting problems (e.g., applying intervention components to crises such as finding housing) as more non-specialist deliver treatments developed for specialists (Cooper & Bailey-Straebler, 2015). More research on improving fidelity by improving these intervention characteristics is needed.

Lastly, workshops on EBPTs may benefit from efforts to improve their effectiveness. Even though supervision or additional follow-up is typically needed to change clinicians’ behavior, recent research has indicated clinicians’ knowledge, skill, and readiness for change may be boosted through active learning strategies (see Beidas, Cross, & Dorsey, 2014; Hodorowicz, Barth, Moyers, & Strieder, 2019; Martino, 2010; Wyman et al., 2008). More research is needed in this area and specifically on exploring the most effective training strategies for clinicians and other provider groups (Martino, 2010).

Conclusion

Clinicians showed good understanding of UP treatment following training with an introductory workshop and viewed the UP favorably with regard to flexibility and relative advantage over standard treatments. Positive attitudes toward EBPTs and fewer years of clinical practice were associated with greater satisfaction with the UP, so those with a CBT orientation and trainees may view this treatment as particularly appealing. Clinicians implemented the UP at moderate rates following workshop training and held negative views toward the manual’s design and packaging. Thus, transdiagnostic and modular treatments, like the UP, may be viewed favorably and may hold appeal over standard EBPTs that treat one disorder. However, some of the perceptions typically associated with traditional EBPTs (e.g., complexity of treatment manuals for community patients) likely continue to impact transdiagnostic treatments. Future research may focus on identifying implementation determinants associated with transdiagnostic treatments and their relationship to implementation outcomes in order to fully understand the advantages these novel treatments offer.