A Mixed-Method Analysis on the Impacts of a System-Driven Implementation of Multiple Child Evidence-Based Practices on Community Mental Health Providers

Abstract

Initiatives to scale up evidence-based practices (EBPs) in routine care are likely to have myriad impacts on community providers, but these impacts have not yet been examined in depth. This is especially true within the context of simultaneous implementation of multiple evidence-based practices. The aim of this study was to characterize the multifaceted impacts on community mental health therapists within a system-driven implementation of multiple EBPs for youth and families. Semistructured interview and survey data were gathered from 60 therapists at 11 agencies contracted with the Los Angeles County Department of Mental Health to deliver EBPs within the Prevention and Early Intervention initiative. Therapists’ accounts of impacts varied, and were either predominately negative, predominantly positive, or mixed-valence. Mixed-methods analyses using Kruskal-Wallis tests showed therapist valence groups varied on mean levels of self-reported burnout on surveys. Themes from qualitative data revealed several favorable (e.g., increased EBP knowledge, structure) and unfavorable (e.g., distress, feeling constrained by EBPs) impacts of county-contracted EBP implementation. These findings inform the development and implementation of future system-driven EBP initiatives that consider therapist perspective to optimize positive impacts and minimize negative impacts on therapists.

Despite recent system-driven efforts to implement multiple evidence-based practices (EBPs) within community mental health settings (Lau & Brookman-Frazee, 2016), little is known about the personal and professional impacts of such efforts on the direct service providers involved. To date, most research dedicated to EBP implementation has focused on benefits and challenges faced by mental health systems, agencies, and clients (e.g., Novins, Green, Legha, & Aarons, 2013). However, the myriad impacts on therapists of EBP implementation initiatives require study, particularly within the increasingly common context of simultaneous implementation of multiple EBPs. Greater attention to impacts at the provider-level is warranted because therapists can derail or support the ultimate success of large-scale investments in their communities. For example, therapists who experience burnout associated with the demands of adopting EBPs are likely to leave their agency (Beidas, Marcus, et al., 2016; Beidas, Stewart, et al., 2016), thus jeopardizing both organizational functioning and EBP sustainment. In contrast, therapists who feel more effective in their duties and confident in their abilities after EBP training may go on to deliver sustained, high-quality care (McCluskey & Lovarini, 2005). The current literature on therapist impacts provides a fragmented representation with little to no discovery-oriented research on the nature of impacts on therapists, or attempts to unpack both positive and negative impacts (e.g., Bride, 2004; Decker, Jameson, & Naugle, 2011; Skriner et al., 2018).

Although shifts toward EBP implementation may affect all providers, therapists likely face impacts unique to community-based, publicly funded mental health contexts. Community therapists often work within large public sectors where they carry large caseloads and provide treatment to clients with complex presentations including clinical severity, poverty, and comorbidity (Southam-Gerow, Chorpita, Miller, & Gleacher, 2008). Each of these factors is linked to therapist burnout (e.g., Kim et al., 2018; Morse, Salyers, Rollins, Monroe-DeVita, & Pfahler, 2012), illustrating the already high vulnerability of community therapists to systemic changes. Additionally, community therapists often provide psychosocial services to trauma-exposed populations, which places them at increased risk of distress or secondary trauma (Bride, 2004; Meldrum, King, & Spooner, 2002). Moreover, top-down pressures to carry larger caseloads, comply with increasing documentation and regulatory demands, and achieve higher levels of clinical productivity via billing (Patterson-Silver Wolf, Dulmus, & Maguin, 2012; Skriner et al., 2018) may create high-stress environments that contribute to low morale, job dissatisfaction, and turnover (Beidas, Marcus, et al., 2016; Hoge et al., 2007; Paris & Hoge, 2010). Within system-driven EBP implementation contexts, there have been mixed findings concerning other workforce impacts. Under specific circumstances, therapist EBP utilization has been linked with detrimental workforce changes (e.g., turnover). Annual turnover rates as high as one in four front-line therapists have been documented in child and adolescent health service EBP implementation contexts (Beidas, Marcus, et al., 2016). However, causal links are unclear and other evidence suggests that EBP implementation has reduced provider turnover in child welfare contexts (Aarons, Sommerfeld, Hecht, Silovsky, & Chaffin, 2009). Additionally, in the same large-scale system shift toward multiple EBP implementation described in the current study, training in larger number of EBPs has been associated with higher emotional exhaustion (Kim et al., 2018) and a greater likelihood of discontinuing delivery of any given EBP (Lau et al., 2018). Indeed, earlier research has shown that some unfavorable attitudes toward treatment manuals among community clinicians with greater years of community treatment experience (Barry et al., 2008). This was echoed in more recent examinations of community providers’ responses to mandated evidence-based treatment delivery, with providers providing predominantly negatively valenced perspectives (Reding et al., 2018). However, positive organizational climates for EBP implementation (e.g., staff recognized and rewarded for using EBPs) result in lower turnover in community mental health agencies undertaking EBP implementation (Beidas, Marcus, et al., 2016). Thus, in routine practice settings where therapists face multiple contextual pressures, introducing requirements for EBP implementation may pose some risk for increased burden on the workforce, but when implementation is well supported, therapists and organizations may reap benefits.

Certainly, it is crucial to examine the potential benefits of EBP implementation on the workforce. Client-level gains in treatment outcomes as a result of single EBP implementation efforts have been documented (e.g., Beidas & Kendall, 2010; Maunder, Milne, & Cameron, 2008), but are not uniform (e.g., Clarke et al., 2005; Southam-Gerow et al., 2010). For therapists, favorable effects of EBP training have included short-term (e.g., Maunder et al., 2008; Scott, Klech, Lewis, & Simons, 2016) and long-term gains in therapist knowledge (e.g., Decker et al., 2011; Drahota, Stadnick, & Brookman-Frazee, 2014), confidence in delivering EBPs (e.g., Beidas & Kendall, 2010; Herschell, Kolko, Baumann, & Davis, 2010), and improved attitudes toward EBPs (Chagnon, Houle, Marcoux, & Renaud, 2007; Rubel, Sobell, & Miller, 2000). In turn, therapist knowledge, attitudes, and perceptions of self-efficacy have been shown to have downstream effects on EBP use, continued use, and fidelity (Beidas, Cross, & Dorsey, 2014; Lau et al., 2018; Okamura, Jackson, & Nakamura, 2019). System mandates and increased EBP trainings have been linked with increased use of EBPs among therapists (Lopez, Osterberg, Jensen-Doss, & Rae, 2011). Therapist adherence and competence in community EBP implementation are also evident when appropriate supports are provided (Beidas & Kendall, 2010; Westbrook, Sedgwick-Taylor, Bennett-Levy, Butler, & McManus, 2008). For instance, therapists who engaged in a system-wide cognitive–behavioral therapy training initiative showed increased openness to EBPs and use of CBT with clients, whereas among nonparticipating therapists, lower knowledge of EBPs drove greater use of psychodynamic (and not CBT) techniques (Beidas et al., 2014). Together, findings from studies of single EBP implementation efforts show impacts on therapist knowledge development, EBP skill acquisition and use, and subjective appraisals of their expertise.

Yet, less is known about how the efforts to scale up and implement multiple EBPs impact the workforce. System-driven implementation of multiple EBPs, rather than a single EBP, is an increasingly prevalent strategy for EBP implementation within state- and county-level mental health systems. Thus, it is crucial to examine how such efforts intended to improve the quality of children’s mental health services in turn affect therapists delivering services within these circumstances (McHugh & Barlow, 2010). Looking to contexts where multiple EBPs have been disseminated and implemented simultaneously, such as Hawaii’s Department of Health (Nakamura et al., 2011), Philadelphia’s Department of Behavioral Health and Intellectual disAbility Services (Beidas, Stewart, et al., 2016), and Los Angeles County Department of Mental Health in California (Lau & Brookman-Frazee, 2016), will allow us to identify impacts that may be similar to singular EBP roll-outs or distinctive to multiple EBP implementation efforts. Although EBP implementation efforts affect the multiple stakeholders involved, front-line therapists involved in direct service to clients and families are perhaps most proximally affected. Such impacts likely encompass significant changes to their practice, allocation of their time and effort, professional decision-making and service delivery procedures, as well as consequences for their job satisfaction and personal wellbeing. For instance, therapists subjected to system-wide implementation of multiple EBPs may be expected to participate in multiple trainings for multiple interventions, each of which may be associated with ongoing certification requirements, demands for EBP-specific assessments and documentation, and new administrative procedures (Regan et al., 2017; Rodriguez, Lau, Wright, Regan, & Brookman-Frazee, 2018). Indeed, understanding the experiences of community therapists delivering EBPs in this context is vital to understanding impacts that may contribute to overall implementation success and failure.

As multiple EBP implementation becomes increasingly prevalent, and a likely model of change for community mental health systems in the future, it is necessary to examine impacts on therapists within these naturalistic experiments. It is precisely in such a burgeoning field that qualitative analysis is opportune. Qualitative examination of therapists’ lived experiences allows for identification of previously unseen and unexpected effects. Analysis of qualitative interviews with primary stakeholders in the multiple EBP implementation process has the potential to identify specific avenues for improving implementation strategies. Leveraging qualitative analysis with quantitative methods, the present study sought to (a) characterize the valence of therapists’ overall perceptions of how system-driven implementation of multiple EBPs have impacted them as therapists, (b) codify the specific types of therapist-described impacts, and (c) contrast the most common themes that emerged in interviews among therapists with overall positive, negative or mixed-valence experiences with implementation.

Method

Study Context

In alignment with the goals of the Mental Health Service Act (MHSA) passed by California voters in 2004, the Los Angeles County Department of Mental Health (LACDMH) launched the Prevention and Early Intervention (PEI) Transformation in 2009. Through PEI, contracted and directly operated agencies were offered the opportunity to be reimbursed for the delivery of approved EBPs. Agencies were offered implementation supports as well as reimbursement for delivery of any of six EBPs through the PEI Transformation. The first round of mass provider trainings was launched in May 2010. Along with implementation requirements and guidelines, LACDMH provided large-scale implementation supports (i.e., initial training and consultation) for the six child and adolescent mental health practices, including Cognitive Behavioral Interventions for Trauma in Schools (CBITS; Stein et al., 2003), Child-Parent Psychotherapy (CPP; Lieberman, Van Horn, & Ippen, 2005), Managing and Adapting Practices (MAP; Chorpita & Daleiden, 2009), Seeking Safety (SS; Najavits, Gallop, & Weiss, 2006), Trauma Focused Cognitive Behavior Therapy (TF-CBT; Mannarino, Cohen, Deblinger, Runyon, & Steer, 2012), and Positive Parenting Program (Triple P; Prinz, Sanders, Shapiro, Whitaker, & Lutzker, 2009). These six practices were selected for supported implementation because of their ability to cover a wide range of common presenting problems and developers’ ability to scale up training (Regan et al., 2017). Here we aim to understand the unique myriad impacts of scaling up multiple EBPs on community therapists across a large behavioral system during the sustainment phase of the PEI Transformation five years after initial implementation. The premise of our study was to understand the implementation of multiple EBPs across an entire system of care, and not necessarily the experience of individual therapists implementing multiple EBPs.

Procedure

Qualitative data for the present study were collected as part of the “In-Depth” portion of the Knowledge Exchange on Evidence-based Practice Sustainment (4KEEPS) research project (Lau & Brookman-Frazee, 2016). The parent 4KEEPS study aimed to examine how multiple EBPs were sustained within the context of this large-scale system reform. Within 28 program sites across 14 agencies, semistructured interviews and surveys were carried out with therapists to gather detailed information about their experiences implementing PEI practices. Therapists received $20 gift cards for completing surveys and $40 gift cards for interviews. Institutional Review Boards at [Masked for Review] approved all procedures for this study.

Participants

Within the In-Depth portion of the 4KEEPS study, 123 therapists who delivered at least one of the six EBPs of interest within the last two months participated in semistructured interviews. All therapists were employed by program sites contracted to implement multiple EBPs. We then drew a stratified random sample of 60 therapist interviews from 20 programs within 11 agencies for transcription and intensive analysis. Stratification was carried out to ensure distribution of interviews describing experience with each of the EBPs. Thus, we first stratified therapists by the practice(s) discussed in the interview, and then sequentially selected interviews at random until a quota of at least 15 therapists per practice (with the exception of CBITS) was selected. We were not able to retain a focus on CBITS because of low rates of penetration and sustainment within the PEI Transformation broadly (Brookman-Frazee et al., 2012). During each interview, therapists discussed an average of 2.23 (SD = .67) practices, with each practice being discussed in as few as one interview (i.e., CBITS) and as many as 35 interviews (i.e., TF-CBT).

Table 1 displays the characteristics of the final sample of 60 therapists included. On average, the number of therapists drawn from each agency was 5.45 (SD = 3.7, Range = 1–13). Most of the therapists identified as female (85.0%), Hispanic/Latinx (60.0%), unlicensed (80%), and having a Master’s degree (83.3%). Almost all therapists had been trained in two or more EBPs tracked within the parent study (90%). Chi-square tests revealed no significant differences in gender, ethnicity, licensure, or education level between the 60 therapists selected to have their interviews analyzed and the 63 therapists who were interviewed but not selected for qualitative analysis.

Table 1.

Therapist Sociodemographic, Professional, and Case-Mix Characteristics by Valence (N = 60)

Characteristic	Positive (n = 21)	Mixed (n = 26)	Negative (n = 11)	None (n = 2)	Total
Age, M (SD)	36.74 (11.45)	35.54 (8.31)	36.55 (8.04)	28.00	34.21 (9.27)
Female, N (%)	15	23	11	2	51 (85.0)
Years as therapist, M (SD)	4.93 (5.21)	4.37 (2.86)	3.88 (2.43)	3.50 (2.12)	4.17 (3.16)
Client caseload, M (SD)	17.05 (4.44)	18.19 (10.01)	18.45 (5.75)	12.50 (6.36)	16.55 (6.64)
Race/Ethnicity, N (%)
Hispanic/Latino	12	17	7	0	36 (60.0)
Non-Hispanic White	4	4	2	1	11 (18.3)
Black	2	3	0	0	5 (8.3)
Asian/Pacific Islander	1	2	1	1	5 (8.3)
Multiracial	0	0	1	0	1 (1.7)
Education level, N (%)
Master’s degree	14	24	11	1	50 (83.3)
Doctoral degree	3	2	0	1	6 (10.0)
<Master’s degree	2	0	0	0	2 (3.3)
Primary discipline, N (%)
Marriage and family therapist	10	16	6	1	33 (55.0)
Social work	5	7	4	0	16 (26.7)
Psychologya	3	3	1	1	8 (13.3)
Other	1	0	0	0	1 (1.7)
Primary theoretical orientation, N (%)
Cognitive/Behavioral	11	17	5	2	35 (58.3)
Family systems	4	5	2	0	11 (18.3)
Eclectic	2	2	2	0	6 (10.0)
Psychodynamic	2	1	1	0	4 (6.7)
Humanistic	0	1	1	0	2 (3.3)
Status, N (%)
Staff	17	23	10	1	51 (85.0)
Trainee	2	3	1	1	7 (11.7)
Licensed	4	6	2	0	12 (20.0)
Language able to deliver services other than English, N (%)
Spanish	9	14	7	1	31 (51.7)
None	10	10	4	1	25 (41.7)
Other	0	2	0	0	2 (3.3)
Primary service setting, N (%)
Outpatient	16	15	9	2	42 (70.0)
School	2	6	1	0	9 (15.0)
Community, home, field-based	0	2	1	0	3 (5.0)
Inpatient	1	2	0	0	3 (5.0)
Group home	0	1	0	0	1 (1.7)
Other	1	0	0	0	1 (1.7)

^aIncludes counseling, clinical, and school psychology.

Measures

Qualitative interview.

The interview guide broadly focused on gauging therapists’ perceptions of and experiences with the six practices of interest and were intended to be completed within 45 min. Based on the guide, interviewers first asked broader questions, followed by more specific follow-up questions. The guide probed for therapists’ thoughts on the initial and ongoing training they received, their general perceptions of the practices in which they were trained, their comments relating to their use of each practice, and the types of adaptations that they made when delivering each practice. Focal to the current qualitative study, therapists were asked an open-ended question about the impact of PEI practices: “How has using PEI practices affected you as a therapist?”

Quantitative survey.

Questions regarding therapist characteristics were administered through an online survey and were adapted from previous work (Brookman-Frazee, Drahota, & Stadnick, 2012). Specifically, therapists were asked about demographic and professional characteristics (age, race/ethnicity, discipline, licensure, and years of clinical experience), caseload characteristics, organizational climate, opinions on and experiences with implementing the six practices, and perceptions of support for implementation within their agency. Therapists were also asked to report on which of the six practices they had received training, delivered, and delivered within the previous two months.

Therapist emotional exhaustion.

A subset of items from the Organizational Social Context Measure (OSC; Glisson, Green, & Williams, 2012; Glisson et al., 2008) was used to assess therapist perceptions of emotional exhaustion in the workplace. Five items from the Emotional Exhaustion subscale of the OSC were used (e.g., “I feel fatigued when I get up in the morning and have to face another day on the job.”). Therapists rated their agreement with each item on a 7-point Likert scale (0 = strongly disagree, 6 = strongly agree). In our sample, the measure had good internal consistency (α = .80).

Therapist openness toward EBPs.

Four items from the psychometrically validated Openness subscale of the Evidence-Based Practice Attitude Scale (EBPAS; Aarons, 2004; Aarons, Cafri, Lugo, & Sawitzky, 2012) was used to assess therapists’ openness to trying new research-based interventions (e.g., “I like to use new types of therapy/interventions to help my clients.”). Therapists rated their agreement with each item using a 5-point Likert scale (0 = not at all, 4 = very great extent). The subscale retained good internal consistency for our current sample (α = .81).

Therapist autonomy.

The Organizational Climate Measure (OCM) examines therapist perceptions of their organization’s policies, practices, and procedures (Patterson et al., 2005). For the current study, therapists employed at the agency completed the Autonomy subscale to describe their perceptions of independence in job-decision making and performance (e.g., “Leaders or agencies keep too tight a reign on the way things are done around here.”). Each item was rated on a four-point Likert scale ranging from 1–4 (1 = definitely false, 4 = definitely true). The subscale retained good internal consistency for the current sample (α = .73).

Data Management and Data Analysis Plan

A “coding, consensus, and comparison” methodology (Willms et al., 1990) that follows an iterative approach was used to analyze the qualitative interview data. All interviews were transcribed and uploaded using QSR International’s NVivo 11 Software. After reviewing a subset of interviews, two master coders (i.e., cotrainers) developed a codebook with eight broad a priori codes. Four transcripts were coded by the two master coders and served as “gold standards,” whereas the remaining 56 were divided among coders to be independently coded following the training phase. Interviews were then independently coded by the coding team of two postbaccalaureate research assistants and one doctoral student. Finally, to ensure consistency and to avoid coder drift, 50% of all independently coded transcripts were randomly selected to be reviewed by the master coders throughout the independent coding process. In instances in which a master coder felt that additional or different codes were needed, the reviewer met with the coder and the final codes for that transcript were revised following the consensus discussion. The codes and definitions were refined throughout this iterative process and resulted in nine broad final codes (practice, reach, perceptions of training, perceptions of practice, implementation supports, practice adaptations, de-adoption, impacts of PEI, and workload/burnout). Qualitative analyses ultimately identified eight distinctive types of impact on therapists (described below).

In addition to coding for impact types, the master coders and coauthors reviewed and categorized each therapist transcript according to one of three “valence” responses to categorize therapists’ perceived PEI impact. Valence was defined as the overall value depicted from therapist’s descriptions of all PEI impacts described. We classified valence into three categories: Positive valence, wherein a therapist’s overall impact of EBP implementation was favorably described; Negative valence, wherein a therapist’s overall impact was unfavorably described; and Mixed valence, wherein a therapist’s overall impact was described in both favorable and unfavorable ways (hereafter referred simply as Positive, Negative, and Mixed valence impact, respectively). A transcript-level code of Mixed valence indicated that a therapist made separate statements in distinct parts of the transcript pertaining to favorable and unfavorable impacts. This did not necessarily mean the described impacts were contradictory in nature or about the same topic, but simply that a therapist made both positive and negative comments about impact during their interview. Only responses to the therapist impact question were used for valence coding. Specifically, therapist impact valence was determined based on each therapist’s responses to questions pertaining to the impact of using EBPs, and not determined based on the overall tone of the entire interview, which was comprehensive and touched on a variety of topics. We utilized this categorization to differentiate emergent themes by valence group. We employed nonparametric tests to explore how therapist-level characteristics (age, years of experience, client caseload, burnout, EBPAS openness, OCM autonomy) may have differed by therapist impact valence using Kruskal-Wallis and chi-square tests for continuous and categorical tests, respectively. To minimize Type I error from conducting multiple paired tests, we adjusted alpha levels using the Holm-Bonferroni sequential method (Holm, 1979).

Results

Themes from the therapist interviews are described below with representative quotes for each of the three study aims. First, we characterized the valence of overall descriptions of the impact of EBP implementation on therapists and reported on differences in the valence of impacts by therapist characteristics (Aim 1). Then, we characterized the specific types of impacts that were described by therapists (Aim 2). Lastly, we contrasted the most common themes that arose as a function of overall therapist impact valence (Aim 3). Refer to Table 2 for a summary of themes.

Table 2.

Proportion of Therapist Sources by Therapist Experience Valence and Impact Theme Types (n, %)

		Therapist experience valence
Impact theme type	Exemplar quote	Positive (n = 21)	Mixed (n = 26)	Negative (n = 11)	Total
1. Increased confidence, self-efficacy, knowledge	“Because I’m mostly CBT, I think it just gave me a better understanding of it. And it made me feel more confident in doing it.” [Mixed]	14, 66.7%	11, 42.3%	0, 0%	25, 41.7%
2. Feels more organized and structured	“I think it makes you, I think it helps with structuring sessions, conceptualizing your case. That helps.” [Positive]	9, 42.9%	11, 42.3%	0, 0%	20, 33.3%
3. Feels more positive about EBPs	“I think my first impressions were maybe, it’s just another practice, we’ll see. Doing it, more interaction with the parents, seeing the progress that parents and kids were making made me realize actually this is a really good practice.” [Mixed]	2, 9.5%	6, 23.1%	0, 0%	8, 13.3%
4. Feels more marketable	“It’s added more to my skills on a resume—If I go for a job I could say, ‘I’m trained in this.’” [Positive]	2, 9.5%	0, 0%	0, 0%	2, 3.3%
Unfavorable
5. Feels distressed	“… so sometimes I’m in my own crisis, so—because these cases are hard. I would have trauma narratives like five in a row and I would want to cry, because it was just too intense.” [Mixed]	0, 0%	11, 42.3%	4, 36.4%	15, 25%
6. Feels limited or forced to use an EBP	“We have to do it for every client. I feel like you’re trying to fit the client to whatever EBPs you were trained in. So, sometimes, that can be limiting. I do think it affects you on a daily basis because it’s ingrained in every—it’s been such a push to have an EBP for every client.” [Negative]	0, 0%	6, 23.1%	7, 63.6%	13, 21.7%
7. Has less time to do other things	“Just to keep up with them … there’s just so much to do that sometimes dashboards just get pushed back. It’s really difficult to stay on top of dashboards. So I fall behind a lot.” [Mixed]	0, 0%	8, 30.8%	3, 27.3%	11, 18.3%
8. Feels less able to build rapport with client	“The very first session, it’s not about rapport building, it’s right off the bat psycho-education, which kind of cuts against my grain. They really don’t give time for rapport, so that would be what I have an issue with.” [Mixed]	0, 0%	3, 11.5%	1, 9.0%	4, 6.7%

Aim 1. Characterizing the Valence of Therapist’s Experiences With EBP Implementation

Each interview was categorized as revealing either Positive, Negative, or Mixed impact of EBP implementation on the therapist. Out of the 60 total therapists, 21 (35%) were classified as Positive, 11 as Negative (18.3%), and 26 as Mixed (43.3%); two interviews (3.3%) were unclassifiable because of reports of no impact on the therapist. Overall, therapists categorized as describing positive impact (hereafter referred to as Positive therapists) described all impacts as being favorable. For example, “I think it’s helped me a lot to really focus in on what is effective for my clients. It helps to have that guideline and so I think it’s affected my self-confidence.” Therapists categorized as describing negative experiences (Negative therapists) detailed all specific impacts of EBP implementation as unfavorable. For example, “It’s restricting. I think all the EBPs give an illusion of—it’s complicated, people are complicated, therapy is complicated, and when you want to make a formula out of it, you’re going to fail … It can be restricting.” Therapists categorized as Mixed described both favorable and unfavorable impacts.

Nonparametric omnibus tests were employed to explore differences in therapist characteristics (i.e., age, race/ethnicity, discipline, licensure, years of clinical experience, client caseload, self-reported burnout, EBPAS openness, and OCM autonomy) by impact valence group. A Kruskal-Wallis test showed a marginally significant omnibus test for difference in therapist burnout score between impact valence groups, χ²(2) = 5.97, p = .051, with a mean burnout score being significantly lower for Positive therapists (M = 19.58, SD = 1.79) than for Negative therapists (M = 27.80, SD = 2.39), U = 43.5, p = .016, with no significant difference from Mixed valence therapists (M = 25.44, SD = 2.28). No other significant differences in therapist characteristics emerged by valence group.

Aim 2. Characterize Specific Therapist Impact Themes

Qualitative analyses of responses to the interview prompt revealed eight specific types of impact codes concerning EBP implementation on therapists. On average, therapists described 2.32 (SD = 1.27, Range = 1–6) distinct types of impact of EBP implementation. Furthermore, each impact type was clearly classifiable as either favorable or unfavorable. These themes are described below. Nine therapist comments about the impact of EBP implementation could not be classified into one of the eight themes, largely because of the vague or general nature of the comments (e.g., “I’m not sure if it does or not, I guess because I don’t really think about, when I’m doing it, like how one is influencing the other. I don’t know if it influences my delivery, to be honest.”).

Favorable Impacts Themes

Four types of favorable impacts emerged within the therapist descriptions of their experience with EBP implementation: (a) Increased confidence, self-efficacy, and knowledge (25 therapists, 41.7%), wherein therapists reported experiencing boosts in confidence with clinical skills and mastery in EBP-related knowledge; (b) More organized and structured (20 therapists, 33.3%), wherein therapists reported feeling more organized and prepared for delivering care to clients and in the overall management of their cases; (c) More positive about EBPs (8 therapists, 13.3%), wherein therapists discussed improved attitudes toward EBPs; and (d) Feels more marketable (two therapists, 3.3%), wherein therapists reported feeling a competitive advantage in the mental health workforce or in their agency as a result of their training or refined specialization. The two most frequently discussed favorable impact themes are discussed in detail next.

Increased EBP confidence, self-efficacy, and knowledge.

Almost half of the Mixed valence therapists and two thirds of Positive therapists (66.7%) described increased knowledge and confidence as a function of their experiences with EBPs. Therapists specifically discussed experiencing improved knowledge and skills in case conceptualization and treatment strategies as a result of EBP training and exposure. One therapist described becoming “knowledgeable in trying to figure out what to do with the child. Having all these interventions with MAP– like where is this destructive behavior, how to model for the caregiver, doing behavior charts with them, implementing other kinds of goals for them. I think I’ve grown a lot as a therapist.” Many also described becoming a “better therapist” after developing EBP knowledge and skills: “I do feel like as a therapist I’m well-rounded in that I can work with various clients and presenting issues and on different populations. I think it’s helped me grow.” Some therapists even highlighted how their EBP expertise has become a part of their professional identity and has generalized to influence their practice as clinicians more generally over time: “It becomes part of who you are. We are CPP therapists, and it’s just integrated and I’m sure it plays out in whatever kind of therapy I am doing.”

In addition to growth in knowledge, many therapists also reported improved self-confidence from implementing EBPs. One therapist stated, “PEI and EBPs actually makes me feel more confident in what I’m doing, because I’m able to talk about it and justify why I’m doing it,” and further explained that training in EBPs helped to boost their credibility and invoke the research base for their interventions: “EBPs gives you language … and justify why I’m doing it, and what I’m doing it for … and this is what has been shown to work.” The structure and resources available for EBPs supported therapist self-efficacy in implementing the intervention while also reducing therapist distress, “When I started using MAP, it made me feel more confident because it gave me a structure and additional resources that I could use. Then I also feel like it’s really supportive, so I can go to somebody to get support, if I have a question, and most of the supervisors here are trained in that EBP.” One therapist described that having access to simplified EBP guidelines facilitated therapist confidence in delivering psychoeducation to caregivers: “It’s helped me be more confident whenever I talk with someone about depression and anger or anxiety. It would help me be more confident explaining to them—with parents—‘This is how active ignoring would help you,’ so be more confident in myself.” Some highlighted the benefits of EBP structure on confidence particularly for beginning therapists: “As a starting clinician, because I just recently graduated and this is my first community job, I feel it gave me like a platform to stand on. I felt like confident that I knew what I was doing. I was going into homes with training and knowing what to do.”

More organized and structured.

Almost half of Mixed (42.9%) and Positive therapists (42.9%) reported improvements in self-organization and structure of the care they provide clients as a result of EBP implementation. Specifically, therapists said it helped with “structuring sessions, conceptualizing your case,” especially “as a newer therapist … it really helped me to feel more prepared and kind of have a plan of action on what I was going to do each session.” Another therapist highlighted how the structure was especially helpful when juggling multiple responsibilities and having little time to think through a plan for all their sessions for each of the clients on their caseload “Okay, it’s already there. I can just go make a copy of this. I already know what’s next on the list of what we’re doing moving through this practice.” One therapist described using elements of PEI EBPs with her non-PEI clients in ways that generalize the structure and content supported by EBP training: “I find myself using some of the elements on some of my other clients, and it actually helps me stay on track. Even with documentation, note writing, I find that I go back and look into the elements and it helps me with my non-MAP clients, so it works out.” Other therapists described how EBP training and materials facilitated their focus in sessions on the core tasks of therapy: “It keeps me very focused whereas some parents will get you in there and they want to talk about the groceries they bought last week and the bills and the this … and I’ll do that for a few minutes and then I have my book in hand and I say, ‘Okay, now what about this? Let’s get to this.’”

Unfavorable Impact Themes

Four types of unfavorable impact themes emerged from therapist descriptions of their experience with EBP implementation: (a) Feels distressed (15 therapists, 25%), wherein therapists described feeling overwhelmed, stressed, or exhausted with the demands of training or implementation; (b) Feels limited or forced to use an EBP (13 therapists, 21.7%), wherein therapists described feeling that their practice is constrained by EBP requirements; (c) Has less time to do other things (11 therapists, 18.3%), wherein therapists discussed feeling that the range of activities involved in EBP implementation results in limited time to devote to their other vital tasks and functions; and (d) Feels less able to build rapport with client (four therapists, 6.7%), wherein therapists discussed feeling that delivering EBPs interfered with their ability to develop strong therapeutic relationships with clients and families. From the transcripts involving unfavorable impact themes, the two most frequent themes are discussed next in detail.

Therapist distress.

Twenty-five percent of all therapists described feeling distressed. The proportion of therapists discussing distress was greatest among Mixed therapists (42.3%) compared with Negative only therapists (36.4%). Distress themes were mentioned in relation to caseload characteristics specific to predominate trauma, non-English language delivery, and EBP cases.

Trauma caseload.

Therapists most notably commented on the psychological distress endured when predominantly treating trauma: “So sometimes I’m in my own crisis, so—because these cases are hard. I would have trauma narratives like five in a row and I would want to cry, because it was just too intense.” Not only was it challenging to treat the trauma directly experienced by child clients, but therapists also reported that caregivers’ psychological distress and multigenerational trauma was a source of stress for therapists: “And it’s a lot of emotions going on and trying to balance—for me what’s hard is trying to balance out that relationship because it’s not one client, it ends up being like two … There’s so much trauma in the past of the caregivers.” Another provider reported in particular, difficulties initiating trauma narratives when delivering trauma-focused EBPs: “I noticed for me it was hard—especially in the beginning—to push my clients into [trauma narrative]—So, I noticed that was my most challenging thing—is getting them to that point—and I’m the one that struggled. Maybe they were ready but I’m the one that struggled in the beginning.” Interestingly, this therapist noted that she may have struggled with the emotional weight of the trauma narrative more than her clients earlier in her implementation of trauma EBPs. Overall, most of the therapists who reported significant distress associated with EBP implementation referenced these impacts as associated with trauma-heavy caseloads.

Non-English language caseload.

Managing non–English-speaking caseloads resulted in major time constraints and general exhaustion for many therapists. These bilingual, predominately Spanish-speaking clinicians described the cumulative additional workload associated with carrying cases in which the client and/or the caregivers were predominately spoke a non-English language. Study therapists often described the translation of EBP materials a time-consuming aspect of delivering EBPs to Spanish-speaking monolingual families, as described by one therapist: “A lot of us were taking our own time and we were going in and we were translating these materials and that was challenging, in itself, because therapeutic language is not what you would use in a session with the people that we’re working with—their education level—and with anybody, really.” Another therapist described the cognitive load and general exhaustion that results from “live” translation of material in-session, especially when also trying to ensure that families have an understanding of concepts that transcends cultural barriers: “Sometimes it’s hard when you’re translating in Spanish and you’re like, ‘How do I explain this in Spanish?’ Spanish is my first language, but sometimes it’s hard to translate what’s going on for the kid, and the cultural understanding.”

EBP-specific demand.

Therapists discussed feeling distress when starting new cases for which they would be delivering an EBPs well after their initial training in a intervention: “Well, this is a kind of personal challenge, that I’ve opened a lot of cases at once, so I’m a bit overwhelmed in that sense. I just started, so I feel challenged in keeping up a little bit with that.” Others described the general exhaustion associated with opening new clients for EBP delivery: “I have been pretty tired I’ll tell you. And I’ve been coming home pretty exhausted, so I think ‘Okay, this is the beginning phase of opening all these cases.” Much of the exhaustion and distress pointed toward distress and burden from EBP-specific documentation requirements, not necessarily the demand of EBP delivery to clients within the session: “It’s the paperwork, and it’s also you want to also have time to reflect yourself. I want to sit down and bring out the forms and just kind of study them for a bit. So, that piece is something that gets lost when you’re trying to be productive with your clients when you have paperwork to do.”

Feels limited or forced to use an EBP.

Twenty-one percent of all therapists described feeling limited or forced to use an EBP as an impact of the PEI initiative, with 63.6% of Negative therapists and 23.1% of Mixed therapists talking about the forced nature of the implementation. Certain characteristics and requirements of EBPs were perceived as generating negative impacts by way of therapists’ sense of decreased autonomy within the workplace and with their clients. Specifically, the structure of EBPs was perceived as limiting therapist originality and as interfering with genuine relationship-building with their clients. One respondent described the EBP structure as hindering therapist creativity: “I feel that it’s limited me … I don’t like that—it makes me feel like I can’t be that creative.” Another described the impacts on rapport-building with clients and families: “But also … it’s been disheartening in the way that it [EBP requirements] exists within the DMH system. Just having something say, ‘You have to do it this way, this way, this way, and say this’ – there’s so much of therapy that exists within the relationship and doesn’t—TF-CBT, I feel leaves room for that, but something like Seeking Safety I don’t feel leaves as much room to be able to build that relationship. It’s more of a teacher-student sort of relationship.”

Aim 3: Contrast of Common Themes by Therapist Impact Valence

We computed the number of therapists that discussed each theme by the respective overall experience valence. As presented in Table 2, Mixed therapists generally commented more than their respective counter groups (Positive, Negative). Regarding favorable impacts reported by Positive versus Mixed therapists, the proportion of therapists describing the impact of experiencing increased confidence, self-efficacy, and knowledge was the highest for both groups. However, a higher proportion of Positive therapists discussed this favorable impact (14, 66.7%) compared with Mixed therapists (42.3%). Negative versus Mixed therapists differed in mentions of unfavorable impacts, such that the proportion of therapists feeling limited or forced to use EBPs was highest among Negative therapists (seven, 63.6%) compared to Mixed therapists (six, 23.1%). Less than half of Mixed therapists talked about feeling distressed (11, 42.3%), while about one third reported themes of distress in the Negative group (four, 36.4%).

Discussion

The current study examined therapists’ perceptions of overall impact of system-driven multiple EBP implementation. Overall, there were more therapists whose interviews revealed who described both positive and negative impacts, followed by those who exclusively described positive reflections on the personal or professional impact of EBP implementation, and therapists who described exclusively negative impacts. Aside from mean levels of self-reported burnout, therapist valence groups did not vary significantly in their demographic or background characteristics. Several favorable (e.g., increased EBP knowledge, structure) and unfavorable (e.g., distress, feeling constrained by EBPs) impact themes emerged across therapist valence groups. However, there were differences in proportions of therapists within valence groups that discussed each impact. These findings may help identify mechanisms that influence the way in which EBP implementation initiatives affect therapists, and thus inform implementation strategies and supports for the community therapist workforce.

First, it is notable that our sample of community mental health therapists was dispersed in how they experience and perceive the impacts of EBP implementation. Our findings showed that Positive therapists experienced significantly less burnout than Negative therapists, but similar levels to Mixed therapists. More research is needed to clarify the directional link between therapist emotional exhaustion and perceived impact of implementation. From previous research, though, we know that burnout is related to overall mental health problems (Ahola et al., 2005), negative staff attitudes toward mental health consumers (Holmqvist & Jeanneau, 2006), and negative attitudes toward EBPs (Kim et al., 2018), all important elements for shaping therapists’ overall self-perceptions of how they are impacted by their use of EBPs. This was apparent in our qualitative findings in that therapists in the Negative impact group were most affected by and reactive to the PEI requirements restricting their autonomy. This compared with therapist interviews that were Mixed or Positively valenced, which often described impacts pertaining to a specific EBP characteristic (e.g., more organized, associated time-burdens). It is notable that therapists in the Negative-valence group did not report the professional benefits discussed by the other groups. It is possible that Negative-valence interviews were with therapists who were unfavorably predisposed from the start and therefore did not perceive structure and organization as benefits of EBP adoption or perhaps the perceived detriments of EBP use overshadowed potential professional benefits. However, this was not probed within the interviews. The characteristic differences across therapists are instrumental in developing targeted workforce strategies.

In fact, therapists discussed a number of favorable and unfavorable impacts that can be leveraged to identify effects of EBP implementation that provide a rich environment in which therapists can thrive. The most commonly discussed favorable impact was improved confidence and knowledge, with two thirds of Positive therapists and almost half of Mixed therapists describing this theme. The fact that many Mixed therapists identified confidence/knowledge as a theme may indicate that improved confidence and mastery of EBP knowledge play a key role in cultivating a balanced perception on self-impacts for these therapists. It is also plausible that confidence/knowledge may buffer against less favorable impacts (e.g., emotional exhaustion). This is consistent with research demonstrating the potency of confidence and self-efficacy among employees in overcoming mental health stressors (e.g., Anthony, 1993; Davidson & Strauss, 1992), improving quality of life (Rosenfield, 1992), and facilitating their undertaking a more proactive approach when faced with stressful situations in the workplace (Jex, Bliese, Buzzell, & Primeau, 2001). It is clear that enhancing therapist confidence and self-efficacy in the implementation of EBPs is crucial, possibly through effective leadership in supervisory contexts, supportive training, and agency-level EBP champion leaders. Therapist confidence and self-efficacy can be improved through sustained and individualized training and consultation, booster training, and supervision (Lopez, Osterberg, Jensen-Doss, & Rae, 2011; Woody, Anderson, & D’Souza, 2015). Increased supported experience using specific EBPs is also associated with self-efficacy (Woody et al., 2015).

Perhaps not surprisingly, given the rapid rollout of multiple EBPs, the most commonly discussed unfavorable theme was workload and psychological distress linked to case characteristics: trauma, EBP-specific requirements, and non–English-speaking cases. Although distress is a well-documented concern for mental health providers (e.g., Garland, Kruse, & Aarons, 2003), it is possible that processes directly affecting case characteristics, such as case triage and assignment procedures, were influenced by PEI requirements to agencies and providers. Use of PEI funds were contingent on implementation of an identified EBP and corresponding client eligibility requirements (e.g., child/youth client age, problem area, treatment session restrictions). Previous findings from the larger parent study found that agency leaders reported challenges in balancing clinician availability, client eligibility, and availability and timing of EBP training resources (Regan et al., 2017). Indeed, casemix characteristics differed across therapists in the current study pool. Perhaps because of this context, these findings are valuable in informing future large-scale implementation efforts in mental health services. For instance, evaluation of implementation efforts or programs should account for casemix index or differences in therapist or organization caseload based on client characteristics (e.g., ethnicity/race, income, symptom presentation, severity, and complexity), so as to reduce biases in comparing program/agency effectiveness (Elliott et al., 2001; Ogles, Carlson, Hatfield, & Karpenko, 2008).

Therapists also described distress relating to therapist innate skill or expertise such as non-English language abilities. Primarily, the delivery of mental health service in a language other than English was described as burdensome for therapists. For example, therapists using a written protocol that is not available in the language or culture of the client is faced with the challenge of adapting the EBP material linguistically or culturally on the spot or outside of session time. These tasks are generally uncompensated, not billable, and unrecognized as a part of their productivity demands. Furthermore, when training, supervision, and manuals are only available in English, multilingual therapists were tasked with adapting the language for in-session delivery themselves. In addition to increasing the cognitive load on therapists to deliver EBPs with fidelity in another language, these modifications and translations need to be done either “live” during sessions with clients or outside of session time, tasks generally uncompensated and unrecognized as a part of their productivity demands. It is important to note that although many of the descriptions of non-English language delivery challenges were specific to Spanish language translation and services (not surprising given the large Latinx population served in LACDMH), this theme is pertinent and generalizable to other non-English languages for which EBP materials are not readily available.

Most notably, and consistent with previous findings (Mealer et al., 2012; Stamm, 2010), there is a clear need for implementation interventions that help diversify and balance therapist casemix to offset emotional burnout and risk of staff turnover. In fact, interventions focused on promoting resilience among therapists (measured as job satisfaction and resilient coping style) by creating supportive organizational climates (e.g., reflective supervision) and fostering a sense of autonomy have proved promising for enhancing well-being and reducing distress among staff (e.g., Beddoe, Davys, & Adamson, 2014; Clarke, 2011). Some ideas for fostering autonomy have been posed, such as implementing a system for integrating, and prioritizing, individualized professional goals and training plans for therapists that take into account professional interests while offering a sense of control (e.g., Rodriguez et al., 2018). Overall, it is clear that caseload assignments and triage should consider system requirements, agency needs, and therapist career development goals.

Previous research has established the importance of organizational climate on emotional exhaustion among therapists (Corrigan et al., 2004; Morse et al., 2012). In the present study, we identified similar patterns. In particular, Mixed and Negative therapists reacted alike to the impacts of the PEI initiative in that both discussed feeling limited by EBPs, highlighting their lack of autonomy within the agency or mental health system. Therapists’ description of a loss of autonomy is not surprising, given the well-documented, rapid nature of the multiple EBP rollout and the associated requirements (Reding et al., 2018). In the current sample, Mixed and Negative therapists, by virtue of overall valence, were more likely to feel restricted in decision-making and feedback, which is consistent with previous research showing that individuals prone to the experience of negative emotions respond less favorably to situations wherein they perceive external locus of control (Larsen & Ketelaar, 1991). This suggests that future EBP implementation initiatives should help therapists perceive autonomy by way of aligning therapist goals, past training experiences, and/or theoretical orientation with agency needs/requirements. There are long-term benefits to supporting therapists’ perceived fit in this way in terms of EBP adoption, fidelity, and sustainment outcomes (Aarons, Wells, Zagursky, Fettes, & Palinkas, 2009; Jensen-Doss, Hawley, Lopez, & Osterberg, 2009; Nelson & Steele, 2007; Reding, Chorpita, Lau, & Innes-Gomberg, 2014). Organizational decisions concerning staffing EBP implementation efforts may have more long-term sustained success if individual providers feel engaged in the process of identifying EBP training opportunities that match career and professional goals (Rodriguez et al., 2018).

Limitations

Some study limitations must be considered. First, therapists were asked to reflect on their experiences several years following the initial EBP implementation rollout, and as a result, therapists reported on its impact retrospectively and not at the outset of the system transformation. Therapists’ responses are thus influenced by their agency’s current practices surrounding EBP implementation and are not intended to capture their initial impressions at the start of the implementation. In fact, many therapists were not working in the system at the beginning of the reform. As such, the impacts described concern the lasting changes that occurred and remained through this sustainment phase as EBP implementation became part of routine practice. Previous reports have documented the considerable strain and acute impacts of the launch of the initiative (e.g., Regan et al., 2017; Reding et al., 2018). Examining perceived impacts during the sustainment phase may provide distinctive information after the tumultuous reform phase. It could be helpful for future studies to capture changes in perceived impacts of large EBP initiatives over time. Second, the semistructured interview protocol included multiple questions intended to capture a variety of challenges and facilitators of EBP implementation and sustainment. Only one question queried about the impact of the implementation initiative on the therapists, the interview guide did not include follow-up probes to interrogate potential impacts in particular areas, as this was not the initial intent of the interview. Targeted follow-up questions around this topic could have yielded additional information and examples supporting each theme. Additionally, our coding of therapist valence was specific to responses on the impact of the therapist, and not based on the overall valence of each therapist’s interview. Broadening the scope of the valence coding could have yielded a different pattern of themes by valence type. Moreover, a large percentage of our therapist interviewees were unlicensed mental health providers. Though not yet licensed, almost all therapist participants had obtained a Master’s or Doctoral degree and were staff (not trainees) at their respective clinics. This is consistent with the background characteristics often seen in community mental health agencies, and thus a strength of the present study. However, in contexts with a different workforce make-up, our findings may not generalize. It is possible that unlicensed providers are impacted differently compared to their more seasoned licensed counterparts in how they are affected by the expectations associated with a multiple EBP scale-up. Therefore, future research with a more diverse range of community therapists is needed to better understand the impacts of such large initiatives on our mental health workforce. Finally, it is possible that factors pertaining to community mental health care and county system as well as interpersonal relations within agencies may have had profound impacts on therapists. However, as our study focused on therapist-level factors, we are limited to drawing conclusions strictly on our current data and not on system-level factors.

Conclusion

The implementation of multiple EBPs within mental health service systems is likely to impact community mental health therapists in favorable and unfavorable ways. This is attributable in part to the characteristics of therapist caseloads that often include complex client presentations of clinical severity, poverty, and comorbidity. Although previous studies have investigated impacts following the rollout of a single EBP, the current study provides a rich, qualitative examination of both favorable and unfavorable impacts on therapists following a system-driven implementation of multiple EBPs for child community mental health services. Our findings provide preliminary evidence for favorable therapist impacts that would be beneficial to bolster and unfavorable impacts that future implementation efforts should work to offset.

Public Significance Statement.

The present study found that community mental health therapists were impacted by the implementation of evidence-based practices in multiple ways, and that in particular, levels of therapist burnout related to how clinicians perceived the impacts. Therapists reported positive impacts of improved knowledge and skill and feeling more structured, as well as less favorable impacts such as feeling more distressed and constrained due to demands of multiple interventions.

Biography

Adriana Rodriguez received her PhD in clinical psychology from Virginia Commonwealth University. She is currently a senior clinical training manager with the Division of Population Behavioral Health, Department of Mental Health-Prevention Center of Excellence, Jane and Terry Semel Institute for Neuroscience and Human Behavior, University of California, Los Angeles. Her areas of professional interest include implementation science, research-community partnerships, mixed-method approaches, and trauma and resilience informed systems of care.

Joanna J. Kim received her PhD in clinical psychology at the University of California, Los Angeles. She is currently a postdoctoral fellow at the REACH Institute, Arizona State University. She is interested in identifying factors that contribute to and counteract disparities in mental health care access, utilization, and delivery for ethnic minority and immigrant youth. She is also interested in family processes that may influence child risk and resilience across diverse cultural contexts.

Chanel Zhan received her BA in chemistry and psychology from Williams College. She is currently a medical student at Duke University School of Medicine. Her professional interests include child and adolescent psychiatry, immigrant and ethnic minority mental health, and global mental health.

Anna S. Lau received her PhD in clinical psychology from the University of California, Los Angeles. She is a professor of psychology at the University of California, Los Angeles. Her research spans the areas of disparities in children’s mental health services, cultural variation in risk and protective factors for child psychopathology, and community implementation of evidence-based treatments (EBTs) for immigrant and ethnic minority families.

Alison B. Hamilton received her PhD in anthropology from the University of California, Los Angeles (UCLA). She is currently a professor in Residence in the Department of Psychiatry and Biobehavioral Sciences at UCLA, and Chief Officer of Implementation & Policy at the VA Center for the Study of Healthcare Innovation, Implementation, & Policy at the VA Greater Los Angeles Healthcare System. Her areas of professional interest include women’s health, mental health, and implementation science.

Lawrence A. Palinkas received his PhD in anthropology from the University of California, San Diego. He is currently the Albert G. and Frances Lomas Feldman Professor of social policy and health in the Suzanne Dworak Peck School of Social Work at the University of Southern California. His areas of professional interest include implementation science, mental health services, and global health.

Resham Gellatly received her MA in psychology from the University of California, Los Angeles and is currently pursuing her last year of doctoral training in clinical psychology. Her research focuses on increasing access to mental health services for underserved youth and families through the dissemination and implementation of evidence-based practices. She is especially interested in working with community stakeholders to understand how adaptations can address issues of cultural fit for ethnically diverse populations.

Lauren Brookman-Frazee received her PhD in counseling, clinical, and school psychology from the University of California, Santa Barbara. She is currently a Professor in the Department of Psychiatry, University of California, San Diego, Associate Director of the Child and Adolescent Services Research Center, Co-Director of the San Diego State University/University of California, San Diego (SDSU/UC San Diego) Joint Doctoral Program in Clinical Psychology, and Director of Education and Training at the UC San Diego Dissemination and Implementation Science Center. Her research involves partnering with mental health and education system leaders, providers, and families to develop, test, and implement scalable psychosocial interventions in community service settings.