Therapists’ Perceived Competence in Trauma-Focused Cognitive Behavioral Therapy and Client Outcomes: Findings From a Community-Based Learning Collaborative

Abstract

This study examined therapists’ perceived competence in Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) and its association with youth treatment outcomes (posttraumatic stress and depression). Participants included 99 community therapists enrolled in a TF-CBT-focused Learning Collaborative (LC), along with one of their randomly selected TF-CBT training cases. Analyzed data included: 1) caregiver/youth-reported posttraumatic stress and depressive symptoms, pre- and post-treatment, and 2) therapist-perceived competence with TF-CBT components across treatment delivery. Youth- and caregiver-reports indicated large, significant pre- to post-treatment decreases in youth posttraumatic stress (ds = 1.10–1.30, ps < .001) and depressive symptoms (d = 1.01, p < .001). Higher therapist-perceived competence with TF-CBT predicted positive treatment responses for posttraumatic stress (ds = 0.38–0.39, ps = .03) and depression (d = 0.25), though only the former association was significant (ps = .03 vs. p = .15). Findings highlight the need to monitor and improve therapists’ competencies to enhance clinical outcomes for trauma-exposed youth and suggest that LCs may be an effective training/implementation model to help achieve those critical goals.

Early exposure to potentially traumatic events, specifically child maltreatment, has been linked to subsequent mental health difficulties, including increased rates of depression (Dunn et al., 2013; Salazar et al., 2011), anxiety (Harpur et al., 2015; Sperry & Widom, 2013), and posttraumatic stress disorder (PTSD; Dias et al., 2017; Messman-Moore & Bhuptani, 2017). While several mental health treatments have been developed to ameliorate these trauma-related symptoms (e.g., Carr, 2019; Dorsey et al., 2020; Dorsey et al., 2017; Gutermann et al., 2016; Leenarts et al., 2013), Trauma-Focused Cognitive Behavioral Therapy (TF-CBT; Cohen et al., 2016) has emerged as one of the most widely utilized (Cohen et al., 2004, 2005, 2011; Deblinger et al., 2006; Leenarts et al., 2013), with robust and long-lasting treatment effect sizes for youth posttraumatic stress (PTS; e.g., ds = 0.80–1.60; Goldbeck et al., 2016; Jensen et al., 2017; Salloum et al., 2016) and depressive symptoms (e.g., d = −0.78; see Lenz & Hollenbaugh, 2015). As a brief treatment (i.e., 8–25 sessions) for trauma-exposed youth, ages 3–18, and their non-offending caregiver(s), TF-CBT includes nine components (“PPRACTICE”): Psychoeducation on trauma exposure and traumatic stress reactions, Parenting skills, Relaxation strategies, Affect identification/modulation, Cognitive coping skills, Trauma narration, In-Vivo exposure of trauma-related cues, Conjoint sessions to share/process the traumatic event(s), and skills to Enhance future safety (see Cohen et al., 2016, for more details).

Dissemination and Implementation of TF-CBT

While continued dissemination of evidence-based practices (EBPs), such as TF-CBT, increases availability, targeted implementation strategies must also be tested to improve adoption and sustained use of these practices within community agencies. Implementation strategies, defined as “systematic intervention processes to adopt and integrate evidence-based health innovations into usual care” (Powell et al., 2012), can include singular (e.g., educational meetings, trainings), as well as multifaceted (i.e., two discrete strategies together such as training/consultation) and blended approaches (i.e., several strategies in a singular package).

Learning Collaboratives

The Learning Collaborative (LC) model, an adaptation of the Institute of Healthcare Improvement’s Breakthrough Series (Kilo, 1999) is a multifaceted implementation strategy to support EBP implementation, usually within single community agencies, by involving clinical teams, comprised of front-line service providers, supervisors, and senior leaders (Cohen & Mannarino, 2008; Ebert et al., 2012; Hanson et al., 2016; Markiewicz et al., 2006; Sigel et al., 2013). The LC model uses a sequential approach, first providing therapists with baseline EBP knowledge via readings and online courses, followed by in-depth training to enhance therapist skill development and introduce quality improvement strategies, and then ongoing support via expert consultation while therapists implement the EBP with training cases (Hanson et al., 2018). Iterations of this model exist, including the Community-Based Learning Collaborative described below (Hanson et al., 2019).

Community-Based Learning Collaborative (CBLC) Model and Outcomes

Project X is an ongoing statewide initiative, whose primary aim is widespread dissemination and implementation of trauma-focused EBPs across youth-serving agencies. A primary focus of Project X has been to train professionals working with children who have experienced child maltreatment. As part of Project X, Hanson and colleagues (2016) enhanced the original LC model to support community-wide implementation of TF-CBT. This augmented LC model, known as the Community-Based Learning Collaborative (CBLC), includes cross-discipline training of broker (i.e., individuals involved in the identification and referral, but not direct treatment, of child mental health needs) and mental health professionals across roles (i.e., frontline providers, supervisors, and senior leaders) from an array of service organizations (e.g., mental health, child welfare, juvenile justice, schools) within a targeted community. Project X’s CBLCs aim to equip service providers with the knowledge and skills to deliver TF-CBT, while also enhancing cross-discipline collaboration to meet the needs of youth/families with trauma-related problems, with an emphasis on children who have experienced abuse and/or neglect.

Prior research suggests that CBLC strategies may significantly increase use of trauma-informed practices for therapists and brokers (Hanson et al., 2019; Helseth et al., 2020), and improve perceptions of organizational support for trauma-focused EBPs, such as TF-CBT (Helseth et al., 2020). Moreover, evidence indicates these individual and organizational benefits are typically maintained, even at 2–4 year follow-up assessments (Helseth et al., 2020). The CBLC has also demonstrated good feasibility for community implementation, as it has been linked to high levels of interdisciplinary engagement, completion, and satisfaction (Hanson et al., 2019), as well as favorable cost-effectiveness data (Dopp et al., 2017). Finally, results indicate CBLCs may significantly, and to a large degree, increase interprofessional collaboration across participant roles (i.e., brokers, clinicians, and senior leaders; ${\mathrm{\eta }}_{\mathrm{p}}^2=.17$) and thereby reduce barriers to evidence-based youth trauma treatment $\left({\mathrm{\eta }}_{\mathrm{p}}^2=.25\right)$, potentially leading to significantly greater utilization of such treatment services (${\mathrm{\eta }}_{\mathrm{p}}^2=.17$; Hanson et al., 2018).

Therapist Challenges in Delivering TF-CBT

Despite these promising preliminary findings, research must still determine if therapists participating in these CBLCs achieve positive outcomes for their training cases and whether therapists demonstrate competence in treatment delivery. This focus on therapist competence stems from multiple studies indicating that higher competence improves clinical outcomes (Amaya-Jackson et al., 2018; Collyer et al., 2019; Kuyken & Tsivrikos, 2009; Trepka et al., 2004; Webb et al., 2010; Zarafonitis-Müller et al., 2014). Furthermore, therapists’ EBP-specific competence is affected by the training they receive (Beidas & Kendall, 2010), which is why it has been recommended that training initiatives include a period of coaching and/or consultation. Although these training strategies are designed to improve therapists’ skills, only one study to date examined whether TF-CBT competence among therapists enrolled in a LC is related to positive training case outcomes. Namely, Amaya-Jackson et al. (2018) found that therapist TF-CBT fidelity (i.e., adherence and competence delivering TF-CBT components) significantly moderated pre- to post-treatment improvements in youth-reported PTS symptoms, such that youth with higher rated therapists improved significantly more than youth receiving TF-CBT from lower rated therapists. Of note, this study combined adherence and competence, using an average score of trainer-ratings of clinician fidelity based on consultation calls and case notes. As such, the isolated impact of TF-CBT competence on youth outcomes remains unknown.

Prior research also indicates that, despite extensive training initiatives, continued difficulties arise in the consistent delivery of TF-CBT treatment components (Ascienzo et al., 2020). For example, a qualitative study with TF-CBT national trainers suggested that therapists do not consistently or competently deliver all TF-CBT components, especially explicit trauma components of the TF-CBT protocol (i.e., trauma narration and processing; hereafter called protocol-specific components), as compared to model-general components (e.g., relaxation, affect identification, cognitive coping; Hanson et al., 2014). These results are consistent with Allen and Johnson (2012) who also found that trauma narration, a protocol-specific component, was one of the least used components reported by therapists delivering TF-CBT. Of note, exposure-based practice elements are often challenging for therapists (Becker-Haimes et al., 2017; Olatunji et al., 2009), especially when their treatment perceived competence is low (Woody et al., 2015a).

As such, the current study examined the relations between therapist competence in TF-CBT delivery and youth treatment response among therapists enrolled in a CBLC. Given prior research on the positive impact of therapist competence on treatment outcomes (Amaya-Jackson et al., 2018; Kuyken & Tsivrikos, 2009; Trepka et al., 2004), we hypothesized that higher therapist-reported competence in TF-CBT would be associated with positive child treatment response. Also, based on prior research (Allen & Johnson, 2012; Hanson et al., 2014; Woody et al., 2015b), perceived competence in delivery of protocol-specific versus model-general components–and their unique relations to treatment response–also were explored.

Method

Context and Procedures

The current study used prospective project evaluation and quality improvement data from Project X, a statewide initiative, currently in its fifth phase, which uses CBLCs to disseminate child trauma-focused EBPs (e.g., TF-CBT) across state agencies. For each of Project X’s CBLCs, data were collected on several outcomes to examine the CBLC as an implementation package (e.g., completion rates of the CBLC, participant-rated utility of CBLC components), individual provider factors relating to EBP use (e.g., attitudes towards EBPs), TF-CBT training outcomes (e.g., knowledge, skills, utilization of the EBP), organizational climate (e.g., barriers to child trauma treatment, support for EBPs) and interprofessional collaboration (e.g., working together to overcome barriers, ensuring clients complete treatment, coordinating services). Data from therapists’ training cases were also collected as part of role-specific CBLC completion requirements (see Hanson et al., 2019). For each training case, therapists provided weekly information on use of the TF-CBT components, their perceived competence with those components, and caregiver participation in treatment. Youth and caregivers completed standardized measures on the youth’s mental health symptoms before and after treatment. The current study included pre- and post-treatment caregiver- and youth-report measures, as well as weekly therapist data regarding their training cases. Data were drawn from Phase 3 of Project X, the project’s largest completed phase which, for therapists, focused on training in TF-CBT to promote sustained delivery with fidelity (see Hanson et al., [2019] for more details on Project X and its phases).

Participants

Data were collected from a total of 338 therapists participating in one of the six, Phase 3 CBLCs from 52 agencies across the state. Of those, 224 (66%) completed all of the CBLC training requirements (i.e., pre-work readings, online TF-CBTweb course; two Learning Sessions, 12 or more consultation calls, and two or more completed TF-CBT cases). The most commonly cited reason for discontinuing CBLC participation was change of employment (44%). The number of participating therapists per agency ranged 1–25, with an average of five (M = 5.3, SD = 6.7). Therapists were predominantly employed by the Department of Mental Health (80%), with fewer therapists employed by child advocacy centers (7%), private practice settings (2%), or other clinical settings (11%). Given that data were collected for program evaluation and quality improvement, demographic information was not collected from participants.

Consistent with CBLC requirements, therapists completed at least two cases utilizing the full TF-CBT protocol (M = 4.1, SD = 0.8; range: 2–5). Data were collected across all therapist cases for the larger study; however, for purposes of this study, one case per therapist was randomly selected via a random number generator (www.random.org) to reduce potential bias resulting from unequal training cases across therapists. The present study utilized three inclusion criteria. First, only those children who met clinical criteria for PTSD at pre-treatment (per youth- or caregiver-report) were included in order to measure a clinically meaningful treatment response to TF-CBT. Second, only youth with complete pre- and post-treatment assessments were included given the study’s focus on the impact of perceived competence on youth treatment response. Third, analyses were restricted to youth, ages 8–18 years, in accordance with validated age ranges for the standardized assessment measures (see below). Based upon those inclusion criteria and selection procedures, the present study analyzed data from 99 youth enrolled in TF-CBT by therapists from one of six Phase 3 CBLCs of Project X. Youth ranged 8–18 years old (M = 13.0, SD = 2.9) and primarily identified as female (56%). Most youth identified racially as White (49%) or Black (39%). Therapists administered pre- and post-treatment clinical assessment measures (see below) to youth and their caregivers and then shared the de-identified measures with the program coordinator, per CBLC protocol. Pre- and post-assessments were completed on average within 150 days of each other (SD = 46.1, range: 39–230).

Measures

Posttraumatic stress symptoms.

The Child PTSD Symptom Scale (CPSS; Foa et al., 2001) is a 24-item measure with youth- and caregiver-report versions that assess the severity of posttraumatic stress among youth ages 8–18. The first 17 items examine type and frequency of PTS (mapping onto DSM-IV criteria for PTSD) using a 3-point scale (1 = “Not At All,” 3 = “Almost Always”) with total scores ranging 0–51, and a clinical cut-off of 15. The CPSS’ other seven items, not counted in the overall score, measure the degree of functional impairment these symptoms cause in various psychosocial domains (0 = absent, 1 = present). The CPSS has shown good internal consistency (α = .89), test-retest reliability (r = .84), and convergent validity (Gillihan et al., 2013).

Depressive symptoms.

The Short Mood and Feelings Questionnaire–Short Version (SMFQ; Angold et al., 1995) assesses depressive symptoms in youth ages 7–18. The SMFQ has child- and caregiver-report versions that evaluate the frequency of 13 depressive symptoms in the past 2 weeks. Depressive symptoms are rated on a 3-point scale (0 = “Not True,” 2 = “True”). Clinical significance is based on the youth’s self-report (total score ≥ 8) or the sum of youth- and caregiver-reports (combined total score ≥ 12). Since caregivers are often unaware of a youth’s internalizing symptoms, caregiver-report is not utilized alone to determine clinical significance. Thus, the SMFQ combined score was utilized for the present study. The SMFQ has demonstrated good reliability in previous studies (Cheng et al., 2009; Rhew et al., 2010; Sharp et al., 2006).

Therapist-reported competence.

For each training case, therapists completed a weekly online survey to assess three treatment aspects: 1) attendance (i.e., whether the youth and/or caregiver participated that week), 2) use of component(s) (i.e., which TF-CBT PPRACTICE components were utilized during sessions that week), and 3) perceived competency in delivering each used component. Therapists self-rated competency on a 5-point scale (0 = “Less than adequate skill,” 4 = “Expert skill”). The survey was modeled after the TF-CBT Practice Checklist–Self Report (Deblinger et al., 2005).

For the present study, average competency scores were calculated for each PRACTICE treatment component, as well as a total average score across all components per case. Internal consistency of overall competence scores was excellent (α = .98). Average competency ratings were then derived for two domains: (1) protocol-specific competence (i.e., competency with explicit trauma exposure components that differentiate TF-CBT from other CBT-based and trauma-focused treatments; i.e. trauma narrative development and processing, in vivo exposures, and conjoint sharing; Dorsey et al., 2017) and (2) model-general competence (i.e., competency with the remaining components that are common elements of CBT-based protocols; psychoeducation, parenting, relaxation, affective regulation, cognitive coping, and enhancing safety; McLeod et al., 2019). Internal consistency of the two derived scales were both excellent (protocol-specific: α = .95; model-general: α = .98). To better compare the relative effect of these competency domains upon treatment response, competency domain scores were standardized by dividing raw sums by the number of TF-CBT components per domain.

Data Analytic Strategy

All analyses were conducted in IBM SPSS Statistics, Version 25. Descriptive statistics and paired samples t-tests were computed to examine hypothesized decreases in PTS and depressive symptoms from pre- to post-treatment. To assess treatment response to TF-CBT, McNemar’s tests were conducted to assess for significant pre- to post-treatment changes, as per CPSS and SMFQ scores (i.e., 0 = “below clinical cutoff” or 1 = “above clinical cutoff”). Operationally, PTS treatment responders were defined as clients with a pre-treatment CPSS score above the clinical cutoff, but a post-treatment CPSS score below the clinical cutoff, as per the same informant. Conversely, PTS treatment non-responders were operationally defined as clients with CPSS scores in the clinical range at both pre- and post-treatment assessments, according to the same informant. Similar procedures were followed for defining depression treatment responders based on the SMFQ combined score (i.e., depression treatment responders were those with a post-treatment score below the clinical cut-off; non-responders were defined as those with a pre- and post-treatment score above the clinical cut-off). Analyses were run for each clinical measure (i.e., CPSS and SMFQ), and twice specifically for CPSS: one based on youth self-report, the other based on caregiver-report. Chi-square and independent t tests assessed whether treatment responders and non-responders differed significantly on demographic (i.e., youth age, gender, race) and clinical variables (i.e., length of treatment, pre-treatment CPSS and/or SMFQ scores, and therapist-perceived competence).

Additionally, descriptive statistics (means, standard deviations) were conducted to examine therapist-reported competence with the overall model, individual components, and protocol-specific versus model-general competence. Binary logistic regression analyses were conducted to assess the relation between overall therapist-reported TF-CBT competence and youth treatment response. Follow-up logistic regression analyses also were conducted, one for each treatment outcome variable (i.e., PTS and depressive treatment response) and each competence domain predictor (i.e., protocol-specific and model-general).

Results

Pretreatment Symptoms and Diagnostic Status

Given the study’s inclusion criteria, all youth had clinically significant PTS scores at pre-treatment by either youth- or caregiver-report. However, youth at pre-treatment tended to have clinical PTS per both youth-report (86.9%, M = 26.2, SD = 9.6, range: 6–50) and caregiver-report on the CPSS (73.5%, M = 22.0, SD = 10.5, range: 0–47). Also, most youth (73.7%) had clinical levels of pre-treatment depressive symptoms, per combined youth- and caregiver-report scores on the SMFQ (M = 21.8, SD = 11.9, range: 1–50). Based on these operational criteria, most youth (69.4%) had comorbid clinical PTS and depressive symptoms, pre-treatment.

Pre- to Post-Treatment Changes in Symptoms and Diagnostic Status

Pre- to post-treatment decrease in scores was statistically significant across informants for PTS (youth-report: t[98] = 12.90, p < .001, d = 1.30; caregiver-report: t[97] = 10.88, p < .001, d = 1.10) and depressive symptoms (t[97] = 9.97, p < .001, d = 1.01). Moreover, the percentage of youth with clinical symptom levels significantly decreased, pre-to post-treatment, for both PTS (youth-report: 86.9% to 41.4%, χ²[1] = 41.19, p < .001; caregiver-report: 73.5% to 26.5%, χ²[1] = 38.94, p < .001) and depression (73.7% to 35.4%, χ²[1] = 26.33, p <.001). Most youth at post-treatment no longer had clinical PTS per youth-report (58.6%, M = 13.2, SD = 9.1, range: 0–41) or caregiver-report (73.5% M = 10.4, SD = 8.1, range:0–35) or depressive symptoms (64.6%, M = 9.2, SD = 7.7, range: 0–35), and most (70.4%) no longer met operational criteria for clinically significant PTS and depressive symptoms (see Table 1 for more details).

Table 1.

Pre- and Post- treatment Child Mental Health Assessment Scores for TF-CBT Training Cases.

Descriptor	Pretreatment			Posttreatment
	CPSS youth	CPSS caregiver	SMFQ	CPSS youth	CPSS caregiver	SMFQ
M	26.2	22.0	21.8	13.18	10.4	9.7
SD	9.7	10.5	11.9	9.1	8.1	7.8
Range	6–50	0–47	1–50	0–41	0–35	0–35
% Clinical	86.9%	73.5%	73.7%	41.4%	26.5%	35.4%

	Pre- to Post-treatment Change
	t	df	p	d
CPSS youth	12.90	98	<.001	1.30
CPSS caregiver	10.88	97	<.001	1.10
SMFQ	9.97	97	<.001	1.01

Note. CPSS represents the Child PTSD Symptom Scale by youth- and caregiver-report, separately. SMFQ represents the Mood and Feelings Questionnaire-Short Version, combined caregiver and child scores. TF-CBT = Trauma-Focused Cognitive Behavioral Therapy. % Clinical represents the percentage of youth that met clinical criteria on the measure at each time point (e.g., pre- or post-treatment).

In terms of treatment response for PTS, 46.5% and 50% of the sample were treatment responders per youth- and caregiver-reported CPSS scores, respectively. For both youth- and caregiver-reported PTS treatment response, youth age, gender, and race; treatment length; and pre-treatment child-reported PTS symptoms did not significant differently between treatment responders and non-responders (ps = .11–.96). However, PTS treatment responders typically had significantly higher pre-treatment caregiver-reported PTS symptoms (M = 25.3, SD = 6.9) and pre-treatment depressive symptoms (M = 24.4, SD = 9.2) than did non-responders (caregiver-reported PTS: M = 18.6, SD = 12.3, t[75.1] = −3.33, p = .001; depression: M = 19.2, SD = 13.5; t[84] = 2.24, p = .03), to a moderate degree (ds = 0.67, and 0.45, respectively).

For depressive symptoms, 61.6% of youth exhibited a treatment response on the SMFQ combined score. Depression treatment responders and non-responders did not significantly differ on youth age or gender; treatment length; or caregiver-reported pre-treatment PTS (ps = .11–.89). However, Black/African American versus White youth were more likely to be depression treatment non-responders than treatment responders (χ²[1] = 5.58, p = .02), to a small degree (ϕ = .29). Furthermore, depression treatment non-responders had significantly higher pre-treatment child-reported PTS (M = 31.5, SD = 9.5) and depressive symptoms (M = 31.1, SD = 11.2) compared to treatment non-responders (PTS: M = 26.3, SD = 8.7, t[71] = 2.42, p = .02, d = 0.58; depression: M = 23.9, SD = 7.4, t[41.6] = 3.04, p = .004, d = 0.80).

Therapist-Perceived Competence and Relation to Treatment Response

Overall, therapists typically reported a moderate amount of perceived competence across TF-CBT treatment delivery (M = 1.9, SD = 0.8, range: 0.5–3.9). Individually, each treatment component had a mean competency within the moderate range (Ms = 1.7–2.0), with In Vivo Mastery having the lowest perceived competence (M = 1.7, SD = 1.0); whereas, Conjoint Parent-Child Sessions and Enhancing Safety components had the highest mean competency ratings (Ms = 2.0, SDs = 0.9). On average, therapists reported significantly greater competence with TF-CBT’s model-general components (M = 1.9, SD = 0.8) than its protocol-specific components (M = 1.8, SD = 0.8; t[97] = −2.35, p = .02), with this difference being small in magnitude (d = 0.24).

Post-traumatic stress treatment response.

Results indicated that overall therapist perceived competence was significantly higher among PTS treatment responders (M = 2.1, SD = 0.8) compared to non-responders (M = 1.8, SD = 0.8), as defined by youth- and caregiver-report (youth-report: t[97] = −1.87, p = .03, d = 0.38; caregiver-report: t[96] = −1.93, p = .03, d = 0.39). Logistic regression results also revealed that higher competence predicted positive PTS treatment response across both reporters (youth: χ²[1] = 5.97, p = .06, Nagelkerke R² = .08, Cox & Snell R² = .06; B = .64, SE = .29, Wald = 4.91, p = .014, OR = 1.90; caregiver: χ²[1] = 13.67, p = .002, Nagelkerke R² = .17, Cox & Snell R² = .13; B = .50, SE = .29, Wald = 2.91, p = .04, OR = 1.65) when controlling for caregiver-reported PTS and depressive symptoms at pre-treatment. Depressive symptoms at pre-treatment did not significantly predict PTS treatment response by youth- or caregiver-report (ps = .12 and .47, respectively); however, higher caregiver-reported PTS at pre-treatment significant predicted positive caregiver-reported PTS treatment response, though only to a trivial degree (B = .07, SE = .03, Wald = 5.63, p = .01, OR = 1.07).

A similarly positive pattern emerged when examining PTS treatment response and different competence domains. According to therapists’ perceptions, protocol-specific competence was significantly higher for PTS treatment responders (M = 2.0, SD = 0.8) than for non-responders (M = 1.7, SD = 0.8; youth-report: t[96] = −2.06, p = .02, d = 0.42; caregiver-report: t[95] = −2.21, p = .02, d = 0.45). Logistic regression analyses indicated that higher protocol-specific perceived competence significantly predicted positive treatment response per both youth- and caregiver-reports of PTS (youth-report: χ²[3] = 7.14, p = .03, Nagelkerke R² = .10, Cox & Snell R² = .07, B = .67, SE = .28, Wald = 5.95, p = .008, OR = 1.96; caregiver-report: χ²[3] = 14.37, p = .001, Nagelkerke R² = .18, Cox & Snell R² = .14, B = .53, SE = .28, Wald = 3.48, p = .03, OR = 1.69) when controlling for caregiver-reported PTS and depressive symptoms at pre-treatment. Again, caregiver-reported PTS at baseline predicted PTS treatment response to a significant but trivial degree (B = .07, SE = .03, p = .01, OR = 1.07).

Finally, therapist-perceived model-general competence was significantly higher for PTS treatment responders (M = 2.1, SD = 0.8) than for non-responders (M = 1.8, SD = 0.8) across informants (youth-report: t[97] = −1.67, p = .048, d = 0.34; caregiver-report: t[96] = −1.82, p = .04, d = 0.37), to a small degree. Moreover, higher model-general competence still significantly predicted PTS treatment response when controlling for caregiver-reported PTS and depressive symptoms at pre-treatment (youth-report: B = .57, SE = .29, Wald = 4.04, p = .02, OR = 1.78; caregiver-report: χ²[3] = 13.48, p = .002, Nagelkerke R² = .17, Cox & Snell R² = .13, B = .48, SE = .29, Wald = 2.74, p = .05, OR = 1.62); however, the overall model for youth-reported treatment response was marginally significant (χ²[3] = 5.00, p = .09, Nagelkerke R² = .07, Cox & Snell R² = .05). Caregiver-reported pre-treatment PTS scores still predicted PTS treatment response to a significant but trivial degree (B = .07, SE = .03, Wald = 5.66, p = .01, OR = 1.07).

Depression treatment response.

Although perceived overall TF-CBT competence was higher for depression treatment responders (M = 2.1, SD = 0.7) than non-responders (M = 1.9, SD = 0.9) to a small degree (d = 0.25), this difference was non-significant (t[71] = −1.04, p = .15). Moreover, overall competence did not significantly predict depression treatment response (B = .37, SE = .37, Wald = .99, p = .16, OR = 1.45) when controlling for youth race, youth-reported PTS, and depressive symptoms pre-treatment. However, the overall model was significant (χ²[7] = 20.68, p = .002, Nagelkerke R² = .34, Cox & Snell R² = .25), with pre-treatment depressive symptoms exhibiting a significant effect on depression treatment nonresponse (B = −.07, SE = .04, Wald = 3.06, p = .04, OR = 0.94), such that higher depressive symptomology at pre-treatment was trivially associated with treatment non-response. Similarly, there was a non-significant difference in protocol-specific competence between depression treatment responders (M = 2.0, SD = 0.8) and non-responders (M = 1.9, SD = 0.9; t[70] = −.57, p = .29, d = 0.14), and protocol-specific competence did not significantly predict treatment response, per the SMFQ (B = .30, SE = .35, Wald = .74, p = .20, OR = 1.36). Again, the overall regression model was significant (χ²[7] = 19.67, p = .003, Nagelkerke R² = .32, Cox & Snell R² = .24), with higher depressive symptoms at pre-treatment significantly but trivially predicting treatment non-response (B = −.07, SE = .04, Wald = 3.19, p = .04, OR = 0.94). Finally, model-general competence did not significantly differ between depression treatment responders (M = 2.1, SD = 0.7) and non-responders (M = 1.9, SD = 0.9; t[71] = −1.16, p = .13, d = 0.28). Despite the overall logistic model being significant (χ²[7] = 20.80, p = .002, Nagelkerke R² = .34, Cox & Snell R² = .25), model-general competence again did not significantly predict depression treatment response (B = .40, SE = .38, Wald. = 1.10, p = .15, OR = 1.49), and higher SMFQ scores at pre-treatment significantly but trivially predicted depression treatment non-response (B = −.07, SE = .03, Wald = 2.96, p = .04, OR = 0.94).

Discussion

While dissemination and implementation of trauma-focused interventions, such as TF-CBT, have increased over recent decades (Bunger et al., 2016; Cohen & Mannarino, 2008; Hanson et al., 2016), limited research has examined the clinical outcomes of multi-faceted implementation strategies, such as the LC and CBLC models, and even less research has established the role of therapist competence on youth and families receiving a targeted EBP, TF-CBT or otherwise (Ebert et al., 2012; Hanson et al., 2018, 2019; Helseth et al., 2020; Powell et al., 2012). Thus, it remains necessary to examine the potential impact these training/implementation strategies have on the therapists, agencies, communities, and families that they are designed to serve. The present study aimed to: (1) present the first findings on clinical outcomes during a CBLC and (2) assess the relation between therapist-reported competence in TF-CBT and treatment outcomes. Given the consistent challenges in training therapists to competently and confidently deliver the trauma-explicit components of TF-CBT (Hanson et al., 2014; Woody, et al., 2015a), therapist-perceived competence was explored for: 1) the overall treatment model, 2) protocol-specific competence, and 3) model-general competence.

CBLC Treatment Outcomes

Study findings indicated that CBLC training cases, overall, had large, statistically significant reductions in posttraumatic stress (PTS; ds = 1.10–1.30) and depressive symptoms (d = 1.01). These outcomes are consistent with Lenz and Hollenbaugh’s (2015) meta-analytic estimates of TF-CBT’s impact on PTS (g = −1.48, 95% CI [−2.13, −0.83]) and depressive symptoms (g = −0.78, 95% CI [−1.41, −0.15]). These findings are particularly notable since they occurred in community, versus university research settings, with therapists new to TF-CBT treating “real world” cases–factors all associated with poorer clinical outcomes, particularly for a specific EBP implementation (e.g., Beidas & Kendall, 2010; McLeod et al., 2019). Indeed, a recent community benchmarking study of TF-CBT outcomes by Rudd et al. (2019) reported a small effect on PTS (d = −0.34), which was significantly smaller than efficacy and effectiveness trial comparisons (i.e., Cohen et al., 2004; Jensen et al., 2014). Although we did not perform formal benchmarking, the studied CBLCs had PTS results with a larger effect than Rudd et al.’s (2019) and appear equal if not superior to most TF-CBT efficacy and effectiveness trials. One potential reason for these larger PTS improvements is that we restricted analyses to youth with clinically significant PTS at pre-treatment. Although it is uncertain the extent to which this inclusion criterion impacted our results, it is noteworthy that our sample’s average pre-treatment CPSS scores across caregiver-(M = 22.0, SD = 10.5) and youth-report (M = 26.2, SD = 9.6) were well within one standard deviation of Rudd et al.’s (2019) (M = 23.8, SD = 11). A second possible explanation is that the CBLCs’ unique components may have improved treatment outcomes. For example, increased interprofessional collaboration and organizational support, important CBLC targets, have been associated with sustained EBP implementation (Aarons et al., 2012; Bai, Wells & Hillemeier, 2009; Nadeem et al., 2016; Palinkas et al., 2012). For this study, CBLC-related improvements in participating brokers’ referral, coordination, and related child-trauma broker practices (e.g., Hanson et al., 2019; Helseth et al., 2020) may have bolstered community clinicians’ TF-CBT implementation and related client outcomes.

The present results are doubly encouraging given that the TF-CBT community cases had not only notable pre-treatment rates of comorbidity but also large, significant decreases in multiple symptomatic domains (i.e., PTS and depression). In contrast, a 50-year meta-analysis of youth psychotherapy outcomes found that most other “multi-problem” treatments tend to have trivial effects (g = −0.15, 95% CI [−0.43, −0.14]; Weisz et al., 2017). Thus, current results provide further evidence that TF-CBT is an effective treatment for youth with clinically significant PTS (Fraser et al., 2013; Jensen et al., 2017) and depression (Cohen et al., 2004; Jensen et al., 2017; Leenarts et al., 2013; Lenz & Hollenbaugh, 2015), and provide the first evidence that TF-CBT can be delivered effectively by community therapists in the context of a CBLC.

Despite this substantial treatment response, a significant portion of youth still had clinical PTS (i.e., 27% per caregiver-report, 40% per youth-report) and depressive symptoms (33%) at post-treatment. Such findings are similar to prior studies indicating that 18%–56% of children continue to exhibit clinically significant PTS following TF-CBT completion (Cohen et al., 2004; Goldbeck et al., 2016; Jensen et al., 2014; Webb et al., 2014). Consistent with extant literature exploring trauma treatment responses across racial groups, Black/African American and White youth were equally likely to have a positive treatment response for PTS (Cohen et al., 2004; Weiner et al., 2009). However, Black youth were significantly less likely to have a positive treatment response for depressive symptoms, to a small degree. These results highlight a need for future research to examine underlying mediators and moderators, including ethnic and racial factors, of treatment response to help minimize the number of children still experiencing significant trauma symptoms at treatment completion.

Therapist-Perceived Competence

In contrast to client demographic or other clinical factors (e.g., youth gender, race, and treatment duration), therapist-perceived competence with TF-CBT emerged as a consistent, significant predictor of PTS treatment response. Namely, therapist-perceived competence was higher for treatment responders than non-responders, and their perceived competence in both protocol-specific and model-general components significantly predicted greater PTS treatment response. As with PTS response, greater competence in both protocol-specific and model-general components of TF-CBT were associated with positive depression treatment response. However, these associations were not significant–potentially due to the reduced sample size of youth with clinical pre-treatment depressive symptoms and the related loss in statistical power compared to the PTS analyses. Yet, regardless of statistical significance, overall, protocol-specific, and model-general competence universally had a smaller effect on depression versus PTS treatment response (i.e., ds = 0.14–0.28 versus ds = 0.34–0.45, respectively). These results indicate a need for future studies to examine mechanisms of depression response, notably with larger samples.

Our overall results are consistent with Amaya-Jackson et al. (2018) who found that therapists with higher TF-CBT fidelity had youth patients with better pre- to post-TF-CBT improvements in PTS. Furthermore, results of Amaya-Jackson et al. (2018) found no moderating effect of therapist fidelity on youth depression treatment response, caregiver distress, or child externalizing symptoms. Despite assessment differences in competence between our study (i.e., self-reported competence implementing specific components during a specific session) and Amaya-Jackson et al. (2018) (i.e., trainer-rated adherence and competence combined based on observed clinician skills, progress notes, and consultation calls), our results cumulatively suggest that greater clinician competence in TF-CBT may indeed predict PTS clinical improvements upon treatment completion, especially during LC initiatives. However, this fidelity may be less impactful for other clinical outcomes, such as depression.

Our results also indicated that therapists’ perceived competence with TF-CBT’s model-general components was significantly higher than with protocol-specific components, a finding consistent with prior studies (Becker-Haimes et al., 2017; Olatunji et al., 2009). Yet, in contrast with those studies, this disparity was small (d = 0.24), potentially because these CBLCs, like other recent TF-CBT training initiatives, included enhanced support for trauma narrative treatment components. Present results affirm this needed enhancement, since competence-PTS outcome associations were consistently larger for protocol-specific (ds = 0.42–0.45) versus model-general competence (ds = 0.34–0.37). In contrast, model-general competence had a larger effect on depression treatment response (d = 0.28) than protocol-specific competence did (d = 0.14)–though this reversal should be interpreted cautiously given the nonsignificance of competence-depression effects. Nevertheless, our results, in conjunction with prior research, highlight the important role of therapist competence with both general CBT components and TF-CBT’s protocol-specific, trauma-explicit components, particularly for PTS outcomes.

Limitations

Despite the present study’s notable strengths (e.g., prospective design, multi-informant assessment of multiple symptom domains, inclusion of community therapists with ecologically valid cases high in comorbidity), several limitations need to be noted. First, data were drawn from an existing training/implementation initiative focused on quality improvement versus research, which precluded randomly assigning child training cases to therapists or control conditions (e.g., therapists not participating in the CBLC). Thus, causal relations between CBLC participation, therapist-reported competence, and treatment outcomes cannot be definitively established. Furthermore, given that data were drawn from a statewide training/implementation initiative, therapists’ demographics, and type and number of youth trauma exposures were not routinely collected and thus not available for analysis. Since child trauma and maltreatment type and frequency often affect youth mental health outcomes (Finkelhor et al., 2007; Hodgdon et al., 2018; Spinazzola et al., 2014), future research should examine whether trauma type or number as well as therapist and client demographics moderate TF-CBT competence-outcome relations.

A second limitation centers on the use of a novel therapist self-report measure of TF-CBT competence. While this allowed for an understanding of therapists’ perceptions of their skills, no objective, psychometrically validated measure was used to determine whether and how perceived competence aligned (or misaligned) with objectively observed skills related to TF-CBT delivery. Also, treatment response was defined as change in PTS or depression from a clinically significant value at pre-treatment to a non-clinically significant score at post-treatment, which may limit our findings’ generalizability. Indeed, response to treatment may be defined in various ways, including reductions in symptoms across mental health domains, decreases in functional impairment, cessation of comorbidity, increases in psychosocial resilience, and improvements in quality of life. Thus, future research on fidelity-treatment response associations should consider a multidimensional view of possible positive treatment outcomes.

Relatedly, generalizability of our findings also may be limited due to our inclusion criteria (i.e., restricting analyzed cases to those with complete pre- and post-treatment data). Our focus was to examine treatment response to a full dose of TF-CBT and to inquire whether, or which, TF-CBT competencies may impact treatment response. Thus, we selected this method to yield the most statistical power to analyze potential differences in treatment outcomes, based on therapist self-reported competence for clients that had received all TF-CBT components (i.e., completed a full dose of treatment). Notwithstanding, our results may not generalize to cases that begin, but do not complete, TF-CBT, as the salience of therapist competencies, TF-CBT-specific or general, may differ depending on whether a therapist is working to prevent premature treatment termination versus ensure positive, but more distal, post-treatment outcomes.

Finally, we should note that for the purposes of this study, we randomly selected one training case for each therapist. It is possible that this method may have limited our ability to examine whether there was additional variability in therapists’ perceived competence in TF-CBT based on the particular training case selected. For example, therapists’ ratings may have been lower with initial versus subsequent training cases, or their ratings may have become more accurate with continued CBLC consultation and client post-treatment feedback. Despite these limitations, our study represents one of the first to have examined TF-CBT competence-outcome relations, as well as the first to present clinical outcomes during a CBLC, TF-CBT or otherwise.

Future Directions

Results of the present study extend prior findings from Project X (Dopp et al., 2017; Hanson et al., , 2016, 2018, 2019; Helseth et al., 2020) and provide additional support for CBLCs as an effective, multifaceted implementation package for EBPs, particularly for TF-CBT. Despite the increase in training and implementation research studies, barriers to EBP adoption still persist (Aarons et al., 2012; Helseth et al., 2020; Nelson & Steele, 2007). This study’s findings on the relation between therapist-perceived competence and treatment outcomes, among those participating in a CBLC, are promising and worthy of replication. Our findings provide a foundation for future research, such as studies examining ways to sustain fidelity, identifying CBLC strategies that improve both protocol-specific and general fidelity, and utilizing intent-to-treat and sensitivity analyses to more formally compare CBLC clinical outcomes to other TF-CBT efficacy and effectiveness benchmarks. Finally, additional research on implementation strategies and how they relate to therapist competence will elucidate how training efforts can best prepare therapists to use complex multi-component EBPs, like TF-CBT, with fidelity and flexibility to bolster their competence and ameliorate mental health symptoms.