Abstract
Objective
Many youth receiving community mental health treatment do not receive evidence-based interventions. Research suggests that community mental health therapists use a broad range of therapeutic techniques at low intensities. The present study examined the relationship between therapist- and client-level predictors on community-based therapists’ report of cognitive, behavioral, psychodynamic, and family techniques within the context of implementation efforts.
Methods
One hundred thirty therapists from 23 organizations in an urban publicly funded behavioral health system implementing evidence-based practices participated. Therapist-level predictors included age, gender, clinical experience, licensure status, and participation in evidence-based practice initiatives. Child-level predictors included therapist-reported child primary disorder (i.e., externalizing, internalizing, or other) and child age. Therapists completed the Therapist Procedures Checklist- Family Revised, a self-report measure of therapeutic techniques used.
Results
Unlicensed therapists were more likely to report use of both psychodynamic and behavioral techniques. Therapists who did not participate in an evidence-based practice initiative were less likely to report use of cognitive techniques. Those with externalizing clients were more likely to report use of behavioral and family techniques. Therapists with the youngest clients (aged 3-7) were most likely to report use of behavioral techniques and less likely to report use of cognitive and psychodynamic techniques.
Conclusions
Results suggest that both therapist and client factors predict self-reported use of therapy techniques. Participating in an evidence-based practice initiative increased report of cognitive techniques. Therapists reported using more behavioral and family techniques for youth with externalizing disorders and fewer cognitive and psychodynamic techniques with young clients.
Keywords: evidence-based practice, community mental health, cognitive-behavioral therapy, usual care, treatment processes
The past fifty years have resulted in a proliferation of evidence-based practices (EBPs) for psychiatric disorders for a range of client populations and presenting problems (1, 2). Despite the development and validation of these treatments, most youth do not have access to EBPs in their communities (3, 4). While the need to translate EBPs from research to practice settings has been identified (5), progress has been slow (6, 7) and hampered by a lack of understanding of community-based usual care (8). The limited research to date examining usual care has focused on characterizing the types of treatment techniques used by community providers (9), that is the procedures employed during the intervention process (e.g., role play, relaxation training; 10). While this has helped elucidate the landscape of usual care, little is known about variables that predict the use of specific therapeutic techniques delivered by community therapists.
Studies focused on usual care of disruptive behavior disorders have demonstrated that community therapists self-report using a wide-range of techniques, both evidence-based and not (11). Observational studies have shown that community therapists generally deliver a broad range of techniques at relatively low intensities (12, 13) in the treatment of externalizing youth. Studies that have examined therapist self-report of usual care in youth receiving treatment for internalizing disorders have corroborated the usual care literature on externalizing disorders (e.g., 14, 15). Observational studies by both McLeod & Weisz (16) and Southam-Gerow and colleagues (17) have demonstrated that community therapists also utilize a variety of techniques in their treatment of youth with internalizing disorders, generally favoring client-centered approaches. Weisz and colleagues (18) found that therapists delivering usual care to youth with depression were observed to utilize more family and psychodynamic techniques when compared to cognitive-behavioral therapy (CBT) techniques.
Increasing the use of EBPs in usual care is an important objective for mental health services research. The literature suggests that children receiving community-based usual care do not clinically improve at the same magnitude as EBPs (19-21). In direct comparisons EBPs generally outperform usual care. For example, in a series of meta-analyses by Weisz and colleagues, differences in effect sizes between usual care and EBPs were approximately .29 (Cohen’s d; 21, 22). Youth receiving usual care in publicly-funded systems have worse treatment trajectories than youth in managed-care systems (20). Taken together, these data suggest considerable room for improvement in usual community-care practices.
To date, few studies have examined contextual predictors of use of evidence-based therapy techniques in usual care. Therapist (e.g., knowledge, attitudes) and client (e.g., presenting disorder) variables may be important predictors of whether a particular therapist uses evidence-based techniques in usual care settings (23, 24). In one recent study, therapist factors (i.e., knowledge, attitudes) were found to be more predictive of use of psychodynamic techniques but organizational factors (i.e., culture and climate) were found to be more predictive of use of cognitive-behavioral and family techniques (25). A limitation of the literature to date is that client factors have been largely ignored as predictors of use of therapy techniques. Understanding both therapist and client factors that predict use of therapy techniques is important because it allows for the identification of factors which can be targeted to change therapist behavior.
The present study allows for exploration of predictors of therapist self-report of techniques that fall into cognitive, behavioral, psychodynamic, and family domains within the context of a system-wide effort to improve usual care through implementing EBPs (see 26). The community mental health therapists in the present sample provide care to youth in a large, urban, publicly-funded system, which supports generalizability. Additionally, we add to the literature by utilizing providers in a system implementing multiple EBPs which provides a unique context to examine usual care within. Specifically, in the publicly-funded mental health system in Philadelphia, therapists are receiving training and consultation in a number of EBPs including cognitive therapy (27), trauma-focused CBT (28), prolonged exposure (29), and dialectical behavior-therapy (30). This study examines whether community-based therapists in this system report using cognitive, behavioral, psychodynamic, and family techniques and if therapist- or client-level factors predict use of these techniques. Given the limited data on predictors of usual care practices, this study was exploratory in nature and no a prior hypotheses were made.
Method
Setting
The Department of Behavioral Health and Intellectual disAbility Services (DBHIDS) in the City of Philadelphia (26) has engaged in pilot CBT implementation initiatives in the public mental health system since 2007. Implementation efforts are supported by a full-time city employee who coordinates the initiative. The training and ongoing consultation provided to therapists closely follows the recommendations of the respective treatment developers and includes post-training consultation. For example, outpatient providers enrolled in the cognitive therapy training participated in 22 hours of didactic workshops followed by six months of weekly group consultation (31). Other initiatives have similarly intensive procedures.
Participants
Purposive sampling was used to recruit the 29 largest agencies within the 100+ community mental health agencies in Philadelphia that provide outpatient services to youth (personal communication, Community Behavioral Health, 2012). These agencies together serve approximately 80% of Philadelphia youth receiving publicly-funded mental health care. Of these 29 agencies, 18 (62%) agreed to participate. An additional agency involved in EBP efforts approached us to participate, resulting in a final sample of 19 agencies representing 23 sites, 130 therapists, 36 supervisors, and 22 executive administrators. There were no exclusion criteria for therapist participation and approximately 60% of therapists from the 23 sites participated. Of participating therapists, 123 (95%) completed all measures on one occasion. Of the agencies enrolled, 16 had participated in DBHIDS sponsored EBP initiatives (mean years of participation = 2.90 ± 2.70); of the therapists enrolled, 54 (41%) had participated in DBHIDS sponsored EBP initiatives. Table 1 provides demographic information about therapists and clients.
Table 1.
Therapist and client demographics
| Variable | Therapist | Client | ||
|---|---|---|---|---|
|
| ||||
| n | % | n | % | |
| Age (M ± SD) | 38.09 ± 11.63 |
10.86 ± 3.70 |
||
| Years at current agency (M ± SD) | 3.35 ± 4.65 |
-- | ||
| Therapist years clinical experience (M ± SD) | 6.89 ± 6.84 |
-- | ||
| Current caseload (M ± SD) | 28.79 ± 22.05 |
-- | ||
| Level of professional burnout (M ± SD) | 4.23 ± 2.58 |
-- | ||
| Hours of supervision received each week (M ± SD) | 1.32 ± 1.21 |
-- | ||
| Gender* | ||||
| Male | 30 | 23% | 69 | 53% |
| Female | 99 | 76% | 56 | 43% |
| Transgender | 1 | 1% | 0 | 0% |
| Hispanic/Latino* | ||||
| Yes | 26 | 20% | -- | -- |
| No | 98 | 75% | -- | -- |
| Ethnicity* | ||||
| Asian | 6 | 5% | 1 | 1% |
| Black or African American | 27 | 21% | 54 | 42% |
| White | 67 | 52% | 12 | 9% |
| Hispanic/Latino | 13 | 10% | 33 | 25% |
| Multiracial | 5 | 4% | 7 | 5% |
| Other | 5 | 4% | 7 | 5% |
| Academic Background* | ||||
| Bachelor’s Degree | 5 | 4% | -- | -- |
| Master’s Degree | 107 | 82% | -- | -- |
| Doctoral Degree | 12 | 9% | -- | -- |
| Professional Background | ||||
| Master’s level counselors | 75 | 58% | ||
| Social workers | 20 | 15% | ||
| Marriage and Family Therapists | 18 | 14% | ||
| Psychologist | 6 | 5% | ||
| Psychiatrist | 1 | <1% | ||
| Other | 4 | 3% | ||
| Licensure status | ||||
| Yes | 33 | 25% | -- | -- |
| Yes | 97 | 75% | -- | -- |
| Participation in EBP Initiatives | ||||
| Yes | 54 | 41% | -- | -- |
| No | 78 | 59% | -- | -- |
| Primary theoretical orientation:* | ||||
| Psychodynamic | 10 | 8% | -- | -- |
| Behavioral | 6 | 5% | -- | -- |
| Cognitive | 5 | 4% | -- | -- |
| Cognitive-Behavioral | 50 | 39% | -- | -- |
| Systemic | 20 | 15% | -- | -- |
| Object Relations | 3 | 2% | -- | -- |
| Other | 4 | 3% | -- | -- |
| Eclectic | 26 | 20% | -- | -- |
| Primary Diagnosis* | ||||
| Internalizing Disorders | -- | -- | 33 | 27% |
| Posttraumatic Stress Disorder | -- | -- | 13 | 11% |
| Major Depressive Disorder | -- | -- | 8 | 7% |
| Anxiety Disorder NOS | -- | -- | 6 | 5% |
| Generalized Anxiety Disorder | -- | -- | 2 | 2% |
| Depressive Disorder NOS | -- | -- | 2 | 2% |
| Dysthymic Disorder | -- | -- | 1 | 1% |
| Adjustment Disorder with Anxiety | -- | -- | 1 | 1% |
| Externalizing Disorders | -- | -- | 71 | 58% |
| Attention-Deficit/Hyperactivity | -- | -- | 47 | 38% |
| Disorder (ADHD) | ||||
| Oppositional Defiant Disorder (ODD) | -- | -- | 7 | 6% |
| co-primary ADHD and ODD | -- | -- | 7 | 6% |
| Conduct Disorder | -- | -- | 6 | 5% |
| Disruptive Behavior Disorder NOS | -- | -- | 4 | 3% |
| Other Disorders | -- | -- | 19 | 15% |
| Adjustment Disorder – Unspecified | -- | -- | 12 | 10% |
| Adjustment Disorder with Mixed Disturbance | -- | -- | 2 | 2% |
| of Emotions and Conduct | ||||
| Reactive Attachment Disorder | -- | -- | 1 | <1% |
| Autistic Disorder | -- | -- | 1 | <1% |
| Asperger’s Disorder | -- | -- | 1 | <1% |
| Psychotic Disorder NOS | -- | -- | 1 | <1% |
| Unspecified substance use disorder | -- | -- | 1 | <1% |
Does not add up to 100% because of missing responses
Procedure
All procedures were approved by and conducted in compliance with the appropriate Institutional Review Boards. See Beidas et al. (25) for a complete description of the procedures of the larger study within which these data were collected. Participants provided informed consent and were compensated $50 for participation. The principles outlined in the Declaration of Helsinki were followed. Data were collected between March 1st and July 25th, 2013. Therapists were informed that their data would be kept confidential, that only aggregated and de-identified information would be shared with agency leadership and DBHIDS.
Measures
Predictor Variables
Therapist characteristics
Therapist demographics were assessed using the Therapist Background Questionnaire [TBQ; 32], a 21-item questionnaire that obtains information on personal characteristics such as age, gender and licensure status. Additionally, therapists indicated whether they had participated in any of the four DBHIDS EBP initiatives (cognitive therapy, trauma-focused CBT, prolonged exposure, and dialectical behavior therapy) on the Evidence-Based Practices Training Survey. We asked specifically whether they had received training and a year of consultation to ensure that they formally participated in the city-sponsored initiative.
Client characteristics
Therapists identified a representative client, defined as a client of their choosing who was most like a typical client on their caseload, and reported on that client’s age, ethnicity, and primary diagnosis. Child primary diagnoses, when necessary, were coded into DSM-IV disorders (e.g., ‘Depression’ was coded as Major Depressive Disorder). To maximize power for statistical analyses, we subsequently coded diagnoses as internalizing, externalizing, and other as there is considerable overlap in the techniques that are evidence-based across the internalizing disorders, and similarly for externalizing disorders (e.g., 33). See Table 1. Therapists then reported on the therapeutic techniques utilized with the representative client they identified.
Dependent Variable
Therapy techniques
The Therapist Procedures Checklist-Family Revised [TPC-FR; 15, 34] is a 62-item self-report therapist checklist that assesses therapeutic techniques from the following modalities: cognitive, behavioral, family, and psychodynamic. The TPC-FR informed the development of the Therapy Process Observational Coding System (TPOCS; 16), a gold-standard observer-rated measure of therapy fidelity. The TPC-FR is a revised version of the TPC (34) developed to also query about family therapy techniques (15). Therapists selected a recent representative client about whom they reported, on one occasion, on which techniques they had used with that client over the course of therapy to date. Each subscale is a mean of the items that fit within that factor, measured on a continuum from 1 to 5 with 1 indicating rarely; 2, seldom; 3, sometimes; 4, often; and 5, most of the time. Higher scores indicate greater self-reported use of the set of techniques. Good internal consistency has been reported for the TPC-FR (15) and the factor structure has been confirmed (35). In our sample, subscale α’s ranged from .84-.94.
Data Analytic Plan
Four mixed-effects linear regression models examined the relationships of the set of variables with self-reported use of specific therapy techniques (i.e., cognitive, behavioral, psychodynamic, family), as measured by the TPC-FR. Mixed-effects models included random intercepts for organization to account for therapists being nested within organizations. Predictor variables included therapist predictors (age; gender = male or female; clinical experience; licensure status = yes or no; EBP initiative participation = yes or no) and child predictors (primary diagnosis = externalizing, internalizing, or other; age = early childhood, middle childhood or adolescence). These predictor variables were included because treatment recommendations vary somewhat for older vs. younger youth and for those with internalizing vs. externalizing disorders and therapists may have differentially participated in training opportunities in EBPs. Missing data were minimal (<10%). Series means were imputed for missing predictor variables. Analyses were conducted using PROC MIXED in SAS 9.0. Effect sizes (Cohen’s f2) were computed using the procedures outlined by Selya and colleagues (36): .02 represents a small effect, .15 a medium effect and .35 a large effect (37).
Results
Therapeutic Techniques Utilized
Specific therapeutic techniques reported were assessed via the TPC-FR (see Table 2 for model parameters and ICCs). Regarding therapist-level predictors, therapists who had participated in EBP initiatives were more likely to report use of cognitive techniques (f2 = .05) while therapists who were not licensed were more likely to report use of behavioral (f2 = .02) and psychodynamic techniques (f2 = .07). Regarding client-level predictors, therapists working with clients with a primary externalizing disorder were more likely to report use of behavioral (f2 = .13) and family (f2 = .02) techniques than therapists working with youth with primary internalizing diagnoses over the course of therapy. Therapists working with early childhood aged youth were more likely to report use of behavioral techniques (f2 = .08) than those working with adolescents; likewise, therapists working with adolescents were more likely to report use of cognitive (f2 = .05) and psychodynamic (f2 = .05) techniques than those working with youth in early childhood, and more likely to report use of psychodynamic techniques (f2 = .05) than those working with youth in middle childhood.
Table 2.
Mixed effects model predicting use of therapy techniques as measured by the Therapist Procedures Checklist-Family Revised (N = 127)
| Variable |
TPC-FR
Behavioral Techniques M±SD= 2.80 ± .88 |
TPC-FR
Cognitive Techniques M±SD= 3.64 ± .75 |
TPC-FR
Family Techniques M±SD= 3.48 ± .96 |
TPC-FR
Psychodynamic Techniques M±SD= 3.42 ± .66 |
|---|---|---|---|---|
| Random Effects | ||||
| Organizational variance | .11 | .09 | .19 | .04 |
| Residual variance | .43 | .41 | .68 | .36 |
| ICC | .20 | .18 | .22 | .1 |
| Fixed Effects | ||||
| Client Factors | ||||
| Client Primary Diagnosis | ||||
| Primary Externalizing Disorder |
.66* | .19 | .44* | .16 |
| Primary Other Disorder | .26 | −.03 | .48 | .08 |
| Primary Internalizing Disorder |
0 | 0 | 0 | 0 |
| Client Age | ||||
| Middle Childhood (8-12) | .22 | −.21 | −.05 | −.31* |
| Early Childhood (3-7) | .65* | −.52* | −.01 | −.36* |
| Adolescence (13-18) | 0 | 0 | 0 | 0 |
| Therapist Factors | ||||
| Therapist Age | .01 | .01 | <.01 | .01 |
| Therapist Gender | ||||
| Gender (male) | −.13 | −.03 | .35 | .15 |
| Gender (female) | 0 | 0 | 0 | 0 |
| Clinical Experience | .02 | −.01 | .03 | −.01 |
| Licensure Status | ||||
| Licensed (no) | .35* | <−.01 | .36 | .45* |
| Licensed (yes) | 0 | 0 | 0 | 0 |
| EBP Initiative Participation | ||||
| EBP Participation (no) | −.15 | −.28* | −.02 | −.07 |
| EBP Participation (yes) | 0 | 0 | 0 | 0 |
Note.
= p < .05. Each dependent variable is a mean of the items that fit within that factor, measured on a continuum from 1 to 5 with 1 indicating rarely; 2, seldom; 3, sometimes; 4, often; and 5, most of the time. Higher scores indicate greater self-reported use of the set of techniques. Client primary diagnosis was dummy coded to indicate whether the primary diagnosis was an Externalizing Disorder (e.g., Attention-Deficit/Hyperactivity Disorder, Oppositional Defiant Disorder, Conduct Disorder), an Internalizing Disorder (e.g., anxiety and depressive disorders, Posttraumatic Stress Disorder) or Other Disorder (i.e., a primary disorder that was not clearly internalizing or externalizing, such as Adjustment Disorder). Child age was dummy coded to indicate whether the target client was in early childhood (i.e., ages 3-7), middle childhood (i.e., ages 8-12) or adolescence (i.e., ages 13 and older). Clinical experience = years in a clinician role. EBP Initiative Participation = years therapist has participated in system-wide training initiatives in evidence based practice.
The structure of the analyses required the use of a reference group for dummy coded variables. Thus, report of techniques used could not be compared between externalizing disorders and other disorders in the above model. When the reference group was changed from primary internalizing disorder to primary other disorder, the results were largely unchanged. Therapists working with clients with a primary externalizing disorder were more likely to report use of behavioral techniques (B = .39, p = .04, f2 = .13) than therapists working with youth with primary other diagnoses over the course of therapy. A significant difference in reported use of family therapy (B = .08, p = .63) was not observed.
Discussion
This study is the first to examine the relationship between therapist- and client-level predictors on community-based therapists’ self-report of techniques that fall into cognitive, behavioral, psychodynamic, and family domains within the context of implementation efforts. Results indicate that both therapist- and client-level variables predicted self-reported use of therapy techniques. Overall, these findings are largely encouraging and suggest that therapists in this system are self-reporting use of evidence-based usual care practices, such as cognitive and behavioral techniques, in certain circumstances. This may be attributed to the system-wide emphasis on implementation of EBPs; indeed the finding that therapists participating in initiatives were more likely to report use of cognitive techniques strengthens this interpretation. This study provides preliminary support for the idea that community therapists can self-report ascribing evidence-based techniques to client needs.
Therapist factors, namely therapist licensure status and participation in EBP initiatives, predicted self-reported use of therapy techniques. Specifically, when compared to licensed providers, therapists who were unlicensed were more likely to report using behavioral and psychodynamic techniques. Given that we controlled for therapist age and years of experience, it is unlikely that these findings are driven by these factors. Unlicensed providers are supervised by licensed providers, so perhaps supervisors in this system are emphasizing behavioral and psychodynamic techniques with supervisees. Understanding the impact of supervisor support on use of therapy techniques is an important area for future inquiry (38). Therapists who had participated in EBP initiatives were more likely to report use of cognitive techniques. Given that there is a longstanding cognitive therapy training initiative in the system (31), this result is not surprising and suggests a positive impact of this EBP initiative on the system.
Importantly, client-level variables also predicted use of therapy techniques. Therapists working with youth with primary externalizing disorders were more likely to report use of behavioral and family techniques when compared to those working with youth with primary internalizing diagnoses. This is consistent with the evidence-base that suggests that at least some behavioral and family based techniques are more effective for youth with externalizing disorders (39, 40). Also of note, therapists were more likely to report use of cognitive and psychodynamic techniques with adolescents than with younger youth (i.e., in early childhood) and more likely to describe the use of behavioral techniques with younger youth compared to adolescents. Cognitive and psychodynamic techniques both involve insight (41) and therapists may be more comfortable using these techniques with older youth.
Although therapists reported using a range of evidence-based cognitive and behavioral therapy techniques, they also reported using non-evidence based techniques, particularly psychodynamic techniques. This is consistent with previous work showing that therapists tend to be eclectic in their approach to therapy and use a variety of therapy techniques (8, 42). Deimplementaion, or exnovation, refers to an organization divesting itself of a previously adopted innovation (43). This represents an area ripe for future research. As unlicensed providers report using psychodynamic techniques more frequently than licensed providers, unlicensed providers may be a particular group worth targeting for exnovation.
These findings are preliminary because we relied on therapist report for two important variables: use of therapy techniques and client presenting diagnosis. First, studies on the concordance of therapist self-report with observation are equivocal. For example, a study conducted by Hurlburt and colleagues (44) casts doubt upon concordance, but deeper examination of this paper suggests that the demand characteristics of the task differed for observers and therapists, calling into question the applicability of these findings to our work. Specifically, therapists reported on their behavior several days after the session, whereas observers scored every minute of treatment sessions. Furthermore, therapists were not trained in how to make ratings of the intensity of their behavior whereas coders had intensive training and detailed instructions on how to score intensity of behavior. On the other hand, a recent study where therapists, youth, caregivers, trained raters, and treatment experts rated therapist adherence to a substance abuse treatment protocol found high concordance between self-reported therapist ratings and trained raters (which showed high concordance with treatment expert ratings). In this study, the methodology was more similar to ours in that therapists were asked to reflect on the overall techniques used in treatment over a larger period of time (1 month; 45). Other studies corroborate this finding (46). Second, therapists reported on their client diagnoses and we did not independently verify these diagnoses using semi-structured interviews; concordance between therapist diagnosis and semi-structured interviews is low (47). However, for the particular question at hand, it is more important to capture the diagnosis which therapists report they are targeting in treatment, whether or not the diagnosis is consistent with research criteria. Therefore, if therapists are differentially implementing therapy techniques to client clinical presentation we would expect to detect that using the methodology employed herein.
The present study is strengthened by its large and diverse sample of community mental health therapists drawn from a major metropolitan area. Several additional limitations exist. Therapists did not provide information about client severity or comorbidity, though a recent study of children receiving services though the publicly-funded mental health system in Philadelphia suggests low levels of comorbidity as reported in Medicaid claims (i.e., < 5%; 48). Information was obtained about a single client on a therapist’s caseload and treatment duration was not reported. It is not known whether treatment duration is associated with amount of intervention techniques utilized by therapists, and future studies would benefit from examining this empirically. Therapists self-selected which clients to report on. Therapists could have been motivated to report on a client who they deemed to be a good candidate for EBP techniques; however we observed a wide range of presenting disorders and ages, suggesting this may not be a significant limitation. Finally, data regarding the TPC’s ability to predict therapist behavior or distinguish between therapists who use varying levels of specific techniques is lacking at present.
Conclusions
Results suggest that both therapist and client factors predict self-reported use of therapy techniques. Most importantly, participating in an EBP initiative increased self-reported use of cognitive techniques, suggesting that a system-wide effort to implement EBPs can result in individual therapist report of evidence-based technique use. Therapists self-reported differentially using techniques, reporting increased use of behavioral and family techniques with youth with externalizing disorders and less use of cognitive and psychodynamic techniques with young clients. Future studies would benefit from utilizing observational methods to ascertain the veracity of these findings. Additionally, studies examining therapists’ use of EBPs with clients with a range of presenting problems (14) and comorbidity are needed, as are those that examine the role of treatment length and sequencing of EBP techniques in community care.
Acknowledgements
Dr. X receives royalties from Oxford University Press and has served as a consultant for Kinark Child and Family Services. Dr. Y has received grant support from Ortho-McNeil Janssen and Forest Research Institute and has served as a consultant to AstraZeneca and Alkermes.
This manuscript was supported by National Institute of Mental Health grants to Dr. Z (MH103955) and Dr. X (MH099179). Additionally, the preparation of this article was supported in part by the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis; through an award from the National Institute of Mental Health (R25 MH080916) and Quality Enhancement Research Initiative (QUERI), Department of Veterans Affairs Contract, Veterans Health Administration, Office of Research & Development, Health Services Research & Development Service. Dr. X was an IRI fellow from 2012-2014. Drs. X, Y, & Z are fellows of the Leonard Davis Institute of Health Economics, University of [OMITTED FOR BLIND REVIEW].
Footnotes
Disclosures
None of the reported disclosures are related to implementation of evidence-based practices for youth in the City of [OMITTED FOR BLIND REVIEW]. The following authors have no disclosures to report: (Dr. Z; Dr. A, Dr. B; Dr. C; Dr. D).
References
- 1.Butler AC, Chapman JE, Forman EM, et al. The empirical status of cognitive-behavioral therapy: A review of meta-analyses. Clinical Psychology Review. 2006;26:17–31. doi: 10.1016/j.cpr.2005.07.003. [DOI] [PubMed] [Google Scholar]
- 2.Chorpita B, Daleiden EL, Ebesutani C, et al. Evidence-based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clinical Psychology-Science and Practice. 2011;18:154–72. [Google Scholar]
- 3.Final Report. U.S. Department of Health and Human Services; Rockville, MD: 2003. New Freedom Commission on Mental Health: Achieving the promise: Transforming mental health care in America. [Google Scholar]
- 4.U.S. Department of Health and Human Services . Mental health: A report of the Surgeon General. National Institute of Mental Health; Rockville, MD: 1999. [Google Scholar]
- 5.Institute of Medicine . Crossing the quality chasm: A new health system for the 21st Century. Author; Washington, DC: 2001. [PubMed] [Google Scholar]
- 6.Stewart RE, Chambless DL. Does psychotherapy research inform treatment decisions in private practice? Journal of Clinical Psychology. 2007;63:267–81. doi: 10.1002/jclp.20347. [DOI] [PubMed] [Google Scholar]
- 7.Weissman MM, Verdeli H, Gameroff MJ, et al. National survey of psychotherapy training in psychiatry, psychology, and social work. Archives of General Psychiatry. 2006;63:925–34. doi: 10.1001/archpsyc.63.8.925. [DOI] [PubMed] [Google Scholar]
- 8.Garland AF, Bickman L, Chorpita BF. Change what? Identifying quality improvement targets by investigating usual mental health care. Administration and Policy in Mental Health and Mental Health Services. 2010;37:15–26. doi: 10.1007/s10488-010-0279-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Garland AF, Haine-Schlagel R, Brookman-Frazee L, et al. Improving community-based mental health care for children: Translating knowledge into action. Administration and Policy in Mental Health and Mental Health Services. 2013;40:6–22. doi: 10.1007/s10488-012-0450-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Goldfried MR. Toward the delineation of therapeutic change principles. The American Psychologist. 1980;35:991–9. doi: 10.1037//0003-066x.35.11.991. [DOI] [PubMed] [Google Scholar]
- 11.Brookman-Frazee L, Garland AF, Taylor R, et al. Therapists’ attitudes towards psychotherapeutic strategies in community-based psychotherapy with children with disruptive behavior problems. Administration and Policy in Mental Health and Mental Health Services. 2009;36:1–12. doi: 10.1007/s10488-008-0195-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Garland AF, Brookman-Frazee L, Hurlburt MS, et al. Mental health care for children with disruptive behavior problems: A view inside therapists' offices. Psychiatric Services. 2010;61:788–95. doi: 10.1176/appi.ps.61.8.788. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Brookman-Frazee L, Haine RA, Baker-Ericzen M, et al. Factors associated with use of evidence-based practice strategies in usual care youth psychotherapy. Administration and Policy in Mental Health and Mental Health Services. 2010;37:254–69. doi: 10.1007/s10488-009-0244-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Weersing VR, Weisz JR. Community clinic treatment of depressed youth: Benchmarking usual care against CBT clinical trials. Journal of Consulting and Clinical Psychology. 2002;70:299–310. doi: 10.1037//0022-006x.70.2.299. [DOI] [PubMed] [Google Scholar]
- 15.Kolko DJ, Cohen JA, Mannarino AP, et al. Community treatment of child sexual abuse: A survey of practitioners in the National Child Traumatic Stress Network. Administration and Policy in Mental Health and Mental Health Services. 2009;36:37–49. doi: 10.1007/s10488-008-0180-0. [DOI] [PubMed] [Google Scholar]
- 16.McLeod BD, Weisz JR. The Therapy Process Observational Coding System for child psychotherapy strategies scale. Journal of Clinical Child and Adolescent Psychology. 2010;39:436–43. doi: 10.1080/15374411003691750. [DOI] [PubMed] [Google Scholar]
- 17.Southam-Gerow MA, Weisz JR, Chu BC, et al. Does CBT for youth anxiety outperform usual care in community clinics? An initial effectiveness test. Journal of the American Academy of Child and Adolescent Psychiatry. 2010;49:1043–52. doi: 10.1016/j.jaac.2010.06.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Weisz JR, Southam-Gerow MA, Gordis EB, et al. Cognitive-behavioral therapy versus usual clinical care for youth depression: an initial test of transportability to community clinics and clinicians. Journal of Consulting and Clinical Psychology. 2009;77:383–96. doi: 10.1037/a0013877. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Manteuffel B, Stephens RL, Sondheimer DL, et al. Characteristics, service experiences, and outcomes of transition-aged youth in systems of care: Programmatic and policy implications. The Journal of Behavioral Health Services & Research. 2008;35:469–87. doi: 10.1007/s11414-008-9130-6. [DOI] [PubMed] [Google Scholar]
- 20.Warren JS, Nelson PL, Mondragon SA, et al. Youth psychotherapy change trajectories and outcomes in usual care: Community mental health versus managed care settings. Journal of Consulting and Clinical Psychology. 2010;78:144–55. doi: 10.1037/a0018544. [DOI] [PubMed] [Google Scholar]
- 21.Weisz JR, Jensen-Doss A, Hawley KM. Evidence-based youth psychotherapies versus usual clinical care: A meta-analysis of direct comparisons. The American Psychologist. 2006;61:671–89. doi: 10.1037/0003-066X.61.7.671. [DOI] [PubMed] [Google Scholar]
- 22.Weisz JR, Kuppens S, Eckshtain D, et al. Performance of evidence-based youth psychotherapies compared with usual clinical care: A multilevel meta-analysis. JAMA Psychiatry. 2013;70:750–61. doi: 10.1001/jamapsychiatry.2013.1176. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Damschroder LJ, Aron DC, Keith RE, et al. Fostering implementation of health services research findings into practice: A consolodated framework for advancing implementation science. Implementation Science. 2009;4:1–15. doi: 10.1186/1748-5908-4-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services. 2011;38:4–23. doi: 10.1007/s10488-010-0327-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Beidas RS, Marcus S, Aarons G, et al. Individual and organizational factors related to community clinicians’ use of therapy techniques in a large public mental health system. JAMA Pediatrics. 2015;169:374–82. doi: 10.1001/jamapediatrics.2014.3736. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Beidas RS, Aarons G, Barg F, et al. Policy to implementation: Evidence-based practice in community mental health-Study protocol. Implementation Science. 2013;8:38. doi: 10.1186/1748-5908-8-38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Stirman SW, Buchhofer R, McLaulin JB, et al. Public-academic partnerships: The Beck Initiative: A partnership to implement cognitive therapy in a community behavioral health system. Psychiatric Services. 2009;60:1302–4. doi: 10.1176/appi.ps.60.10.1302. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Cohen JA, Deblinger E, Mannarino AP, et al. A multisite, randomized controlled trial for children with sexual abuse-related PTSD symptoms. Journal of the American Academy of Child and Adolescent Psychiatry. 2004;43:393–402. doi: 10.1097/00004583-200404000-00005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Foa EB, McLean CP, Capaldi S, et al. Prolonged exposure vs supportive counseling for sexual abuse-related PTSD in adolescent girls: A randomized clinical trial. The Journal of the American Medical Association. 2013;310:2650–7. doi: 10.1001/jama.2013.282829. [DOI] [PubMed] [Google Scholar]
- 30.Linehan MM, Comtois KA, Murray AM, et al. Two-year randomized controlled trial and follow-up of dialectical behavior therapy vs therapy by experts for suicidal behaviors and borderline personality disorder. Archives of General Psychiatry. 2006;63:757–66. doi: 10.1001/archpsyc.63.7.757. [DOI] [PubMed] [Google Scholar]
- 31.Creed TA, Stirman SW, Evans A, et al. A Model for Implementation of Cognitive Therapy in Community Mental Health: The Beck Initiative. The Behavior Therapist. 2014;37:56–64. [Google Scholar]
- 32.Weisz JR. Therapist Background Questionnaire. University of California; Los Angeles, CA: 1997. [Google Scholar]
- 33.Chorpita BF, Daleiden EL, Weisz JR. Identifying and selecting the common elements of evidence based interventions. A distillation and matching model. Mental Health Services Research. 2005;7:5–20. doi: 10.1007/s11020-005-1962-6. [DOI] [PubMed] [Google Scholar]
- 34.Weersing VR, Weisz JR, Donenberg GR. Development of the Therapy Procedures Checklist: A therapist-report measure of technique use in child and adolescent treatment. Journal of Clinical Child and Adolescent Psychology. 2002;31:168–80. doi: 10.1207/S15374424JCCP3102_03. [DOI] [PubMed] [Google Scholar]
- 35.Do M-C, Warnick E, Weersing VR, et al. Examination of the psychometric properties of the Therapy Procedures Checklist - Family Revised in Association for Behavioral and Cognitive Therapies (ABCT) National Harbor, MD: 2012. [Google Scholar]
- 36.Selya AS, Rose JS, Dierker LC, et al. A practical guide to calculating Cohen's f2, a measure of local effect size, from PROC MIXED. Frontiers in Psychology. 2012;3:1–6. doi: 10.3389/fpsyg.2012.00111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Cohen J. Statistical Power Analysis for the Behavioral Sciences. Erlbaum; Hilldale, NJ: 1988. [Google Scholar]
- 38.Dorsey S, Pullmann MD, Deblinger E, et al. Improving practice in community-based settings: a randomized trial of supervision – study protocol. Implementation Science. 8:89. doi: 10.1186/1748-5908-8-89. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Chronis AM, Jones HA, Raggi VL. Evidence-based psychosocial treatments for children and adolescents with attention-deficit/hyperactivity disorder. Clinical Psychology Review. 2006;26:486–502. doi: 10.1016/j.cpr.2006.01.002. [DOI] [PubMed] [Google Scholar]
- 40.Mattejat F. Evidence-based family therapy: which family-based interventions are empirically supported? Kindheit und Entwicklung. 142005:3–11. [Google Scholar]
- 41.Beck AT. Cognitive Therapy and the Emotional Disorders. Meridian; New York, NY: 1979. [Google Scholar]
- 42.Beidas RS, Mychailyszyn MP, Edmunds JM, et al. Training School Mental Health Providers to Deliver Cognitive-Behavioral Therapy. School Mental Heath. 2012;4:197–206. doi: 10.1007/s12310-012-9074-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Kimberly JR, Evanisko Organizational innovation: The influence of individual, organizational, and contextual factors on hospital adoption of technological and administrative innovations. Academy of Management Journal. 1981;24:689–713. [PubMed] [Google Scholar]
- 44.Hurlburt MS, Garland AF, Nguyen K, et al. Child and family therapy process: Concordance of therapist and observational perspectives. Administration and Policy in Mental Health and Mental Health Services. 2010;37:230–44. doi: 10.1007/s10488-009-0251-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Chapman JE, McCart MR, Letourneau EJ, et al. Comparison of youth, caregiver, therapist, trained, and treatment expert raters of therapist adherence to a substance abuse treatment protocol. Journal of Consulting and Clinical Psychology. 2013;81:674–80. doi: 10.1037/a0033021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Hogue A, Dauber S, Henderson CE, et al. Reliability of therapist self-report on treatment targets and focus in family-based intervention. Administration and Policy in Mental Health and Mental Health Services. 2014;41:697–705. doi: 10.1007/s10488-013-0520-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Lewczyk CM, Garland AF, Hurlburt MS, et al. Comparing DISC-IV and clinical diagnoses among youths receiving public mental health services. Journal of the American Academy of Child and Adolescent Psychiatry. 2003;42:349–56. doi: 10.1097/00004583-200303000-00016. [DOI] [PubMed] [Google Scholar]
- 48.Kang-Yi CD, Locke J, Marcus SC, et al. A comparison of school-based behavioral health service use and associated expenditures for children with autism with those of children with other disorders. Psychiatric Services. doi: 10.1176/appi.ps.201400505. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]