Individual and organizational factors related to community clinicians’ use of therapy techniques in a large public mental health system

Abstract

Importance

Few studies have examined the effects of both clinician and organizational characteristics on the use of evidence-based practices in mental healthcare. Improved understanding of these factors could guide future implementation efforts to ensure effective adoption, implementation, and sustainment of evidence-based practices.

Objective

To estimate the relative contribution of clinician and organizational factors on clinician self-reported use of cognitive-behavioral, family, and psychodynamic techniques within the context of a large-scale effort to increase use of evidence-based practices in an urban public mental health system serving youth and families.

Design

Observational and cross-sectional. Data collected in 2013.

Setting

Twenty-three organizations.

Participants

We used purposive sampling to recruit the 29 largest child-serving agencies, which together serve approximately 80% of youth receiving publically funded mental health care. The final sample included 19 agencies with 23 sites, 130 therapists, 36 supervisors, and 22 executive administrators.

Main Outcome Measures

Clinician self-reported use of cognitive-behavioral, family, and psychodynamic techniques, as measured by the Therapist Procedures Checklist – Family Revised.

Results

Linear mixed-effects regression models were used; models included random intercepts for organization to account for nesting of clinicians within organization. Clinician factors accounted for the following percentage of the overall variation: cognitive-behavioral (16%), family (7%), psychodynamic (20%). Organizational factors accounted for the following percentage of the overall variation: cognitive-behavioral (23%), family (19%), psychodynamic (7%). Older clinicians and clinicians with more open attitudes were more likely to endorse use of cognitive behavioral techniques, as were those in organizations that had spent fewer years participating in evidence-based practice initiatives, had more resistant cultures, and had more functional climates. Female clinicians were more likely to endorse use of family techniques, as were those in organizations employing more fee-for-service staff and with more stressful climates. Clinicians with more divergent attitudes and less knowledge about evidence-based practices were more likely to use psychodynamic techniques.

Conclusions & Relevance

This study suggests that both clinician and organizational factors are important in explaining clinician behavior and the use of evidence-based practices, but that their relative importance varies by therapeutic technique.

Implementation science frameworks posit that both clinician (e.g., knowledge and attitudes) and organizational (e.g., culture and climate) characteristics affect the delivery of evidence-based practices (EBPs) in health and mental health.¹ Little is known about the relative contributions of these two sets of characteristics. This study estimates the relative contribution of clinician and organizational factors on clinicians’ use of cognitive-behavioral, family, and psychodynamic therapy techniques, within the context of a large-scale effort to increase the use of cognitive-behavioral therapy (CBT) in an urban public mental health system.

Literature supports the role of both clinician and organizational factors in the delivery of children’s mental health services. For example, clinician factors such as attitudes towards EBPs^2,3 predict the extent to which clinicians deliver EBPs as designed. Similarly, organizational factors such as organizational culture (i.e., shared employee perceptions around expectations and norms)⁴ and organizational climate (i.e., psychological impact of the work environment on individual well-being)⁵ have been linked to quality of services⁶ and youth mental health outcomes.^6,7 Previous research has largely focused on either clinician or organizational factors. Both sets of studies find evidence for the predictive validity of their constructs of interest. Clinician and organizational factors are correlated,⁸ making it difficult to disentangle the individual contribution of each set. Further, different outcomes have been examined in these two sets of studies, making it challenging to compare results.

To address these issues, we measured the relationship between clinician and organizational characteristics and use of therapy techniques in clinicians in an urban public mental health system engaged in a large-scale effort to increase the use of CBT. We explore the relative contribution of clinician and organizational characteristics on therapist self-reported use of cognitive-behavioral, family, and psychodynamic techniques, widely endorsed techniques by therapists in usual care.⁹ CBT has a large body of evidence to support its effectiveness in addressing youth psychiatric disorders.¹⁰ Family therapy has also been found to be effective for some youth psychiatric disorders, particularly when family-oriented components are combined with cognitive-behavioral techniques.¹¹ Psychodynamic therapy has little evidence to support its effectiveness for youth psychiatric disorders.¹⁰

Methods

Setting

Since 2007, the Philadelphia Department of Behavioral Health and Intellectual disAbility Services has supported the implementation of therapy techniques in the public mental health system based on the principles of CBT. Implementation support includes a full-time city employee who coordinates implementation, and training and consultation by treatment developers.

Agencies and Participants

There are over 100 community mental health agencies in Philadelphia that provide outpatient services to youth (Cathy Bolton, PhD, email communication, January 3^rd, 2013). We used purposive sampling to recruit the 29 largest child-serving agencies, which serve approximately 80% of youth receiving publically funded mental health care. Of these 29 agencies, 18 (62%) agreed to participate. Additionally, one agency involved in EBP efforts asked to participate, resulting in a final sample of 19 agencies (23 sites, 130 therapists, 36 supervisors, 22 executive administrators). Each site (N = 23), rather than each agency (N = 19), was treated as a distinct organization because of different leadership structures, locations, and staff. Going forward, we refer to site as “organization.” The leader of each organization was invited to participate as the “executive administrator.” There were no exclusion criteria for participation. Of the organizations enrolled in this study, 16 had participated in city-sponsored EBP initiatives.

Procedure

All procedures were approved by appropriate Institutional Review Boards. We approached the executive administrator of each organization for participation. Executive administrators completed their questionnaires using REDcap, a secure web-based application that supports online data collection.¹² For supervisors and clinicians, we scheduled a one-time, two-hour meeting at each organization, at which we provided lunch, obtained informed consent, and completed data collection. Approximately 60% of therapists employed by the 23 organizations participated in the study. Participants received $50.

Measures

Participant Characteristics

Executive administrators provided information on their age, gender, ethnicity/race, and educational background. Clinician and supervisor demographics were assessed using the Therapist Background Questionnaire (TBQ), ¹³ a 21-item demographics questionnaire. We also asked participants to report on employment status (i.e., fee-for-service or salaried).

Clinician attitudes were assessed using the Evidence-Based Practice Attitude Scale (EBPAS)¹⁴ a 15-item self-report questionnaire that assesses constructs related to appeal of EBP, requirements to use EBP, general openness to new practices, and divergence between EBP and usual practice. Each subscale is an average of the items within that factor which is measured on a continuum from 0 (not at all) to 4 (very great extent). The EBPAS demonstrates good internal consistency¹⁵ and validity.¹⁶

Clinician knowledge about EBP was measured using the Knowledge of Evidence-Based Services Questionnaire (KEBSQ)¹⁷ a 40-item self-report instrument. Knowledge is measured on a continuum from 0–160 where higher scores are indicative of more knowledge of evidence-based services for youth. Psychometric data suggests temporal stability, discriminative validity, and sensitivity to training.¹⁷

Organizational Characteristics

Supervisors provided information on the number of clinicians in their unit and their employment status. From that information, we determined program size (i.e., number of clinicians in their unit) and percentage of fee-for-service staff. We also assessed the number of years the organization had formally been involved in city-sponsored EBP initiatives.

Organizational culture and climate were measured from the perspectives of clinicians, supervisors, and executive administrators using the Organizational Social Context Measurement System (OSC),¹⁸ a 105-item measure of the social context of mental health organizations. Organizational culture includes proficiency, rigidity, and resistance, and climate includes engagement, functionality, and stress. Proficient cultures are those in which clinicians prioritize the well-being of clients and are expected to keep up competencies. Rigid cultures are those in which clinicians have little autonomy and discretion. Resistant cultures refer to ones in which clinicians are expected to be apathetic to. Engaged climates refer to ones in which clinicians feel they can accomplish worthwhile things and remain invested in their work. Functional climates are ones in which clinicians feel that they are able to get their job done effectively. Stressful climates refer to ones in which clinicians feel emotionally exhausted. Organizational culture and climate are measured with T-scores with a mean of 50 and standard deviation of 10 based on a normed sample of 100 community mental health clinics.¹⁸ The OSC has strong psychometric properties.¹⁹

Implementation climate was measured from the perspective of clinicians, supervisors, and executive administrators, using the Implementation Climate Scale (ICS)²⁰ an 18-item scale that measures climate for EBP implementation including focus on EBP, educational support for EBP, recognition for using EBP, rewards for using EBP, selection of staff for EBP, and general organizational openness. Each subscale is an average of the items within that factor which is measured on a continuum from 0 (not at all) to 4 (very great extent). Psychometrics are strong with excellent reliability and validity.²⁰

Implementation leadership was measured from the perspective of clinicians, rating their direct supervisor, using the Implementation Leadership Scale (ILS),²¹ a 12-item scale that measures leader behaviors relevant to implementation of EBP including proactive, knowledgeable, supportive, and perseverant leadership. Each subscale is an average of the items within that factor which is measured on a continuum from 0 (not at all) to 4 (very great extent). The ILS has strong psychometric properties with excellent internal consistency and validity.²¹

Dependent Variables

Clinician use of cognitive-behavioral, family, and psychodynamic techniques was measured using the Therapist Procedures Checklist - Family Revised (TPC-FR),²² a 62-item self-report checklist. Clinicians were asked to respond in reference to a representative client that they were currently treating. Clinicians could endorse using strategies from all three families of techniques. Each dependent variable is an average of the items that fit within that factor which is measured on a continuum from 1–5 where the anchors refer to 1 (rarely); 2 (seldom); 3(sometimes); 4(often); 5(most of the time). Higher scores are indicative of more utilization of the set of techniques. Factor structure has been confirmed, test-retest reliability is strong, and the instrument is sensitive to within-therapist changes in technique use.^22,23

Data Analytic Plan

Organizational measures are constructed by aggregating individual responses within the organization, if there is enough agreement among individuals. To determine agreement, we used average within-group correlation (awg, rwg) statistics.^24,25 On all organizational variables, statistics were above the suggested .60 level;^25,26 therefore participant responses to organizational constructs were averaged within each organization. Missing data for both independent and predictor variables were minimal (<10%); series means were imputed for missing predictor variables.

We used three sets of linear mixed-effects regression models to determine the associations of clinician and organizational factors (independent variables) with self-reported use of cognitive-behavioral, family, and psychodynamic techniques (dependent variables). All linear mixed-effects models included random intercepts for organization to account for nesting of clinicians within organizations and fixed effects for clinician and organizational factors. Clinician factors included participant demographics [gender, age, clinical experience, employment status (i.e., fee-for-service or salaried)], attitudes [EBPAS subscales] and knowledge [KEBSQ total score]. Organizational factors included organization demographics [cumulative years organization participated in city-sponsored EBP initiatives, program size (i.e., number of therapists in the unit), organization type (percentage of fee-for-service staff)], implementation climate [ICS subscales], implementation leadership [ILS subscales], and organizational culture and organizational climate [OSC subscales]. Dependent variables included use of cognitive-behavioral [TPC-CBT], family [TPC-Family], and psychodynamic [TPC-Psychodynamic] techniques. Therapist case-mix and clinician ethnicity were initially included in the model as covariates but were removed because associated coefficients were not statistically significant.

Analyses were conducted using PROC MIXED in SAS 9.0. Four separate models were conducted for each of the dependent variables. In the first unconditional model, only the organization random effect was included to provide an estimate of both the organizational and residual variance. This model allows us to estimate how much of the total variance could possibly be attributed to the organization and to calculate the intraclass correlation coefficient (ICC). In the second model, only clinician fixed effects were included to estimate the total variance attributable to clinician fixed effects. This model allowed us to calculate the proportion of the total variation in the model explained by clinician factors. In the third model, only organizational fixed effects were included to estimate the total variation attributable to organizational fixed effects. In the fourth model, both clinician and organizational fixed effects were included; these models are reported upon in the results. Our goal was to understand how much of the overall variation in our dependent variable (s) was explained by the set of organizational factors and individual factors, respectively (i.e., separately). The proportion of variation explained by clinician factors was calculated by subtracting the total variation from the clinician factors model (Model 2) from the variance of the unconditional model (Model 1) and dividing by the total variation from Model 1 (i.e., %Var_clinician = (Var_{unconditional} - Var_clinician)/Var_{unconditional}); the same technique was used separately to calculate the proportion of variation for organizational factors (i.e., %Var_organization = (Var_{unconditional} – Var_organization)/Var_{unconditional}).²⁷ Our analyses of the contributions of individual and organizational factors focus on the unique effect of each factor after controlling for all other factors in the model.

Results

Participants

Table 1 provides demographic information about clinicians. Half of executive administrators were male; they identified as Asian (9%), African American (18%), Caucasian (55%), Multiracial (9%), or missing ethnicity/race (9%). Fifteen percent identified as Hispanic/Latino. Supervisors were primarily female (69%); they identified as African American (17%), Caucasian (56%), Hispanic/Latino (14%), other (3%), or missing (10%).

Table 1.

Descriptive Statistics for Variables

Variable	N	Median [Inter-Quartile Range; 25th–75th percentile] or percentage (%)
Dependent Variable – Therapist Procedures Checklist – Family Revised
Cognitive-behavioral Techniquesa	127	3.15 [2.78–3.70]
Family Techniquesa	127	3.47 [3.00–3.80]
Psychodynamic Techniquesa	127	3.57 [2.89–4.25]
Clinician Factors
Demographics
Ethnicity/Raceb	123	Asian (5%), African-American (22%), Caucasian (55%), Hispanic/Latino (11%), Multiracial (4%), Other (4%)
Educational Backgroundb	124	Bachelors (4%), Masters (86%), Doctoral (10%)
Years at Current Organization	124	2.00 [1.00–4.00] years
Genderb	129	Male (23%); Female (76%)
Age (Years)	122	35.00 [29.00–47.00] years
Years Clinical Experience	122	5.00 [2.00–10.00] years
Employment Statusb	119	FFS (52%); Salaried (40%)
Attitudes
Requirements: Extent to which a therapist would adopt EBP if it were requiredc	129	3.00 [2.00–3.67]
Appeal: Extent to which a therapist would adopt EBP if it were appealingc	129	3.25 [2.67–3.75]
Openness: Extent to which a therapist is open to trying EBPc	129	3.00 [2.50–3.75]
Divergence: Extent to which EBP is not clinically usefulc	129	1.25 [.75–1.75]
Knowledge
Total Knowledge of EBP for Youthd	127	94.00 [89.50–101.00]
Organizational Factors	Org. N
Cumulative Years Participating in EBP initiativese	23	3.00 [0.00–5.00] years
Program Size (Number of Therapists)	23	9.50 [7.00–25.00] therapists
Percentage of Staff that are Employed Using a Fee-for-Service Model	23	65%
Implementation Climate
Focus on EBPs: Extent to which an organization values and emphasizes EBPc	23	2.38 [1.79–2.89]
Educational support: Extent to which an organization provides educational support for EBPc	23	1.58 [1.25–1.95]
Recognition: Extent to which an organization recognizes staff implementing EBPc	23	2.00 [1.70–2.61]
Reward: Extent to which an organization financially rewards staff implementing EBP c	23	.39 [.31–.95]
Staff selection: Extent to which an organization selects staff based on ability to implement EBPc	23	2.33 [2.00–2.90]
Openness: Extent to which an organization is generally open to innovationc	23	2.92 [2.33–3.42]
Implementation Leadership
Proactive: Extent to which leader developed a plan to facilitate EBP implementationc	23	2.12 [1.75–2.89]
Knowledgeable: Extent to which leader is knowledgeable about EBPc	23	2.89 [2.25–3.33]
Supportive: Extent to which leader is supportive around EBP implementationc	23	3.04 [2.67–3.44]
Perseverant: Extent to which leader is perseverant through ups and downs of EBP implementationc	23	2.79 [2.36–3.33]
Organizational Social Context
Proficient culture: Extent to which clinicians are expected to remain knowledgeable and competentf	23	55.60 [45.83–59.40]
Rigid culture: Extent to which clinicians have little autonomy and discretionf	23	57.97 [52.95–63.18]
Resistant culture: Extent to which clinicians are apathetic to changef	23	64.22 [56.82–74.70]
Engaged climate: Extent to which clinicians feel like they can accomplish worthwhile thingsf	23	54.17 [48.82–58.72]
Functional climate: Extent to which clinicians feel like they can function effectivelyf	23	62.14 [55.33–72.19]
Stressful climate: Extent to which clinicians are emotionally exhaustedf	23	55.46 [51.82–59.15]

^aMeasured on a continuum from 1–5 where the anchors refer to 1 (rarely); 2 (seldom); 3(sometimes); 4(often); 5(most of the time). Higher scores are indicative of more utilization of the set of techniques.

^bDoes not sum to 100% due to missing data.

^cMeasured on a continuum from 0–4 where the anchors refer to 0 (not at all); 1 (slight extent); 2 (moderate extent); 3 (great extent); 4 (very great extent). Higher scores are indicative of more positive attitudes, implementation climate, and implementation leadership.

^dMeasured on a continuum from 0–160 where higher scores are indicative of more knowledge of evidence-based services for youth.

^eTo calculate this variable, we added up the total number of years spent participating in an EBP initiative. For example, if an agency participated in one initiative for 2 years, and another initiative for 3 years, the total score for this variable would be 5 years. Higher numbers are indicative of more time spent in EBP initiatives.

^fOrganizational culture and climate are measured with T-scores with a mean of 50 and standard deviation of 10 based on a normed sample of 100 community mental health clinics. Higher scores on proficiency, engagement, and functionality are reflective of more positive culture or climate. Higher scores on rigidity, resistance, and stress are reflective of more negative culture and climate.

Abbreviations: EBP = evidence-based practice, FFS = fee-for-service; * = adds up to greater than 100% because of rounding error, ^ not included in linear mixed-effects mode

See Table 1 for descriptive statistics for all variables included in the models. See eTable 1 for a correlation matrix documenting correlations between predictors and outcomes.

Clinician Use of Cognitive-Behavioral Techniques

See Table 2 for model parameters. Organizational factors accounted for 23% of the variance in clinicians’ use of cognitive-behavioral techniques; clinician factors accounted for 16%. Three organizational variables were associated with use of cognitive-behavioral techniques. Clinicians in organizations that had participated for fewer years in city-sponsored EBP initiatives, had more resistant cultures, and had more functional climates were more likely to use cognitive-behavioral techniques. Two clinician variables were associated with use of cognitive-behavioral techniques. Older clinicians and clinicians with more open attitudes towards new practices were more likely to use cognitive-behavioral techniques.

Table 2.

Variation in and Factors Associated with Use of Cognitive-Behavioral, Psychodynamic, and Family Therapy Strategies

Variable	Cognitive-Behaviorala	Psychodynamica	Familya
Variance Components in Random Effects Only Model
Organizational variance	.13	.04	.17
Residual variance	.35	.39	.75
ICC	.27	.09	.19

Clinician factors parameter estimates from regression analyses [confidence intervals, alpha = .05]
Demographics	Mean Diffb	Lower CI	Upper CI	Mean Diffb	Lower CI	Upper CI	Mean Diffb	Lower CI	Upper CI
Male Gender (male vs. female)	−.13	−.41	.15	−.26	−.56	.03	−.49	−.90	−.08
Age (years; 1-year difference)	.02	.00	.03	.01	−.01	.02	.01	−.01	.03
Clinical Experience (years; 1-year difference)	−.00	−.02	.02	−.01	−.03	.15	.03	−.00	.06
Salaried Employment Status (salaried vs. FFS)	−.03	−.34	.29	−.01	−.32	.33	.03	−.42	.48
Attitudes
Requirements: Extent to which a therapist would adopt EBP if it were required (1-point difference)c	−.04	−.15	.08	−.01	−.13	.11	−.08	−.24	.08
Appeal: Extent to which a therapist would adopt EBP if it were appealing (1-point difference)c	.07	−.12	.27	.09	−.11	.30	.19	−.10	.48
Openness: Extent to which a therapist is open to trying EBP (1-point difference)c	.21	.03	.40	.13	−.06	.32	.06	−.21	.32
Divergence: Extent to which EBP is not clinically useful (1-point difference)c	.14	−.02	.30	.23	.06	.40	.05	−.18	.28
Knowledge
Total Knowledge of EBP for Youth (1-point difference)d	.00	−.01	.01	−.02	−0.03	−.00	−.01	−.03	.01

Organizational factors parameter estimates from regression analyses [confidence intervals, alpha = .05]
	Mean Diffb	Lower CI	Upper CI	Mean Diffb	Lower CI	Upper CI	Mean Diffb	Lower CI	Upper CI
Cumulative Years Participating in EBP initiatives (1-year difference)e	−.13	−.22	−.04	−.05	−.14	.04	−.12	−.24	.01
Program size (number of therapists; 1-therapist difference)	−.02	−.04	.00	−.01	−.03	.01	.01	−.04	.02
Organization Type (% FFS staff; 1-percentage difference)	.75	−.08	1.58	.55	−.30	1.41	1.26	.07	2.46
Implementation Climate
Focus on EBP: Extent to which an organization values and emphasizes EBP (1-point difference)c	−.13	−.70	.43	.10	−.49	.69	.01	−.81	.82
Educational support: Extent to which an organization provides educational support for EBP (1- point difference)c	−.27	−.94	.39	.24	−.45	.93	−.20	−1.17	.76
Recognition: Extent to which an organization recognizes staff implementing EBP (1-point difference)c	.19	−.37	.75	.13	−.45	.71	.13	−.68	.93
Reward: Extent to which an organization financially rewards staff implementing EBP (1-point difference)c	.04	−.49	.56	−.06	−.61	.48	.51	−.24	1.27
Staff selection: Extent to which an organization selects staff based on ability to implement EBP (1-point difference)c	−.01	−.65	.62	−.37	−1.03	.29	−.76	−1.68	.16
Openness: Extent to which an organization is generally open to innovation (1-point difference)c	−.40	−.99	.18	.13	−.47	.73	.21	−.64	1.04
Implementation Leadership
Proactive: Extent to which leader developed a plan to facilitate EBP implementation (1-point difference)c	.08	−.45	.61	.01	−.54	.56	−.04	−.80	.73
Knowledgeable: Extent to which leader is knowledgeable about EBP (1-point difference)c	.01	−.36	.38	.11	−.28	.49	−.24	−.78	.29
Supportive: Extent to which leader is supportive around EBP implementation (1-point difference)c	−.43	−1.05	.19	−.42	−1.06	.23	−.25	−1.14	.65
Perseverant: Extent to which leader is perseverant through ups and downs of EBP implementation (1-point difference)c	.35	−.45	1.16	.16	−.67	1.00	.54	−.62	1.70
Organizational Social Context
Proficient culture: Extent to which clinicians are expected to remain knowledgeable and competent (1-point difference)f	−.03	−.08	.01	−.02	−.06	.02	−.02	−.08	.04
Rigid culture: Extent to which clinicians have little autonomy and discretion (1-point difference)f	−.02	−.05	.01	−.01	−.05	.02	−.04	−.08	.01
Resistant culture: Extent to which clinicians are apathetic to change (1-point difference)f	.05	.02	.08	.02	−.01	.05	.03	−.02	.07
Engaged climate: Extent to which clinicians feel like they can accomplish worthwhile things (1-point difference)f	.01	−.04	.05	.01	−.03	.05	.03	−.03	.09
Functional climate: Extent to which clinicians feel like they can function effectively (1-point difference)f	.09	.02	.16	.03	−.04	.10	.08	−.02	.17
Stressful climate: Extent to which clinicians are emotionally exhausted (1-point difference)f	.05	−.00	.11	.03	−.03	.09	.12	.04	.20

^aMeasured on a continuum from 1–5 where the anchors refer to 1 (rarely); 2 (seldom); 3(sometimes); 4(often); 5(most of the time). Higher scores are indicative of more utilization of the set of techniques.

^bMean diff refers to the mean differences in the outcome between groups (for categorical variables) or for a 1-point difference (for continuous variables.

^cMeasured on a continuum from 0–4 where the anchors refer to 0 (not at all); 1 (slight extent); 2 (moderate extent); 3 (great extent); 4 (very great extent). Higher scores are indicative of more positive attitudes, implementation climate, and implementation leadership.

^dMeasured on a continuum from 0–160 where higher scores are indicative of more knowledge of evidence-based services for youth.

^eTo calculate this variable, we added up the total number of years spent participating in an EBP initiative. For example, if an agency participated in one initiative for 2 years, and another initiative for 3 years, the total score for this variable would be 5 years. Higher numbers are indicative of more time spent in EBP initiatives.

^fOrganizational culture and climate are measured with T-scores with a mean of 50 and standard deviation of 10 based on a normed sample of 100 community mental health clinics. Higher scores on proficiency, engagement, and functionality are reflective of more positive culture or climate. Higher scores on rigidity, resistance, and stress are reflective of more negative culture and climate.

Abbreviations: ICC = intraclass correlation coefficient; EBP = evidence-based practice; FFS = fee-for-service

Clinician Use of Family Techniques

Organizational variables accounted for 19% of the variance in the use of family techniques; clinician variables accounted for 7%. Two organizational variables were associated with use of family techniques. Organizations employing more fee-for-service staff and organizations with more stressful climates were more likely to use family techniques. One clinician variable was associated with use of family techniques: female clinicians were more likely to use family techniques.

Clinician Use of Psychodynamic Techniques

Clinician factors accounted for 20% of the variance in the use of psychodynamic techniques; organizational factors accounted to 7%. Two clinician factors were associated with use of psychodynamic techniques. Clinicians with more divergent attitudes on the perceived difference between evidence-based practices and current practices and less knowledge about EBP were more likely to use psychodynamic techniques.

Variance Attributable to Clinician and Organizational Factors Collectively

For use of cognitive-behavioral techniques, collectively clinician and organizational factors explained 30% of the overall variation; for use of psychodynamic strategies, collectively clinician and organizational factors explained 18% of the overall variation; for use of family strategies, collectively clinician and organizational factors explained 26% of the overall variation (data not shown).

Discussion

This study provides information on what predicts clinicians’ use of therapy techniques in a large public mental health system supporting implementation of CBT. Organizational factors accounted for more of the variance in clinicians’ use of cognitive-behavioral and family techniques. Conversely, clinician factors accounted for more of the variance in their use of psychodynamic techniques. CBT and family therapy are both evidence-based techniques for childhood disorders whereas psychodynamic techniques have less rigorous evidence to support them.¹⁰ Taken collectively, these findings suggest that organizational factors are more likely to drive use of EBPs, whereas clinician attributes are more likely to drive use of non-EBP therapy techniques.

Consistent with the literature,¹⁶ clinicians with more open attitudes were more likely to use cognitive-behavioral techniques. However, inconsistent with the literature,¹⁴ older clinicians were more likely to use cognitive-behavioral techniques. Older therapists may have more experience changing treatment modalities according to demand because they have been in the system longer. Paradoxically, organizations that had spent fewer years participating in city-sponsored EBP initiatives were more likely to use cognitive-behavioral techniques. It is possible that organizations that seek out CBT training have clinicians less experienced in CBT. Alternatively, it is possible that organizations spending more years participating in EBP initiatives experience “evidence-based practice fatigue” (i.e., innovation fatigue²⁸), or the stress involved with competing demands of difficult human service jobs coupled with lack of clarity of how these initiatives fit with the role of therapist. Further research is needed to understand potential unintended consequences of EBP efforts such as innovation fatigue.

Organizations with more resistant cultures and organizations with more functional climates were more likely to have clinicians who endorsed using cognitive-behavioral techniques. Organizations with more resistant cultures may be more likely to participate in initiatives to increase use of innovation. Clinicians’ perceptions that they are functioning effectively appear to contribute to more use of cognitive-behavioral techniques. Organizations with more stressful climates and with more fee-for-service staff were more likely to have clinicians who used family techniques. It is possible that organizations with more stressful climates serve a more chaotic population, suggesting the potential utility of family techniques, which are indicated for youth with psychiatric disorders and chaotic family environments.²⁹

Consistent with the literature, clinicians who were less knowledgeable about EBP and held more divergent attitudes towards EBP were more likely to use psychodynamic techniques.³⁰ Also consistent with previous findings,⁹ clinicians reported using cognitive-behavioral and other therapy techniques (i.e., eclecticism) at the same time suggesting a potential exnovation problem. Exnovation refers to “the process whereby an organization decides to divest itself of an innovation that it had previously adopted”.³¹ Further research is needed to understand how organizations can plan for EBP fit with existing practices.

Interestingly, implementation climate and leadership, constructs hypothesized to be related to implementation outcomes,^8,32 did not predict use of EBP. However, the present study did not examine more complex interactive or mediational processes to account for how these constructs may work together with molar culture or climate to predict the outcome utilized in this study.³³

Some study limitations should be noted. The primary outcome variables are based on clinician self-report of use of therapeutic techniques and clinicians are not always accurate reporters of use of therapeutic techniques.^34,35 Second, we did not have 100% participation, creating a potential selection bias at both the therapist- and organizational-level. Third, we used a random intercepts only model and did not allow the slopes to vary by organization due to sample size limitations.

These findings offer important implications. Organizational variables accounted for more of the variance than individual variables in predicting use of EBP, suggesting that organizational-level implementation strategies³⁶ will be more effective in increasing the use of EBP than implementation strategies that directly target the clinician. Clinician factors account for more of the variance than organizational factors in clinician use of psychodynamic techniques. Efforts to implement EBP may need to be accompanied by efforts to encourage clinicians to divest themselves of outdated practices. While these two activities may be seen as two sides of the same coin, the results of this study suggest that the process of exnovation may be driven by different factors than the process of implementation. Of perhaps equal importance, the variables included in our measurement model represent many of the constructs posited to predict implementation, and yet separately they accounted only for a maximum of 23% of the overall variation in outcomes, suggesting there are a number of unmeasured constructs. Finally, this study highlights the need for prospective studies to test the relative contributions and interactions of clinician and organizational focused implementation strategies on adoption, fidelity, and sustainment of EBP.

These findings suggest the nuanced impact of clinician and organizational factors on clinician use of therapy techniques. This study suggests that both “where you work” and “who you are” matter in understanding clinician behavior in context, and that improving the effectiveness of implementation strategies should consider both approaches.³⁷