Therapist and organizational factors associated with participation in evidence-based practice initiatives in a large urban publicly-funded mental health system

Abstract

The goal of this study was to identify therapist and organizational characteristics associated with participation in evidence-based practice (EBP) training initiatives sponsored by a large publicly funded mental health system. Self-report data from therapists (N = 247) nested within 28 mental health clinics was collected in 2015. Results from regression analyses indicated that length of time employed at an organization was associated with individual therapist participation in an EBP initiative. Organizational and implementation climate were associated with organizational participation in an EBP initiative. Organizations characterized by higher levels of stress were more likely to participate in EBP initiatives and organizations characterized by higher engagement and educational support for EBPs were less likely to participate. Implications include the need for systems to consider organizational characteristics when sponsoring organizations in EBP initiatives.

In recent years, there has been an emphasis on the implementation of evidence-based practices (EBPs) into practice settings. This focus is due to a growing understanding that EBP are not reaching the most vulnerable, particularly those served by the public health and mental health systems. ^1,2 For example, despite there being over 1200 evidence-based treatments for childhood psychiatric disorders,³ these treatments are rarely found in public mental health settings..^3,4 Implementation science, the study of methods to promote the systematic uptake of research,⁵ was developed to understand how to more effectively implement EBP in community settings. Conceptual frameworks,^6,7 corroborated by empirical evidence,^8–10 suggest that EBP implementation necessitates an understanding of the individuals (e.g., provider knowledge and attitudes),^11–13 organizations (e.g., organizational culture and climate)^14,15 and systems (e.g., financing)^16,17 involved; as well as the particular stage of implementation (i.e., exploration, preparation, implementation, and sustainment; EPIS framework).^6,7 However, despite the growing body of literature on characteristics of implementation success, there has been less attention paid to understanding which individuals and organizations choose to implement (i.e. adopt) EBP in the community, particularly within the context of systems who do not mandate EBP implementation.

Determining which individuals and organizations adopt EBP is an important piece of the dissemination and implementation puzzle, particularly in the exploration and preparation phases of the implementation process. Prior research indicates that individual clinician characteristics are associated with adoption, including positive attitudes towards EBP,¹⁸ being an older provider, and having higher educational attainment.^18,19 In otherwise supportive organizations, reasons why individuals chose not to adopt EBP include barriers such as time, resources, and misconceptions of EBP.^20,21 At the organizational level, the evidence is equivocal. Some studies have found that adoption and implementation of EBP correlates with more positive organizational climates^22,23 whereas another study found a negative relationship.¹⁸ Importantly, it is also possible that the implementation of an EBP can in itself cause disruptions in organizational culture and climate.²⁴

Given that the literature on EBP adoption is limited, conflicting, and often theoretical, empirical research investigating characteristics of individuals and organizations who do and do not participate in EBP implementation in public behavioral health settings is needed. It is particularly relevant and timely given recent efforts by state and county behavioral health systems to increase the availability of EBP through legislation, mandates, and system-funded training opportunities.^25–29 The Philadelphia Department of Behavioral Health and Intellectual disAbility Services (DBHIDS) is one such behavioral health system that has engaged in an effort to encourage the implementation of EBP. Since 2007, DBHIDS has supported and funded a number of training initiatives to increase the availability of cognitive behavioral therapy (CBT) throughout behavioral health organizations in Philadelphia. These efforts present a natural laboratory within which to investigate characteristics associated with individuals and organizations who participate in EBP initiatives.

The goals of the current study are to examine (1) whether therapist characteristics predict therapist participation in EBP initiatives and (2) whether organizational characteristics predict organizational participation in EBP initiatives in public mental health settings. Variables were selected based on previous research and implementation science frameworks highlighting the impact of both individual and organizational factors on the implementation and delivery of EBPs.^6,30,31 Therapist characteristics included demographics (e.g., age, educational background, and length of time at organization), burnout, attitudes toward and knowledge of EBP, and use of specific therapy skills. Organizational level variables included organizational culture and climate, and implementation climate. Organizational culture and climate are two factors shown to be related to attitudes towards adoption of innovation in general, and toward EBP in particular.¹⁹ Implementation climate, defined as staff beliefs that innovation adoption, implementation, and use is expected, rewarded, and supported by the organization,³² may also be an important predictor of participation in an EBP initiative. Given limited existing research, hypotheses are exploratory.

Method

Setting

DBHIDS has supported four specific EBP initiatives: cognitive therapy^33,34 prolonged exposure, trauma-focused cognitive behavioral therapy³⁵ and dialectical behavior therapy across a variety of care settings (e.g., outpatient, inpatient, residential), with two of the initiatives focused on child services (i.e., cognitive therapy and trauma-focused cognitive-behavioral therapy). A full-time employee within DBHIDS coordinates implementation, training, and ongoing consultation by treatment developers. Collectively, the initiatives have supported and financed training, expert consultation, and ongoing technical assistance in over 50 organizations in the Philadelphia system.³⁶ Over the course of implementation, the process employed by DBHIDS for selecting organizations to participate in initiatives has become more formal. Initially, selection of organizations for participation was not uniform and largely guided by DBHIDS (e.g. larger organizations selected, excelling organizations selected), whereas more recently organizations have applied for participation through a competitive request-for-applications process and have been chosen for participation by DBHIDS.³⁶

Agencies and Participants

More than 100 community mental health organizations in Philadelphia provide outpatient services to youth (data provided by the City of Philadelphia Community Behavioral Health; Sarah Chen, PhD, MSW, personal communication, October 30^th, 2014). Thirty-one of the 100 organizations that serve the largest proportion of youth were purposively sampled. Of these 31 organizations, 22 (73%) agreed to participate. Several agencies had multiple locations, resulting in 28 sites. Twenty-three of the 28 sites had participated in at least one child-focused training initiatives (site is referred to as organization from this point forward). The sample consisted of 73 organizational leaders, (i.e., 22 supervisors, 29 clinical directors, and 22 executive directors), and 257 therapists. Approximately 58% of therapists employed by the 22 organizations participated in the study. There were no exclusion criteria for therapist participation. Previous investigation utilizing the data from organizational leaders in this sample has been conducted and reported elsewhere.^37–39 Data from therapists only were used for the current investigation.

Procedure

This study was approved by the University of Pennsylvania and City of Philadelphia Institutional Review Boards. The person(s) identified as the leader of the organization was approached to solicit his/her organization’s participation. A one-time two-hour meeting was scheduled with potential participants, during which lunch was provided. During this meeting, the research team gave an overview of the study, obtained written informed consent, and collected all measures. All were compensated $50 for participation in the study.

Measures

Individual-level measures

Participant demographics

Participants completed a brief demographics questionnaire that assessed age, race/ethnicity, educational background, years of experience, years at current organization, hours worked weekly, caseload, and worker status (independent contractor, salaried, other).⁴⁰ Therapists indicated whether they had formally participated in any of the four DBHIDS EBP initiatives. Participants were asked to verbally confirm that they understood that the question about initiative participation was referring to the one-year training and ongoing consultation provided through DBHIDS.

Burnout

The Maslach Burnout Inventory Human Services Survey (MBI-HSS)⁴¹ is a 22-item self-report measure of therapist burnout that measures emotional exhaustion, depersonalization, and reduced personal accomplishment. Items are rated on a 0 (Never) to 6 (Every day) scale. Higher scores on emotional exhaustion and depersonalization and lower scores on personal accomplishment (reverse scored for the total score) indicate higher levels of burnout. The MBI is the most widely used instrument to assess burnout and has shown satisfactory internal consistency and discriminant and factorial validity.^42–44

Attitudes

The Evidence-Based Practice Attitude Scale (EBPAS)¹³ is a 15-item self-report measure with four subscales assessing attitudes toward adoption of EBP. The four subscales represent four theoretically derived dimensions of attitudes toward EBP adoption: appeal (is EBP intuitively appealing), requirements (would EBP be used if required by organization, supervisor, or state), openness (openness to innovation in general), and divergence (perceived divergence between EBP and current practices). Items are rated on a 0 (Not at all) to 4 (Very great extent) scale. Higher scores indicate more positive attitudes, with the exception of divergence, which is reverse coded. The EBPAS has national norms, and has demonstrated validity and good internal consistency with subscale alphas ranging from .67–.91.⁴⁵

Knowledge

The Knowledge of Evidence-Based Service Questionnaire (KEBSQ) ⁴⁶ is a 40-item self-report instrument that measures knowledge of various evidence-based therapy techniques across several youth problem areas. Knowledge is measured on a continuum from 0–160 where higher scores are indicative of more knowledge of EBP. Psychometric data suggests temporal stability, discriminant validity, and sensitivity to training.⁴⁶

Use of therapy strategies

The Therapist Procedures Checklist – Family Revised (TPC-FR)⁴⁷ is a 62-item self-report checklist measuring use of therapy strategies from four modalities: cognitive, behavioral, family, and psychodynamic. Therapists were asked to respond in reference to a representative client that they were currently treating. Items are rated on a 1 (Rarely) to 5 (Most of the time) scale and higher scores are indicative of more utilization of the set of strategies for a given domain. The factor structure has been confirmed, test-retest reliability is strong, and the instrument is sensitive to within-therapist changes in strategy use.^47,48

Organizational-level measures

Organizational evidence-based practices

DBHIDS provided the research team with a list of the organizations that had participated in one of the four city-sponsored EBP initiatives, the year they began participating, and their completion year (if applicable).

Organizational culture and climate

The Organizational Social Context Measurement System (OSC),⁴⁹ includes 105 items measuring six dimensions that assess organization culture and organizational climate in mental health and social service organizations.^49a Organizational culture is defined as shared beliefs and expectations of a work environment and includes three primary dimensions: proficiency, rigidity, and resistance. Proficient cultures are those in which therapists prioritize the well-being of clients and are expected to be competent. Rigid cultures are those in which therapists have little autonomy. Resistant cultures are those in which therapists are expected to show little interest in changes or new ways of providing services. Organizational climate refers to shared perceptions about the work environment’s impact on worker well-being and functioning and is measured on three dimensions: engagement, functionality, and stress. Engaged climates refer to settings in which therapists feel they can accomplish worthwhile goals and remain invested in their work. Functional climates are those in which therapists can get their job done effectively and have a well-defined understanding of how they fit into the organization. Stressful climates refer to settings in which therapists feel emotionally exhausted. Organizational culture and climate scores are profiled with T scores (mean ± SD = 50 ± 10) established from a national sample of 100 mental health organizations.⁴⁹ The measurement model, within-system agreement, reliability and between-system differences, are strong.⁵⁰

Implementation climate

The Implementation Climate Scale (ICS),³² is an 18-item self-report measure of multiple factors that contribute to successful implementation including: organizational focus on EBP (e.g., view EBP implementation as important); educational support for EBP (e.g., provide support for trainings or training materials); recognition for using EBP (e.g., use of EBP held in esteem); rewards for using EBP (e.g., financial incentives for EBP use); selection of staff for EBP (e.g., selection based on previous experience or formal education in EBP); and selection of staff for openness (e.g., flexible, adaptable). Items are rated on a 0 (Not at all) to 4 (Very great extent) scale with higher scores indicating a more positive implementation climate indicating the organization values and is supportive of EBP. Psychometric evaluation suggests good reliability and validity.³²

Data analytic plan

Organizational measures are included in analyses by aggregating individual responses within the organization, if there is enough agreement. Average within-group correlation (awg, rwg) statistics were used.^51,52 On all organizational variables, both statistics were above the suggested .60 level;^51,53 therefore participant responses to organizational constructs were averaged within each organization. Missing data for predictor variables were less than 10%; series means were imputed for missing predictor variables. Consistent with previous studies,⁵⁴ each site (K = 28), rather than each organization (K = 22), was treated as a distinct unit because of largely different leadership structures, locations, and staff.

Initial descriptive statistics were calculated using SPSS version 23. Main analyses were conducted using PROC MIXED and PROC LOGISTIC in SAS 9.0. First, a mixed-effects linear regression model was conducted to identify if therapist demographic variables (i.e., clinical experience, age, years at organization, hours worked weekly, caseload, worker status [independent contractor, salaried, other]) were related to participation in EBP initiatives (continuous; 0–4 initiatives). Second, a mixed-effects linear regression model was conducted to identify if therapist knowledge (KEBSQ),⁴⁶ attitudes (EBPAS; requirements, openness, appeal, and divergence),¹³ burnout (MBI; emotional exhaustion, depersonalization, and personal accomplishment),⁴¹ and self-reported use of therapy strategies (TPC-FR; behavioral, cognitive, family, psychodynamic) ⁴⁷ were related to therapist participation in an initiative (yes/no). Mixed-effects models included random intercepts for organization to account for nesting of therapists within organizations and fixed effects for therapist factors.

Third, seventeen logistic regression models were run to identify if organizational culture (OSC; proficiency, rigidity, resistance),⁴⁹ organizational climate (OSC; engagement, functionality, stress),⁴⁹ and implementation climate (ICS; focus on EBPs, educational support, recognition, reward, selection of staff, organization openness),³² were related to organizational participation in an initiative (dichotomous; yes/no). Because analyses were underpowered to detect effects at the organizational level, trend relationships (p < .10) were included. Adjustment for multiple models were not made given the exploratory nature of the analyses.⁵⁵

Results

See Table 1 for descriptive statistics of therapist demographic variables. The sample was predominantly female and ethnically diverse. The majority of therapists had master’s degrees and were either not licensed or in the process of obtaining licensure. Forty-seven percent of therapists participated in an EBP initiative (data missing for four); 82% of organizations participated in an EBP initiative. Of therapists who participated in at least one EBP initiative, 26.7% participated in one, 13.4% in two, and 6.4% in three or more.

Table 1.

Therapist (N = 247) demographics

Variable	Frequency (%)
Gendera
Male	21.6%
Female	77.7%
Hispanic/Latinoa
Yes	21.1%
No	76.9%
Ethnicitya
Asian	4.5%
Black or African American	30%
White	40.9%
Hispanic/Latino	16.2%
Multiracial	4.0%
Other	2.8%
Academic backgrounda
Bachelor’s degree	6.5%
Master’s degree	77.7%
Doctoral degree	13.8%
Other
Licensure statusa
Yes	19.8%
No	39.3%
In process	38.9%

^aDoes not add up to 100% because of missing responses

Demographic variables predicting individual participation

Table 2 presents the results of analyses predicting individual therapist participation in EBP initiatives. Only one variable was significant; as years spent at the organization increased, the number of initiatives therapists participated in also increased.

Table 2.

Therapist demographic variables predicting EBP initiative participation (N = 243)

Variable	M (SD) [Range] or percentage	Parameter Estimate	Standard Error
Dependent variable: Therapist initiative participation	.77 (1.01) [0–4]	--	--
Therapist age	38.74 (11.93) [23–76]	1.07	.74
Clinical experience	10.05 (8.60) [0–44]	−1.31	1.04
Years at organization	3.3 (3.9) [0–30]	.60*	.04
Hours worked per week	25.13 (15.89) [0–90]	.32	.50
Weekly caseload	28.73 (20.63) [1–125]	.28	.34
Worker status	salaried (20.6%); contractor (76.6%); other (9.7%)	8.08	13.35

Abbreviations: EBP, Evidence-Based Practice

^*p < .05

Therapist participation in EBP initiatives

Table 3 presents the results of analyses assessing whether therapist knowledge, attitudes, burnout, and self-reported use of therapy strategies was associated with therapist participation in EBP initiatives. Descriptive statistics for each therapist-level measure are also presented in Table 3. Nearly all scale and subscale means were in the expected range for this setting and population and in line with available norms.^41,45–47 On the EBPAS, the mean for the divergence subscale was somewhat higher than national norms, indicating therapists in this sample tended to view EBPs as less clinically useful than clinical experience.⁴⁵ None of the variables were significantly related to therapist participation.

Table 3.

Therapist knowledge, attitudes, burnout, and strategy use predicting EBP initiative participation (N = 243)

Variable	M (SD) [Range] or percentage	Odds Ratio	CI (95%)
Dependent variable: Therapist initiative participation	Yes (46.6%) No (51.8%)	--	--
TPC-FR
Behavioral	2.9 (.80) [1.13–4.8]	1.16	.72–1.88
Cognitive	3.67 (.67) [1.69–5]	1.05	.60–1.85
Psychodynamic	3.38 (.67) [1.33–5]	.85	.47–1.54
Family	3.39 (.87) [1.07–4.93]	1.13	.71–1.81
Knowledge (KEBSQ)	96.12 (8.9) [72–121]	.98	.94–1.02
Attitudes (EBPAS)
Requirements	2.86 (1.0) [0–4]	.81	.60–1.10
Appeal	3.17 (.68) [1–4]	1.31	.81–2.12
Openness	3.08 (.71) [.75–4]	.78	.49–1.24
Divergence	2.73 (.74) [1–4]	.95	.63–1.43
Burnout (MBI)
Emotional Exhaustion	19.10 (12.5) [0–54]	1.01	.98–1.04
Depersonalization	4.09 (4.7) [0–24]	1.00	.93–1.08
Personal Accomplishment	38.86 (6.6) [12–48]	1.02	.97–1.07

Abbreviations: EBP, Evidence-Based Practice; TPC-FR, Therapist Procedures Checklist-Family

Revised; KEBSQ, Knowledge of Evidence-Based Service Questionnaire; EBPAS, Evidence-Based Practice Attitudes Scale; MBI, Maslach Burnout Inventory

Organizational participation in EBP initiatives

Table 4 presents the results of analyses predicting organizational participation in initiatives. Descriptive statistics for each organizational-level measure are also presented in Table 4. All means were in the expected range for this setting and population.^32,49 Organizational climate was marginally related to organizational participation in EBP initiatives. Each 1 point increase in engagement decreased the odds that an organization participated in an EBP initiative by 11% (OR = .89, p = .08). Each 1 point increase in stress increased the odds that an organization participated in an EBP initiative by 28% (OR = 1.28; p = .06). Implementation climate was marginally related to organizational participation in EBP initiatives. Each 1 point increase in educational support for EBPs decreased the odds that an organization participated in an EBP initiative by 78% (OR = .22, p = .09).

Table 4.

Organizational model predicting organization EBP initiative participation (N = 28)

Variable	M (SD) [Range] or percentage	Odds Ratio	CI (95%)
Dependent variable: Organization EBP initiative participation	Yes (82.1%) No (17.9%)	--	--
Organizational Culture (OSC)a
Proficiency	53.86 (9.3) [26.01–70.07]	.95	.84–1.08
Rigidity	58.63 (7.2) [41.74–71.49]	1.03	.90–1.18
Resistance	62.40 (9.7) [47.92–82.02]	1.03	.92–1.14
Organizational Climate (OSC)
Engagement	54.38 (9.07) [39.80–70.22]	.89+	.78–1.02
Functionality	63.34 (9.52) [44.32–81.48]	1.01	.91–1.12
Stress	53.89 (6.61) [39.39–66.80]	1.28+	.99–1.66
Implementation Climate (ICS)
Focus on EBPs	2.67 (.47) [1.8–3.46]	.11	.01–1.69
Educational support	2.07 (.70) [.80–3.48]	.22	.04–1.27
Recognition	1.96 (.59) [.80–3.06]	.26	.03–2.06
Reward	0.55 (.58) [0–1.90]	.77	.15–3.9
Staff selection	1.87 (.64) [0–2.98]	.44	.08–2.42
Openness	2.92 (.52) [2–3.78]	.41	.05–3.26

Abbreviations: EBP, Evidence-Based Practice; OSC, Organizational Social Context Measurement

System; ICS, Implementation Climate Scale

^aOSC values are T-scores

⁺p < .10

Discussion

This study examined characteristics of therapists and organizations that did or did not participate in system-sponsored EBP initiatives in an urban publically-funded behavioral health system engaged in a large-scale effort to increase the use of EBP. Consistent with themes from previous qualitative research,⁵⁶ therapist number of years at an organization was associated with participation in a greater number of EBP initiatives. Organizational climate factors were marginally associated with organizational EBP initiative participation, including engagement and stress, also consistent with previous research.²⁴ Implementation climate, specifically educational support, was marginally associated with organizational participation. These results suggest that organizational characteristics (i.e. inner context variables) may differentiate those who participate in EBP initiatives, which has important implications for future efforts to take EBP implementation to scale in large systems. These results are consistent with the EPIS framework, which suggests that EBP implementation and sustainment are impacted by both inner context (e.g., organizational and provider characteristics) and outer context (system-level factors such as policy and service environment).⁶ The current results provide additional evidence that inner context variables impact EBP implementation.

The only therapist characteristic associated with participation in EBP initiatives was length of time employed at the organization. The most parsimonious explanation for this finding is that the longer a therapist is employed at an organization, the more opportunities there are to participate in EBP initiatives. Therapist participation was determined differently across time and setting (i.e. in some instances therapists volunteered and in other instances they were selected by organizational leadership), so additional explanations for the results also exist. Results are consistent with research from the organizational psychology literature suggesting positive associations between job tenure and voluntary learning (i.e., non-mandatory training or learning opportunities).⁵⁷ It may be that staff with a longer tenure are perceived as less risky in terms of return on investment given concerns about therapist turnover.⁵⁸ They may also be viewed as more connected to the organization or effective in their positions and thus more likely to remain employed.⁵⁹ However, results are not consistent with literature suggesting that that the longer a therapist is employed at an organization, the less open they are to EBP adoption.¹⁹ Thus, more research is necessary to better understand the relationship between job tenure and EBP adoption.

Previous literature suggests that therapist characteristics, such as attitudes towards and knowledge about EBPs and level of training are related to adoption.^18,19,60 Thus, it was surprising that therapist characteristics did not differentiate EBP initiative participation. It may be that the sample of therapists was heterogeneous making it difficult to identify meaningful differences. Reasons for a heterogeneous sample include that the selection of therapists was not uniform across organizations, that therapists may not have had equal opportunity to participate dependent on when they joined an organization, and that even within participating organizations not all therapists participated in the initiative. Future research that clarifies individual differences in participation in system level implementation efforts is warranted.

Organizations that participated in EBP initiatives had organizational climates characterized by lower levels of engagement and higher levels of stress. Thus, therapists in EBP participating organizations reported feeling less like they can accomplish worthwhile things and remain invested and may be experiencing emotional exhaustion and feeling overwhelmed with work. These results are consistent with previous research. Patterson and colleagues²⁴ compared organizational climates among mental health programs within a large organization that were offering and not offering EBP. They found that programs offering EBP were characterized by less engagement and more stress than programs without any EBP. Items used to measure the “engagement” construct in the present study included “I feel I treat some of the clients I serve as impersonal objects (reverse coded)” and “I have accomplished many worthwhile things in this job.” It is possible that higher engagement scores reflect a strong internal mission and confidence in current treatment approaches, and such organizations might not see the need to bring in new models of treatment.

With regard to organization stress, there are several potential interpretations. The implementation process itself, which requires devoting time to training, ongoing support, and other operations (e.g., altering clinical documentation practices) may increase organizational stress.²⁴ Several items measuring the “stress” construct could be interpreted as reflective of the implementation process (e.g., “Interests of the clients are often replaced by bureaucratic concerns such as paperwork,” and “The amount of work I have to do keeps me from doing a good job”). Alternatively, an organization may participate because either internal or system leaders believe that the implementation of EBP might mitigate some of the stress already present in an organization. Given the cross-sectional nature of this study, the directionality of this relationship cannot be determined but the results provide fodder for future inquiry. Finally, organizations fostering an implementation climate that provided educational support were less likely to participate in an EBP initiative. It may be that organizations providing more internal resources such as EBP workshops do not feel the need to participate in system-led (or external) training opportunities. Understanding differences between organizations that seek out EBP training versus those who participate in system-led efforts is another interesting area for future research.

Limitations

Several limitations exist. First, selection of both individuals and organizations for participation in initiatives was not systematic. Second the data were cross-sectional and this precludes us from making causal inferences. Third, data about length of time at the organization since participating in training for each therapist were not collected. Fourth, data were self-report, which can be limited by recall and social desirability biases. Fifth, analyses were underpowered to detect effects at the organizational level, however, results identified several marginally significant findings that were corroborated by previous research. Sixth, all organizations and providers were from a single large system, potentially limiting generalizability. Finally, study analyses were exploratory and results need replication.

Implications for Behavioral Health

This study has important implications for understanding the implementation of EBP within large public behavioral health systems, especially when EBP adoption is not mandated. This exploratory study investigated individual and organizational factors associated with participation in EBP initiatives in a large, urban public mental health system and found that factors at both the therapist and organizational levels may be related to participation in system-sponsored EBP initiatives. The large-scale implementation of EBP requires significant investment of time and financial resources. Understanding which therapist and organizational factors impact adoption of EBP has the potential to inform how systems and organizations can most effectively allocate those resources. More research is needed to understand directionality of impact; however, results suggest that it is particularly important to attend to the climate within an organization when implementing EBPs. More systematic investigation of how individuals and organizations are selected or choose to participate in EBP initiatives is needed to help tailor implementation strategies to support large-scale implementation.