Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Nov 26.
Published in final edited form as: J Am Acad Child Adolesc Psychiatry. 2012 Apr;51(4):10.1016/j.jaac.2012.01.018. doi: 10.1016/j.jaac.2012.01.018

Leadership, Innovation Climate, and Attitudes toward Evidence-Based Practice during a Statewide Implementation

Gregory A Aarons 1,*, David H Sommerfeld 2
PMCID: PMC3841107  NIHMSID: NIHMS525835  PMID: 22449648

Abstract

Objective

Leadership is important in practice change, yet there are few studies addressing this issue in mental health and social services. This study examined the differential roles of transformational (i.e., charismatic) leadership and leader member exchange (i.e., the relationship between a supervisor and their direct service providers) on team innovation climate (i.e., openness to new innovations) and provider attitudes toward adopting evidence-based practice (EBP) during a statewide evidence-based practice implementation (EBPI) of an intervention to reduce child neglect.

Methods

Participants were 140 case-managers in 30 teams providing home-based services to families in a statewide child-welfare system. Teams were assigned by region to EBPI or services as usual (SAU) conditions. Multiple group path analysis was used to examine associations of transformational leadership and leader member exchange with innovation climate and attitudes toward adoption and use of EBP.

Results

Transformational leadership predicted higher innovation climate during implementation while leader member exchange predicted higher innovation climate during SAU. Innovation climate was, in turn, associated with more positive attitudes toward EBP for the EBPI group.

Conclusions

Strategies designed to enhance supervisor transformational leadership have the potential to facilitate implementation efforts by promoting a strong climate for EBPI and positive provider attitudes toward adoption and use of EBP.

Keywords: leadership, implementation, evidence-based practice, innovation climate, evidence-based practice attitude scale

Introduction

Evidence-based practices (EBPs) are those health interventions that are supported by rigorous scientific research, allow for clinical expertise in their application, and provide for consumer choice, preference, and culture.1, 2 The ability to adopt, implement, and sustain EBPs is becoming increasingly important for mental health and social service organizations and providers.3 Leaders can impact the capacity to foster (or slow) change and innovation.47 The role of leadership in EBP implementation (EBPI) is often discussed but rarely empirically examined. In this study we address two different aspects of leadership, “transformational leadership” and “leader-member exchange” and their associations with innovation climate and subsequent provider attitudes toward adopting EBP.

Transformational Leadership

Transformational leadership is motivational, individually considerate, intellectually stimulating, and can engage staff in supporting the mission and vision of a leader and promote a climate for innovation and change.8 Transformational leadership8 has been extensively researched and validated in many different organizational settings and cultures9, 10 and reflects a visionary or charismatic form of leadership in which followers are inspired to adopt the goals, values, and behaviors of the leader.11 Transformational leaders encourage analytical engagement and challenge standard ways of conducting work processes.12,13 The heightened sense of intrinsic motivation that transformational leaders are able to establish is expected to result in greater creative and innovative efforts to support organizational goals.5 Transformational leadership is important in organizational outcomes both during change and during stable operations,14, 15 and predicts improved unit performance.16 Transformational leadership compared to traditional leadership styles (e.g., democratic, task-oriented, relation-oriented) predicts greater unit performance and employee job performance and satisfaction.17 Finally, transformational leadership is associated with child and adolescent mental health clinicians’ attitudes toward adopting EBP18 and with mental health service team performance and client outcomes.19

Leader-Member Exchange

Leader-member exchange represents the relationship between a leader and follower that can enhance buy-in and willingness to perform well on the job.20 Leader-member exchange is also important in organizational functioning and job satisfaction20 and differs from transformational leadership in that it focuses on bi-directional relationships between leader and follower similar to “working alliance” in clinical practice.21 It is the nature of the relationship and the reciprocity that develops through ongoing exchanges that influences subordinate actions. Leader-member exchange has been shown to act as a mediator of the effect of transformational leadership on staff performance and behavior.22, 23

Differential Leadership Effects during Change

Eggleston and Bhagat24 suggest that different leadership approaches are required dependent upon whether an organization is in a stable operating state or undergoing change. Some studies have shown that charismatic leadership emerges as the most important approach for leading an organization when experiencing significant organizational change.25 Similarly, the perceived effectiveness of charismatic leaders increases when employees experience greater levels of organizational change.26 This begs the question of whether different aspects of leadership may produce different effects on teams and providers experiencing change (i.e., implementing EBP) compared to providing services as usual (SAU).

Innovation Climate

Innovation climate is a type of strategic organizational climate. Strategic climates are those that support a particular organizational goal such as adopting an EBP. Innovation climate is associated with organizational outcomes including improved implementation of ideas,27 greater organizational innovation,5 and more general benefits such as employee and consumer satisfaction28 and perceived service effectiveness by consumers.29 Both transformational leadership and leader-member exchange may impact innovation climate.

Attitudes toward Evidence-Based Practice

One potential mechanism for translating organizational or team support for innovation into employee behaviors is through influencing employee attitudes. Employee attitudes are important in the actual adoption and implementation of innovations 30, 31 although this process is complex.32 More favorable attitudes towards change and scientific information distinguish early adopters of innovations from late adopters.33 Mental health provider attitudes towards EBP are related to organizational context, provider characteristics, and leadership.18, 34, 35 In addition, employee attitudes toward adoption and use of EBP can be considered an important “implementation outcome” distinct from clinical outcomes.36

The Present Study

The present study compares the associations of transformational leadership and leader-member exchange with innovation climate and employee attitudes toward adoption and use of EBPs for children’s service providers implementing an EBP compared to those delivering SAU. The service system contracts with community-based organizations (CBOs) to provide in-home services. Organizations in the study had been providing mental health and/or social services for over 20 years and those implementing the EBP were in the “active implementation” phase.3 The EBP was SafeCare®, an intervention shown to reduce child neglect37 and to improve service provider retention and reduce emotional exhaustion.38, 39

SafeCare incorporates a generalized problem solving approach into three modules: 1) home safety/accident prevention, 2) child health, and 3) parent-child and/or parent-infant interaction. In contrast to SAU, SafeCare is highly structured and manualized and was delivered by Bachelor and Master’s degree level service providers in teams reporting to the same supervisor. More detailed information can be found in the SafeCare manual.40 Agencies in regions selected to implement SafeCare had previously been providing SAU, thus the EBPI involved adding SafeCare to their model of care. Service providers in the EBPI condition received a minimum of a week-long didactic and interactive (i.e., practice. role-play) training in SafeCare typically followed by at least monthly in-vivo coaching. The SafeCare coach accompanies the provider to the client’s home, observes, models behaviors where appropriate, and provides supportive feedback.

SAU was less structured/more ideographic, often driven by emergent client concerns and crises. SAU focused primarily on parenting, family crises, household management and child caregiving, and tended to approach this via discussion and general advice rather than the more structured, behavioral skill-based SafeCare approach. SAU used a variety of parenting materials and guides, some published and some developed within a given CBO.37

Based on our literature review we developed the following hypotheses: H1) transformational leadership will be positively associated with greater innovation climate, H2) transformational leadership will be positively associated with leader-member exchange, H3) leader-member-exchange will be positively associated with greater innovation climate, H4) the effect of transformational leadership on innovation climate will be mediated by leader-member exchange, H5) transformational leadership will have a stronger positive relationship with innovation climate for the EBPI group, while leader-member exchange will be more important during SAU, and H6) more positive innovation climate will be associated with more positive provider attitudes toward adopting EBP. To our knowledge this is the first study to examine associations of transformational leadership and leader member exchange with innovation climate and attitudes toward EBP in a statewide implementation study.

Methods

Study Context

The data for this study were collected as part of a longitudinal mixed-methods study of a statewide EBPI in the Oklahoma Children’s Services system (OCS).41 The OCS contracts with CBOs to provide in-home services in all six regions of the state. Service providers were employees of the CBOs but treatment model (SafeCare vs. SAU) was manipulated at the region level (n=6). Assignment to treatment condition began by randomizing the two urban regions. For the four rural regions all possible assignment permutations were evaluated and assignment to condition was based on the best pre-implementation balance.37 Teams within regions were then randomized to coaching or no coaching however the research questions in this study focus on EBP condition without regard to coaching.

Service providers and clinical supervisors employed by CBOs participated in biannual (spring/fall) web-based surveys. Each survey took approximately 45–90 minutes to complete. Participants received a written description of the study and informed consent was obtained prior to beginning the survey. Participants received a gift certificate for participating in the study. This study was approved by the appropriate institutional review boards.

The current study utilizes data from three waves of data collection (waves 2,3 and 4) after the adoption decision/planning phase and during active implementation of SafeCare.3 While SafeCare was implemented prior to Wave 1, data from the first wave was not used because it did not include all of the measures needed for these analyses. Changes to the contracting and service delivery system instituted after wave four precluded use of data from subsequent waves. Thus, we limited our study to the three waves providing the needed data and relevant timing of changes in the service system. Each respondent is included in the analyses only once, even if they participated in multiple waves of the study. Where multiple observations were present we selected the last observation, ensuring that the respondent had the longest possible time within their CBO for exposure to their supervisor’s leadership and team innovation climate. This strategy also allowed for respondents in teams implementing SafeCare to have the longest EBPI experience.

Sample

Participants were 140 service providers working in 30 teams providing comprehensive home-based services to families involved with the child-welfare system. Most (96.4%) respondents worked full time, mean age was 37.6 years (SD = 11.4; Range=22–68), and 87.1% were female. Respondents were Caucasian (62.9%), African-American (20.0%), Hispanic (5.0%), and American Indian (12.1%). Almost half (49.3%) had a Bachelors degree, 22.1% some graduate training, and 28.6% a Masters degree. Almost half indicated social work (47.1%) as their primary discipline followed by psychology (16.4%), human relations (13.6%), child development (10.7%), marriage and family therapy (3.6%), or other disciplines (5.6%), and missing responses (2.9%). Respondents worked at their present agency an average of 2.3 years (SD = 3.7). The only group difference in demographics was that there were more female providers in the SAU (94.5%) vs. SafeCare (82.4%) condition (p<.05). All providers were eligible to participate and the response rate across the three waves averaged 94.7% (range=90.7%–96.8%).

Measures

Transformational Leadership was assessed with the Multifactor Leadership Questionnaire (MLQ) 45-item Form 5X42 through which service providers rated their immediate supervisor’s transformational leadership behaviors. The Cronbach’s alphas from the present study data indicated high reliability for the overall measure of transformational leadership (α=0.97) and the four subscales of idealized influence (α=0.93), inspirational motivation (α=0.92), intellectual stimulation (α=0.79), and individual consideration (α=0.92). The validity of the MLQ is supported by studies demonstrating its prediction of important organizational outcomes14, 15 and performance.16 Example items include “spends time teaching and coaching” and “articulates a compelling vision of the future”. Service providers indicated the extent to which their supervisor exhibited specific behaviors related to each question on a 5 point Likert-type scale from “Not at all” to “To a very great extent.” Subscale scores were averaged to create an overall mean score for each participant.

Leader Member Exchange was assessed with the Leader-Member Exchange Scale,20 a single-factor 7-item scale that assesses the quality of the relationship between a leader and subordinate. Consistent with prior research46, the leader-member exchange scale demonstrated good psychometric properties (α =0.94). The leader-member exchange scale has been used in numerous studies with demonstrated validity in studies of organizational support, climate, work attitudes, and staff turnover.4345 Example items include “How well does your supervisor understand your job problems and needs” and “I have enough confidence in my supervisor that I would defend and justify his/her decision if he/she were not present to do so.” Participants indicated their response to each item on a 5 point Likert-type scale and the mean score was used in our analyses.

Innovation Climate was assessed with the Support for Innovation subscale of the Team Climate Inventory.47 The subscale is comprised of eight items including “In this team, we take the time needed to develop new ideas” and “This team is open and responsive to change”. The scale demonstrated very high reliability in the current study (α =0.95). The scale has well established reliability and validity and has been validated across international contexts29 and for large and small groups.47, 48 Responses were indicated on a five point Likert scale ranging from “Not at all” to “To a very great extent”.

The Evidence-Based Practice Attitude Scale (EBPAS) was used to assess provider attitudes toward adopting EBP34, 49, 50. The EBPAS has 15 items and four subscales, Appeal (the intuitive appeal of EBP), Requirements (the likelihood of adopting EBPs given supervisor, organizational, or system requirements to do so), Openness (general openness to new practices), and Divergence (perceived divergence between research-based/academically developed interventions and current practice) and a total scale score representing respondents’ global attitude toward adopting and using EBP. Respondents indicated their level of agreement to each statement using the Likert-type scale described above. Reliability analyses for the EBPAS were consistent with prior research50 with a high overall alpha of α = 0.86 and subscale alphas of 0.87 (Appeal), 0.94 (Requirements), 0.84 (Openness), and 0.71 (Divergence). EBPAS validity is supported by associations with clinic structure and policies,34 organizational culture and climate,35 and leadership.18 Validity is supported by studies showing that EBPAS scores predict adoption and use of EBPs.51, 52 The total EBPAS mean score was utilized in the analyses.

Data Analytic Approach

Aggregation Analyses

Transformational leadership, leader-member exchange, and innovation climate are believed to operate at the team level (i.e., level 2) and attitudes toward EBP at the individual level (i.e., level 1).18, 34, 53 In order to examine these assumptions we computed awg, ICC(1) and ICC(2) for each construct. The awg inter-rater agreement statistic54, 55 was used to assess the degree to which members within each of the participating teams agreed in their responses to the leadership and innovation climate scales and to assess whether composing the individual-level responses to higher-level constructs was justified. In order to facilitate interpretability and comparability to other reliability/consistency measures we scaled the awg statistic with a range of 0 to 154 where 1 indicates perfect agreement and values of .70 indicate moderate agreement.55

Path Analyses

We conducted multigroup path analyses that adjusted for the nested data structure (i.e., providers nested in teams) to evaluate the research hypotheses. As shown in Figure 1, we proposed and tested a model in which the effects of transformational leadership on innovation climate are partially mediated through leader member exchange. Transformational leadership is anticipated to have direct effects on leader-member exchange and innovation climate, and indirect effects on innovation climate through leader member exchange. We also tested whether there was a significant effect of innovation climate on provider attitudes toward adopting EBP. Our primary interest was whether these associations differ during active implementation of an EBP vs. SAU. Specifically we expected that transformational leadership would be more likely to affect innovation climate during active implementation efforts. We conducted multigroup path analyses in which we compared the two conditions (EBPI vs. SAU) by examining the magnitude and significance of path coefficients between the two conditions. Next, we tested each path by estimating multigroup models, one in which the path of interest was constrained to be equal, compared to a model in which the paths were each freely estimated. The constrained and free models were compared using scaled chi-square difference test statistics.56

Figure 1.

Figure 1

Clustered Path Analysis: Association of Transformational Leadership and Leader-Member Exchange with Innovation climate and Innovation climate with Provider Attitudes Toward Evidence-Based Practice compared to SAU. Note: N=139; EBPI =evidence-based practice implementation (n=85)/SAU=services as usual (n=54); EBPI coefficients appear to left of slash/SAU coefficients appear to the right of slash; χ2 (4)=1.074; p=.90; CFI=1.000, TLI=1.000, RMSEA=0.000, SRMR=0.012; *p<.05, ***p<.001

We used maximum likelihood estimation with robust standard errors to account for the nested design (k=30 clusters). All models were estimated using the Mplus software.57 One SAU case was excluded due its missing data pattern. However, missing data among the remaining cases was low (96.3% covariance coverage) and we utilized full information maximum likelihood estimation for missing data to retain a sample size of 139 for the path analyses. In additional analyses (available by request) we estimated structural equation models that utilized a transformational leadership latent variable comprised of MLQ subscales. The subscale factor loadings were similar to one another and overall results were essentially identical to the path analysis approach. Thus, we report the more parsimonious path analysis. Indicators of model fit were assessed using the comparative fit index (CFI), Tucker-Lewis index (TLI), root mean square error of approximation (RMSEA), and the standardized root mean square residual (SRMR) where CFI and TLI values greater than .90, RMSEA values less than .10, and SRMR values less than .08 indicate acceptable model fit.5861

Results

Sample characteristics

Of the 140 providers, 85 were in the EBPI group and 55 were in the SAU group. Descriptive statistics and correlations are presented for the EBPI and SAU groups in Table 1. The bivariate correlations demonstrated generally similar patterns between the two groups, although the absolute strength of certain associations differed somewhat. This was particularly evident in the relationship between leader-member exchange and innovation climate (SAU r= .77, p< .001; EBPI r=.54, p < .001). Study hypotheses were examined in the multivariate framework of the path analysis.

Table 1.

Descriptive Statistics and Correlations

Variables Mean SD 1 2 3
1. Transformational leadership EBPI (n=85) 2.51 0.96
SAU (n=54) 2.68 0.89
2. Leader-Member Exchange EBPI (n=84) 2.74 0.98 .83***
SAU (n=52) 2.81 0.92 .88***
3. Innovation climate EBPI (n=85) 2.42 0.75 .65*** .54***
SAU (n=55) 2.58 1.06 .68*** .77***
4. Attitudes Towards EBP EBPI (n=85) 2.79 0.54 .20 .20 .29**
SAU (n=55) 2.74 0.56 .19 .15 .23

Notes: EBPI = Evidenced-based practice Implementation, SAU = Services as Usual; N(EBPI) = 85, N(SAU) =55;

**

p<.01,

***

p<.001

Aggregation analyses

Our analyses supported leadership and innovation climate as characteristics of the team. This held for transformational leadership (awg = .70, ICC(1) = .069, ICC(2) = .274), leader-member exchange (awg = .72, ICC(1) = .133, ICC(2) = .315), and innovation climate (awg = .79, ICC(1) = .075, ICC(2) = .302).

Multiple Group Path analysis

Figure 1 shows the path model results. Path coefficients for the EBPI group are to the left of the slash and SAU coefficients to the right of the slash. We found partial support for H1 in that transformational leadership was positively associated with innovation climate, for the EBPI group (.70, SE=.13, p < .001) but not for the SAU group (−.13, SE=.16, ns). The two groups differed significantly for the path coefficients from transformational leadership to innovation climate with the EBPI group having a significantly stronger effect (χ2 dif = 7.43, 1 df, p<.01).

As expected (H2), we identified a strong positive relationship between transformational leadership and leader-member exchange for both the EBPI and SAU groups (.83, SE=.05, p<.001 and .90, SE=.02, p<.001 respectively). The strength of the relationship did not differ between the two groups differ (χ2 dif = 0.60, 1 df).

Partial support was found for H3 as leader-member exchange was significantly associated with innovation climate only in the SAU group (.90, SE=.15, p < .001; EBPI = −.05, SE=.16, ns). The paths from leader-member exchange to innovation climate differed between the groups with a significantly stronger effect for the SAU group (χ2 dif = 28.97, 1 df, p<.001).

Next, H4 was partially supported in that leader-member exchange fully mediated the association of transformational leadership with innovation climate only for the SAU group. No mediation was found between transformational leadership and innovation climate by leader-member exchange for the EBPI group.

For H5 active implementation (i.e., EBPI condition) resulted in a stronger positive association between transformational leadership and innovation climate (EBPI = .70, p<.001, SAU = −.13; χ2 dif = 7.43, 1 df, p<.01). Conversely, for providers in a more stable environment (i.e., SAU condition) we found a stronger positive association between leader-member exchange and innovation climate (EBPI = −.05, SAU =.90, p<.001; χ2 dif = 28.97, 1 df, p<.001).

Finally, H6 was partially supported as only the EBPI group demonstrated a positive significant relationship between innovation climate and more positive attitudes toward EBP while the SAU group had a non-significant association between these variables (EBPI = .29, SE=.14, p< .05,; SAU = .20, SE=.17, ns). However, the coefficients for the pathway from innovation climate to attitudes toward EBP were not significantly different between the two groups (χ2 dif = 0.41, 1 df).

Model fit indices computed from the single overall multigroup path analysis model which simultaneously generated the separate path coefficients for the EBPI and SAU groups indicated excellent fit between the proposed model and the observed data (χ2 (4) = 1.074; p=.90; CFI=1.000, TLI=1.000, RMSEA=0.000, SRMR=0.012).

Discussion

We found that transformational leadership had a strong direct association with innovation climate, but only for the teams actively implementing SafeCare. For these teams, innovation climate was also associated with more positive staff attitudes toward adopting EBP. This suggests that transformational leadership may influence innovation climate and more positive attitudes toward adopting EBP during the active implementation phase.3 For SAU, the quality of the leader-provider relationship was important in supporting a positive innovation climate during more stable periods of service provision. This suggests that organizations and leaders should be cognizant of leadership approaches while at the same time considering their organization’s EBPI phase.

The strong positive association of transformational leadership with innovation climate for the EBPI (but not the SAU) group reinforces the importance of more direct effects of transformational leadership during active implementation of innovation and organizational change. This finding is highly congruent with research demonstrating that leader support for innovation implementation is important in improving organizational climate for implementation of innovation.62 It is also congruent with findings supporting the association of transformational leadership with attitudes toward EBP in mental health settings.18 The present study adds to the evidence that leadership is important in moving innovations into large public service systems and community-based service organizations.3, 63 Literature suggests that innovation climate and attitudes toward EBP can be considered “implementation outcomes” because they provide the context for more distal outcomes such as reach to the provider or service population, fidelity, and/or clinical change.36 Leadership and climate can facilitate improved implementation and can be improved through organizational intervention.63

It appears that context matters for EBPI. In the present study, teams in both conditions were performing essentially the same function (i.e., providing home visitation services to at-risk children and families), so it is not surprising that the innovation climate was more strongly related to attitudes toward EBP during active implementation for the EBPI group relative to SAU. Although the magnitude of differences of path coefficients was small, this suggests that it is during organizational change that transformational leadership should be bolstered to positively impact the implementation process, and ultimately, staff willingness to adopt EBP.

Mean differences on most measures between groups were small. In addition, bivariate correlations were similar across groups with some exceptions. It is only when the multivariate analyses were conducted that group differences became more apparent. Further studies are needed to verify the results found here. In particular, studies are needed to examine the impact of leader development strategies on innovation climate, staff attitudes toward adopting EBPs, and EBP implementation efficiency and sustainability.

Limitations

Some limitations of the present study should be noted. First, our data relied on self-report from providers in the service system. As such, common source bias could impact results.64 However, the experimental design allays this concern as providers were assigned to conditions and the key questions for the study relate to study condition (EBPI vs. SAU). Future studies would benefit from the inclusion of objective or observational measures of leadership. Second, the small sample size may have impacted our ability to find significant effects for the association of innovation climate with attitudes toward EBP for the SAU group. Future studies should examine these issues with larger sample sizes. Third, the high intercorrelation of transformational leadership and leader-member exchange suggests overlap of these two constructs. While there is overlap, the constructs still represent differing aspects of leadership: leadership behaviors on the one hand vs. the relationship of leaders with their subordinates on the other. Future research should examine these differences and determine if additional work on measure development is needed. Finally, variances on all measures across groups were not significantly different except for Innovation Climate (lower variance for EBPI condition). However, we would expect strong transformational leadership to be associated with greater congruence of responses within a subordinate group.

Clinical Implications

The implementation and use of EBPs holds promise to improve clinical practice. EBPs should integrate clinical expertise and judgment to meet the needs of clients and patients.1, 2 Improving leadership to improve subsequent team and provider buy-in, adoption, and use of EBPs should improve clinical outcomes. However, EBPI is not a simple undertaking and there are many issues facing clinicians in today’s mental health service context. For example, insurance reimbursements, productivity requirements, and quality assurance mandates are just a few of the concerns facing clinicians. However, more effective leadership should help organizations and providers to implement and sustain the most effective clinical practices.

Conclusions

Within health, mental health, and social service settings there is growing momentum to implement EBPs.6567 These efforts are only beginning to be informed by the considerable literature examining the development and implementation of innovation in business settings30 and the growing literature focusing on innovation implementation within service organizations in health, mental health, and social service settings.31 However, many challenges persist and there remains a need to examine and improve factors that can facilitate EBPI. Public sector mental health and social service settings often use a team structure; however the relationship of leadership and team perceptions and functioning continues to be understudied.68 Similarly, the study of organizational climates that support EBP adoption and implementation should be vigorously pursued. Within public sector service organizations, successful adoption and implementation of EBPs is highly dependent upon a number of factors32, 69, 70 but individual service providers are the primary vehicle through which knowledge and skills are applied to the needs of clients and consumers.71 Indeed, a comprehensive review of the research and resulting implementation models identified individual “adopters” (i.e., those directly involved in changing their service delivery methods) as critical players in the implementation process.31 Understanding how leadership impacts the values, beliefs, and motivation of individual providers to adopt EBPs is crucial for effective implementation. Of particular relevance are findings demonstrating that supervisor leadership18 and organizational culture and climate are also associated with more positive attitudes toward adopting EBPs.35

Future research should focus on identifying ways to improve leadership during organizational change. For example, managers could receive training and coaching in order to improve transformational and transactional leadership skills. In addition, leadership training could include an emphasis on creating a positive climate for implementation.72 Such approaches hold promise to improve the quality of care and outcomes of health and human services provided in the public sector.

Acknowledgments

This study was supported by National Institute of Mental Health grants R01MH072961, R21MH082731, and P30MH074678. The authors thank the community-based organizations, case-managers, and supervisors that made this study possible. Dr. Aarons was a co-editor for this special section on implementation research in children’s mental health. The authors declare no other conflicts of interest.

Contributor Information

Gregory A. Aarons, University of California, San Diego, Department of Psychiatry.

David H. Sommerfeld, Child and Adolescent Services Research Center at, Rady Children’s Hospital San Diego

Reference List

  • 1.Institute of Medicine. Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press; 2001. [PubMed] [Google Scholar]
  • 2.American Psychological Association. Report of the 2005 Presidential Tasks Force on evidence-based practice. American Psychological Association; 2005. [Google Scholar]
  • 3.Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in child welfare. Adm Policy Ment Health and Ment Health Serv Res. 2011;38(1):4–23. doi: 10.1007/s10488-010-0327-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Damanpour F, Schneider M. Phases of the adoption of innovation in organizations: Effects of environment, organization and top Managers. British Journal of Management. 2006;17(3):215–236. [Google Scholar]
  • 5.Jung DI, Chow C, Wu A. The role of transformational leadership in enhancing organizational innovation: Hypotheses and some preliminary findings. The Leadership Quarterly. 2003;14:525–544. [Google Scholar]
  • 6.Gumusluoglu L, Ilsev A. Transformational leadership, creativity, and organizational innovation. Journal of Business Research. 2009;62(4):461–473. [Google Scholar]
  • 7.Scott SG, Bruce RA. Determinants of innovative behavior: A path model of individual innovation in the workplace. Academy of Management Journal. 1994;37(3):580–607. [Google Scholar]
  • 8.Avolio BJ, Bass BM, Jung DI. Re-examining the components of transformational and transactional leadership using the Multifactor Leadership Questionnaire. Journal of Occupational and Organizational Psychology. 1999;72:441–462. [Google Scholar]
  • 9.Bass BM. Utility of managerial self-planning on a simulated production task with replications in twelve countries. Journal of Applied Psychology. 1977;62(4):506–509. [Google Scholar]
  • 10.Judge TA, Piccolo RF. Transformational and transactional leadership: A meta-analytic test of their relative validity. Journal of Applied Psychology. 2004;89(5):755–768. doi: 10.1037/0021-9010.89.5.755. [DOI] [PubMed] [Google Scholar]
  • 11.Bass BM. Leadership and performance beyond expectations. New York: Free Press; 1985. Leadership: Good, Better, Best. [Google Scholar]
  • 12.Bass B, Avolio B. MLQ: Multifactor leadership questionnaire (Technical Report) Binghamton University, NY: Center for Leadership Studies; 1995. [Google Scholar]
  • 13.Tichy N, Ulrich D. The leadership challenge: Call for the transformational leader. Sloan Management Review. 1984:59–69. [Google Scholar]
  • 14.Groves KS. Leader emotional expressivity, visionary leadership, and organizational change. Leadership and Organization Development Journal. 2006;27(7):565–582. [Google Scholar]
  • 15.Hinkin TR, Tracey JB. The relevance of charisma for transformational leadership in stable organizations. Journal of Organizational Change Management. 1999;12(2):105–119. [Google Scholar]
  • 16.Howell JM, Avolio BJ. Transformational leadership, transactional leadership, locus of control, and support for innovation: Key predictors of consolidated-business-unit performance. J Appl Psychol. 1993;78(6):891–902. [Google Scholar]
  • 17.Molero F, Cuadrado E, Navas M, Morales JF. Relations and effects of transformational leadership: A comparative analysis with traditional leadership styles. The Spanish Journal of Psychology. 2007;10(2):358–368. doi: 10.1017/s1138741600006624. [DOI] [PubMed] [Google Scholar]
  • 18.Aarons GA. Transformational and transactional leadership: Association with attitudes toward evidence-based practice. Psychiatric Services. 2006;57(8):1162–1169. doi: 10.1176/appi.ps.57.8.1162. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Corrigan PW, Diwan S, Campion J, Rashid F. Transformational leadership and the mental health team. Adm Policy Ment Health. 2002;30(2) doi: 10.1023/a:1022569617123. [DOI] [PubMed] [Google Scholar]
  • 20.Graen GB, Uhl-Bien M. Relationship-based approach to leadership: Development of leader-member exchange (LMX) theory of leadership over 25 years: Applying a multilevel multi-domain perspective. Leadership Quarterly Special Leadership: The multiple-level approaches (Part I) 1995;6(2):219–247. [Google Scholar]
  • 21.Horvath AOS, BD Relation between working alliance and outcome in psychotherapy: A meta-analysis. Journal of Counseling Psychology. 1991;38:139–149. [Google Scholar]
  • 22.Wang H, Law KS, Hackett RD, Wang D, Chen ZX. Leader-member exchange as a mediator of the relationship between transformational leadership and followers’ performance and organizational citizenship behavior. Acad Manage J. 2005;48(3):420–432. [Google Scholar]
  • 23.Yukl G, O’Donnell M, Taber T. Influence of leader behaviors on the leader-member exchange relationship. Journal of Managerial Psychology. 2009;24(4):289–299. [Google Scholar]
  • 24.Eggleston K, Bhagat R. Organizational contexts and contingent leadership roles: A theoretical exploration. Human Relations. 1993;46(10):1177–1191. [Google Scholar]
  • 25.Nadler DA, Tushman ML. Beyond the charasmatic leader: Leadership and organizational change. California Management Review. 1990;32(2):77–96. [Google Scholar]
  • 26.Groves KS. Linking leader skills, follower attitudes, and contextual variables via an integrated model of charismatic leadership. Journal of Management. 2005;31(2):255–277. [Google Scholar]
  • 27.Clegg C, Unsworth K, Epitropaki O, Parker G. Implicating trust in the innovation process. Journal of Occupational and Organizational Psychology. 2002;75:409–422. [Google Scholar]
  • 28.Proudfoot J, Jayasinghe U, Holton C, et al. Team climate for innovation: What difference does it make in general practice? International Journal for Quality in Health Care. 2007;19(3):164–169. doi: 10.1093/intqhc/mzm005. [DOI] [PubMed] [Google Scholar]
  • 29.Mathisen GE, Einarsen S, Jorstand K, Bronnick K. Climate for work group creativity and innovation: Norwegian validation of the team climate inventory (TCI) Scandinavian Journal of Psychology. 2004;45:383–392. doi: 10.1111/j.1467-9450.2004.00420.x. [DOI] [PubMed] [Google Scholar]
  • 30.Frambach RT, Schillewaert N. Organizational innovation adoption: A multi-level framework of determinants and opportunities for future research. Journal of Business Research. Special Issue: Marketing theory in the next millennium. 2002;55(2):163–176. [Google Scholar]
  • 31.Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Quarterly. 2004;82(4):581–629. doi: 10.1111/j.0887-378X.2004.00325.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Aarons GA, Sommerfeld DH, Walrath-Greene CM. Evidence-based practice implementation: The impact of public vs. private sector organization type on organizational support, provider attitudes, and adoption of evidence-based practice. Implement Sci. 2009;4(83):1–13. doi: 10.1186/1748-5908-4-83. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Rogers EM. Diffusion of innovations. 5. New York, NY: Free Press; 2003. [Google Scholar]
  • 34.Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS) Ment Health Serv Res. 2004;6(2):61–74. doi: 10.1023/b:mhsr.0000024351.12294.65. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Aarons GA, Sawitzky A. Organizational culture and climate and mental health provider attitudes toward evidence-based practice. Psychological Services. 2006;3(1):61–72. doi: 10.1037/1541-1559.3.1.61. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research questions. Adm Policy Ment Health and Ment Health Serv Res. 2011;38(2):65–76. doi: 10.1007/s10488-010-0319-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Chaffin M, Hecht D, Bard D, Silovsky J, Beasley WH. A statewide trial of the SafeCare home-based services model with parents in Child Protective Services. Pediatrics. doi: 10.1542/peds.2011-1840. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Aarons GA, Sommerfeld DH, Hecht DB, Silovsky JF, Chaffin MJ. The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: Evidence for a protective effect. J Consult Clin Psychol. 2009;77(2):270–280. doi: 10.1037/a0013223. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Aarons GA, Fettes DL, Flores LE, Sommerfeld DH. Evidence-based practice implementation and staff emotional exhaustion in children’s services. Behav Res Ther. 2009;47(11):954–960. doi: 10.1016/j.brat.2009.07.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Lutzker JR, Bigelow KM. Reducing child maltreatment a guidebook for parent services. New York: Guilford Press; 2002. [Google Scholar]
  • 41.Aarons GA, Fettes DL, Sommerfeld DH, Palinkas LA. Mixed-methods for implementation research: Application to understanding staff turnover. doi: 10.1177/1077559511426908. In review. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Bass BM, Avolio BJ. MLQ: Multifactor leadership questionnaire (Technical Report) Binghamton University, NY: Center for Leadership Studies; 1995. [Google Scholar]
  • 43.Erdogan B, Kraimer ML, Liden RC. Work value congruence and intrinsic career success: The compensatory roles of leader-member exchange and perceived organizational support. Personnel Psychology. 2004;57(2):305–332. [Google Scholar]
  • 44.Harris KJ, Kacmar KM, Witt LA. An examination of the curvilinear relationship between leader-member exchange and intent to turnover. Journal of Organizational Behavior. 2005;26(4):363–378. [Google Scholar]
  • 45.Janssen O, Van Yperen NW. Employees’ goal orientations, the quality of leader-member exchange, and the outcomes of job performance and job satisfaction. Academy of Management Journal. 2004;47(3):368–384. [Google Scholar]
  • 46.Gerstner CR, Day DV. Meta-analytic review of leader-member exchange theory: Correlates and construct issues. Journal of Applied Psychology. 1997;82(6):827–844. [Google Scholar]
  • 47.Anderson NR, West MA. Measuring climate for work group innovation: Development and validation of the Team Climate Inventory. Journal of Organizational Behavior. 1998;19(3):235–258. [Google Scholar]
  • 48.Bain PG, Mann L, Pirola-Merlo A. The innovative imperative: The relationships between team climate, innovation, and performance in research and development teams. Small Group Research. 2001;32:55–73. [Google Scholar]
  • 49.Aarons GA, Glisson C, Hoagwood K, Kelleher K, Landsverk J, Cafri G. Psychometric properties and United States national norms of the Evidence-Based Practice Attitude Scale (EBPAS) Psychological Assessment. 2010;22(2):356–365. doi: 10.1037/a0019188. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Aarons GA, McDonald EJ, Sheehan AK, Walrath-Greene CM. Confirmatory factor analysis of the Evidence-Based Practice Attitude Scale (EBPAS) in a geographically diverse sample of community mental health providers. Administration and Policy in Mental Health and Mental Health Services Research. 2007;34:465–469. doi: 10.1007/s10488-007-0127-x. [DOI] [PubMed] [Google Scholar]
  • 51.Henggeler SW, Chapman JE, Rowland MD, et al. Statewide adoption and initial implementation of contingency management for substance abusing adolescents. J Consult Clin Psychol. 2008;76(4):556–567. doi: 10.1037/0022-006X.76.4.556. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Smith BD, Manfredo IT. Frontline counselors in organizational contexts: A study of treatment practices in community settings. J Subst Abuse Treat. 2011;41(2):124–136. doi: 10.1016/j.jsat.2011.03.002. [DOI] [PubMed] [Google Scholar]
  • 53.Aarons GA, Glisson CA, Green PD, et al. Association of clinician attitudes toward evidence-based practice with the organizational social context of mental health services: A national study. 2011 doi: 10.1186/1748-5908-7-56. Manuscript submitted for publication. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Brown RD. Interrater agreement reconsidered: The role of maximum possible variance. 15th Annual Conference of the Society for Industrial and Organizational Psychology; New Orleans, LA. 2000. [Google Scholar]
  • 55.Brown RD, Hauenstein NM. Interrater agreement reconsidered: An Alternative to the rwg indices. Organizational Research Methods 2005. 2008;8:165. [Google Scholar]
  • 56.Satorra A, Bentler PM. A scaled difference chi-square test statistic for moment structure analysis. Psychometrika. 2001;66(4):507–514. doi: 10.1007/s11336-009-9135-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Muthén LK, Muthén BO. Mplus user’s guide. 3. Los Angeles: Author; pp. 1998–2008. [Google Scholar]
  • 58.Dunn G, Everitt B, Pickles A. Modelling covariances and latent variables using EQS. London and New York: Chapman and Hall; 1993. [Google Scholar]
  • 59.Hu L, Bentler PM. Fit indices in covariance structure modeling: sensitivity to underparameterized model misspecification. Psychological Methods. 1998;3:424–453. [Google Scholar]
  • 60.Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling. 1999;6(1):1–55. [Google Scholar]
  • 61.Kelloway EK. Using Lisrel for Structural Equation Modeling: A Researcher’s Guide. Thousand Oaks, CA: Sage Publications; 1998. [Google Scholar]
  • 62.Klein KJ, Conn AB, Sorra JS. Implementing computerized technology: An organizational analysis. Journal of Applied Psychology. 2001;86(5):811–824. doi: 10.1037/0021-9010.86.5.811. [DOI] [PubMed] [Google Scholar]
  • 63.Glisson C, Schoenwald SK, Hemmelgarn A, et al. Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. J Consult Clin Psychol. 2010;78(4):537–550. doi: 10.1037/a0019160. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Podsakoff PM, MacKenzie SB, Podsakoff NP. Common method biases in behavioral research: A critical review of the literature and recommended remedies. J Appl Psychol. 2003;88(5):879–903. doi: 10.1037/0021-9010.88.5.879. [DOI] [PubMed] [Google Scholar]
  • 65.Goldman HH, Ganju V, Drake RE, et al. Policy implications for implementing evidence-based practices. Psychiatric Services. 2001;52(12):1591–1597. doi: 10.1176/appi.ps.52.12.1591. [DOI] [PubMed] [Google Scholar]
  • 66.Ringeisen H, Hoagwood K. Clinical research directions for the treatment and delivery of children’s mental health services. In: Marsh DT, Fristad MA, editors. Handbook of serious emotional disturbance in children and adolescents. New York: John Wiley & Sons; 2002. pp. 33–55. [Google Scholar]
  • 67.Schoenwald SK, Hoagwood K. Effectiveness, transportability, and dissemination of interventions: What matters when? Psychiatric Services. 2001;52(9):1190–1197. doi: 10.1176/appi.ps.52.9.1190. [DOI] [PubMed] [Google Scholar]
  • 68.Dionne SD, Yammarino FJ, Atwater LE, Spangler WD. Transformational leadership and team performance. Journal of Organizational Change Management. 2004;17(2):177–193. [Google Scholar]
  • 69.Green AG, Aarons GA. A comparison of policy and direct practice stakeholder perceptions of factors affecting evidence-based practice implementation using concept mapping. Implement Sci. 2011;6:104. doi: 10.1186/1748-5908-6-104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Aarons GA, Wells RS, Zagursky K, Fettes DL, Palinkas LA. Implementing evidence-based practice in community mental health agencies: A multiple stakeholder analysis. American Journal of Public Health. 2009;99(11):2087–2095. doi: 10.2105/AJPH.2009.161711. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Hasenfeld Y. Human services as complex organizations. Newbury Park, CA: Sage Publications; 1992. [Google Scholar]
  • 72.Klein KJ, Sorra JS. The challenge of innovation implementation. Academy of Management Review. 1996;21(4):1055–1080. [Google Scholar]

RESOURCES