Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Sep 1.
Published in final edited form as: Adm Policy Ment Health. 2016 Sep;43(5):629–639. doi: 10.1007/s10488-015-0670-9

Leadership, Organizational Climate, and Perceived Burden of Evidence-Based Practice in Mental Health Services

Kim C Brimhall 1, Karissa Fenwick 1, Lauren R Farahnak 2,3, Michael S Hurlburt 1,3, Scott C Roesch 4, Gregory A Aarons 2,3,
PMCID: PMC4706502  NIHMSID: NIHMS706340  PMID: 26152770

Abstract

The use of evidence-based practices (EBPs) is associated with favorable client outcomes, yet perceived burden of using EBPs may affect the adoption and implementation of such practices. Multilevel path analysis was used to examine the associations of transformational leadership with organizational climate, and their associations with perceived burden of using EBPs. Results indicated significant relationships between transformational leadership and empowering and demoralizing climates, and between demoralizing climate and perceived burden of EBPs. We found significant indirect associations of leadership and perceived burden through organizational climate. Findings suggest that further research is needed to examine the extent to which improving leadership and organizational climate may reduce perceived burden and use of EBPs with the ultimate goal of enhancing quality of care.

Keywords: Evidence-based practice, Transformational leadership, Organizational climate, Attitudes


Improving the quality of mental health services is of vital importance in the United States and abroad (Aarons et al. 2012; Beidas and Kendall 2014). The American Academy of Child & Adolescent Psychiatry and the American Psychological Association recommend and support the use of evidence-based practices (EBPs) as an important approach to improving mental health services (American Academy of Child & Adolescent Psychiatry 2006; American Psychological Association 2005; Beidas and Kendall 2014). Although considerable funding and resources have been dedicated to encouraging the use of EBPs (Aarons et al. 2012; Bruns et al. 2008; Cashin et al. 2008; Glasgow et al. 2012; McHugh and Barlow 2010), substantial barriers to dissemination and implementation remain (Beidas and Kendall 2010, 2014). At the organizational level, these barriers may include insufficient resources, inadequate leadership support, and provider beliefs, perceptions, and attitudes about EBPs (Aarons and Palinkas 2007; Shapiro et al. 2011; Torrey et al. 2012).

Research has demonstrated that provider (i.e., clinician and case-manager) attitudes toward EBPs are associated with the implementation process and the uptake and use of EBPs (Aarons et al. 2007, 2010; Henggeler et al. 2008). Already faced with the demands of heavy caseloads, documentation requirements, and crisis management, community mental health providers who perceive EBPs as an added burden may be less likely to adopt, implement, and sustain EBPs (Aarons et al. 2009), especially in the early phases of implementation where there is greater emphasis on the learning and practice of new knowledge and skills. Factors that can reduce provider perceptions of EBP-related burden may increase the likelihood that attitudes toward EBPs are improved, and that adoption and implementation of EBPs is facilitated and ultimately, the quality of care is improved. Organizational factors such as leadership and climate have previously been shown to be associated with provider attitudes toward EBPs in mental health and system-wide behavioral health transformation efforts (Aarons 2006; Aarons et al. 2011). Because leadership is associated with employee perceptions of the work environment and job performance, it is likely to have a role in influencing provider perceptions of organizational context, and organizational initiatives and job tasks such as the use of EBPs.

Conceptual Model

Transformational leadership can be described as the leader’s ability to recognize the unique talents of individual employees through individualized consideration, encourage new ways of thinking and solving problems through intellectual stimulation, create a shared sense of purpose among employees through inspirational motivation, and provide role modeling in a way that sets a positive example through idealized influence (Aarons et al. 2011; Bass and Avolio 1995). To explore the relationships of leadership and climate on perceived burden, we built on previous work that found positive effects of transformational leadership on organizational outcomes (Bass and Avolio 1995). For example, research has found transformational leadership to moderate associations of empowering and demoralizing dimensions of workplace climate (i.e., positive and/or negative shared employee perceptions of and emotional responses to the work environment) and subsequent behavioral health system employee outcomes during a statewide behavioral health system reform in New Mexico (Aarons et al. 2011). In the present study we extended this work to examine the direct and indirect associations of transformational leadership with staff perceptions of burden related to EBP use. Figure 1 illustrates the conceptual model for the current study.

Fig. 1.

Fig. 1

Conceptual model of leadership, climate, and perceived burden of evidence-based practices. Level 2 = team level variables (i.e., transformational leadership, empowering climate and demoralizing climate); Level 1 = individual level variable (i.e., provider-perceived burden of EBPs)

Leadership and Climate

Extensive research exists on the positive outcomes of transformational leadership (Krishnan 2012; Wang et al. 2011). A recent meta-analysis found transformational leadership to be associated with improved employee work attitudes and aspects of the work environment including job satisfaction, organizational commitment, employee motivation, work effort, and overall team performance (Wang et al. 2011). Although research has been limited, there is empirical evidence that transformational leadership is associated with provider attitudes toward EBPs (Aarons 2006). Given the positive associations of transformational leadership on employee work life, we hypothesized that similar associations would be found for perceived burden.

Hypothesis 1 Greater transformational leadership will be associated with lower perceived burden of using EBPs.

In addition to the associations of transformational leadership with various work outcomes (e.g., provider attitudes, job satisfaction, organizational commitment; Wang et al. 2011), some research has suggested that transformational leadership may affect work outcomes through its influence on organizational climate (Aarons and Sawitzky 2006; Aarons et al. 2011; Reichers and Schneider 1990; Zohar 1980; Zohar and Luria 2010). Previous research from a number of sources supports construing the work environment as being empowering (i.e., positive) or demoralizing (i.e., negative). In general, an empowering climate is one where employees feel fairly rewarded and recognized for their work efforts (Mok and Au-Yeung 2002) while also reporting equal access to information (Laschinger et al. 2001; Spreitzer 1996). This is represented by the fairness dimension of an empowering climate as described by Aarons et al. (2011). The second aspect of empowering climate is evident when employees perceive that there are ample opportunities to learn and grow in their job and work (Fisher 2014; Laschinger et al. 2001; Seibert et al. 2004) and this dimension is assessed as growth and advancement. The third dimension of empowering climate is that of role clarity where organizational goals and work-related expectations are clearly defined (Mok and Au-Yeung 2002; Seibert et al. 2004; Spreitzer 1996). Thus, for this study we define empowering workplace climate as a function of high scores on fairness, perceived opportunities for growth and advancement, and role clarity. These three dimensions indicate a climate that supports and empowers employees to be engaged in their daily work.

A demoralizing work environment is also assessed along three dimensions and has been described as an organizational climate where employees perceive that they are treated as impersonal objects, are exploited by management, and feel significant distrust (Bastien et al. 1995). The first dimension is depersonalization where employees feel removed from their work and those they serve. The second aspect of demoralizing climate is emotional exhaustion where employees feel emotionally and physically drained, as well as frustrated and tired from their jobs (Warren-Gross 2009; Siu et al. 1999). The third aspect of demoralizing climate is role conflict where employees feel ambiguity in job expectations, there are unclear policies and strategic plans, and confusion between what management communicates should be done and what is actually done in day-to-day work (Bastien et al. 1995; Donnelly 1994). These three dimensions represent a challenging climate in which it is difficult for employees to function effectively. Thus, we define demoralizing climate as being indicated by high levels of depersonalization, emotional exhaustion, and role conflict (Aarons et al. 2011; Glisson and Hemmelgarn 1998; Maslach et al. 2001).

More positive leadership has been shown to be associated with more positive or less negative organizational climates (Schneider et al. 2013). More specifically, transformational leadership has been shown to be associated with higher levels of empowering climate and lower levels of demoralizing climate (Aarons et al. 2011). Therefore, we hypothesized that transformational leadership would be positively associated with empowering climate, and negatively associated with demoralizing climate.

Hypothesis 2a Transformational leadership will be positively associated with empowering climate scores.

Hypothesis 2b Transformational leadership will be negatively associated with demoralizing climate scores.

Evidence indicates that the organizational climate can influence employee attitudes and subsequent employee behavior and outcomes (Glisson et al. 2008a; Schneider et al. 2013). For example, high service climate (i.e., a work environment that focused on customer service) was associated with more service oriented employee behavior, which ultimately resulted in higher organizational financial performance and improved customer satisfaction (Schneider et al. 2013). Likewise, positive organizational climates have been linked with improved employee work attitudes, such as job satisfaction and organizational commitment, which have then been linked to improved employee performance (Glisson and Green 2011). Therefore, based on past research that has demonstrated positive associations of leadership with more positive organizational climates and subsequent employee outcomes (Aarons and Sawitzky 2006; Aarons et al. 2011; Reichers and Schneider 1990; Zohar 1980; Zohar and Luria 2010), the current study further hypothesized that transformational leadership would have indirect associations on perceived burden of EBPs mediated through the work environment.

Hypothesis 3a Transformational leadership will be associated with lower perceived burden of EBPs through higher empowering climate scores.

Hypothesis 3b Transformational leadership will be associated with lower perceived burden of EBPs through lower demoralizing climate scores.

Exploring both the direct and indirect associations of transformational leadership with the perceived burden of EBPs contributes to the existing literature by identifying another potential benefit of transformational leadership in the EBP implementation process. In addition, examining how organizational dynamics are associated with provider attitudes will draw attention to the often-neglected role of the organization in implementation processes.

Methods

Participants

Participants were recruited from public sector mental health clinics in San Diego County, California. Based on administrative data, 99 teams within mental health programs were initially identified. Of the 99 teams, 26 were considered ineligible because they provided residential treatment and/or information was not provided to identify their supervisor and thus could not be nested within teams for analysis (i.e., a single supervisor for the team was not identified). In addition, one team was excluded due to nonresponse despite repeated contact attempts, resulting in a total of 72 eligible and responsive teams. Of the 72 eligible teams, 68 provided child, adolescent, and/or family outpatient services and agreed to participate (89 % response rate). We defined teams as groups of direct service providers who: a) shared the same primary work supervisor; and b) regularly interacted with one another in order to accomplish work objectives. All participating teams had at least one team leader or supervisor. Of the 440 eligible staff providers on those teams, 435 agreed to participate (98.9 % response rate). Administrative staff members (n = 15) were excluded from the study because they did not directly use EBPs. Data from 57 individual participants were excluded due to participants not providing information sufficient to identify their assigned work team (necessary for appropriate aggregation), resulting in a final sample of 363 service providers from 68 teams. Average team size was approximately 5.34 (SD = 3.4) providers.

Procedure

This study was approved by the appropriate institutional review boards and informed consent was obtained prior to survey administration. Program managers and/or team leaders were contacted by telephone or email by the corresponding author or a research associate. The study was described in detail and any questions were answered. In some cases in-person meetings were held to further describe the study and answer any questions. Once permission was obtained at the program level, research staff contacted team supervisors (in some cases the same person as the program manager) and in-person data collection meetings were scheduled during a regular team meeting. Trained research assistants administered the survey in paper format to participants in organizational meetings at each of the program locations. Supervisors and participants completed their questionnaires in separate rooms. The survey took approximately 60 min on average (range = 45–180 min). If participants did not finish during the allotted time research staff members and the participant would agree on a designated time (usually a week later) that the research assistant would return to collect completed surveys. This data collection was not supported by specific grant funds and there was no fiscal support for participant payments. However, all teams were provided light refreshments and feedback in consideration of their participation (Blinded for review).

Measures

Leadership

The Multifactor Leadership Questionnaire (MLQ; Bass and Avolio 1995) form 45× was used to assess transformational leadership. The 45× is a short form of the MLQ and was selected to help reduce the burden of completing surveys for the study participants. The MLQ asks respondents to indicate the extent to which their supervisor engages in specific leadership behaviors. Consistent with past research (Bass and Avolio 1997; Kearney and Gebert 2009), we used the following four domains to represent transformational leadership: (1) individualized consideration (e.g., “Treats you as an individual rather than just as a member of the group”); (2) intellectual stimulation (e.g., “Seeks differing perspectives when solving problems”); (3) inspirational motivation (e.g., “Expresses confidence that goals will be achieved”); and (4) idealized influence (e.g., “Emphasizes the importance of having a collective sense of mission”). Response options ranged from 0 (not at all) to 4 (to a very great extent) on a 5-point Likert scale. Previous research supports the internal consistency reliability for each domain (Aarons et al. 2011). The current study found good to excellent Cronbach’s alphas for all MLQ subscales including individualized consideration (α = 0.85), intellectual stimulation (α = 0.81), inspirational motivation (α = 0.90), idealized influence (α = 0.85), and the total scale (α = .95).

Empowering Climate and Demoralizing Climate

Items from the Children’s Services Survey (Glisson and Hemmelgarn 1998) were used to measure both empowering and demoralizing climate. Response options for each item ranged from 0 (not at all) to 4 (to a very great extent) on a 5-point Likert scale. Following previous research done by Aarons et al. (2011), empowering climate is composed of three subscales: (1) fairness (e.g., “The salary I receive is fair in light of my job performance and responsibilities”); (2) growth and advancement (e.g., “This agency rewards experience, determination, and hard work”); and (3) role clarity (e.g., “I know what people in my agency expect of me”). In the current study, Cronbach’s alphas for each domain were as follows: fairness (α = 0.63); growth and advancement (α = 0.81); and role clarity (α = 0.86). Although the current Cronbach’s alpha for the fairness domain was higher than previously found in other studies, it was slightly lower than the other domains for empowering climate. This may be due to the fact that the fairness domain was the only domain with reverse-scored items (3 out of 6 items were reverse-scored). Although all items were correctly reverse-scored, reverse-scored items may result in lower reliability estimates (Carlson et al. 2011; Conrad et al. 2004; Rodebaugh et al. 2007). The overall Cronbach’s alpha for empowering climate was 0.71.

Demoralizing climate also has three domains: (1) depersonalization (e.g., “I treat some of the children I serve as ‘impersonal objects”’); (2) emotional exhaustion (e.g., “I feel like I’m at the end of my rope”); and (3) role conflict (e.g., “How often do you have to bend a rule in order to carry out an assignment?”). In the current study, Cronbach’s alphas for each domain were as follows: depersonalization (α = 0.81); emotional exhaustion (α = 0.91); and role conflict (α = 0.88). The overall Cronbach’s alpha for demoralizing climate was 0.85.

Perceived Burden

All four items from the Evidence Based Practice Attitude Scale 50 (EBPAS-50; Aarons et al. 2012) “Burden” sub-scale were used to measure perceived burden. Perceived burden represents employee perceptions of the time and administrative burden of learning and using EBPs. Participants were asked to indicate the extent to which they agreed with each statement, and response options ranged from 0 (not at all) to 4 (to a very great extent) on a 5-point Likert scale. For example, respondents were asked to respond to: “I don’t know how to fit EBP into my administrative work.” Previous research found that all four items represented one factor, with loadings ranging from .51 to .72 and an overall Cronbach’s alpha of 0.76 (Aarons et al. 2012). The current study found a Cronbach’s alpha for perceived burden of EBPs of 0.76.

Statistical Procedures

To investigate the direct and indirect effects of leadership on employee-perceived burden through both empowering and demoralizing climates, a cross-level multigroup path analysis was conducted using Mplus 6.1. Missing data were minimal and patterns were explored prior to analysis. Transformational leadership had approximately 98 % (n = 354) of values present whereas 2 % (n = 9) were missing; empowering climate had approximately 86 % (n = 313) of values present, whereas 14 % (n = 50) were missing; demoralizing climate had approximately 88 % (n = 318) of values present, whereas 12 % (n = 45) were missing; and perceived burden of EBPs had approximately 94 % (n = 343) of values present, whereas 6 % (n = 20) were missing. Maximum likelihood estimation has been a recommended method to handle missing data due to its ability to use all available data to provide optimal model parameter estimation (McArdle 2013). In order to use maximum likelihood estimation, data must be considered missing at random (Schafer and Graham 2002; Scheffer 2002). Data can be reasonably considered missing at random if missingness in the outcome variable does not depend on the outcome itself (Rubin 1996). A logistic regression was examined that predicted missingness in the outcome including all study variables as predictors (McArdle 2013). Results were nonsignificant and therefore full information maximum likelihood estimation was used to handle missing data. The strength of the indirect association of transformational leadership and burden of EBPs through demoralizing and empowering climate were tested using PRODCLIN 2 (Tofighi and MacKinnon 2011), which corrects for the non-normal distribution of the indirect effect. This program provides a confidence interval for the indirect effect, which allows significance testing of a multilevel mediated relationship. Significance is indicated where the confidence interval does not contain 0.

Regarding the multilevel nature of the analyses, transformational leadership and organizational climate (i.e., empowering and demoralizing dimensions) were hypothesized to operate at the team level (i.e., level 2), influencing perceived burden of using EBPs at the individual level (i.e., level 1). To examine these assumptions and determine if the data supported team-level aggregation, we computed intraclass correlation coefficients (ICCs; Shrout and Fleiss 1979) and the average correlation within groups (awg; Brown 2000; Brown and Hauenstein 2005) for transformational leadership and dimensions of organizational climate. ICCs indicate the amount of dependency among observations within groups and awg assesses the degree to which providers within each team agree in their responses to the leadership and organizational climate scales. To ease interpretability, we scaled the awg statistic using a range of 0–1, with 1 indicating perfect agreement and .70 indicating moderate agreement (Brown 2000). Higher levels of agreement suggest that aggregation of individual-level responses (i.e., level 1) to higher-level constructs (i.e., level 2) is justified (Brown and Hauenstein 2005).

Results

Sample

Table 1 shows the sample descriptive statistics for categorical variables and Table 2 shows means and standard deviations for continuous variables. The greatest proportion of study participants self-reported as Caucasian, followed by Hispanic or Latino. Most members of the sample were women and most of the providers reported being in a registered intern position (i.e., currently earning licensing hours towards their psychology degree in the state of California). Average caseload size per provider was approximately 15 clients (SD = 13.24). In terms of education, the largest proportion of participants had obtained a master’s degree, followed by those with a bachelor’s degree. The average age of study participants was approximately 36 years (SD = 10.44), with an average of approximately 2.5 years of job tenure with the organization. The mean score for transformational leadership was 2.41 (SD = 0.86) on a scale of 0 to 4, with 4 indicating high transformational leadership. On the same 0–4 scaling with higher scores representing higher levels of climate, the average score for empowering climate was 2.13 (SD = 0.61), whereas demoralizing climate was relatively low with a mean score of 1.06 (SD = 0.70). Likewise, provider-perceived burden of using EBPs appeared to be low for this sample (M = 0.99, SD = 0.82).

Table 1.

Descriptive statistics for categorical variables

Variable n %
Race and ethnicity
 African American 27 7.50
 Asian 16 4.44
 Hispanic or Latino 85 23.61
 Native American 1 0.28
 Caucasian 188 52.22
 Other 43 11.94
Gender
 Female 293 81.16
 Male 68 18.84
Position in organization
 Student intern 27 7.58
 Registered intern 157 44.10
 Licensed provider 86 24.16
 Unlicensed provider 59 16.57
 Other 27 7.58
Education
 Some high school 1 0.28
 High school graduate 2 0.55
 Some college 9 2.49
 Associate degree 7 1.94
 Bachelor’s degree 49 13.57
 Some graduate education 23 6.37
 Master’s degree 240 66.48
 PhD or MD 23 6.37
Other 7 1.94

N = 363

Table 2.

Mean, standard deviation, and range for continuous variables

Variable M (SD) Range
Age 35.64 (10.44) 21–66
Job tenure (years) 2.42 (3.76) 0–29
Caseload 15.24 (13.12) 0–80
Leadership 2.41 (0.86) 0–4
Empowering climate 2.13 (0.61) 0–4
Demoralizing climate 1.06 (0.70) 0–4
Staff-perceived burden 0.99 (0.82) 0–4

N = 363

Aggregation Analyses

ICC measured the extent to which responses from providers in the same team were influenced or clustered by team (Bliese 2000; Kenny and Judd 1996) while awg indicated the extent to which providers in the same team agreed with each other in their responses to transformational leadership and climate survey questions (Brown 2000). The ICC for transformational leadership (.27) indicated a moderate degree of dependency among provider responses within teams, whereas ICCs for empowering climate (.15) and demoralizing climate (.15) indicated a lower degree of dependency. The average awg(J) value was .77 (range = .65–.91) for transformational leadership; .77 (range = .66–.90) for empowering climate; and .74 (range = .62–.93) for demoralizing climate. Together, the results of these analyses indicated that provider perceptions of transformational leadership, empowering climate, and demoralizing climate were sufficiently shared to be treated as team-level constructs in this study.

Direct Associations

Figure 2 shows the standardized direct associations of the multilevel path analytic model. Contrary to hypothesis 1, we did not find a significant direct association between transformational leadership and perceived burden. As hypothesized (2a) greater transformational leadership was associated with higher scores on empowering organizational climate (β = .37, t = 6.47, p < .001). Also as hypothesized (2b), greater transformational leadership was associated with lower scores on demoralizing organizational climate (β = −.26, t = −2.70, p <.01). Finally, higher scores on demoralizing climate were associated with higher burden scores (β = .55, t = 4.48, p < .001), however the path from empowering climate to perceived burden was not significant.

Fig. 2.

Fig. 2

Multilevel model of leadership, climate, and perceived burden of evidence-based practices. N = 363; Path coefficients are standardized effects; *p < .01, **p < .001; Level 2 = team level variables (i.e., transformational leadership, empowering climate and demoralizing climate); Level 1 = individual level variable (i.e., provider-perceived burden of EBPs)

Indirect Associations

Table 3 shows the estimates of the direct association of transformational leadership with perceived burden and its indirect associations mediated through empowering and demoralizing dimensions of organizational climate. Results revealed that transformational leadership had significant indirect association with perceived burden through both empowering (β = −.185, SE = .062, 95 % CI −.317, −.075) and demoralizing climate (β = −.143, SE = .063, 95 % CI −.280, −.035). In other words, as levels of transformational leadership increased, perceived burden scores decreased through higher empowering climate scores and lower demoralizing climate scores. Although the direct association of empowering climate and perceived burden was not significant in the path analysis, indirect associations of leadership and perceived burden through empowering climate were supported.

Table 3.

Standardized direct, indirect, and total effect estimates for perceived burden

Predictor Direct
Indirect
β SE β SE
Empowering climate −.05 .14 −.19* .06
Demoralizing climate .55* .12 −.14* .06
Leadership .11 .08 N/A

N/A not available

*

p < .001

Discussion

Our findings demonstrate that higher levels of transformational leadership are associated with lower perceived EBP burden and that this association is mediated by demoralizing workplace climate. Rather than directly affecting perceived burden (Hypothesis 1), transformational leadership was associated with lower perceived burden through both higher empowering climate scale scores (Hypotheses 2a and 3a) and lower demoralizing climate scale scores (Hypotheses 2b and 3b). Although the direct association between empowering climate and provider-perceived burden was not significant in the path analysis, results from the mediation analysis indicated significant indirect associations from transformational leadership to perceived burden through empowering climate. While this result may appear at odds with traditional assumptions regarding requirements for determining mediation, more current analytic approaches (as applied in this study) demonstrate that this was a significant indirect association (Hayes 2009). This supports the conclusion that more positive transformational leadership may be associated with lower levels of provider-perceived EBP burden through empowering organizational climate. Further research is needed to determine if and how changing leadership behaviors may change workplace climate and subsequent provider-perceived burden of EBPs, either directly or indirectly.

Transformational leadership may also influence other aspects of provider-perceived burden of EBPs. The way in which leaders introduce and frame EBPs, including efforts to mitigate negative work consequences associated with the use of EBPs, likely has an impact on whether providers initially agree to explore the use of EBPs. Although the current study focused on only one aspect of how providers experience burden associated with EBPs, these findings suggest that transformational leadership may reduce perceived burden of EBP implementation and use through influencing workplace climate.

Mental health organizations hoping to improve the implementation of EBPs by improving provider attitudes toward EBPs should consider interventions that focus on transformational leadership and implementation leadership. For example, interventions could utilize 360 degree assessments and personal development plans to foster and improve transformational and other types of strategic leadership (Aarons et al. 2015; Arthur and Hardy 2014). Transformational leadership consists of several aspects, each of which is considered to be malleable. For example, individualized consideration can be developed by encouraging and coaching leaders to take time to listen to and be responsive to each employee’s needs (Bass 1999; Bass and Avolio 1995). This is applicable to EBP implementation. Leaders who listen to concerns voiced by providers using EBPs and offer individualized feedback on ways to improve the use of EBPs (i.e., individualized consideration) communicate the value of provider opinions and the importance of using EBPs (Aarons et al. 2014).

Transformational leadership is one of the most widely studied and supported leadership approaches (Aarons et al. 2011; Bass 1999; Bass and Riggio 2006; Kearney and Gebert 2009; Wang et al. 2011), and there are several existing workshops and trainings designed to help leaders of organizations embrace this approach (Arthur and Hardy 2014; Barling et al. 1996; Kirkbride 2006; Wang et al. 2011). Although many leadership development approaches are not empirically tested and fall short of improving leadership, some recent approaches have documented positive change in implementation leadership (Aarons et al. 2015) and transformational leadership (Arthur and Hardy 2014). The current study provided further evidence regarding the role transformational leadership can play in the context of mental health organizations striving to improve workplace climate and perceived burden of using EBPs. The present work also suggests how leaders and organizations might intervene in order to reduce the perceived burden of EBP through leadership and organizational development activities (Aarons et al. 2015).

Strengths and Limitations

Extensive research has demonstrated the positive influences of transformational leadership on various aspects of the work environment (Wang et al. 2011). For example, transformational leadership has been associated with improved organizational retention, job satisfaction, commitment, overall performance, employee motivation, and work effort (Krishnan 2012; Wang et al. 2011). Transformational leadership has also been associated with improving aspects of organizational climate (Aarons and Sawitzky 2006; Aarons et al. 2011). Previous studies have found associations between transformational leadership and attitudes toward EBPs (Aarons 2006) and transformational leadership and organizational climate (Aarons et al. 2011; Sarros et al. 2008). The current study contributes to the existing literature by affirming associations between transformational leadership and organizational climate, and extends previous research by examining employees’ sense of burden associated with the use of EBPs; an aspect of employee attitudes not explored in previous studies. Future research using prospective designs is needed to examine the causal relationships between leadership, workplace climate, and perceived burden of EBPs.

The current study analyzed a large number of organizational teams to increase our understanding of the influence of leadership on attitudes toward EBPs; however it was not without limitations. The measurement of demoralizing climate and perceived burden may contain some overlap, introducing the risk of collinearity (Tabachnick and Fidell 2013) and criterion contamination (Darkes et al. 1998). For example, the items “I feel I am working too hard at my job” and “I feel at the end of my rope” on the demoralizing climate scale may correspond to similar processes as the items “I don’t have time to learn anything new” and “I can’t meet my obligations” on the perceived burden scale. In addition, this was a cohort study in which all variables were measured at the same time and thus causality cannot be inferred.

The fact that all providers responded for all constructs using the same method (i.e., self-report) introduces potential bias inherent in self-report measures, especially when asking about internal psychological states. Consistent with recommendations by Podsakoff et al. (2003), items that may potentially exhibit common source bias have proximal and methodological separation in that they are measured in different ways and in different substantive sections of the survey. In this study, the MLQ questions related to ratings of observable manager behavior, measured on a Likert-like type scale. Another set of questions assessed perceptions of organizational climate and yet another set for perceived burden of EBP. Each set of questions were embedded in a series of questions with a different substantive focus (i.e., leadership, climate, attitudes to EBP). To promote accurate and unbiased responses and minimize any social pressures or expectations, the survey was conducted voluntarily and confidentially (Podsakoff et al. 2003). Future research using more objective measures and other corroborative methods should be used to confirm results found using self-report data (Podsakoff and Organ 1986; Spector 2006). Finally, caution is warranted in regard to generalizability of results as the sample obtained may not be representative of all mental health organizations/providers. For example, roughly half of the respondents from this sample had somewhat limited work experience and many were working toward licensure as a practicing professional (i.e., student interns n = 27, 7.58 %, and registered interns n = 157, 44.10 %) and this is greater than that found in a nationally representative sample of mental health programs where 13.11 % of respondents were BSW social workers working toward licensure (Glisson et al. 2008a, b). Further research is needed to determine the degree to which these findings generalize to professionals at various career stages, and to other health and allied health settings where more or less experience and credentialing is required.

Conclusions

Results support the positive associations of transformational leadership with dimensions of the work environment and contribute to the existing literature by highlighting another aspect of employee’s attitudes to EBP (i.e., perceived burden of EBPs) that may be positively affected by transformational leadership. Although results only indirectly address the association between leadership and perceived burden toward EBPs, past research has demonstrated the positive influences of transformational leadership on beneficial outcomes for team performance and overall workplace environments (Grant 2012; Keller 2006; Sivasubramaniam et al. 2002; Mullen and Kelloway 2009). In addition, past research suggests that policy and contracting have a profound influence on the types of practices that are expected, supported, and rewarded in service systems and organizations, with the notion that implementation/organizational climate can influence the uptake of practices (Jacobs et al. 2014). The current study supports the notion that leaders can be change agents who can influence the work environment, which may then influence targets for change (Zohar and Polachek 2014) such as behavioral health providers’ attitudes towards EBPs. Implementing a leadership intervention that increases supervisor transformational or implementation leadership behaviors (Aarons et al. 2015) could not only improve the work environment, but might also improve provider attitudes toward EBPs, ultimately improving overall workforce outcomes and outcomes of clinical services in behavioral health service settings.

Acknowledgments

This work was supported by National Institute of Mental Health Grants R01MH072961, K01MH001695, and R01MH092950. The authors thank the participating community based organizations and clinicians for their participation in this study.

References

  1. Aarons GA. Transformational and transactional leadership: Association with attitudes toward evidence-based practice. Psychiatric Services. 2006;57:1162–1169. doi: 10.1176/appi.ps.57.8.1162. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Aarons GA, Cafri G, Lugo L, Sawitzky A. Expanding the domains of attitudes towards evidence-based practice: The Evidence Based Practice Attitude Scale-50. Administration and Policy in Mental Health and Mental Health Services Research. 2012;39:331–340. doi: 10.1007/s10488-010-0302-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Aarons GA, Ehrhart MG, Farahnak LR, Hurlburt MS. Leadership and organizational change for implementation (LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science. 2015;10(1):11. doi: 10.1186/s13012-014-0192-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Aarons GA, Ehrhart MG, Farahnak LR, Sklar M. Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health. 2014;35:255–274. doi: 10.1146/annurev-publhealth-032013-182447. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Aarons GA, Glisson C, Hoagwood K, Kelleher K, Landsverk J, Cafri G. Psychometric properties and U.S. national norms of the Evidence-Based Practice Attitude Scale (EBPAS) Psychological Assessment. 2010;22:356–365. doi: 10.1037/a0019188. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Aarons GA, McDonald EJ, Sheehan AK, Walrath-Greene CM. Confirmatory factor analysis of the Evidence-Based Practice Attitude Scale (EBPAS) in a geographically diverse sample of community mental health providers. Administration and Policy in Mental Health and Mental Health Services Research. 2007;34:465–469. doi: 10.1007/s10488-007-0127-x. [DOI] [PubMed] [Google Scholar]
  7. Aarons GA, Palinkas LA. Implementation of evidence-based practice in child welfare: Service provider perspectives. Administration and Policy in Mental Health and Mental Health Services Research. 2007;34:411–419. doi: 10.1007/s10488-007-0121-3. [DOI] [PubMed] [Google Scholar]
  8. Aarons GA, Sawitzky AC. Organizational climate partially mediates the effect of culture on work attitudes and staff turnover in mental health services. Administration and Policy in Mental Health and Mental Health Services Research. 2006;33:289–301. doi: 10.1007/s10488-006-0039-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Aarons GA, Sommerfeld DH, Willging CE. The soft underbelly of system change: The role of leadership and organizational climate in turnover during statewide behavioral health reform. Psychological Services. 2011;8:269–281. doi: 10.1037/a0026196. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Aarons GA, Wells RS, Zagursky K, Fettes DL, Palinkas LA. Implementing evidence-based practice in community mental health agencies: A multiple stakeholder analysis. American Journal of Public Health. 2009;99:2087–2095. doi: 10.2105/AJPH.2009.161711. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. American Academy of Child & Adolescent Psychiatry. Evidence based practice. 2006 Retrieved from: http://www.aacap.org/AACAP/Policy_Statements/2006/Evidence_Based_Practice.aspx.
  12. American Psychological Association. Policy statement on evidence-based practice in psychology. 2005 Retrieved from http://www.apa.org/practice/guidelines/evidence-based-statement.aspx.
  13. Arthur CA, Hardy L. Transformational leadership: A quasi-experiment study. Leadership & Organization Development Journal. 2014;35:38–53. doi: 10.1108/LODJ-03-2012-0033. [DOI] [Google Scholar]
  14. Barling J, Weber T, Kelloway EK. Effects of transformational leadership training on attitudinal and financial outcomes: A field experiment. Journal of Applied Psychology. 1996;81:827–832. [Google Scholar]
  15. Bass BM. Two decades of research and development in transformational leadership. European Journal of Work and Organizational Psychology. 1999;8:9–32. [Google Scholar]
  16. Bass B, Avolio B. MLQ: multifactor leadership questionnaire: technical report. Palo Alto, CA: Mind Garden; 1995. [Google Scholar]
  17. Bass BM, Avolio BJ. Full range leadership development: manual for the multifactor leadership questionnaire. Palo Alto, CA: Mind Garden; 1997. [Google Scholar]
  18. Bass BM, Riggio RE. Transformational leadership. 2. Mahwah, NJ: Lawrence Erlbaum; 2006. [Google Scholar]
  19. Bastien DT, McPhee RD, Bolton KA. A study and extended theory of the structuration of climate. Communications Monographs. 1995;62(2):87–109. [Google Scholar]
  20. Beidas RS, Kendall PC. Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice. 2010;17:1–30. doi: 10.1111/j.1468-2850.2009.01187.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Beidas RS, Kendall PC. Dissemination and implementation of evidence-based practices in child and adolescent mental health. New York, NY: Oxford University Press; 2014. [Google Scholar]
  22. Bliese PD. Within-group agreement, non-independence, and reliability: Implications for aggregation and analysis. In: Klein KJ, Kozlowski SWJ, editors. Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions. San Francisco, CA: Jossey-Bass; 2000. pp. 349–381. [Google Scholar]
  23. Brown RD. Interrater agreement reconsidered: The role of maximum possible variance. Paper presented at the 15th Annual Conference of the Society for Industrial and Organizational Psychology; New Orleans, LA. 2000. Apr, [Google Scholar]
  24. Brown RD, Hauenstein NMA. Interrater agreement reconsidered: An alternative to the rwg indices. Organizational Research Methods. 2005;8:165–184. [Google Scholar]
  25. Bruns EJ, Hoagwood KE, Rivard JC, Wotring J, Marsenich L, Carter B. State implementation of evidence-based practice for youths, part II: Recommendations for research and policy. Journal of the American Academy of Child and Adolescent Psychiatry. 2008;47:499–504. doi: 10.1097/CHI.0b013e3181684557. [DOI] [PubMed] [Google Scholar]
  26. Carlson M, Wilcox R, Chou C, Chang M, Yang F, Blanchard J, et al. Psychometric properties of reverse-scored items on the CES-D in a sample of ethnically diverse older adults. Psychological Assessment. 2011;23:558–562. doi: 10.1037/a0022484. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Cashin C, Scheffler R, Felton M, Adams N, Miller L. Transformation of the California mental health system: Stakeholder-driven planning as a transformational activity. Psychiatric Services. 2008;59:1107–1114. doi: 10.1176/ps.2008.59.10.1107. [DOI] [PubMed] [Google Scholar]
  28. Conrad KJ, Wright BD, McKnight P, McFall M, Fontana A, Rosenheck R. Comparing traditional and Rasch analyses of the Mississippi PTSD scale: Revealing limitations of reverse-scored items. Journal of Applied Measurement. 2004;5:15–30. [PubMed] [Google Scholar]
  29. Darkes J, Greenbaum PE, Goldman MS. Sensation seeking-disinhibition and alcohol use: Exploring issues of criterion contamination. Psychological Assessment. 1998;10:71–76. doi: 10.1037/1040-3590.10.1.71. [DOI] [Google Scholar]
  30. Donnelly M. Managing people and quality in a climate of change. Managing Service Quality: An International Journal. 1994;4(4):41–44. doi: 10.1108/09604529410065261. [DOI] [Google Scholar]
  31. Fisher DM. A multilevel cross-cultural examination of role overload and organizational commitment: Investigating the interactive effects of context. Journal of Applied Psychology. 2014;99(4):723. doi: 10.1037/a0035861. [DOI] [PubMed] [Google Scholar]
  32. Glasgow RE, Vinson C, Chambers D, Khoury MJ, Kaplan RM, Hunter C. National Institutes of Health approaches to dissemination and implementation science: Current and future directions. American Journal of Public Health. 2012;102:1274–1281. doi: 10.2105/AJPH.2012.300755. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Glisson C, Green P. Organizational climate, services, and outcomes in child welfare settings. Child Abuse and Neglect. 2011;35:582–591. doi: 10.1016/j.chiabu.2011.04.009. [DOI] [PubMed] [Google Scholar]
  34. Glisson C, Hemmelgarn A. The effects of organizational climate and interorganizational coordination on the quality and outcomes of children’s service systems. Child Abuse and Neglect. 1998;22:401–421. doi: 10.1016/S0145-2134(98)00005-2. [DOI] [PubMed] [Google Scholar]
  35. Glisson C, Landsverk J, Schoenwald S, Kelleher K, Hoagwood KE, Mayberg S Research Network on Youth Mental Health. Assessing the organizational social context (OSC) of mental health services: Implications for research and practice. Administration and Policy in Mental Health and Mental Health Services Research. 2008a;35:98–113. doi: 10.1007/s10488-007-0148-5. [DOI] [PubMed] [Google Scholar]
  36. Glisson C, Shoenwald SK, Kelleher K, Landsverk J, Hoagwood KE, Mayberg S, et al. Therapist turnover and new program sustainability in mental health clinics as a function of organizational culture, climate, and service structure. Administration and Policy in Mental Health and Mental Health Services Research. 2008b;35:124–133. doi: 10.1007/s10488-007-0152-9. [DOI] [PubMed] [Google Scholar]
  37. Grant AM. Leading with meaning: Beneficiary contact, prosocial impact, and the performance effects of transformational leadership. Academy of Management Journal. 2012;55:458–476. doi: 10.5465/amj.2010.0588. [DOI] [Google Scholar]
  38. Hayes AF. Beyond Baron and Kenny: Statistical mediation analysis in the new millennium. Communication Monographs. 2009;76(4):408–420. doi: 10.1080/03637750903310360. [DOI] [Google Scholar]
  39. Henggeler SW, Chapman JE, Rowland MD, Halliday-Boykins CA, Randall J, Shackelford J, Schoenwald SK. Statewide adoption and initial implementation of contingency management for substance-abusing adolescents. Journal of Consulting and Clinical Psychology. 2008;76:556–567. doi: 10.1037/0022-006X.76.4.556. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Jacobs SR, Weiner BJ, Bunger AC. Context matters: measuring implementation climate among individuals and groups. Implementation Science. 2014;9:46. doi: 10.1186/1748-5908-9-46. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Kearney E, Gebert D. Managing diversity and enhancing team outcomes: The promise of transformational leadership. Journal of Applied Psychology. 2009;94:77–89. doi: 10.1037/a0013077. [DOI] [PubMed] [Google Scholar]
  42. Keller RT. Transformational leadership, initiating structure, and substitutes for leadership: A longitudinal study of research and development project team performance. Journal of Applied Psychology. 2006;91:202–210. doi: 10.1037/0021-9010.91.1.202. [DOI] [PubMed] [Google Scholar]
  43. Kenny DA, Judd CM. A general procedure for the estimation of interdependence. Psychological Bulletin. 1996;119(1):138–148. doi: 10.1037/0033-2909.119.1.138. [DOI] [PubMed] [Google Scholar]
  44. Kirkbride P. Developing transformational leaders: The full range leadership model in action. Industrial and Commercial Training. 2006;38:23–32. doi: 10.1108/00197850610646016. [DOI] [Google Scholar]
  45. Krishnan VR. Transformational leadership and personal outcomes: Empowerment as mediator. Leadership & Organization Development Journal. 2012;33:550–563. doi: 10.1108/01437731211253019. [DOI] [Google Scholar]
  46. Laschinger HKS, Finegan J, Shamian J. The impact of workplace empowerment, organizational trust on staff nurses’ work satisfaction and organizational commitment. Health Care Management Review. 2001;26(3):7–23. doi: 10.1097/00004010-200107000-00002. [DOI] [PubMed] [Google Scholar]
  47. Maslach C, Schaufeli WB, Leiter MP. Job burnout. Annual Review of Psychology. 2001;52:397–422. doi: 10.1146/annurev.psych.52.1.397. [DOI] [PubMed] [Google Scholar]
  48. McArdle JJ. Dealing with longitudinal attrition using logistic regression and decision tree analyses. In: McArdle JJ, Ritschard G, editors. Contemporary issues in exploratory data mining in the behavioral sciences. New York: Taylor & Frances; 2013. pp. 282–311. [Google Scholar]
  49. McHugh RK, Barlow DH. The dissemination and implementation of evidence-based psychological treatments: A review of current efforts. American Psychologist. 2010;65:73–84. doi: 10.1037/a0018121. [DOI] [PubMed] [Google Scholar]
  50. Mok E, Au-Yeung B. Relationship between organizational climate and empowerment of nurses in Hong Kong. Journal of Nursing Management. 2002;10(3):129–137. doi: 10.1046/j.1365-2834.2002.00285.x. [DOI] [PubMed] [Google Scholar]
  51. Mullen JE, Kelloway EK. Safety leadership: A longitudinal study of the effects of transformational leadership on safety outcomes. Journal of Occupational and Organizational Psychology. 2009;82(2):253–272. doi: 10.1348/096317908X325313. [DOI] [Google Scholar]
  52. Podsakoff PM, MacKenzie SB, Podsakoff NP. Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology. 2003;88(5):879–903. doi: 10.1037/0021-9010.88.5.879. [DOI] [PubMed] [Google Scholar]
  53. Podsakoff PM, Organ DW. Self-reports in organizational research: Problems and prospects. Journal of Management. 1986;12(4):531–544. [Google Scholar]
  54. Reichers AE, Schneider B. Climate and culture: An evolution of constructs. In: Schneider B, editor. Organizational climate and culture. San Francisco, CA: Jossey-Bass; 1990. pp. 5–39. [Google Scholar]
  55. Rodebaugh TL, Woods CM, Heimberg RG. The reserve of social anxiety is not always the opposite: The reverse-scored items of the social interaction anxiety scale do not belong. Behavior Therapy. 2007;38:192–206. doi: 10.1016/j.beth.2006.08.001. [DOI] [PubMed] [Google Scholar]
  56. Rubin DB. Multiple imputation after 18+ years. Journal of American Statistical Association. 1996;91:473–489. doi: 10.2307/2291635. [DOI] [Google Scholar]
  57. Sarros JC, Cooper BK, Santora JC. Building a climate for innovation through transformational leadership and organizational culture. Journal of Leadership & Organizational Studies. 2008;15:145–158. [Google Scholar]
  58. Schafer JL, Graham JW. Missing data: Our view of the state of the art. Psychological Methods. 2002;7:147–177. doi: 10.1037/1082-989X.7.2.147. [DOI] [PubMed] [Google Scholar]
  59. Scheffer J. Dealing with missing data. Research Letters in the Information and Mathematical Sciences. 2002;3:153–160. [Google Scholar]
  60. Schneider B, Ehrhart MG, Macey WH. Organizational climate and culture. Annual Review of Psychology. 2013;64:361–388. doi: 10.1146/annurev-psych-113011-143809. [DOI] [PubMed] [Google Scholar]
  61. Seibert SE, Silver SR, Randolph WA. Taking empowerment to the next level: A multiple-level model of empowerment, performance, and satisfaction. Academy of Management Journal. 2004;47(3):332–349. http://www.jstor.org/stable/20159585. [Google Scholar]
  62. Shapiro CJ, Prinz RJ, Sanders MR. Facilitators and barriers to implementation of an evidence-based parenting intervention to prevent child maltreatment: the triple p-positive parenting program. Child Maltreatment. 2011;17:86–95. doi: 10.1146/annurev-clinpsy-032511-143104. [DOI] [PubMed] [Google Scholar]
  63. Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin. 1979;86:420–428. doi: 10.1037//0033-2909.86.2.420. [DOI] [PubMed] [Google Scholar]
  64. Siu OL, Lu L, Cooper CL. Managerial stress in Hong Kong and Taiwan: A comparative study. Journal of Managerial Psychology. 1999;14(1):6–25. doi: 10.1108/02683949910254675. [DOI] [Google Scholar]
  65. Sivasubramaniam N, Murry WD, Avolio BJ, Jung DI. A longitudinal model of the effects of team leadership and group potency on group performance. Group and Organization Management. 2002;27:66–96. doi: 10.1177/1059601102027001005. [DOI] [Google Scholar]
  66. Spector PE. Method variance in organizational research truth or urban legend? Organizational Research Methods. 2006;9:221–232. [Google Scholar]
  67. Spreitzer GM. Social structural characteristics of psychological empowerment. Academy of Management Journal. 1996;39(2):483–504. http://www.jstor.org/stable/256789. [Google Scholar]
  68. Tabachnick BG, Fidell LS. Using multivariate statistics. 6. Boston, MA: Pearson; 2013. [Google Scholar]
  69. Tofighi D, MacKinnon DP. RMediation: An R package for mediation analysis confidence intervals. Behavior Research Methods. 2011;43:692–700. doi: 10.3758/s13428-011-0076-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Torrey WC, Bond GR, McHugo GJ, Swain K. Evidence-based practice implementation in community mental health settings: The relative importance of key domains of implementation activity. Administration and Policy in Mental Health and Mental Health Services Research. 2012;39:353–364. doi: 10.1007/s10488-011-0357-9. [DOI] [PubMed] [Google Scholar]
  71. Wang G, Oh IS, Courtright SH, Colbert AE. Transformational leadership and performance across criteria and levels: A meta-analytic review of 25 years of research. Group and Organization Management. 2011;36:223–270. doi: 10.1177/1059601111401017. [DOI] [Google Scholar]
  72. Warren-Gross L. How does a failing school stop failing? Knowledge Quest. 2009;38(2):40–43. [Google Scholar]
  73. Zohar D. Safety climate in industrial organizations: Theoretical and applied implications. Journal of Applied Psychology. 1980;65:96–102. [PubMed] [Google Scholar]
  74. Zohar D, Luria G. Group leaders as gatekeepers: Testing safety climate variations across levels of analysis. Applied Psychology. 2010;59:647–673. doi: 10.1111/j.1464-0597.2010.00421.x. [DOI] [Google Scholar]
  75. Zohar D, Polachek T. Discourse-based intervention for modifying supervisory communication as leverage for safety climate and performance improvement: A randomized field study. Journal of Applied Psychology. 2014;99(1):113. doi: 10.1037/a0034096. [DOI] [PubMed] [Google Scholar]

RESOURCES