Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Sep 1.
Published in final edited form as: Adm Policy Ment Health. 2016 Sep;43(5):640–649. doi: 10.1007/s10488-015-0673-6

A prospective examination of clinician and supervisor turnover within the context of implementation of evidence-based practices in a publicly-funded mental health system

Rinad S Beidas a,✉, Steven Marcus b,c, Courtney Benjamin Wolk a, Byron Powell a, Gregory A Aarons d, Arthur C Evans a,e, Matthew O Hurford a,e,f, Trevor Hadley a, Danielle R Adams a, Lucia M Walsh a, Shaili Babbar a, Frances Barg g, David S Mandell a
PMCID: PMC4715798  NIHMSID: NIHMS708503  PMID: 26179469

Abstract

Staff turnover rates in publicly-funded mental health settings are high. We investigated staff and organizational predictors of turnover in a sample of individuals working in an urban public mental health system that has engaged in a system-level effort to implement evidence-based practices. Additionally, we interviewed staff to understand reasons for turnover. Greater staff burnout predicted increased turnover, more openness toward new practices predicted retention, and more professional recognition predicted increased turnover. Staff reported leaving their organizations because of personal, organizational, and financial reasons; just over half of staff that left their organization stayed in the public mental health sector. Implications include an imperative to focus on turnover, with a particular emphasis on ameliorating staff burnout.

Keywords: turnover, implementation, mental health services

Rates of turnover (i.e., the rate at which employees leave an organization; Ben-Dror, 1994; Paris & Hoge, 2010) in publicly-funded mental health settings range from 30–60% annually (Mor Barak, Nissly, & Levin, 2001). Turnover can compromise continuity of care, destabilize agencies, and diminish quality of services (Glisson & James, 2002; Knudsen, Johnson, & Roman, 2003; Mor Barak et al., 2001). Turnover also can derail efforts to implement evidence-based practices (EBPs; Ganju, 2003; Isett et al., 2007; Woltmann et al., 2008), because systems and organizations invest substantial resources in training and supporting mental health workers (e.g., see Beidas et al., 2013) and the skilled workforce is then lost (Droege, Scott, & Hoobler, 2003), resulting in decreased fidelity of implementation (Rollins, Salyers, Tsai & Lydick, 2010). Although most relevant research documents the negative impact of turnover, turnover also may have positive effects because more engaged and cooperative staff can be hired (Wieder & Kruszynski, 2007; Woltmann & Whitley, 2007).

Staff turnover is a well-studied phenomenon. A recent meta-analytic review of 82 studies proposed and tested an organizational framework of turnover (Heavey, Holwerda, & Hausknecht, 2013). Organizational actions that demonstrated employer commitment to investing in their employees, positive employee attitudes towards the job and organization, and positive perceptions of work group and supervisor relationships were associated with lower turnover. Several of these organizational factors, such as organizational climate (i.e., shared employee perceptions around the psychological impact of their work environment on their own well-being; Glisson et al., 2008; Williams & Glisson, 2014) also have been found to predict turnover in public service sectors. Organizations with the best climates demonstrate lower turnover (Glisson et al., 2008). Organizational climate also has been found to partially mediate the association between organizational culture (i.e., organizational norms and expectations), work attitudes and subsequent turnover (Aarons & Sawitzky, 2006). Strong organizational citizenship behavior (Campbell, Perry, Maertz, Allen, & Griffeth, 2013) and organizational supports (DePanfilis & Zlotnik, 2008) also have been found to be associated with lower turnover.

Individual-level predictors of turnover, including employee burnout, high among public mental health workers (e.g., Evans et al., 2006; Jenkins & Elliott, 2004), consistently have been associated with high turnover in a number of industries (e.g., Choi, Cheong, & Feinberg, 2012; Firth & Britton, 1989; Leiter & Maslach, 2009). However, few studies have examined this link empirically in employees in mental health settings (Paris & Hoge, 2010), particularly within the context of implementation of EBPs. Implementation efforts may result in high levels of EBP fatigue and contribute to clinician burnout (Beidas et al., 2015). Additionally, therapist attitudes towards EBPs may differentially impact turnover, in that individuals with more positive attitudes may be more willing to “stay the course,” even in the face of implementation challenges, than individuals with poor attitudes towards EBPs. Alternatively, burnout may result in negative attitudes towards EBPs (Salyers et al., 2015).

Studying turnover in the context of implementation efforts is important because turnover potentially thwarts implementation and sustainability in that it can reduce the number of staff who are trained in an EBP and organizational return on investment. The stress and fatigue associated with the implementation process may increase staff turnover rates (Beidas et al., 2015). Alternatively, implementation of EBPs may result in lower turnover rates because staff may feel more invested in and may feel less burnout when they gain new tools and can treat their clients more effectively. A small body of literature has examined turnover within the context of implementation efforts. In a national study, turnover rates were high but not different across implementation of different EBPs. Turnover negatively affected both fidelity to and penetration of EBPs (Woltmann et al., 2008). During the implementation of Assertive Community Treatment, turnover rates (30%) were comparable or better than turnover rates in other settings and were inversely related to treatment fidelity (Rollins et al., 2010). A study examining turnover in the implementation of SafeCare found that intervention implementation in conjunction with fidelity monitoring was associated with lower rates of therapist turnover, suggesting a potentially protective effect of implementation with the proper supports (Aarons et al., 2009).

The literature in this area published to date has several limitations. First, little research has examined predictors of turnover during implementation efforts. Constructs proximal to implementation, such as attitudes towards EBPs, participation in EBP initiatives, and implementation climate have been largely absent from these studies. Second, studies have not examined where staff go when they leave their agencies. Given the substantial investments in training, it is important to know whether clinicians leave the public systems entirely or instead seek other opportunities within the public sector which would suggest less of a potential loss of return on investment from a system perspective. Finally, the majority of studies examining turnover have not used prospective designs, and have examined turnover intention rather than actual turnover (Woltmann et al., 2008).

The aim of the current study is twofold. First, we quantitatively and prospectively examine staff and organizational predictors of turnover in a sample of clinicians and supervisors working in an urban public mental health system engaged in a large-scale system effort to increase the use of EBPs. We hypothesized that we would observe high rates of turnover and that burnout, attitudes towards EBPs, participation in EBP initiatives, implementation climate, organizational culture, and organizational climate would predict turnover. We were particularly interested in exploring the interaction between participation in EBP practice initiatives and relevant variables to understand if implementation amplified the impact of previously found significant relationships (e.g., burnout). Second, we used semi-structured interviews to learn why staff left their organizations and where they were currently employed. Given the exploratory nature of these interviews, we did not specify a priori hypotheses.

Method

Setting

Since 2007, the Philadelphia Department of Behavioral Health and Intellectual disAbility Services (DBHIDS) has engaged in an effort to widely implement therapy techniques in the public mental health system based on the principles of cognitive behavioral therapy (CBT). These efforts include four specific initiatives: cognitive therapy (Stirman et al., 2010; Creed, Stirman, Evans, & Beck, 2014), prolonged exposure, trauma-focused cognitive behavioral therapy (Beidas et al., in review), and dialectical behavior therapy. A full-time city employee coordinates implementation, training, and ongoing consultation by treatment developers for each of the four initiatives.

Agencies

More than 100 community mental health agencies in Philadelphia provide outpatient services to youth (Cathy Bolton, PhD, email communication, January 3rd, 2013). We used purposive sampling to recruit the 29 largest child-serving agencies, which together serve approximately 80% of youth receiving publicly-funded mental health care. Of these 29 agencies, 18 (62%) agreed to participate. Additionally, another organization involved in EBP efforts asked if they could participate, resulting in a final sample of 19 agencies with 23 sites, 130 clinicians, 36 supervisors, and 22 executive administrators. Consistent with previous studies (Aarons, Sommerfeld, Hecht, Silovsky, & Chaffin, 2009), each site (K = 23), rather than each organization (K = 19), was treated as a distinct organization because of different leadership structures, locations, and staff. There were no exclusion criteria for clinician or supervisor participation. Approximately 60% of therapists employed by the 23 organizations participated in the study. Of the 23 organizations enrolled in this study, 16 had participated in city-sponsored EBP initiatives.

Participants

Table 1 provides demographic information about clinicians and supervisors. Half of executive administrators were male. They self-identified as Asian (9.1%), African American (18.2%), Caucasian (54.5%), Multiracial (9.1%), or missing ethnicity/race (9.1%). Fifteen percent identified as Hispanic/Latino. Highest educational degree attained included bachelor’s degree (9.1%), master’s degree (50.0%), and doctoral degree (31.8%). Data were missing for 9.1%.

Table 1.

Clinician (N=130) and Supervisor (N=36) Demographics

Variable Clinician Frequency (%)or Mean (SD) Supervisor Frequency (%)or Mean (SD)
Gendera
 Male 23% 22.2%
 Female 76% 69.4%
 Transgender 1% 0%
Hispanic/Latinoa
 Yes 20% 19.4%
 No 75% 69.4%
Ethnicitya
 Asian 4.9% 0%
 Black or African American 22% 16.7%
 White 54.5% 55.6%
 Hispanic/Latino 10.6% 13.9%
 Multiracial 4.1% 0%
 Other 4.1% 2.8%
Academic backgrounda
 Bachelor’s degree 3.8% 0%
 Master’s degree 82.3% 75.0%
 Doctoral degree 9.2% 13.9%
Licensure statusa
 Yes 24.6% 52.8%
 No 39.2% 25%
 In process 31.5% 13.9%
Age 38.09 (11.63) 46.09 (10.44)
Years at current organization 3.35 (4.65) 7.48 (6.88)
Open in a new tab

Note. EBP = Evidence Based Practice;

a

Does not add up to 100% because of missing response

Procedure

This study was approved by appropriate Institutional Review Boards. The person identified as the leader of the organization was approached via email to solicit his/her organization’s participation. A one-time two-hour meeting was scheduled, during which we provided lunch, gave an overview of the study, and obtained written informed consent. Consented clinicians and supervisors completed self-report measures during this meeting. Executive administrators completed their self-report measures via REDcap, a web-based survey technology. All participants were compensated $50.

One year following this meeting, we contacted organization leadership, who identified clinicians and supervisors who had left their agencies since the initial meeting. Thirty-nine clinicians and supervisors out of 166 participants reportedly left their agencies. We contacted all individuals to obtain information about their reasons for leaving as well as their new employment setting via semi-structured interview. Twenty nine (74%) participants completed the interviews.

Measures

Individual-Level Quantitative Measures

Demographics

Participants provided demographic information. We also asked clinicians and supervisors a standardized question from the Therapist Background Questionnaire (Weisz, 1997) about their level of professional burnout by asking, “How often, if ever, do you experience a feeling of professional burnout?” ranging from 0 (never) to 10 (constantly) (Weisz, 1997). While this one item question does not have the methodological rigor of psychometrically validated measures such as the Maslach Burnout Inventory (MBI; Maslach, Jackson, & Leiter, 1996), our preliminary analyses from another wave of data collection suggest a high correlation with this item and the three subscales of the MBI: professional efficacy (r = .41), exhaustion (r = .67) and cynicism (r =.60).

Evidence-Based Practices Training Survey

Therapists indicated whether they had participated in any of the four DBHIDS EBP initiatives (cognitive-therapy, trauma-focused CBT, prolonged exposure, and dialectical behavior therapy). We asked specifically whether they had received training and a year of consultation to ensure that they formally participated in the city-sponsored initiative.

Attitudes

The Evidence-Based Practice Attitude Scale (EBPAS; Aarons, 2004), completed by clinicians and supervisors, is a 15-item self-report measure of constructs related to EBP implementation, including appeal, requirements, openness, and divergence. The range of each subscale is 0 to 4, where 0 = not at all; 1 = slight extent; 2 = moderate extent; 3 = great extent; and 4 = very great extent. Higher scores indicate more positive attitudes, with the exception of divergence, which is reverse coded. The EBPAS has national norms and demonstrated good internal consistency (subscale alphas range from .67–.91) (Aarons et al., 2010).

Organizational-Level Quantitative Measures

Organizational Evidence-Based Practices

The City of Philadelphia DBHIDS provided us with a list of the organizations that had participated in one of the four city-sponsored EBP initiatives, the year they began participating, and their completion year (if applicable). From that information, we calculated the cumulative years each organization participated in DBHIDS EBP initiatives.

Organizational Culture and Climate

The Organizational Social Context Measurement System (OSC; Glisson et al., 2008), completed by executive administrators, supervisors, and clinicians, assesses the social context of mental health and social services organizations. The 105-item OSC measures organizational culture and organizational climate. Organizational culture is composed of proficiency, rigidity, and resistance. An organization’s T-scores on these three dimensions can be used to create profiles that represent worst (proficiency scores 2 or more standard deviations below rigidity and resistance scores), average, and best culture (proficiency scores that are 2 or more standard deviations above rigidity and resistance scores; Glisson et al., 2008). Organizational climate is composed of engagement, functionality, and stress. Similarly, an organization’s T-scores on these three dimensions can be used to create profiles that represent worst (stress score is 2 or more standard deviations above functionality and engagement), average, and best climate (functionality and engagement scores are 2 or more standard deviations above stress; Glisson et al., 2008). Psychometric properties, including the measurement model, within-system agreement, reliability and between-system differences, are acceptable (Glisson, Green, & Williams, 2012).

Implementation Climate

The Implementation Climate Scale (ICS; Ehrhart, Aarons, & Farahnak, 2014), completed by executive administrators, supervisors, and clinicians, is an 18-item self-report measure of implementation climate that assesses the following constructs: focus on EBPs; educational support for EBPs; recognition for using EBPs; rewards for using EBPs; selection of staff for EBPs; and selection of staff for openness. The range of each subscale is 0 to 4, where 0 = not at all; 1 = slight extent; 2 = moderate extent; 3 = great extent; and 4 = very great extent. Higher scores indicate a more positive implementation climate. Psychometric evaluation suggests good reliability and validity (Ehrhart et al., 2014).

Qualitative Measure

Turnover

A semi-structured phone interview was composed of one open-ended prompt, “We would like to learn more about why you left your position as a clinician or supervisor at ORGANIZATION NAME.” The interviewer ensured that information about when the individual left the organization, why the individual left, and where the individual was currently working, if applicable, was reported. We also gathered information about whether the clinician or supervisor was currently working in the public (e.g., community mental health clinics with public funding) or private sector (e.g., private practice), the location of their new position, and whether their exit was voluntary.

Data Analytic Plan

Quantitative analysis

Organizational measures generally are included in analyses by aggregating individual responses within the organization, if there is enough agreement. We used average within-group correlation (awg, rwg) statistics (Brown & Hauenstein, 2005; James, Demaree, & Wolf, 1984). On all organizational variables, both statistics were substantially above the suggested .60 level (Bliese, 2000; Brown & Hauenstein, 2005); therefore participant responses to organizational constructs were averaged within each organization. Missing data for predictor variables were less than 10%; series means were imputed for missing predictor variables.

We used one mixed-effects linear regression model and one fixed-effects regression model. The first mixed effects model estimated the impact of staff factors (i.e., participation in an EBP initiative (yes/no), burnout, and attitudes) on individual turnover (i.e., whether a particular clinician or supervisor left his/her organization). This mixed-effects model included random intercepts for organization to account for nesting of clinicians and supervisors within organizations and fixed effects for staff factors. The second fixed effects model estimated the impact of organizational factors (number of years participating in EBP initiatives, implementation climate, organizational culture (best, average, worst), and organizational climate (best, average, worst) on organizational (i.e., site) level turnover (i.e., percentage of turnover in one year at the organization). For both sets of analyses, we conducted additional, separate models that included an interaction term between variables observed to be significant in the initial model and the implementation variables. Analyses were conducted using PROC GLIMMIX and PROC MIXED in SAS 9.0.

Qualitative

Phone interviews were transcribed and entered into QSR NVivo 10.0 for data management and coding. Authors RSB and DRA independently read a subset of transcripts and created a set of codes using content analysis. An initial codebook was established and refined through consensus meetings. DRA coded all interviews and RSB dual-coded 20% for reliability. The inter-rater reliability function in NVivo was utilized to determine agreement (kappa = .99).

Results

Quantitative results

See Table 1 for descriptive statistics for predictor and dependent variables.

Turnover

Descriptives

Twenty-five percent of the 130 therapists (n = 33) and 17% of the 36 supervisors (n = 6) left their agencies in the year following our site visit (N = 39). Of those 39 participants, we were able to collect follow-up information from 29 individuals. Fifty-five percent of those 29 participants left their agencies for new jobs in the public sector. Thirty-five percent of participants left their agencies for jobs in the private sector. Ten percent of participants who left their agencies were not currently employed. Seventy-nine percent of participants voluntarily left their agencies, 17% involuntarily left their agencies (for one participant, voluntary or involuntary exit information was not provided). Forty-eight percent of participants were still employed in Philadelphia, while 24% of participants worked out-of-state, 17% worked in the suburbs, and eleven percent were not working. Turnover rates ranged from 0% to 67% among organizations.

Individual turnover

Table 2 presents the results of the analyses predicting turnover at the staff level. Two variables predicted individual clinician and supervisor turnover. For each unit increase in burnout (range = 0–10), the odds of staff leaving their organization in the following year were .15 times greater than for those who experienced less burnout. For each unit increase in openness to new practices (range = 0 to 4), the odds of staff remaining at their organization in the following year were 2.0 times greater than for those who were held less positive attitudes.

Table 2.

Individual Model Predicting Individual Turnover (N = 166)

Variable N M (SD) [Range] or percentage% Odds Ratio CI (95%) P
Dependent variable: Individual turnover 166 Not at organization (23.5%);
Remained at organization (76.5%)		 -- -- --
Participated in EBP initiative 158 Yes (43%)
No (57%)		 .71 .33–1.53 .38
Burnout 149 4.01 (2.56) [0–10] .85 .72–.99 .04
Attitudes
Requirements 163 2.78(1.07) [0–4] 1.02 .68–1.55 .91
Appeal 163 3.14 (.66) [1.50–4] .60 .30–1.22 .16
Openness 163 3.09 (.68) [1.25–4] 2.01 1.06–3.81 .03
Divergence 163 1.26 (.71) [0–4.00] .83 .46–1.48 .52
Open in a new tab

We conducted two additional analyses to test if burnout and openness to innovation differentially predicted turnover in individuals based on participation in EBP initiatives. In the first model, we included an interaction term between burnout and EBP participation and in the second model, we included an interaction term between openness to innovation and EBP participation. The resulting coefficients were not statistically significant and therefore were not retained in the reported model (results not shown).

Organizational turnover

Table 3 presents the results of the analyses predicting the percentage of staff turnover at each organization. Organizations that provided more recognition for staff implementing EBP had higher turnover in the year following. We conducted a separate analysis to test if recognition for implementing EBP differentially predicted turnover in organizations implementing EBPs by including an interaction term between recognition and organizational EBP participation. The resulting coefficient was not significant and therefore was not retained in our model (results not shown).

Table 3.

Organizational Model Predicting Organization Turnover (K = 23)

Variable N M (SD) [Range] or percentage% Estimate Standard Error P
Dependent variable: Organizational turnover 23 24.24% (17.54) [0–66.7%] -- -- --
Number of years participating in EBP initiatives 23 3.09 (2.70) [0–7.0] .64 1.38 .65
Implementation Climate
Focus on EBPs 23 2.35 (.70) [1.00–3.61] 2.63 9.77 .79
Educational support 23 1.81 (.73) [1.00–3.56] −15.68 9.48 .12
Recognition 23 2.03 (.77) [0.00–3.21] 18.99 7.20 .02
Reward 23 .72 (.68) [0.00–3.33] 3.59 7.42 .64
Staff selection 23 2.42 (.57) [1.30–3.44] −10.32 9.70 .31
Openness 23 2.91 (.62) [1.17–3.67] −4.06 7.01 .57
Organizational Social Context
Culture 23 Worst (22%)
Average (78%)
Best (0%)		 −4.62 11.02 .68
Climate 23 Worst (13%)
Average (78%)
Best (9%)		 −18.66 11.02 .11
Open in a new tab

Note. EBP = Evidence Based Practice

Qualitative results

See Table S1 for illustrative quotes. Participants could provide more than one reason for leaving their agency. Of those respondents who completed the brief interview, fifty-five percent reported personal reasons for leaving. Personal reasons included family illness, childcare, wanting to study for licensure, and not feeling challenged.

Forty-five percent of participants leaving reported that organizational factors affected their decision to leave their organization. These factors included organizational expectations, disorganization, and lack of organizational support. Participants noted that expectations were too high, specifically pertaining to case management requirements and paperwork. Additionally, participants noted that disorganization in the workplace made it difficult to do their job effectively. Participants also reported that lack of support impacted their decision to leave their organization because they did not receive needed supervision or consultation and their organization did not provide incentives to promote retention.

Forty one percent of participants reported that financial reasons affected their decision to leave their organization. Participants reported that they did not have enough money to support their lifestyle and often were not paid for all of the hours they worked. Second, fee-for-service issues were cited by a number of participants, who reported the financial instability of being a contract worker. Third, budget cuts were noted by staff who were laid off.

Thirty one percent of participants reported leaving their organization for a more desirable position in another organization. Several participants noted that there was no room for growth, which made it necessary for them to leave their organization.

Twenty four percent of participants reported leadership as a factor that affected their decision to leave, including dishonest or corrupt leadership. Ten percent of participants reported that they stayed at their organization as long as they did due to positive leadership experiences.

Twenty one percent of participants reported client-related factors that affected their decision to leave their agencies. Participants reported frustration with clients that often broke appointments, and client engagement. Participants also noted that complicated cases with comorbid psychosocial difficulties made it difficult to be effective. Finally, participants noted that their case-load also factored into their decision, with most indicating that their case-load was too high.

Discussion

This study provides information on predictors of turnover in a sample of clinicians and supervisors working with youth in an urban publicly-funded mental health system engaged in a large-scale effort to increase the use of EBP. The results provide insights on where clinicians and supervisors go after leaving their place of employment, as well as their reasons for leaving. Twenty-four percent of staff left their organizations within one year. This percentage is comparable to other studies of turnover within the context of EBP implementation efforts (e.g., Rollins et al., 2010), and is lower than estimates of turnover in community mental health settings more generally (Jayaratne & Chess, 1984; Mor Barak et al., 2001). Staff-level predictors of turnover included burnout and openness towards EBPs; organizational predictors of turnover included implementation recognition. In semi-structured interviews, staff reported a number of factors that influenced their decision to leave their organization including personal, organizational, financial, opportunity, leadership, and client factors.

Notably, the turnover rate was lower in our sample than found in studies not conducted within the context of implementation (Mor Barak et al., 2001). A previous study found a protective impact from EBP implementation with fidelity monitoring (Aarons et al., 2009). We did not identify a significant effect of implementing an EBP at the individual- or organizational-level on turnover, nor did we find any significant interactions between implementing an EBP and burnout, openness to innovation or recognition of staff). One potential explanation is that implementing EBPs is not a panacea for workforce issues and staff retention – simply implementing EBPs may not be enough of an intervention to impact these issues. The lower turnover rate in the Philadelphia system may reflect the City of Philadelphia’s efforts to create a positive system-wide culture and climate, which may incentivize clinicians to stay at their agencies or within the system. Interestingly, of those individuals who left their agencies, more than half were still employed in public service sectors, suggesting that concerns about system-level return on investment may be unfounded because staff are staying in the same system.

Priori literature is consistent with our finding that burnout predicts turnover (Jayaratne & Chess, 1984; Maslach, Jackson, & Leiter, 1996) and suggests the importance of attending to burnout in staff employed in mental health setting. This finding was corroborated by our semi-structured qualitative interviews. Many of the themes that emerged in the interviews related to burnout (e.g., feeling unsupported by one’s organization, being overwhelmed by paperwork burden and caseloads) and should be measured quantitatively in future studies. Participants with more positive attitudes toward EBPs were two times more likely to stay at their agencies in the year following. One potential explanation for this finding is that having a generally open attitude towards new practices may be correlated with openness to experience, a personality trait measured by the five factor model of personality (Costa & Mccrae, 1992) that predicts retention in employees in organizations (Salgado, 2002). The transactional relationship between burnout and openness to new practices is an important one for future research, given the finding that burnout can diminish the motivation to try new practices (Salyers et al., 2015).

Organizations fostering a positive implementation climate in the form of providing recognition for employees engaging in EBP had higher turnover rates, on average, even after controlling for number of years the organization has spent in city-sponsored EBP initiatives. The items used to measure the “recognition” construct included “recognizing clinicians who use EBPs as clinical experts,” “holding clinicians in high esteem for using EBPs,” and “being more likely to be promoted because of using EBPs.” Organizations that provide recognition for employees engaging in EBPs may have staff who perceive themselves to be highly competent, actively seek opportunities for learning and growth, and consequently, may have more opportunities to be lured away by other organizations. This stands in contrast to previous findings that found that lack of advancement and growth opportunities indirectly predicted turnover (Alexander, Lichtenstein, Joo Oh, Ullman, 1998). It also is inconsistent with findings that organizational investment in staff negatively predicts turnover (Heavey et al., 2013). An alternative explanation may be that staff experience increased responsibilities in implementing EBPs without increases in pay and may be more motivated to move on to positions where EBPs are not being implemented or where EBPs are more established.

The qualitative interviews suggest the importance of a number of variables that did not emerge as quantitatively significant, or were not measured. For example, although quantitative measures of organizational social context did not predict turnover, organizational social context was raised as an important factor repeatedly during the interviews, consistent with the literature (Glisson & Green, 2011; Glisson et al., 2008; US General Accounting Office, 2003). We may have been underpowered to uncover small effects at the organizational level. In addition, the range of scores on the measure of organizational culture was limited and left-skewed which also may have hampered our ability to detect an association. Many clinicians reported being unable to make ends meet with their salary, particularly those working within fee-for-service environments. Approximately half of the organizations in this sample employed a fee-for-service staffing model (i.e., clinicians as contractors). In tight economic times, organizations may increase their workforce of part-time contractors in lieu of hiring full-time clinicians. Participant report suggests that this type of staffing model may threaten the potential for individuals to remain employed in public mental health settings.

There are a number of study limitations that should be noted. First, we used only one item to measure clinician and supervisor burnout rather than a gold-standard established measure such as the Maslach Burnout Inventory (Maslach et al., 1996). Second, the burnout item may be correlated with other predictors (e.g., organizational climate) and may have resulted in collinearity. Third, we conducted separate analyses of individual- and organizational-level factors because of concerns about statistical power, but it would have been preferable to use a single multi-level model. To address this limitation, we conducted a single multi-level analysis with individual and organizational characteristics in the same model. The magnitude of the coefficients was similar, but no longer statistically significant at p < .05, although the p values approached .05. This finding suggests that our results were not spurious, but rather resulted from a lack of statistical power. Fourth, we did not measure all of the potential variables that could have contributed to turnover, especially performance, income and work-life balance; these variables may have been associated with predictors included in our models (e.g., burnout). Fifth, organizations were not randomly assigned to participation in EBP initiatives, therefore there may be unobserved differences between organizations that participated and those that did not (e.g., better functioning organizations). Finally, reasons for participant turnover were based on self-report (i.e., were not subject to independent verification) and the qualitative interview responses could have been subject to recall bias.

Despite these limitations, the findings from this study have important implications. First, much has been written about turnover in public mental health service settings, but little has been done to ameliorate this problem. Identifying potential targets for turnover interventions is an important next step. A recent review points to several promising burnout interventions (Morse, Salyers, Rollins, Monroe-DeVita, Pfahler, 2012) that are a potential avenue for future research and warrant randomized controlled trials. A number of practical suggestions have been made to reduce burnout (e.g., better salaries, financial incentives, clinical supervision; Paris & Hoge, 2010), which could also be bundled with implementation strategies (Proctor, Powell, & McMillen, 2013) to improve implementation efforts. Implementation of EBPs themselves are likely not powerful enough of an intervention to influence workforce issues.

Although not directly measured in this study, turnover can potentially pose a threat to organizational and systematic efforts to implement evidence-based practices (Ganju, 2003; Isett et al., 2007; Woltmann et al., 2008) due to the substantial resources that organizations and systems invest in training and supporting employees. Based on the turnover observed in this study, it seems likely that some organizations will experience close to 100% turnover after any 4-year period, suggesting that current models of one-time training and short-term follow-up consultation (Beidas & Kendall, 2010) are unlikely to be effective. Although turnover may be detrimental, in some instances it may allow for agencies to recruit replacements who bring valued knowledge about EBPs. It is heartening that many of the clinicians leaving their agencies are going to other public sector agencies, thus taking their new knowledge and skills with them. However, without an organizational infrastructure to support EBPs, it may be difficult for clinicians to implement EBPs in a quality manner. Given the ongoing and significant consequences of turnover, research that allows for better understanding of how to support and stabilize the workforce in publicly-funded mental health service settings is needed.

Supplementary Material

10488_2015_673_MOESM1_ESM

Acknowledgments

We are especially grateful for the support that the Department of Behavioral Health and Intellectual DisAbility Services has provided for this project, and for the Evidence Based Practice and Innovation (EPIC) group. Funding for this research project was supported by the following grants from NIMH: (K23 MH099179, Beidas; F32 MH103955, Benjamin Wolk). Additionally, the preparation of this article was supported in part by the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis; through an award from the National Institute of Mental Health (R25 MH080916) and Quality Enhancement Research Initiative (QUERI), Department of Veterans Affairs Contract, Veterans Health Administration, Office of Research & Development, Health Services Research & Development Service. Dr. Beidas was an IRI fellow from 2012–2014. We would like to thank for the following experts who provided their time and input on this project: Dr. Marc Atkins, Dr. Ross Brownson, Dr. David Chambers, Dr. Bruce Chorpita, Dr. Charles Glisson, Dr. Nicholas Ialongo, Dr. John Landsverk, Dr. Enola Proctor, Dr. Ronnie Rubin.

References

  1. Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS) Mental Health Services Research. 2004;6(2):61–74. doi: 10.1023/b:mhsr.0000024351.12294.65. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Aarons GA, Hoagwood K, Landsverk J, Glisson C, Kelleher K, Cafri G. Psychometric properties and US National norms of the Evidence-Based Practice Attitude Scale (EBPAS) Psychological Assessment. 2010;22(2):356–365. doi: 10.1037/A0019188. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Aarons GA, Sawitzky AC. Organizational climate partially mediates the effect of culture on work attitudes and staff turnover in mental health services. Administration and Policy in Mental Health and Mental Health Services Research. 2006;33(3):289–301. doi: 10.1007/s10488-006-0039-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Aarons GA, Sommerfeld DH, Hecht DB, Silovsky JF, Chaffin MJ. The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: Evidence for a protective effect. Journal of Consulting and Clinical Psychology. 2009;77(2):270–280. doi: 10.1037/a0013223. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Alexander JA, Lichtenstein R, Joo Oh H, Ullman E. A causal model of voluntary turnover among nursing personnel in long-term psychiatric settings. Research in Nursing & Health. 1998;21:415–427. doi: 10.1002/(sici)1098-240x(199810)21:5<415::aid-nur5>3.0.co;2-q. [DOI] [PubMed] [Google Scholar]
  6. Beidas RS, Aarons G, Frances B, Evans A, Hadley T, Hoagwood K, Mandell D. Policy to implementation: Evidence-based practice in community mental health- Study protocol. Implementation Science. 2013;8(38):1–9. doi: 10.1186/1748-5908-8-38. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Beidas RS, Adams DR, Kratz HE, Jackson K, Berkowitz S, Zinny A, Evans A. Philadelphia Alliance for Child Trauma Services (PACTS): Building a trauma-informed system of care in a large urban public mental health system. (In review) [Google Scholar]
  8. Beidas RS, Kendall PC. Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology. 2010;17(1):1–30. doi: 10.1111/j.1468-2850.2009.01187.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Beidas RS, Marcus S, Aarons GA, Hoagwood K, Schoenwald S, Evans A, Mandell DS. Individual and organizational factors related to community clinicians’ use of therapy techniques in a large public mental health system. JAMA Pediatrics. 2015;169(4):374–382. doi: 10.1001/jamapediatrics.2014.3736. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Ben-Dror R. Employee turnover in community mental health organization: A developmental stages study. Community Mental Health Journal. 1994;30(3):243–257. doi: 10.1007/BF02188885. [DOI] [PubMed] [Google Scholar]
  11. Bliese PD. Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In: Klein K, Kozlowski S, editors. Multilevel Theory, Research, and Methods in Organizations. San Francisco, CA: Joseey-Bass; 2000. pp. 349–380. [Google Scholar]
  12. Brown RD, Hauenstein NMA. Interrater agreement reconsidered: An alternative to the r(wg) indices. Organizational Research Methods. 2005;8(2):165–184. doi: 10.1177/1094428105275376. [DOI] [Google Scholar]
  13. Campbell NS, Perry SJ, Maertz CP, Allen DG, Griffeth RW. All you need is... resources: The effects of justice and support on burnout and turnover. Human Relations. 2013;66(6):759–782. doi: 10.1177/0018726712462614. [DOI] [Google Scholar]
  14. Choi S, Cheong K, Feinberg RA. Moderating effects of supervisor support, monetary rewards, and career paths on the relationship between job burnout and turnover intentions in the context of call centers. Managing Service Quality. 2012;22(5):492–516. doi: 10.1108/09604521211281396. [DOI] [Google Scholar]
  15. Costa PT, Mccrae RR. The 5-factor model of personality and its relevance to personality-disorders. Journal of Personality Disorders. 1992;6(4):343–359. [Google Scholar]
  16. Creed TA, Stirman SW, Evans AC, Beck AT. A model for implementation of cognitive therapy in community mental health: The Beck Initiative. The Behavior Therapist. 2014;37(3):56–65. [Google Scholar]
  17. DePanfilis D, Zlotnik JL. Retention of front-line staff in child welfare: A systematic review of research. Children and Youth Services Review. 2008;30(9):995–1008. [Google Scholar]
  18. Droege SB, Hoobler JM. Employee turnover and tacit knowledge diffusion: A network perspective. Journal of Managerial Issues. 2003;15(1):50–64. [Google Scholar]
  19. Ehrhart MG, Aarons GA, Farahnak LR. Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ICS) Implementation Science. 2014;9(1):157. doi: 10.1186/s13012-014-0157-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Evans S, Huxley P, Gately C, Webber M, Mears A, Pajak S, Katona C. Mental health, burnout and job satisfaction among mental health social workers in England and Wales. The British Journal of Psychiatry. 2006;188:75–80. doi: 10.1192/bjp.188.1.75. [DOI] [PubMed] [Google Scholar]
  21. Firth H, Britton P. Burnout, absence and turnover amongst British nursing staff. Journal of Occupational Psychology. 1989;62(1):55–59. [Google Scholar]
  22. Ganju V. Implementation of evidence-based practices in state mental health systems: Implications for research and effectiveness studies. Schizophrenia Bulletin. 2003;29(1):125–131. doi: 10.1093/oxfordjournals.schbul.a006982. [DOI] [PubMed] [Google Scholar]
  23. Glisson C, Green P. Organizational climate, services, and outcomes in child welfare systems. Child Abuse & Neglect. 2011;35(8):582–591. doi: 10.1016/j.chiabu.2011.04.009. [DOI] [PubMed] [Google Scholar]
  24. Glisson C, Green P, Williams NJ. Assessing the Organizational Social Context (OSC) of child welfare systems: Implications for research and practice. Child Abuse & Neglect. 2012;36(9):621–632. doi: 10.1016/j.chiabu.2012.06.002. [DOI] [PubMed] [Google Scholar]
  25. Glisson C, James LR. The cross-level effects of culture and climate in human service teams. Journal of Organizational Behavior. 2002;23(6):767–794. [Google Scholar]
  26. Glisson C, Schoenwald SK, Kelleher K, Landsverk J, Hoagwood KE Research Network on Youth Mental Health. Therapist turnover and new program sustainability in mental health clinics as a function of organizational culture, climate, and service structure. Administration and Policy in Mental Health and Mental Health Services Research. 2008;35(1–2):124–133. doi: 10.1007/s10488-007-0152-9. [DOI] [PubMed] [Google Scholar]
  27. Heavey AL, Holwerda JA, Hausknecht JP. Causes and consequences of collective turnover: A meta-analytic review. Journal of Applied Psychology. 2013;98(3):412–453. doi: 10.1037/a0032380. [DOI] [PubMed] [Google Scholar]
  28. Isett KR, Burnam MA, Coleman-Beattie B, Hyde PS, Morrissey JP, Magnabosco J, Goldman H. The state policy context of implementation issues for evidence-based practices in mental health. Psychiatric Services. 2007;58(7):914–921. doi: 10.1176/ps.2007.58.7.914. [DOI] [PubMed] [Google Scholar]
  29. James LR, Demaree RG, Wolf G. Estimating within-group interrater reliability with and without response bias. Journal of Applied Psychology. 1984;69(1):85–98. doi: 10.1037//0021-9010.69.1.85. [DOI] [Google Scholar]
  30. Jayaratne S, Chess WA. Job-satisfaction, burnout, and turnover - A national study. Social Work. 1984;29(5):448–453. doi: 10.1093/sw/29.5.448. [DOI] [PubMed] [Google Scholar]
  31. Jenkins R, Elliott P. Stressors, burnout and social support: Nurses in acute mental health settings. Journal of Advanced Nursing. 2004;48(6):622–631. doi: 10.1111/j.1365-2648.2004.03240.x. [DOI] [PubMed] [Google Scholar]
  32. Knudsen HK, Johnson JA, Roman PM. Retaining counseling staff at substance abuse treatment centers: Effects of management practices. Journal of Substance Abuse Treatment. 2003;24(2):129–135. doi: 10.1016/s0740-5472(02)00357-4. [DOI] [PubMed] [Google Scholar]
  33. Leiter MP, Maslach C. Nurse turnover: The mediating role of burnout. Journal of Nursing Management. 2009;17(3):331–339. doi: 10.1111/j.1365-2834.2009.01004.x. [DOI] [PubMed] [Google Scholar]
  34. Maslach C, Jackson SE, Leiter MP. The Maslach Burnout Inventory Manual. 3. Palo Alto: Consulting Psychologists Press; 1996. [Google Scholar]
  35. Mor Barak ME, Nissly JA, Levin A. Antecedents to retention and turnover among child welfare, social work, and other human service employees: What can we learn from past research? A review and metanalysis. Social Service Review. 2001;75:625–661. [Google Scholar]
  36. Morse G, Salyers MP, Rollins AL, Monroe-DeVita M, Pfahler C. Burnout in mental health services: A review of the problem and its remediation. Administration and Policy in Mental Health and Mental Health Services Research. 2012;39(5):341–352. doi: 10.1007/s10488-011-0352-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Paris M, Hoge MA. Burnout in the mental health workforce: A review. Journal of Behavioral Health Services & Research. 2010;37(4):519–528. doi: 10.1007/s11414-009-9202-2. [DOI] [PubMed] [Google Scholar]
  38. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: Recommendations for specifying and reporting. Implementation Science. 2013;8:139. doi: 10.1186/1748-5908-8-139. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Rollins AL, Salyers MP, Tsai J, Lydick JM. Staff turnover in statewide implementation of ACT: Relationship with ACT fidelity and other team characteristics. Administration and Policy in Mental Health and Mental Health Services Research. 2010;37(5):417–426. doi: 10.1007/s10488-009-0257-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Salgado J. The big five personality dimensions and counterproductive behaviors. International Journal of Selection and Assessment. 2002;10:117–125. [Google Scholar]
  41. Salyers MP, Hood BJ, Schwartz K, Alexander AO, Aalsma MC. The experience, impact, and management of professional burnout among probation officers in juvenile justice settings. Journal of Offender Rehabilitation. 2015;54(3):175–193. doi: 10.1080/10509674.2015.1009967. [DOI] [Google Scholar]
  42. Stirman SW, Spokas M, Creed TA, Farabaugh DT, Bhar SS, Brown GK, Beck AT. Training and consultation in evidence-based psychosocial treatments in public mental health settings: The ACCESS Model. Professional Psychology, Research and Practice. 2010;41(1):48–56. doi: 10.1037/a0018099. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. US General Accounting Office. Child welfare: HHS could play a greater role in helping child welfare agencies recruit and retain staff. Washington DC: U.S. General Accounting Office; 2003. [Google Scholar]
  44. Weisz JR. Therapist Background Questionnaire. Los Angeles, CA: University of California; 1997. [Google Scholar]
  45. Wieder BL, Kruszynski R. The salience of staffing in IDDT implementation: One agency’s experience. American Journal of Psychiatric Rehabilitation. 2007;10:103–112. [Google Scholar]
  46. Williams NJ, Glisson C. The role of organizational culture and climate in the dissemination and implementation of empirically supported treatments for youth. In: Beidas RS, Kendall PC, editors. Dissemination and Implementation of Evidence-Based Practices in Child and Adolescent Mental Health. 1. New York: Oxford University Press; 2014. pp. 61–81. [Google Scholar]
  47. Woltmann EM, Whitley R. The role of staffing stability in the implementation of integrated dual disorders treatment: An exploratory study. Journal of Mental Health Administration. 2007;16:757–769. [Google Scholar]
  48. Woltmann EM, Whitley R, McHugo GJ, Brunette M, Torrey WC, Coots L, Drake RE. The role of staff turnover in the implementation of evidence-based practices in mental health care. Psychiatric Services. 2008;59(7):732–737. doi: 10.1176/appi.ps.59.7.732. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

10488_2015_673_MOESM1_ESM
NIHMS708503-supplement-10488_2015_673_MOESM1_ESM.docx (17.8KB, docx)

RESOURCES