Behavioral health employee turnover: HR-data machine learning and job wellbeing predictabilities

Abstract

Objective:

Identifying employees at high turnover risk is critical to support retention among community behavioral health organizations (CBHOs). The current study proposed a method that integrated 1) machine learning (ML) models trained with historical Human Resource (HR) data and 2) survey-based job wellbeing indicators to predict current employee turnover.

Methods:

The ML models trained with historical HR data (going back to 2011 for the oldest cases) were applied to current employees’ HR data to obtain predicted turnover probability (${\widehat{p}}_{HR}$). Additionally, 12 job wellbeing indicators were collected from the current employees via organization-wide surveys, and their actual turnover status was obtained 12 months later. Logistic regressions were used to test the predictability of combined with the job wellbeing indicators for predicting future turnover.

Results:

The study included 303 CBHO employees with an average age of 43.2±13.2, 72% female, and 60% White. Seventy-two (24%) left the organization voluntarily by the time of turnover outcome extractions (12 months later). The predicted actual turnover, but adding three critical job wellbeing indicators (i.e., career advancement opportunities, expectation alignment, importance of supervision) improved the prediction accuracy (area under the curve improved from .63 to .76). Particularly, (OR=2.15) and career advancement opportunities (OR=0.61) significantly predicted employee turnover.

Conclusions:

The current study provides a viable and low labor-intensive method for identifying high turnover-risk employees by applying the ML models trained with historical HR to future data for predicting actual turnover and identifying additional survey data that could augment the prediction.

Introduction

Many community behavioral health organizations (CBHOs) struggle with extensive employee turnover. The reported rates range from 25% to 60% annually across studies (1–3), contributing to the significant and chronic behavioral health provider shortage. The continuing workforce shortage is projected to enlarge the gaps in meeting the increased demand for timely and quality behavioral health care access (4). Identifying high-turnover risk employees is a critical first step to providing workforce support for retention.

Employee turnover has been studied for decades (5–9). Predictors include demographics (e.g., age, gender, education) (10), insufficient salary (11), excessive job demands (12), increased burnout, decreased job satisfaction, lack of professional development opportunities, emotional labor burden, and low organizational trust (13). Other predictors include a lack of organizational support (5), reduced positive workplace climates, fair performance appraisal, job autonomy, workplace psychological safety (14), and increased workplace interpersonal conflict (15).

As such, while we have a general understanding of turnover factors, we do not yet have an established strategy to locally apply the prediction models to our routine human resource (HR) management for identifying employees at high turnover risk. The major challenges include methodological limitations. For example, most existing studies are survey-based, cross-sectional, and correlational studies with a limited number of predictors tested for turnover intention. Although turnover intention correlates with actual turnover, their predictors can differ (16). Further, prediction weights and priorities of turnover factors for individuals in different organizational settings may differ. In particular, CBHOs consist of diverse employees with varying personal backgrounds (e.g., demographics), job characteristics (e.g., positions, salary), and job experiences (e.g., supervisory support).

To overcome these challenges, machine learning (ML) has gained popularity in predicting employee turnover (17). ML is a broad framework for extracting regularities from data using powerful automated algorithms for data-driven and person-centered outcome prediction (18, 19). A group of researchers and CBHO employees partnered to develop a strategy that applied ML models to HR data to predict employee turnover (20, 21). Their main finding was that the HR data-based ML with Random Forest (22) predicted employee turnover with good prediction accuracy (i.e., AUC [area under the curve] >.8).

However, the ML approach has two major limitations. First, the models were developed and tested with historical data with known outcomes (i.e., turnover vs non-turnover [stay]). Testing the models with new data (active cases) to predict future turnover outcomes is important. Second, although HR’s commonly collected data (such as compensation, employee demographics, job characteristics) may be predictors for actual turnover, employees’ internal/personal job experiences may indicate their future turnover through increased/decreased turnover intention (16). Because of the urgency for addressing employees’ workplace wellbeing (23), many organizations integrate job wellbeing surveys/check-ups into their HR management operations. However, employee surveys could be challenging without understanding key job wellbeing variables, avoiding the added burden for employees (who may already be overwhelmed) to complete a lengthy survey. It is essential to identify a few key job wellbeing variables with a minimum survey response burden and to integrate them into the HR data collection system to identify high-turnover risk employees.

The current study integrated HR data-based ML models and job wellbeing indicators to predict CBHO employees’ turnover probabilities within the next 12 months. The three specific aims are: testing turnover predictability of HR data using ML, identifying key job wellbeing indicators, and evaluating the gain in prediction by adding those indicators over the prediction from the ML model. Our hypotheses were (1) the ML models built based on historical data can predict current employees’ future turnover, (2) among the ML models (random forest [RF], logistic regression [LGR], lasso regression [LR]), RF will provide the best prediction based on the original ML development (20, 21) and (3) the prediction accuracy (i.e., AUC) will improve by adding job wellbeing survey items. The current study will make a significant contribution to the field by proposing an innovative approach to predicting CBHO employee turnover without placing a substantial data collection burden on employees.

Methods

Implementation Setting and Procedure

The current study was conducted at a CBHO that employed approximately 300 employees in an urban area in the Midwestern United States, which served about 2,500 clients annually. The CBHO provides evidence-based interventions to support recovery among people with severe mental illness and substance use disorders, including assertive community treatment, supported employment, medication management, and other psychosocial services. With its mission to achieve behavioral health excellence and ensure broad access to evidence-based interventions, the organization also offers training, coaching, and consultation services to promote evidence-based practices among both internal and external staff across its network.

We obtained HR and other administrative data of 354 current employees from the CBHO. The data included employees’ age, gender, race, education, marital status, exempt status, job position, past work years, hourly wage, work hours, training hours, the proportion of male clients, the proportion of clients with schizophrenia diagnoses, and average client age with whom the employees worked. The original ML models that the current study applied were trained with these variables (21).

We used the HR data-based ML models (i.e., RF, LGR, LR) for turnover prediction, which were developed with historical employees’ data (going back to 2011 for the oldest cases) (21). Briefly speaking, RF is a nonparametric method for prediction that is an extension of classification trees. LGR is a traditional regression approach for classification, and LR is a regularization technique that extends LGR by adding a penalty term for regression coefficients to improve the generalizability of the estimates. The AUC of the RF model in the original study exceeded .8 (refer to the original study for details (21)). We applied these ML models to the current employees’ data (N=354) and obtained each employee’s predicted turnover probability (, based on RF, LGR, or LR) within the next 12 months. Using provides two important benefits: 1) it carries the information from historical HR data, accounting for complex nonlinear relationships among the data due to the use of ML approaches, and 2) it reduces the number of predictors in the model for the new data, creating room for adding additional meaningful predictors.

Additionally, we sent a voluntary, confidential Qualtrics job wellbeing survey to 342 employees whose email addresses were correctly identified. The job wellbeing survey items (11 indicators rated on a 6-point scale) were developed through a community-based participatory research process (which involved collaboratively generating and validating critical job wellbeing indicators with stakeholders based on data from exit surveys, stay interviews, and employee workgroups). Each item was designed for single-item use with content validity to assess employee job wellbeing status (24). The indicators were found to be sensitive in capturing variations in job wellbeing across different staff and job characteristics. The items included perceived clarity of communication with upper management, involvement in decision making, job fairness, expectation alignment, professional development opportunities, work flexibilities, supervisory support (quality of work, wellbeing), the perception that degree pays off, the perception about being valued, and job stress, with higher scores indicating better job wellbeing except for the stress indicator. An example indicator statement is: “I have opportunities to advance my career here.” Additionally, we included an item asking how important it would be for employees to be able to talk about the 11 job wellbeing matters with their supervisor (see Appendix in online supplement). Given the short length of the survey, compensation was not provided.

Finally, after we applied the ML models to estimate the current employees’ predicted turnover probability (${\widehat{p}}_{HR}$) and collected the job wellbeing indicators, we obtained the employees’ actual turnover outcomes 12 months later. The Indiana University institutional review board approval was obtained for the study. Informed consent was obtained from the participants, and a waiver of informed consent was applied to the data already being collected by the organization.

Analyses

We first conducted a logistic regression analysis to predict the turnover outcome using the turnover probabilities predicted from HR data (${\widehat{p}}_{HR}$) with each ML method (i.e., RF, LGR, LR). Note that, of the initial 354 cases in the HR record, 98 (28%) left the organization, while 256 (72%) remained. Following the original ML development protocol, we removed cases of involuntary turnover (e.g., deceased, unsatisfactory performance, misconduct/violation, job abandonment) and those rehired, resulting in 303 cases (72[24%] voluntary turnover). Predicted turnover probabilities (${\widehat{p}}_{HR}$) were available for all 303 cases, while 162 (54%) had job wellbeing data. There was no statistically significant difference in the turnover outcome between those who completed the survey and those who did not (p=.6). Second, we conducted independent samples t-tests between the 12 job wellbeing indicators and turnover outcome for those who completed the survey. Third, we selected the significant or marginally significant job wellbeing indicators from the previous step and added them to the initial logistic regression with . Our intention was to identify a minimum number of critical job wellbeing predictors while maximizing the overall prediction accuracy. We evaluated the improvement of prediction accuracy according to AUC, based on which the original ML models were trained. AUC ≥ .8 is typically considered good (25). Because of the scale differences, all predictors were standardized.

Results

Table 1 shows the demographics and job characteristics of the 303 employees. Table 2 provides the descriptive statistics of those who responded to the job wellbeing survey. The initial logistic regression revealed that the predicted turnover probability from HR data (${\widehat{p}}_{HR}$) predicted the turnover probability significantly for RF (turnover M=.59±.26 vs. stay M=.47±.24; OR=1.56, p <.001, AUC= .63) and LR (turnover M=.48±.09 vs. stay M=.46±.07; OR=1.30, p=.04, AUC= .57). The prediction was marginally significant for LGR (turnover M=.48±.20 vs. stay M=.43±.18; OR=1.27, p=.08, AUC= .56). Among the competing models, RF demonstrated the best performance. Based on this model, those who left the organization within the next 12 months had higher ${\widehat{p}}_{HR}$.

TABLE 1.

Descriptive statistics of the predictors

Categorical Variable	N	%
Marital Status
Married	98	32
Single	203	67
Missing	2	1
Education
Associate and below	94	31
Bachelor	90	30
Masters and above	119	39
Exempt Status
Exempt	67	22
Non-exempt	236	78
Race
Black	119	39
White	181	60
Other	3	1
Employment Type
Clinical	235	78
Non-clinical	68	22
Gender
Female	219	72
Male	83	27
Position
Admin Assist/Assistant/coordinator	12	4
Direct Provider	87	29
Finance	14	5
Leadership	13	4
Nurse	18	6
Outreach/Community	14	5
Peer Provider	7	2
Residential Service	28	9
Team Leader/Manager	63	21
Therapist/Prescriber	32	11
Other	15	5
Continuous Variable	M	SD
Age	43.24	13.24
Past work years	4.83	6.01
Hourly wage	23.33	15.05
Work hours/week	35.68	4.36
Training hours	128.33	114.74
Client average age	44.30	11.48
Proportion of male clients	.58	.23
Proportion of clients with schizophrenia diagnoses	.45	.29

Note. The reference category for each categorical predictor is in italics.

TABLE 2.

Descriptive statistics of job wellbeing indicators by stay and turnover cases

Indicator	Stay Case			Turnover Case
	N	M	SD	N	M	SD	p
Communication from upper management is clear	115	3.70	1.27	37	3.54	1.14	.510
People are included in decision-making	116	3.28	1.17	38	3.16	1.22	.594
Employees are treated fairly	116	4.23	1.09	40	3.95	.99	.150
My expectations align well with the expectations of management	113	4.12	1.16	39	3.72	1.30	.075 ꝉ
My degree pays off	99	3.92	1.45	32	3.47	1.52	.133
I have opportunities to advance my career here	107	3.84	1.30	39	3.28	1.28	.022*
My work is flexible to accommodate my personal/family needs	116	4.69	1.25	40	4.43	1.34	.258
My supervisor cares about my well-being	116	4.96	1.43	40	4.90	1.22	.822
My supervisor supports the quality of my work	115	5.02	1.27	40	4.75	1.10	.238
I am acknowledged and my input is valued here	115	4.27	1.43	39	4.05	1.47	.414
Stress overweighs positive feelings that may come from my work	115	3.99	1.04	40	3.98	1.31	.937
How important would it be for you to be able to talk about these with your supervisor	115	3.60	1.21	39	3.18	1.27	.065 ꝉ

Note: The first 11 indicators were rated from Never (1) to Aways (6), where a higher rating indicates better job wellbeing (the stress indicator is reverse coded). The 12^th indicator was rated from Not important (1) to Very important (5), where a higher rating indicates that the rater perceived more importance of talking about their job wellbeing with their supervisors.

^*p < .05

^ꝉp < .10.

The independent samples t-tests showed that those who left the organization within the next 12 months rated career advancement opportunities significantly lower (p<.05). Additionally, expectation alignment and the importance of supervision marginally (p<.10) predicted the turnover outcome. Adding the three job well-being indicators to the initial logistic regression model (with only ${\widehat{p}}_{HR}$) improved the prediction accuracy (AUC) from .63 to .76 (AUC did not further improve by adding all 12 indicators). AUC = .76 means that a randomly selected employee who left the organization had a predicted probability score larger than that for a randomly chosen employee who stayed about 76% of the time. In the model, and career advancement opportunities significantly predicted the turnover outcome, controlling for the other predictors (see Table 3). Specifically, for every SD increase in ${\widehat{p}}_{HR}$, the odds of turnover increased by 115% (OR=2.15), and for every SD increase in career advancement opportunities, the odds of turnover reduced by about one-third (OR=0.61).

TABLE 3.

Key Predictors of Employee Turnover

Predictor	OR	95% CI	p
Predicted turnover probability (${\widehat{p}}_{HR}$, from RF model)	2.15	1.39–3.33	.001
My expectations align well with the expectations of management	.81	.50–1.30	.377
I have opportunities to advance my career here	.61	.38–.97	.036
How important would it be for you to be able to talk about these with your supervisor	.77	.52–1.16	.211

Note. OR = odds ratio; The predictors are standardized in the model; AUC=0.76.

Discussion

Supporting job wellbeing is a critical priority for employee retention. While there is a general understanding of factors contributing to turnover, identifying employees at high risk of leaving is challenging, impeding or delaying timely support in real work settings. The behavioral health workforce is diverse, encompassing a wide range of occupations and responsibilities. To address the challenges, the current study proposes a solution by leveraging historical and existing HR and administrative records combined with low-response-burden job wellbeing indicators. This approach demonstrates that machine learning models trained on HR and administrative data, supplemented with job wellbeing indicators, can effectively predict employee turnover within the next 12 months. Notably, the study found that employees who perceive fewer opportunities for career advancement are much more likely to leave the organization.

The RF model (17) predicted the future turnover of current employees. In other words, the predicted turnover probabilities (${\widehat{p}}_{HR}$) tended to be higher for those who left the organization within the next 12 months. More specifically, employees are slightly more than twice as likely to leave when the predicted probability increases by one unit. However, the predictive accuracy was less optimal (AUC =.63) than the original study with the historical data (AUC >.8). Therefore, the current study did not replicate the same level of prediction accuracy. This finding is not surprising. Although the historical HR data contain a relatively large number of employees’ information to inform turnover prediction, new employees may have patterns of relationships that are unique to the current time and environment. HR data alone is not sufficient to optimize prediction accuracy, given the fluidity and complexity of the turnover processes (20, 21).

Employees’ perceptions of their job wellbeing are important in turnover prediction. Accordingly, we incorporated job wellbeing indicators to enhance overall prediction accuracy. The prediction accuracy substantially improved (AUC increased from .63 to .76). The finding aligns with our hypothesis and existing literature. For example, while employee demographics and job characteristics can predict actual turnover among CBHO employees, perceived job wellbeing experiences are more closely linked to turnover through changes in turnover intention (16). Turnover intention is known as one of the strongest turnover predictors (6). Job wellbeing indicators may help capture employees’ workplace experiences while avoiding direct turnover questions (e.g., “How likely are you to leave?”), which are often suboptimal. Identifying early signs of dissatisfaction through job wellbeing indicators may provide important cues for addressing future turnover.

The current study suggests a few key job wellbeing indicators for predicting future turnover while minimizing the psychological burden of the inquiry about the reasons behind turnover intention. Among the 12 candidate job wellbeing indicators, three indicators emerged as significant (i.e., career advancement opportunities) or marginally significant (expectation alignment and importance of supervision) predictors for turnover within the next 12 months. This is consistent with the previous studies, where career advancement opportunities (26, 27) and expectation alignment in their job role (28) have been identified as critical turnover or retention predictors. The perceived importance of talking with supervisors about job wellbeing challenges is also consistent with the literature. For example, supervisors’ support can help reduce actual turnover via reduced turnover intention by addressing job wellbeing (29). Employees who perceive greater value in discussing job wellbeing with their supervisors may be better positioned to address workplace challenges than those who feel such conversations are ineffective (e.g., due to limited supervisory support or strained relationships). After controlling for all factors, we found that employees with more career development opportunities are significantly less likely to leave the organization—specifically, about one-third less likely for every one standard deviation increase in perceived opportunities.

There are some limitations to be acknowledged for this study. First, the study was conducted at a single site with a limited sample size; thus, the findings may not generalize to other settings. For instance, although the career advancement opportunity indicator was identified as a strong predictor for turnover in our study site, this can differ for different organizations. Second, while our approach can identify high turnover-risk individuals, further investigations are needed to understand the fine mechanisms to develop individually tailored retention strategies. This is a common challenge of the ML classification approach (17), where the prediction mechanisms are often hidden in the black box.

Despite the limitations, the current study has significant contributions. To our knowledge, this is the first study that integrated ML models and employee job wellbeing indicators to predict current employees’ future turnover. Specific contributions follow. First, our approach leverages existing HR data, which carries many years of information from a greater sample, to identify high turnover-risk individuals for intervention. Second, the study identified critical job wellbeing indicators for HR to incorporate with a relatively low response burden for employees. By adding the three job wellbeing questions, HRs are more likely to identify high-risk employees with greater prediction accuracy. Third, the job wellbeing items can be used as brief measures for general job wellbeing and turnover surveys. Adding job wellbeing indicators to surveys may help predict turnover outcomes and understand the factors.

Implications for future study directions follow. To advance this work, it is important to implement the proposed approach across diverse settings. Integrating job wellbeing indicators into the HR’s routine data collection and conducting a large-scale multi-site trial would provide robust validation of the prediction model. Such trials may uncover both common and context-specific patterns in turnover prediction. While shared patterns could inform broader workforce retention policies, organization-specific insights may guide tailored interventions. To enhance scalability—especially for resource-limited CBHOs— a dashboard could be developed to deploy the current approaches in a timely manner through internal efforts (e.g., uploading HR data would automatically generate individual predicted turnover probabilities[${\widehat{p}}_{HR}$]). Additionally, artificial intelligence (AI) could be employed to distribute and collect job wellbeing surveys, which may not only increase procedural efficiency but also encourage honest responses by anonymizing data before it is used by employers to estimate individual turnover probabilities. Importantly, predicted turnover probabilities should be used to support, not penalize, employees. For instance, HR teams may analyze patterns among individuals with elevated turnover risk (e.g., demographics such as age, gender, race, marital status; departments/teams; client populations, including their age and intensity of care needs; work locations; team leaders/supervisors) to identify structural or cultural issues (e.g., workload distribution, equity concerns). If risks are concentrated within specific demographics, job roles, departments, teams, or under particular supervisors, then targeted training and adequate resource allocation may help address underlying issues and improve staff retention.

Conclusions

A major innovation lies in the application of ML prediction models to new data for predicting actual turnover within the next 12 months, given that many ML evaluations use historical data with a known outcome. ML models trained with historical data may have limited external validity when applied to future cases. The study integrated HR data-based ML and job wellbeing indicators to enhance turnover prediction. The proposed approach offers a practical, low-effort method for identifying employees at elevated risk of turnover, potentially enabling timely, context-based, and targeted retention strategies.

Highlights.

• Identifying employees at high turnover risk with a viable and low labor-intensive method can be a significant contribution to the psychiatric service field, which has struggled with excessive turnover and workforce shortages.

• The study demonstrates the predictabilities of HR-data machine learning and job wellbeing indicators for behavioral health employee turnover within the next 12 months.

• Lack of job development opportunities, misalignment of expectations in the job, and lower expectations towards supervisors about discussing job wellbeing could lead to future turnover.

• HR-data machine learning and job wellbeing indicators are promising approaches for supporting the behavioral health workforce.