Using Machine Learning Methods to Examine Turnover Rates in State Health Agencies

Abstract

Context:

High turnover rates in the public health workforce pose ongoing challenges to maintain essential services and institutional knowledge. Recent studies indicate that job dissatisfaction, burnout, and structural barriers have intensified following the COVID-19 pandemic. While prior studies have identified key predictors of turnover intention, the potential of machine learning (ML) to improve predictive accuracy and guide targeted interventions remains underexplored.

Objective:

This study applied ML techniques to examine the predictors of turnover intent and to simulate the impact of workplace satisfaction improvements among state health agency employees.

Methods:

We used data from 4 waves of the nationally representative Public Health Workforce Interests and Needs Survey: 2014, 2017, 2021, and 2024. Focusing on state health agency central office staff, we trained 3 ML models—Lasso Regression, Random Forest, and Gradient Boosting—to predict intent to leave one’s organization within the next year. Models were trained separately by year using cross-sectional data and evaluated. Variable importance was assessed, and a simulation was conducted to evaluate the potential reduction in predicted turnover following targeted improvements in job satisfaction, organizational satisfaction, and pay satisfaction.

Results:

All models demonstrated strong performance, with area under the receiver operating characteristic curve values ranging from 0.78 to 0.85. Job satisfaction consistently emerged as the most important predictor across all models and years, followed by organizational and pay satisfaction. Lasso Regression generally achieved the highest sensitivity and accuracy. Simulation results showed that modest improvements in satisfaction variables could substantially reduce predicted turnover intent, particularly among early- and mid-career staff.

Conclusion:

This study highlights the value of ML for identifying key predictors of turnover intention. Findings reinforce the importance of job satisfaction, organizational climate, and compensation in retaining public health staff. ML-driven tools can support more proactive, data-informed retention strategies in the governmental public health system.

Introduction

The public health workforce is critical to safeguarding and improving population health, yet it faces persistent challenges with retaining skilled professionals. In 2013, local health departments experienced a mean turnover rate of 10%,1 and from 2008 to 2019, the number of local health department full-time equivalents declined from 162 000 to 136 000—a decrease of approximately 16%.2 Adding to these concerns, approximately one-third of the public health workforce has reported contemplating leaving their positions at some point in the future.3 Against this backdrop of rising turnover and shrinking staff, 1 recent estimate suggests that public health agencies nationwide required an additional 80 000 full-time employees to adequately deliver foundational public health services prior to COVID-19.4 Factors such as budget cuts, the COVID-19 pandemic, and workplace dissatisfaction have only exacerbated these patterns.5 The federal policy and funding environment has also created substantial tumult in the workforce.

High turnover creates significant challenges for both organizations and individuals. From an organizational perspective, public health agencies incur additional costs for recruitment, hiring, and training,6,7 while the loss of institutional knowledge and temporary drops in productivity can undermine essential health services.8-10 Meanwhile, individuals who leave may face financial instability, career interruptions, and reduced well-being.11

Several studies have identified pay dissatisfaction and overall job satisfaction as consistent predictors of turnover intentions.6-13 In addition, factors such as heavy workloads, limited opportunities for advancement, emotional labor, low organizational trust, interpersonal conflicts, and unsupportive workplace environments have been shown to increase turnover risk.13 Conversely, a positive workplace climate, fair performance appraisals, strong organizational support, effective supervision, a clear path for advancement, and a sense of psychological safety have been found to reduce turnover intentions.12-14

These findings highlight the multifaceted and systemic nature of turnover in the public health workforce, further complicated by external shocks like the pandemic. Addressing these issues calls for a combination of policy reform, organizational change, and data-driven tools to better anticipate and mitigate staff departures. While the practical application of machine learning (ML) in public health workforce planning is still emerging, ML offers considerable promise due to its ability to process large, complex, and heterogeneous datasets. These techniques rely on computational algorithms that systematically identify patterns in data, allowing models to learn from past observations and make informed, data-driven predictions or decisions.15 As such, ML holds potential for uncovering nuanced predictors of workforce behavior that might be overlooked by traditional analytical methods.

In this study, we applied ML methods to the Public Health Workforce Interests and Needs Survey (PH WINS) from 2014, 2017, 2021, and 2024 to predict employee intent to leave. Focusing on state health agency staff, we developed and evaluated predictive models to identify key variables associated with turnover. To further explore the practical implications of these findings, we conducted simulations in which we altered key predictors to assess how changes in these factors would influence predicted turnover rates.

Methods

This project utilizes data from the PH WINS, a nationally representative online survey administered by the de Beaumont Foundation in collaboration with the Association of State and Territorial Health Officials.

We restricted the dataset to include only respondents employed at the State Health Agency-Central Office settings because local data only became available in PH WINS starting in 2017. We conducted a complete case analysis using data from 4 survey waves: 2014 (n = 10 794), 2017 (n = 18 491), 2021 (n = 14 981), and 2024 (n = 18 110). Missing responses for this item were modest, ranging from 3.2% to 8.2% across survey waves, and were excluded from the analysis.

The primary outcome of interest is intent around voluntary separation for reasons other than retirement, based on the question: “Are you considering leaving your organization within the next year?” Across survey years, most participants reported no intention to leave. However, this share has varied over time: 79% in 2014, 69% in 2017, 71% in 2021, and 76% in 2024 (see Supplementary Digital Content Appendix Table A1, available at http://links.lww.com/JPHMP/B607).

We identified both categorical and continuous variables as predictors (commonly referred to as “features” in the ML literature). To align with conventions in the social and behavioral sciences, we use the term predictors throughout. Categorical predictors included gender, race/ethnicity, age-group, education level, supervisory status, mental health status, job satisfaction, organizational satisfaction, workplace engagement, and perceptions of training and development. The primary continuous predictor was tenure (in years).

It is important to note that not all predictors were consistently available across all survey years. Therefore, models were estimated separately for each wave—2014, 2017, 2021, and 2024. Supplementary Digital Content Appendix Table A2, available at http://links.lww.com/JPHMP/B607 summarizes the availability of each variable by survey year. All analyses were performed using cross-sectional data from each PH WINS wave independently, to account for potential year-specific effects, particularly the 2021 survey, which may reflect workforce dynamics during the COVID-19 pandemic. For 2014-2021, variables were included based on previously published literature. In 2024, all job satisfaction and workplace environment variables were included for ML analysis, along with demographic variables. All data processing and analysis were conducted in R version 4.4.1.

ML Applications

We employed both parametric and nonparametric ML approaches in this study. As our parametric method, we used Lasso regression due to its close connection to logistic regression, a widely used traditional classification technique. Lasso enhances logistic regression by incorporating a regularization component that penalizes the absolute values of regression coefficients, encouraging sparsity and simpler models.16,17 Although this introduces some bias, it reduces prediction variance, thereby improving the model’s generalizability.18 Lasso is particularly useful in handling high-dimensional data, allowing for more complex models and effective variable selection, including interaction terms, compared with standard logistic regression.

For the nonparametric approach, we applied 2 ensemble tree-based methods: random forest and gradient boosting. Both extend the traditional classification tree model, which splits data into homogeneous groups based on predictor variables and captures complex nonlinear relationships. However, single decision trees are often sensitive to noise and prone to overfitting.19,20 Random forest addresses these limitations by constructing multiple trees on different resampled subsets of the data and averaging their predictions.21 Additionally, it randomly selects a subset of predictors at each split, which helps reduce redundancy and improves the model’s stability and generalization performance.21 Gradient boosting, on the other hand, builds trees sequentially, with each new tree trained to correct the errors of the existing ensemble.22 This iterative refinement typically yields higher predictive accuracy but requires careful tuning to avoid overfitting. When well-calibrated, gradient boosting is particularly effective for fine-grained prediction tasks, making it a strong complement to random forest.23,24

To further investigate the impact of modifiable workplace factors, we conducted a simulation analysis using the PH WINS 2024 test dataset. This simulation was informed by the results of our ML models—Lasso regression, random forest, and gradient boosting, which consistently identified job satisfaction, organizational satisfaction, and pay satisfaction as among the most important predictors of turnover intention, based on variable importance rankings. To assess how improvements in these key domains might affect predicted turnover, we simulated targeted increases in satisfaction levels within each age-group (≤35, 36-45, 46-60, and ≥61). Specifically, for each satisfaction domain, 20% of individuals who originally selected “Disagree” (score = 2) were reassigned to “Agree” (score = 4), and 20% of those who selected “Agree” (score = 4) were reassigned to “Strongly Agree” (score = 5). The trained models were then rerun on the simulated test dataset to estimate the predicted probability of leaving under these improved satisfaction scenarios. This simulation framework allowed us to examine the potential impact of targeted satisfaction improvements on turnover risk across different age groups.

Results

Table 1 summarizes the performance of 3 ML models—Lasso Regression, Random Forest, and Gradient Boosting—applied to 4 waves of PH WINS data (2014, 2017, 2021, and 2024). Each model was trained and evaluated separately using consistent performance metrics: area under the receiver operating characteristic curve (AUC), sensitivity, specificity, overall accuracy, and the P-value comparing model accuracy to the no information rate (Acc > NIR).

TABLE 1.

Performance Assessment of Parametric and Nonparametric Machine Learning Models

Year 2014
Criterion	Random Forest	Gradient Boosting	Lasso Regression
AUC	0.80	0.78	0.80
Sensitivity	0.75	0.70	0.76
Specificity	0.75	0.71	0.69
Accuracy	0.74	0.70	0.75
P value (Acc > NIR)	.99	.99	.99
Year 2017
Criterion	Random Forest	Gradient Boosting	Lasso Regression
AUC	0.81	0.80	0.81
Sensitivity	0.73	0.74	0.72
Specificity	0.74	0.73	0.75
Accuracy	0.73	0.73	0.73
P value (Acc > NIR)	6.00e-11	1.33e-12	1.62e-11
Year 2021
Criterion	Random Forest	Gradient Boosting	Lasso Regression
AUC	0.82	0.81	0.83
Sensitivity	0.77	0.76	0.85
Specificity	0.73	0.72	0.65
Accuracy	0.76	0.75	0.80
P value (Acc > NIR)	5.96e-07	2.20e-16	2.00e-16
Year 2024
Criterion	Random Forest	Gradient Boosting	Lasso Regression
AUC	0.84	0.83	0.85
Sensitivity	0.81	0.82	0.80
Specificity	0.72	0.69	0.74
Accuracy	0.79	0.79	0.78
P value (Acc > NIR)	2.10e-07	2.56e-07	1.79e-05

Abbreviations: AUC, area under the receiver operating characteristic curve; NIR, no information rate.

The area under the receiver operating characteristic curve, which measures the model’s ability to distinguish between employees who do and do not intend to leave, ranged from 0.78 to 0.85 across all models and years, indicating generally strong performance. Lasso regression achieved the highest AUC and accuracy in both 2021 (AUC = 0.83, accuracy = 0.80) and 2024 (AUC = 0.85, accuracy = 0.78), and also demonstrated the highest sensitivity—the ability to correctly identify those likely to leave—especially in 2021 (0.85). This suggests that it is particularly useful for flagging potential leavers. Random Forest consistently delivered the highest specificity—the ability to correctly identify those likely to stay—particularly in 2014 and 2021 (both 0.75 and 0.73, respectively). This makes it well suited for minimizing false positives in retention targeting. Gradient Boosting generally offered the most balanced trade-off between sensitivity and specificity, especially in 2017 and 2021. In 2017, it matched the accuracy of the other models (0.73) and had nearly equal sensitivity (0.74) and specificity (0.73). In 2021, while slightly trailing Lasso in accuracy, it still performed well across both metrics.

While overall model performance was strong, statistical significance varied by year. In 2014, none of the models significantly outperformed the No Information Rate, indicating limited predictive power during that period. In contrast, from 2017 onward, all models achieved statistically significant accuracy improvements. For instance, Lasso regression’s P-values dropped to 1.62e-11 in 2017 and 2.00e-16 in 2021, reinforcing its strong predictive utility.

In addition to evaluating predictive performance, we explored the key drivers of turnover intention by examining variable importance across all 4 survey years. Figure 1 displays the top-ranked predictors for each model—Random Forest, Gradient Boosting, and Lasso Regression—highlighting their relative contributions to model predictions for the year 2024. Variable importance plots for 2014, 2017, and 2021 are included in Supplementary Digital Content Appendix Figure A1, available at http://links.lww.com/JPHMP/B607 for reference. For Random Forest and Gradient Boosting, variable importance reflects each predictor’s role in improving model accuracy through splits and node purity. For Lasso Regression, importance is represented by the magnitude of standardized regression coefficients.

FIGURE 1.

Variable Importance Plot

Job satisfaction consistently emerged as the most influential predictor across all years and models, underscoring its foundational role in turnover intention. In 2014, job satisfaction, organizational satisfaction, and workplace recommendation were already among the top predictors across models. Demographic factors such as age and race appeared with moderate importance in Lasso but were largely absent from the tree-based models. By 2017, the influence of job satisfaction became even more pronounced. Organizational satisfaction and pay satisfaction rose in prominence, consistently ranking in the top 5 features. The relative importance of demographic characteristics further declined, especially in Random Forest and Gradient Boosting. In 2024, job satisfaction remained dominant. Psychological safety, burnout, and supervisor satisfaction gained greater importance. New predictors like “love my job” and “voice concerns” ranked highly, particularly in Random Forest and Gradient Boosting. In contrast, demographic and structural variables—such as age, race, and tenure—had diminished influence across all models.

To assess the potential impact of improving workplace satisfaction, we conducted a simulation in which job satisfaction, organizational satisfaction, and pay satisfaction were modestly increased across 4 age-groups: <35, 36 to 45, 46 to 60, and 61+. We chose to stratify by age because age often aligns with career stage and potential responsiveness to workplace interventions, and prior research has shown age to be positively related to job satisfaction through differences in work values, job rewards, and broader life-course processes.25-28

Specifically, for each domain, 20% of respondents who initially selected “Disagree” were reassigned to “Agree,” and 20% who selected “Agree” were reassigned to “Strongly Agree.” Table 2 presents the results of this simulation across the 3 modeling approaches: Random Forest, Gradient Boosting, and Lasso Regression.

TABLE 2.

Predicted Number of Leavers Before and After Simulated Satisfaction Improvements, by Age-Group and Model

Age-Group	Model	Original Predicted Leavers	Simulated Predicted Leavers	% Reduction
<35	Random Forest	432	426	1.39
	Gradient Boosting	446	419	6.05
	Lasso Regression	473	417	11.84
36-45	Random Forest	394	379	3.81
	Gradient Boosting	383	350	8.62
	Lasso Regression	447	388	13.20
46-60	Random Forest	435	424	2.53
	Gradient Boosting	392	350	10.71
	Lasso Regression	412	349	15.29
61+	Random Forest	72	72	0.00
	Gradient Boosting	72	55	23.61
	Lasso Regression	58	51	12.07

Across all age-groups and models, simulated improvements in satisfaction led to a reduction in the number of staff predicted to leave. The <35 age-group experienced the most pronounced reduction in the Lasso Regression model, with a predicted decline of 11.84%, compared to 6.05% in the Gradient Boosting model and 1.39% in the Random Forest model. A similar pattern was observed in the 36-45 groups, where Lasso predicted a 13.20% reduction, Gradient Boosting 8.62%, and Random Forest 3.81%. For the 46 to 60 age-group, reductions were again strongest in the Lasso model at 15.29%, followed by 10.71% in Gradient Boosting and 2.53% in Random Forest. Even among those aged 61 and older—who had the lowest baseline turnover risk—improvements in satisfaction were associated with predicted declines in intent to leave: Gradient Boosting showed a 23.61% reduction, Lasso 12.07%, while Random Forest showed no change.

Discussion

This study used 4 waves of PH WINS data to examine predictors of turnover intention among state health agency staff using 3 ML approaches. Across all years, job satisfaction emerged as the most influential predictor, with organizational satisfaction and pay satisfaction also ranking highly. These findings reinforce longstanding evidence that satisfaction-related factors are central to public health workforce retention. These variables, consistently led across multiple models and years, highlight their enduring relevance.

While all models performed well, Lasso regression generally delivered higher sensitivity and accuracy, particularly in 2021 and 2024. This suggests that regularized linear models can be just as effective—if not more so—than more complex ensemble methods in certain applied settings. Gradient Boosting and Random Forest also produced strong results, each offering different strengths in capturing predictor relationships and handling feature interactions.

The simulation analysis offered additional insight into how improvements in workplace satisfaction could reduce predicted turnover. Across most age-groups, the Lasso Regression model consistently produced the largest percentage reductions in intent to leave, suggesting that it was particularly sensitive to changes in satisfaction-related variables. Gradient Boosting also demonstrated strong effects, including the largest reduction among employees aged 61 and older, while Random Forest generally yielded smaller but consistent improvements. The most pronounced reductions were observed among younger and mid-career staff—groups often more responsive to workplace interventions. These findings suggest that even modest gains in job, organizational, and pay satisfaction could translate into meaningful decreases in predicted attrition, reinforcing the potential of targeted strategies to enhance workforce retention.

From a workforce planning perspective, these findings support targeted, data-informed strategies for retention. Rather than focusing solely on structural or demographic factors, agencies may achieve more by improving workplace climate, communication, and perceived support. In the context of state health agencies, this could include strategies such as enhancing opportunities for career advancement and training, implementing mentorship programs for early- and mid-career staff, improving transparency in decision making, and regularly assessing and addressing employee satisfaction. Strengthening supervisor-employee relationships and expanding recognition and reward systems may also help retain skilled staff.

It is also important to acknowledge that retention strategies operate within a broader policy and funding environment. State health agencies are heavily reliant on federal funding, primarily from the US Department of Health and Human Services, to support personnel and program activities.29 This dependency can create vulnerability when federal priorities shift or funding levels change, influencing workforce stability and the scope of agency activities. Consequently, workforce retention efforts should be paired with advocacy for sustainable, flexible funding streams that allow agencies to maintain critical positions and invest in long-term capacity-building, even during periods of federal funding uncertainty. While such interventions require time and investment, the potential return, reduced recruitment and training costs, continuity in services, and a more engaged and stable workforce are substantial.

Limitations

Several limitations warrant consideration. First, the analysis is based on cross-sectional data, so it cannot speak to actual turnover or long-term behavioral change. Second, simulated changes are hypothetical and assume uniform response to satisfaction improvements. Real-world interventions may yield more variable effects. Third, some variables were not consistently available across years, which could affect comparability.

Future research should explore longitudinal data to track actual turnover outcomes and test the impact of organizational interventions over time. Expanding the model to include agency-level factors—such as leadership, funding shifts, or public health emergencies—may also help capture important context for workforce dynamics.

Implications for Policy & Practice

• Job satisfaction, organizational satisfaction, and pay satisfaction are the strongest predictors of turnover intent across state health agencies.

• Targeted efforts to improve satisfaction—especially among early- and mid-career staff—could significantly reduce attrition.

• Predictive models can reliably identify employees at risk of leaving, allowing agencies to design proactive retention strategies.

• Leveraging national organizations, internal leadership, and peer learning networks can guide the identification and implementation of proven retention strategies in governmental public health.

Conclusion

The governmental public health enterprise is facing monumental challenges—to funding, political support, and a workforce that may well feel attacked in a post-COVID environment. With a loss of tens of thousands of jobs through involuntary turnover, limiting voluntary turnover is of paramount importance. Over the last decade, PH WINS has allowed for a robust investigation of intent to leave in the public health workforce. Researchers have found, consistently, that job satisfaction and workplace engagement are critical in retention, and recent PH WINS data also highlight the role of self-reported mental health—particularly among employees under age 35—and the moral injury experienced by many in the field.30 The size and scope of PH WINS allows for sophisticated ML techniques to examine these assumptions even more closely. Machine learning offers a valuable lens for understanding and anticipating turnover risk in public health. By identifying the predictors that matter most and simulating the effects of workplace changes, we have identified practical directions for retention strategies in state health agencies and the workforce more broadly.

In governmental health agencies, where institutional change may be challenging, identifying best practices to improve satisfaction and engagement can involve leveraging existing resources, as well as collaborating with human resources, workforce development units, and leadership teams within the agency. Peer learning collaboratives, staff climate surveys, and analysis of high-performing divisions can also help surface actionable strategies tailored to the agency context.