The influence of field-based training on caseworker turnover

Abstract

Caseworker turnover is a persistent problem for child welfare agencies. This study examines whether field-based pre-service training decreases turnover and examines which organizational factors mediate the effect of training on turnover. We used the population of caseworkers (N = 2365) hired into three caseworker roles during the transition from classroom-based to field-based training in a large U.S. state to compare differences in turnover between classroom-trained and field-trained caseworkers using discrete-time logistic regressions. We find that field-trained conservatorship caseworkers have 39% lower odds of leaving the agency within 18 months of hire compared to similar classroom-trained caseworkers.

We examined whether organizational factors explain the effect of training on turnover rates among a sample of surveyed conservatorship caseworkers (72% response rate). We conducted decomposition models to determine the direct and indirect effects of training on turnover. We found that job satisfaction fully mediates the turnover effect and caseworker burnout partially mediates the effect of field-based training on turnover. Though we observed no effect on turnover, field-trained caseworkers in investigations and family-based safety services reported higher job satisfaction. The findings provide the first empirical support for an industry trend toward field-based training.

Maintaining a high-quality, professional, and stable workforce is integral to a child welfare agency’s capacity to keep children safe from abuse and neglect. Recruitment and retention of high-quality staff are persistent challenges for child welfare agencies, however. In the United States, between 14% and 22% of frontline caseworkers at state child welfare agencies leave annually (Edwards & Wildeman, 2018). Caseworkers commonly leave because of low pay, high caseloads and heavy administrative burdens, lack of supervisor support, experiencing burnout, and inadequate training (Curry, McCarragher, & Dellmann-Jenkins, 2005; Mor Barak, Nissly, & Levin, 2001; Pietrowiak & Gambino, 2003; Zlotnick et al., 2005). Caseworker turnover disrupts service provision to children and families and increases workloads of remaining staff. Recruiting and training staff is also costly for child welfare agencies (Strolin, McCarthy, & Caringi, 2007).

One way that agencies work to decrease turnover is through improvements to pre-service training. In the past several decades, most U.S. child welfare agencies have implemented a field-based approach to pre-service training, including elements such as peer mentorship, scenario role play, and job shadowing along with classroom training (Armstrong, Coy, McNeish, Menendez, & Policella, 2013). The goal of these approaches is to allow caseworkers to assess whether the position is what they expected and facilitate skill-building by allowing new caseworkers the opportunity to apply what they learn in classroom training directly to practice (Armstrong et al., 2013).

In this study we examine how the introduction of a field-based training model in a large U.S. state influenced caseworker turnover rates during the first 18 months on the job. The first 18 months of job tenure is a meaningful measure because, nationwide, the average caseworker leaves before reaching two years of tenure (Edwards & Wildeman, 2018). Additionally, we examine the extent to which organizational factors mediate the relationship between training and turnover to better understand the pathways by which training can influence retention.

1. Background

High turnover among caseworkers inhibits the ability of child welfare agencies to provide high-quality services to children and families. Turnover is particularly harmful for social service agencies because turnover disrupts casework and client-worker relationships (Strolin et al., 2007). Turnover also burdens remaining staff members who must cover the cases of exiting workers and requires agencies to spend additional funds on recruitment, training, and onboarding (Strolin et al., 2007). High turnover rates can also discourage new workers from entering the child welfare field because turnover contributes to a perception of instability and low job satisfaction within agencies (Balfour & Neff, 1993; Mor Barak et al., 2001).

Many agencies and child welfare experts promote field-based training as a way to increase retention, but existing research does not examine whether field-based pre-service training models for new caseworkers are associated with lower caseworker turnover (Armstrong et al., 2013; Cross, Goulet, Evans, & Tittle, 2017; Curry et al., 2005; Radey, Schelbe, & King, 2019). Similarly, past research identifies numerous organizational characteristics that promote retention of child welfare workers, but does not examine whether field-based pre-service training is an effective way to improve the organizational characteristics linked to retention and intended retention (Barth, Lloyd, Christ, Chapman, & Dickinson, 2008; Benton, 2016; Chen & Scannapieco, 2010; Curry et al., 2005; Kim & Kao, 2014; Lee, Forster, & Rehner, 2011; Mor Barak et al., 2001; Scannapieco & Connell-Carrick, 2007; Zlotnick et al., 2005).

Previous research has consistently found that caseworkers who were more satisfied with their jobs were less likely to express an intention to quit (Chen & Scannapieco, 2010; Kim & Kao, 2014; Mor Barak et al., 2001; Weaver, Chang, Clark, & Rhee, 2007; Zlotnick et al., 2005). Despite a consistent link between job satisfaction and intention to quit, however, Weaver et al. (2007) demonstrate that though caseworkers who are more satisfied with their jobs are less likely to report an intention to leave, job satisfaction is not associated with lower odds of actually leaving. Field-based training provides caseworkers with the opportunity to experience their job responsibilities firsthand during training, which could increase job satisfaction by allowing caseworkers to understand and set realistic expectations for their role early on.

Prior research also consistently links a positive work environment to retention. Kim and Kao (2014) found that caseworkers with more positive perceptions of their workplace climate were less likely to report that they intended to leave. Positive perceptions of workplace organizational culture, defined by the quality of supervision, level of coworker support, and a climate of professional commitment, have been linked to higher intentions to remain employed in child welfare (Lee et al., 2011). Similarly, previous studies have linked higher levels of supervisor support to increased retention (Benton, 2016) and higher rates of intended retention (Chen & Scannapieco, 2010). Additionally, research has demonstrated that retained caseworkers were more likely to report that their supervisors facilitated learning and enthusiasm, and found that retained caseworkers spent more time with their supervisors than exiting caseworkers (Barth et al., 2008; Scannapieco & Connell-Carrick, 2007), demonstrating that supportive supervision can influence retention decisions.

Past research also links coworker support to retention. Griffiths, Royse, Culver, Piescher, and Zhang (2017) found that satisfaction with peer support was one of the key factors influencing workers’ overall job satisfaction and driving decisions to remain employed. Curry et al. (2005) found that coworker support was significantly associated specifically with the retention of less experienced workers, especially during the training stage. However, He, Phillips, Lizano, Rienks, and Leake (2018) found that above-average frequency of peer support is linked to higher burnout, theorizing that perhaps struggling caseworkers are more likely to seek out peer support.

Importantly, several studies have demonstrated that support from coworkers and supervisors can mitigate job stress or burnout, as well as potentially influence job satisfaction. Caseworkers who reported higher levels of job burnout, defined as the sense of “emotional exhaustion, depersonalization, and reduced personal accomplishment that can occur among individuals who work with other people in some capacity” (Maslach, Jackson, & Leiter, 1996) were more likely to intend to leave (Mor Barak et al., 2001; Zlotnick et al., 2005). Benton (2016) found that supportive factors, particularly supportive supervision, mitigates the influence of burnout on turnover, and He et al. (2018) demonstrated that high-quality supervision can mitigate the effects of job stress to protect against early exit.

Similarly, Lee et al. (2011) found that a high level of organizational support for caseworkers is associated with higher job satisfaction and an increased ability to positively cope with job stress. Field-based training places caseworkers in real-life job scenarios surrounded by their colleagues and supervisor during training. Training in a real-life environment potentially increases the extent to which caseworkers feel supported and able to cope with challenges by allowing caseworkers to develop relationships with other staff who they can lean on later when they are conducting casework independently.

Caseworkers’ perception of their own skills is also linked to retention. Specifically, caseworkers who reported higher levels of self-efficacy, confidence, and competence in their skills were more likely to remain in their position (Chen & Scannapieco, 2010; Mor Barak et al., 2001). By providing caseworkers with extensive opportunities to directly practice skills needed for their job, field-based training should lead to higher competence among new caseworkers.

2. Conceptual model

This study fills a gap in the existing child welfare workforce literature by testing the effectiveness of a field-based pre-service training model in decreasing caseworker turnover. We compare the turnover rates of caseworkers trained in a field-based model to caseworkers trained in a classroom-based model during the same timeframe. We also connect existing research on organizational factors and turnover by examining which subjective job experience factors mediate the effectiveness of field-based training. We analyze the hazard of exiting the caseworker role for all new caseworkers hired in a large U.S. state between October 2014 and April 2016 during the regional rollout of a field-based pre-service training program.

2.1. Classroom-based and field-based training

In the classroom training model, all training took place in a classroom setting with caseworkers spending most of the three-month training period with other new caseworkers and professional trainers. The new training model (field-based training) changed pre-service training in two key ways. First, field-based training shifted the focus of pre-service training from classroom-based training to training in the field in real-life scenarios and, second, it introduced a competency-based approach that emphasizes mastery of skills, rather than simply knowledge recitation.

Field-based training combines some classroom training with extensive field training opportunities. Caseworkers spend three to four weeks in the classroom and 10 weeks in the field. During the time in the field, new caseworkers shadow a veteran caseworker mentor. Caseworker mentors demonstrate casework activities to new caseworkers and gradually allow the new caseworkers to complete portions of the mentors’ casework, building the new caseworker’s skills and confidence in preparation for independent casework. In addition to exposing new caseworkers to actual job scenarios, field-based training provides new caseworkers more time to develop a relationship with their new unit and supervisor early in training to begin building the rapport and trust that can help caseworkers feel more supported when they begin casework. In contrast, in the classroom-based training, caseworkers spent the training period with other new caseworkers and staff trainers, isolated from their unit and supervisor.

Another important shift is that field-based training is competency-based, meaning that caseworkers demonstrate mastery of competencies in the field, in addition to demonstrating knowledge of key concepts through a written assessment. Additionally, the field-based model incorporates a formal graduated caseload policy in which caseworkers are supposed to work one-third of the average caseload after completing three months of training and work up to a full caseload by their sixth month on the job. By providing caseworkers with extensive field-training opportunities focused on skill- and competency-building throughout pre-service training, the field-based model was designed to give caseworkers a more realistic picture of the job early on in training and increase the competence of caseworkers when they complete training.

2.2. Caseworker specialties

Under the field-based model, newly-hired caseworkers complete training broadly relevant to frontline child welfare staff, as well as training specific to their role, or specialty. Caseworkers are hired to work in one of three main specialties: conservatorship, family-based safety services (family-based), or investigations. Conservatorship caseworkers monitor the care of children who are in the custody of the agency, ensure families receive needed services, search for placements for children in care, and make recommendations to the court about the child’s permanent placement. Family-based caseworkers work with families when risk factors for abuse and neglect are present to provide services and reduce the risk of maltreatment while the children remain at home. Investigative caseworkers investigate alleged abuse and neglect to determine if the allegation(s) can be substantiated and whether the child must be removed from his or her home because of ongoing risk of abuse and neglect.

The caseworker position is a demanding role with a high level of responsibility, long and sometimes irregular hours, and difficult job requirements. The specific demands caseworkers face on the job, however, differ across specialties. In addition to variation in the demands placed on caseworkers, variation in supervision experiences, coworker relationships, and work culture may also influence caseworkers’ decisions to leave or stay. Each of these factors may make it difficult for training to substantively decrease turnover, particularly once the training period ends.

2.3. Purpose of current study

The purpose of the current study is to examine whether a new field-based training model leads to lower turnover among newly-hired caseworkers compared to the old classroom training model. The study also identifies any organizational factors that mediate, or explain, the relationship between training and turnover. Specifically, the current study addresses the following research questions:

1. Turnover Analysis: Are caseworkers trained under the field-based training model less likely to leave the agency prior to reaching 18 months of tenure? Does the training model affect turnover similarly across caseworker specialties?

2. Mediation Analysis: Do organizational factors explain the relationship between training and turnover? Does field-based training influence turnover indirectly through higher job satisfaction, lower burnout, higher organizational support, higher competence, and more coworker and supervisor support?

The current study fills the gap in existing literature by examining the link between field-based training and turnover through a natural experiment with a large sample representing the full population of new caseworkers hired across one and a half years to the state child welfare agency. Additionally, the study adds to the research base by examining turnover longitudinally over 18 months and examining caseworkers with different job functions separately. This study also connects to previous research that links organizational factors to retention by examining the extent to which job experience factors mediate the relationship between training and turnover to better understand the pathways by which training can improve turnover.

3. Method

3.1. Data sources

This study uses confidential human resources administrative data maintained by the state child welfare agency and provided to the research team through a restricted data use agreement. The human resources administrative data include: dates of hire and exit, date of birth, race and ethnicity, gender, caseworker specialty, and region of hire for all staff employed by the agency during the time period. The researchers also obtained data on the caseload of caseworkers from the state’s case management administrative data. Both data sources provide current information through December 31, 2017.

This study also uses caseworker survey data from the Statewide Survey, a repeated cross-sectional survey developed by the authors and administered to all new caseworkers (including classroom-trained and field-trained caseworkers) hired between January 1, 2015, and March 31, 2016. We developed the Statewide Survey as part of a program evaluation of the new training model and administered a repeated cross-section in November 2015 and May 2016. The survey assesses caseworkers’ training experience and early job experience through a series of items assessing organizational factors. Several of the organizational factor scales consist of items pulled from existing scales, including the Job Satisfaction Survey (Spector, 1985), the Survey of Employee Engagement (Lauderdale, 1999), and Maslach’s Burnout Inventory (Maslach et al., 1996), and the remaining items were developed specifically for the evaluation of the field-based training program.

In total, we received 1763 new caseworker responses from 2463 surveys administered, resulting in a 72% response rate, including 551 classroom-trained caseworker responses (67% response rate) and 1212 field-trained caseworker responses (74% response rate). For this study, we pooled both cross sections and used the first survey completed after the caseworker completed training, if the caseworker was surveyed twice. Data from the human resources data, case management data, and the Statewide Survey were matched using employee identification numbers.

3.2. Sample

We used two samples for this research, both consisting of first-time caseworkers. The first sample is used for the turnover analysis and a smaller sub-sample is used for the mediation analysis.

The turnover analysis sample consists of the population of all first-time caseworkers hired to the state child welfare agency between October 1, 2014 and April 30, 2016, during the transition from classroom-based to field-based training (N = 2414). The transition to field-based training rolled out progressively across 11 regions over this time period, providing a sample of field-trained and classroom-trained caseworkers hired during similar time periods. We limited the sample to the three largest specialties (conservatorship, family-based, and investigations) because job responsibilities in each specialty differ substantially and we do not have a large enough sample to analyze the smaller specialties separately (including foster and adoptive home development, alternative response, kinship care, and senior investigations). Therefore, we excluded 49 new caseworkers hired into smaller specialties. Excluded caseworkers from smaller specialties are demographically similar to caseworkers included in the sample, with the exception that the excluded caseworker group included more males and more caseworkers age 49 and older. The final sample for the turnover analysis includes 1549 field-trained caseworkers and 816 classroom-trained caseworkers covering the full population of conservatorship, family-based, and investigations caseworker (N = 2365). This sample includes 712 conservatorship, 360 family-based, and 1293 investigations caseworkers.

For the mediation analysis, we began with the turnover sample (N = 2365) and excluded caseworkers who did not complete a Statewide Survey after their training period. The hiring date range used to select caseworkers to receive the Statewide Survey (January 2015 through March 2016) is more narrow than the date range used for the turnover analysis (October 2014 through April 2016), and therefore 635 caseworkers are not included in the mediation sample because they were hired too early or too late to receive the Statewide Survey. We excluded 205 caseworkers who were on leave or had already exited their position at the time of survey administration and 326 caseworkers who completed the survey during their training period. Finally, we excluded 186 caseworkers who did not complete at least two-thirds of the survey’s organizational factor measures that assess the caseworker’s subjective experience of the job and work environment. Descriptive analyses show that caseworkers excluded from the mediation analyses are demographically similar overall to caseworkers included in the sample, though excluded caseworkers tend to be younger (age 25–29). The final mediation sample includes 306 classroom-trained caseworkers and 707 field-trained caseworkers, including 345 conservatorship caseworkers, 185 family-based caseworkers, and 483 investigative caseworkers (n = 1013).

Classroom-trained caseworkers in the mediation sample had systematically longer tenure than field-trained caseworkers at the time they completed the Statewide Survey (see Table 2, bottom panel). Because classroom-trained caseworkers, on average, were hired slightly earlier than field-trained caseworkers, a disproportionate number of classroom-based caseworkers were not surveyed because they had already left when the survey was administered compared to field-trained caseworkers. Because we expect caseworkers who left early in their tenure to be most dissatisfied with their jobs, the organizational factors of the classroom-trained caseworkers are likely positively skewed, resulting in a conservative estimate of the effect of field-based training on turnover through these mediating organizational factors.

Table 2.

Descriptive statistics for turnover & mediation sample characteristics, by caseworker specialty and training model.

	Turnover sample (n = 2365)
	All		Conservatorship		Family-Based		Investigations
%	Classroom-Trained	Field-Trained	Classroom-Trained	Field-Trained	Classroom- Trained	Field-Trained	Classroom-Trained	Field-Trained
Exit within first 18 months	48.7**	43.1**	43.8*	35.4*	44.6	37.6	51.9	49.3
Gender  Male	16.8**	21.6**	9.9	14.0	10.8	13.6	21.5**	28.6**
Female	83.2**	78.4**	90.2	86.1	89.2	86.4	78.5**	71.4**
Race/Ethnicity  White	37.8	36.9	40.4	41.5	33.8	33.0	37.8	35.0
African American	41.2***	33.8***	39.4*	30.3*	43.9	36.2	41.1*	35.3*
Hispanic	19.4***	27.6***	17.7*	26.3*	21.6	29.4	19.4***	27.8***
Other	1.7	1.8	2.5	2.0	0.7	1.4	1.7	1.8
Age group  Less than 25	28.1	25.8	28.6	30.3	24.5	24.0	28.9*	23.4*
25–29	28.7	27.4	32.5	26.5	35.3	29.4	25.1	27.5
30–39	24.9	25.7	22.7	24.4	21.6	24.0	26.8	27.0
40–49	13.7	14.9	11.3	13.2	15.8	14.5	14.1	16.0
50 and above	4.7	6.3	4.9	5.7	2.9*	8.1*	5.1	6.1
Total	816	1549	203	509	139	221	474	819
	Mediation sample (n = 1013)
	All		Conservatorship		Family-Based		Investigations
%	Classroom-Trained	Field-Trained	Classroom-Trained	Field-Trained	Classroom-Trained	Field-Trained	Classroom-Trained	Field-Trained
Exit within first 18 months	33.3	28.3	27.8	20.8	32.3	28.3	37.1	34.0
Gender  Male	15.4	18.8	7.8	13.7	10.8	13.3	21.9	24.7
Female	84.6	81.2	92.2	86.3	89.2	86.7	78.2	75.3
Race/Ethnicity  White	38.9	37.6	44.4	40.8	29.2	32.5	39.7	37.1
African American	41.5**	31.1**	33.3	29.8	49.2	38.3	43.1**	29.5**
Hispanic	17.0***	29.4***	17.8	27.5	20.0	28.3	15.2***	31.3***
Other	2.6	1.8	4.4	2.0	1.5	0.8	2.0	2.1
Age group  Less than 25	26.8	25.6	35.6	31.0	26.2	25.0	21.9	21.7
25–29	28.1	26.2	28.9	23.5	32.3	28.3	25.8	27.4
30–39	27.1	26.0	25.6	25.9	27.7	23.3	27.8	27.1
40–49	13.4	14.9	7.8	12.6	12.3	14.2	17.2	16.9
50 and above	4.6	7.4	2.2	7.1	1.5*	9.2*	7.3	6.9
Mean tenure at survey (months)	7.5***	5.7***	7.6***	5.7***	7.4***	5.5***	7.6***	5.7***
Organizational factors (mean)  Job Satisfaction	2.6***	2.8***	2.5***	2.9***	2.7*	2.9*	2.5**	2.7**
Burnout	2.7***	2.6***	2.8***	2.5***	2.6	2.5	2.8*	2.6*
Organizational Support	2.9***	3.0***	2.9*	3.1*	2.9	3.1	2.8*	3.0*
Competence	0.56*	0.62*	0.46**	0.59**	0.60	0.69	0.60	0.62
Supervisor Support	3.1	3.2	3.1	3.2	3.1	3.2	3.1	3.1
Coworker Support	3.2	3.2	3.3	3.2	3.2	3.3	3.2	3.2
Total	306	707	90	255	65	120	151	332

^*p < 0.05.

^**p < 0.01.

^***p < 0.001.

3.3. Measures

3.3.1. Outcome

The dependent variable used throughout this study is a binary indicator of employee exit during the first 18 months of employment, measured in two ways. The turnover analysis uses a time-varying measure of exit for each month of employment, up through the 18th month of tenure. The mediation analysis uses a binary indicator of whether the caseworker left at any point during the first 18 months of tenure. This 18-month timeframe allows us to observe all sample caseworkers through the first three months of training, two months of graduated caseload, and up to thirteen months of working a full caseload.

3.3.2. Primary independent variable

The main independent variable in both analyses is a binary measure of whether the worker participated in classroom-based training or field-based training. Caseworker training is identified using caseworker hire date and the implementation date of field-based training in the region of hire. Before January 2015, all new caseworkers received classroom-based training and beginning in November 2015, all newly hired caseworkers received field-based training. We individually verified training model for caseworkers hired near the transition between training programs in their region.

3.3.3. Mediators

To examine which organizational factors mediate the relationship between training and turnover, we created six mean scales for job satisfaction, burnout, organizational support, supervisor support, coworker support, and competence. Each mean scale, except competence, measures respondent opinions as Strongly Disagree (1), Disagree (2), Agree (3), and Strongly Agree (4). The competence scale measures respondent opinions as Not at All Confident, A Little Confident, Somewhat Confident, and Very Confident. We recoded the competence scale to a binary measure such that 1 = Very Confident and 0 = all other responses because the responses were dichotomously distributed at A Little Confident and Very Confident.

We used Spearman Correlations to verify that each scale represents a distinct organizational factor. Although most of the scales were significantly correlated (p < 0.05), we found that the job satisfaction and organizational support scales were highly correlated (r = 0.79). Because of the high correlation of the scales and similar theoretical underpinnings of the subscale items, we conducted confirmatory factor analysis to examine whether the job satisfaction and organizational support subscale items represented two distinct constructs. As a result of the factor analysis, we re-coded several organizational support subscale items for inclusion in the job satisfaction scale and dropped the organizational support subscale items that did not fit well in either scale. No job satisfaction items were dropped or moved to the organizational support scale. Table 1 lists example sub-scale items for each of the six mean scales.

Table 1.

Organizational factor mean scales and example items.

Mean Scale	Scale Item Examples	Cronbach’s Alpha
Job Satisfaction	My job meets my expectations. I feel safe when performing job duties away from CPS. I believe I have a career with CPS.	0.88
Burnout	I feel emotionally drained from my work. Rules and procedures make doing my job difficult. I have to spend too much time on documentation.	0.77
Organizational Support	Upper management effectively communicates important information. There is a basic trust among employees and supervisors. I believe people are generally treated fairly.	0.90
Supervisor Support	My supervisor provides me with a clear understanding of my work responsibilities. My supervisor takes an active interest in developing my knowledge and skills. My supervisor gives me accurate feedback about my performance.	0.91
Coworker Support	People in my unit cooperate to get the job done. The people I work with care about my well-being. The people I work with treat each other with respect.	0.85
Competence	Delivering age-appropriate explanations to children. Using assessment tools accurately. Defining safety in the family in measurable ways.	0.94

Reliability analysis of the six mean scales (job satisfaction, burnout, organizational support, supervisor support, coworker support, and competence) found mean subscale alphas ranging from 0.77 to 0.94. Subscale alphas suggest acceptable to high internal consistency for each of the mean scale mediators (Kline, 1993).

3.3.4. Covariates

We controlled for average caseload in the turnover analysis. Using the case management administrative data, we calculated average daily caseload in each tenure month for each caseworker, providing a measure of average caseload that accounts for daily and monthly fluctuations in caseload. We also controlled for caseworker gender and race/ethnicity in the turnover analysis models.

In the mediation analysis, we controlled for tenure at time of survey administration to account for the tenure differences between classroom-trained and field-trained caseworkers.

3.4. Analytic strategy

3.4.1. Turnover analysis

We first examined the hazard of exit by training model using Kaplan-Meier curves for all caseworkers and then separately for caseworkers in the conservatorship, family-based, and investigation specialties. We then modeled the hazard of exit for the full sample of caseworkers using a discrete-time logistic regression that incorporates a set of indicator variables for all tenure months under study and caseworker-related covariates. This parametric model allowed us to assess the effect of training model on turnover holding constant other observed factors that may also be related to turnover. The tenure month indicator variables allowed us to estimate the baseline hazard function without imposing any constraints.

We checked the specification of our model by testing for violations of the proportional hazards assumption implicit in the discrete-time logistic model and tested whether the turnover sample should be pooled across caseworker specialties (Singer & Willett, 2003).

We estimated a discrete-time logistic regression with interactions between all variables and an indicator for the caseworker’s full caseload months (not shown) and found that the hazard of exit is not proportional across all caseworker tenure months. Consequently, we stratified the tenure periods in all analyses into a graduated caseload period (months 1–5) and a full caseload period (months 6–18).

We estimated a second diagnostic model with all covariates fully interacted with a caseworker specialty indicator to determine whether separate models for training on turnover should be fit for each specialty group (not shown). A Wald test found that the interactions are jointly significant at the 0.001 level. As such, we estimate discrete-time logistic models for all caseworkers and then for caseworkers in each specialty (see Table 3).

Table 3.

Discrete-time logit model of turnover during full caseload (months 6–18), by caseworker specialty (odds ratios).

Outcome: Exited within first 18 months	All Caseworkers	Conservatorship	Family-based	Investigations
Field-based training	0.808**	0.610**	0.837	0.915
Female	0.636***	0.574**	1.373	0.656***
Race/ethnicity  African American	1.072	1.129	0.932	0.954
Hispanic	0.835	0.744	0.451*	0.946
Other	1.311	1.574	–	1.468
Monthly average caseload	–	1.009	1.013	1.031***
Intercept	0.082***	0.067***	0.021***	0.073***
Person-month observations	19,308	6204	3090	9966
AIC	5947.5	1520.4	871.6	3522.0

Note: Estimated odds ratios for person months not shown.

^*p < 0.05.

^**p < 0.01.

^***p < 0.001

In the final discrete-time logistic regressions, we modeled turnover as a function of training model, average monthly caseload, gender, and race/ethnicity for the full sample of caseworkers and separately by caseworker specialty. We stratified the tenure periods for the final discrete-time logistic models to tenure months one through five and months six through 18.

3.4.2. Mediation analysis

As a precursor to any mediation analysis, we first assessed the distribution of each organizational factor mean scale for normality. We found that the competence mean scale was not normally distributed, thus we cubed the measure to achieve a more normal distribution. For mean scales that are non-normal, we ran additional mediation regressions with transformed versions of the scales that normalize the distribution of the scale values.

We used t-tests to compare average mean scale scores by training model for each of the six organizational factors across all three caseworker specialties. A significant difference in mean scale score between training models indicates that a given organizational factor may mediate training model and turnover. When we found a direct effect of training on turnover for a caseworker specialty and a difference in mean scale score by training model for an organizational factor, we tested the organizational factor as a mediator of training and turnover within that specialty. We used the ldecomp module in Stata 15, a decomposition model for dichotomous outcomes, to test the mediation of the organizational factors on turnover.

The ldecomp module in Stata 15 assigns counterfactual values of mediators to each value of the main explanatory variable, and vice versa, to decompose the indirect and direct effects of the main explanatory variable (in this case, training model) on the outcome (exit) in a mediated model (Buis, 2010). The module estimates the predicted probabilities of exit using a logistic regression that includes the training model type and the mediator of interest. The average predicted probabilities for the mediator over classroom-trained and field-trained caseworkers can be calculated, transformed back to log odds, and then used as a counterfactual value for both caseworker groups. The module also computes bootstrapped standard errors of the estimated direct and indirect effects, which provides more accurate inferential testing.

4. Results

4.1. Turnover analysis

Descriptive statistics (see Table 2, panel 1) showed that significantly more classroom-trained caseworkers (48.7%) exited within their first 18 months of tenure than field-trained caseworkers (43.1%). Subsetting the turnover sample by caseworker specialty, we found a significant difference in exit by training model for conservatorship caseworkers only.

We used Kaplan-Meier failure curves (see Fig. 1) to examine observed differences in turnover over time by training model without controlling for any other factors affecting turnover or the composition of caseworkers receiving field-based versus classroom-based training. In the full sample of caseworkers, we observed a small difference in the hazard of exit between the training models starting after tenure month 10. Subsetting the sample by caseworker specialty, we observed that classroom-trained caseworkers had higher hazards of exit than field-trained caseworkers from tenure month 10 and on in the conservatorship and family-based specialties.

Fig. 1.

Kaplan-Meier failure curves, by training model and caseworker specialty.

We explored whether the introduction of a formal graduated caseload in the field-based training model resulted in meaningful caseload differences that may have influenced turnover. Fig. 2 shows that the average monthly mean caseload is similar for classroom-trained and field-trained caseworkers in each specialty. Logistic regressions of the effect of the training model interacted with monthly mean caseload on the hazard of exit show no significant effect of training-related caseload differences on turnover (not shown).

Fig. 2.

Average monthly mean caseload over tenure months, by training model and caseworker specialty.

Table 3 presents the estimated results for discrete-time logistic models of the hazard of exit in tenure months six through 18, controlling for caseworker gender, race/ethnicity, and monthly average caseload. In the analysis of the full sample, we found a significant effect of field-based training on turnover. On average, field-trained caseworkers had 19% lower odds of exiting by 18 months than classroom-trained caseworkers.

Within caseworker specialties during tenure months six through 18, we found the field-based training program significantly affected the hazard of exit for the conservatorship caseworkers only. The odds of exit were 39% lower for field-trained conservatorship caseworkers as compared to classroom-trained conservatorship caseworkers. Average caseload did not significantly predict turnover for conservatorship or family-based caseworkers, but among investigators, the odds of exiting increased 3.1% with each additional investigation added to an investigator’s average monthly caseload.

We also examined the hazard of exit in the first five months of caseworker tenure; this period included each caseworker’s time in training plus the initial months in which they carry a graduated caseload. Analysis of the exit hazard in the first five months of tenure showed no significant effect of training model on turnover for either the full caseworker sample or for any of the specialty sub-samples (not shown).

4.2. Mediation analysis

We used organizational factor mean scales from the Statewide Survey to test the paths by which field-based training affects caseworker turnover. In the turnover analysis, we found that field-based training was associated with 39% lower odds of exiting within 18 months of hire for conservatorship caseworkers, but not for family-based caseworkers or investigators. As a sensitivity check, we re-ran the discrete-time logistic regression model on the mediation sample and found that the turnover difference between classroom-trained and field-trained caseworkers is similar to the turnover sample results. The significant direct effect of training on turnover for conservatorship caseworkers indicated that there was a potential for mediation in the conservatorship sub-sample.

We first examined differences in the means of the organizational factors by training model to assess whether there were significant differences in the potential mediators by training. We found that field-trained conservatorship caseworkers had significantly higher mean job satisfaction, organizational support, and competence compared to their classroom-trained counterparts (see panel 2 of Table 2). Field-trained conservatorship caseworkers also reported significantly lower average levels of burnout than classroom-trained caseworkers.

The finding of a significant training effect on turnover and significant associations between these four organizational factors and the training model indicated there may be mediation of the training model effect on turnover for conservatorship caseworkers. We tested mediation using a decomposition model for the four organizational factors for which we found a training effect; the decomposition results are presented for conservatorship caseworkers only (see Table 4).

Table 4.

Mediation analysis of training model on turnover within 18 months, for conservatorship caseworkers.

Conservatorship	Direct Effect of Training on Turnover	Indirect Effect of Training on Turnover
	(odds ratios)	(odds ratios)
Job Satisfaction	0.603	0.658***
Burnout	0.478*	0.828**
Org. Support	0.458*	0.863
Competence	0.380**	1.045
Observations	345	345

Bootstrapped standard errors used in significance testing for decomposition models.

^*p < 0.05.

^**p < 0.01.

^***p < 0.001

4.2.1. Job satisfaction

In the decomposition model, we found that job satisfaction fully mediated the effect of field-based training on turnover for conservatorship caseworkers (see Table 4). These results indicate that the training model influenced turnover through elements of job satisfaction, including through meeting caseworkers’ expectations for their job, clearly defining responsibilities, providing a sense of safety at work, and providing work-life balance. If classroom-trained caseworkers had the same average level of perceived job satisfaction as their field-trained colleagues, their odds of exiting would be reduced by 36%.

4.2.2. Burnout

The decomposition model revealed that burnout partially mediated the relationship between training and turnover for conservatorship caseworkers. Caseworkers with classroom training would have 18% lower odds of exit if they had lower levels of burnout similar to their field-trained colleagues. Because burnout only partially mediated the relationship between training and turnover, the relationship is explained both by a direct effect of field-based training on turnover and by the indirect mechanism of caseworker burnout. On average, the indirect effect of burnout on the training and turnover relationship accounted for 20% of the total effect of caseworker training on turnover.

4.2.3. Organizational support

We found no evidence that organizational support significantly mediated the effect of training on turnover for conservatorship caseworkers. Though we saw a significant difference in mean organizational support by training model in the descriptive analysis, the decomposition analysis found no significant indirect effect of organizational support on turnover.

4.2.4. Competence

Similarly, the decomposition analysis found no significant mediation effect of competence on the effect of training on caseworker turnover, despite the significant difference in mean competence across training models in the descriptive statistics (see Table 2, panel 2). A decomposition model with the transformed version of the competence scale also found no evidence of mediation.

4.2.5. Investigations and family-based caseworker organizational factors

Though we could not examine mediation without a direct effect between training and turnover among family-based workers and investigators, we examined mean scale scores for each organizational factor for investigations and family-based caseworkers. We found that field-trained family-based workers had significantly higher job satisfaction than classroom-trained caseworkers and field-trained investigators reported higher job satisfaction, lower burnout, and higher organizational support than their classroom-trained peers (see Table 2, panel 2).

5. Discussion

Research examining how caseworker training affects job experience and turnover has not kept pace with the shift to field-based learning in child welfare pre-service training. To our knowledge, this is the first study to examine the impact of field-based training on turnover, and the first study to examine how organizational factors, such as job satisfaction and burnout, mediate the training and turnover relationship. By examining the factors that mediate training and turnover, we explore potential mechanisms by which field-based training improves workforce stability. We studied a large, population-level sample of caseworkers and took advantage of the natural experiment design of the training rollout to account for unobserved differences in new caseworkers. Unlike prior studies, we used longitudinal human resources data to analyze caseworker turnover rather than taking a cross-sectional approach. We also stratified the analysis by caseworker role to account for variation among caseworkers with different roles within the child welfare system.

5.1. Turnover analysis

Overall, we found that field-based training increased worker stability among conservatorship caseworkers during their first 18 months of tenure, but did not influence worker stability for family-based or investigations workers. The types of tasks and emotional strain faced by investigations and family-based caseworkers may be different, and perhaps less influenced by improvements to training, than the stressors that conservatorship caseworkers experience. Additionally, specialty-specific training quality or mentorship quality may differ across specialties, resulting in a training effect on turnover for conservatorship caseworkers, but not caseworkers from other specialties. Variation in training quality and mentor quality across specialties is outside the scope of the current study, indicating opportunities for future research.

Variation in caseload may also limit the extent to which training can decrease turnover rates among investigators. Though caseload did not significantly influence turnover among conservatorship or family-based caseworkers, investigators with higher caseloads were more likely to exit prior to reaching 18 months of tenure. Each additional investigation on an investigator’s average daily caseload increased his or her odds of exit by 3.1% in a given month. Child welfare agencies must consider pre-service training in the larger context of the caseworker role, especially for investigators. High-quality pre-service training may be most effective when provided in conjunction with other measures to decrease turnover, such as ensuring caseloads are reasonable.

5.2. Mediation analysis

We also assessed the extent to which job satisfaction, burnout, organizational support, supervisor support, coworker support, and competence mediate training and turnover for conservatorship workers.

We found that job satisfaction fully explained the effect of training on turnover for conservatorship caseworkers, indicating that higher job satisfaction among field-trained conservatorship caseworkers leads to decreased turnover. The job satisfaction scale consists of items that measure the extent to which the position meets the caseworker’s expectations and basic needs for a job: safety, role clarity, opportunity for advancement, appropriate workload, and fair compensation (see Table 1). Our findings support the theory that providing conservatorship workers a more realistic job preview and increased understanding of their responsibilities in training improves their likelihood of remaining on the job for the first year and half of tenure.

Burnout partially mediated the relationship between training and turnover among conservatorship workers. The burnout scale measures the extent to which caseworkers feel emotionally exhausted from their work. Importantly, in this study we assessed burnout three to five months after caseworkers completed training, indicating that the experience of field training better prepares caseworkers to deal with stress and job challenges once training is over, and that lowering burnout may be an effective pathway to decrease turnover.

We found no evidence that organizational support, supervisor support, coworker support, and competence mediated training and turnover. Specifically, field-based training is not associated with higher levels of supervisor support or coworker support for conservatorship workers. Conservatorship caseworkers report higher competence and higher organizational support when they went to field training, but neither higher competence nor higher organizational support are associated with turnover. Thus, none of these organizational factors mediate training and turnover.

Though we could not examine mediation without a direct effect between training and turnover among family-based workers and investigators, we found that job satisfaction is improved by field-based training among family-based workers and investigators, and additionally field-based training is associated with higher organizational support and lower burnout among investigators.

Overall, this study provides support for targeting job satisfaction and burnout through field-based training to decrease early caseworker turnover. However, agencies must consider caseworker training in the context of specific roles in the agency, rather than treating all specialized caseworker jobs the same, to better understand how to tailor training experiences to prepare and support caseworkers with different job functions.

Despite the large sample size and rigorous methodology, this study has several limitations. Field-based training was part of a larger package of agency-wide improvements that began during the study period, and we cannot account for each change in our model. Though numerous changes took place, these period effects occurred agency-wide and therefore all caseworkers would have experienced similar changes, regardless of training model. One specific agency-wide change included a pay raise for all caseworkers that was first announced in October 2016. Very few caseworkers, however, were still under observation at this time. Another limitation is that we rely on self-reported organizational factor measures for the mediation analysis. Although participant responses could be biased as a result, we would not expect respondents in one training group to provide more biased responses than respondents from the other training group. Lastly, future studies should examine turnover over a longer timeframe than 18 months to assess whether the differences in turnover rates between field-trained and classroom-trained caseworkers continue to grow over time.

The job of frontline child welfare staff is inherently challenging. Caseworker training cannot necessarily make the job less difficult but can better equip caseworkers to handle the challenges of their job, both in terms of skill development and through supporting caseworkers to better cope with the challenges. The present study demonstrates that job satisfaction and burnout are factors that can be effectively targeted through field-based pre-service training to decrease caseworker turnover, but also highlights the importance of considering specialized caseworker roles individually when designing training and monitoring turnover to adequately support caseworkers working in different roles within an agency.