Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Nov 1.
Published in final edited form as: Child Youth Serv Rev. 2013 Sep 13;35(11):10.1016/j.childyouth.2013.09.002. doi: 10.1016/j.childyouth.2013.09.002

Reducing turnover is not enough: The need for proficient organizational cultures to support positive youth outcomes in child welfare

Nathaniel J Williams a,✉, Charles Glisson b
PMCID: PMC3834965  NIHMSID: NIHMS524988  PMID: 24273363

Abstract

Objective

High caseworker turnover has been identified as a factor in the poor outcomes of child welfare services. However, almost no empirical research has examined the relationship between caseworker turnover and youth outcomes in child welfare systems and there is an important knowledge gap regarding whether, and how, caseworker turnover relates to outcomes for youth. We hypothesized that the effects of caseworker turnover are moderated by organizational culture such that reduced caseworker turnover is only associated with improved youth outcomes in organizations with proficient cultures.

Methods

The study applied hierarchical linear models (HLM) analysis to the second National Survey of Child and Adolescent Well-being (NSCAW II) with a U.S. nationwide sample of 2,346 youth aged 1.5- to 18-years-old and 1,544 caseworkers in 73 child welfare agencies. Proficient organizational culture was measured by caseworkers’ responses to the Organizational Social Context (OSC) measure; staff turnover was reported by the agencies’ directors; and youth outcomes were measured as total problems in psychosocial functioning with the Child Behavior Checklist (CBCL) completed by the youths’ caregivers at intake and at 18 month follow-up.

Results

The association between caseworker turnover and youth outcomes was moderated by organizational culture. Youth outcomes were improved with lower staff turnover in proficient organizational cultures and the best outcomes occurred in organizations with low turnover and high proficiency.

Conclusions

To be successful, efforts to improve child welfare services by lowering staff turnover must also create proficient cultures that expect caseworkers to be competent and responsive to the needs of the youth and families they serve.

Keywords: turnover, organizational culture, proficiency, child welfare, youth outcomes

1. Introduction

Child welfare systems in the United States are under increasing pressure to improve outcomes for the children and families they serve. Wide variation in the success of child welfare systems nationally has led to the search for system-level explanations of differential outcomes for youth (U.S. Department of Health and Human Services, 2011). Of the many system-level factors cited as potential causes of ineffectiveness, high caseworker turnover has received the most attention (Strand, Spath, & Bosco-Ruggiero, 2010; Strolin, McCarthy, & Caringi, 2007; U.S. General Accounting Office [GAO], 2003). High turnover is endemic in U.S. child welfare systems (Cyphers, 2001, 2005; Drake & Yadama, 1996; GAO, 2003) and, given the central role played by caseworkers in the care process (Ryan, Garnier, Zyphur, & Zhai, 2006; Westbrook, Ellett, & Asberg, 2012), numerous scholars have hypothesized that high turnover is detrimental to youth outcomes (DePanfilis & Zlotnik, 2008). However, almost no empirical research has examined the relationship between caseworker turnover rates and youth outcomes in child welfare systems (Strolin et al., 2007). Thus, there is an important knowledge gap regarding how caseworker turnover relates to outcomes for youth served by child welfare systems.

An understanding of the association between caseworker turnover and youth outcomes is important for child welfare systems because of its implications for administrative practice, resource expenditures, and intervention development (Collins-Camargo, Ellett, & Lester, 2012). Child welfare administrators control limited resources and exercise circumscribed influence over the many factors that affect the well-being of children who are referred, assessed, and treated for abuse and neglect (Webb, Dowd, Harden, Landsverk, & Testa, 2010). Accordingly, administrators must prioritize malleable and impactful causes of system ineffectiveness. To the extent that high turnover contributes to poor youth outcomes, investments in caseworker retention strategies represent cost-effective methods to enhance youth well-being (GAO, 2003; Kacmar, Andrews, Van Rooy, Steilberg, & Cerrone, 2006). However, these strategies may represent wasted resources and significant opportunity-costs if the turnover-outcomes relationship is contingent on other unaddressed agency factors or if caseworker turnover is not associated with youth outcomes.

Studies of the relationship between organizational turnover and performance in the broader organizational literature support the view, derived from human capital (Strober, 1990) and social capital theories (Shaw, Duffy, Johnson, & Lockhart, 2005), that employee turnover rates are negatively and linearly associated with a wide range of organizational outcomes. However, these studies also indicate that moderators play an important role in the strength of this relationship (Park & Shaw, 2013). For example, organizational culture has recently been identified as a moderator of the turnover-performance relationship in human services organizations (Mohr, Young, & Burgess, 2012). Studies of child welfare systems indicate that organizational culture is both an important predictor of service quality (Glisson & Green, 2006; Glisson & James, 2002; Yoo & Brooks, 2005) and associated with the development of caseworker human and social capital (DePanfilis & Zlotnik, 2008; Westbrook et al., 2012). These studies suggest a contingent relationship may exist between caseworker turnover and youth outcomes depending on the quality of organizational culture within the agency (Glisson, Green, & Williams, 2012). This study advances knowledge on strategies for increasing child welfare system effectiveness by examining the relationship between caseworker turnover and youth outcomes as moderated by organizational culture in a national sample of child welfare agencies.

1.1. Turnover and Organizational Performance

Studies of employee turnover in the organizational and child welfare literatures have focused primarily on understanding the antecedents to turnover, emphasizing such factors as employee characteristics (e.g., age, education, burnout, organizational commitment), supervisory relationships (e.g., supportiveness), and job characteristics (e.g., caseload, flextime polices, on-call requirements, and compensation) (Griffeth, Hom, & Gaertner, 2000; Stolin et al., 2007). Much less well-developed is the body of literature examining the consequences of employee turnover. Recent reviews have characterized this literature as underdeveloped, minimally integrated, and in need of further study (Park & Shaw, 2013; Shaw, 2011). Surveying the turnover literature in child welfare settings, Strolin et al. (2007, p.29) noted “the empirical research on the effects of workforce turnover in child welfare is scant” and called for large-scale, empirical studies that test the hypothesized cross-level links between caseworker turnover and youth outcomes in child welfare systems.

The relationship between employee turnover and organizational performance is most often explained using human and social capital theories (Shaw et al., 2005; Strober, 1990). Human capital theory contends that more experienced employees accumulate specialized tacit and formal knowledge and skills through extended task-specific practice, training, and experience (Kacmar et al., 2006). The loss of these experienced employees through turnover damages organizational performance because of the loss of their accumulated expertise. Although organizations can replace employees who leave, newer employees require time to develop similar levels of competence and organizational performance suffers as a result.

Social capital theory emphasizes the importance of interpersonal relationships and the quality of social ties to an organization’s effectiveness (Shaw et al., 2005). Networks of interpersonal relationships developed by employees over time (i.e., social capital) contribute to organizational performance by facilitating the development and application of knowledge, uniting employees around collective goals, building mutual trust, and generating a store of goodwill and mutually reinforcing favors among employees (Dess & Shaw, 2001). Increased turnover impedes organizational performance by disrupting these networks of relationships and social ties. Both human and social capital theories predict a negative linear relationship between turnover rates and organizational performance. Human and social capital theories have strong empirical support for predicting organizational effectiveness (Crook, Todd, Combs, Woehr, & Ketchen, 2011; Dess & Shaw, 2001) and for explaining the relationship between employee turnover and organizational performance (Park & Shaw, 2013).

Human and social capital are believed to be especially important to the effectiveness of human services organizations such as child welfare systems because these organizations rely on relationship-based and person-centered technologies to achieve targeted outcomes (Collins-Camargo et al., 2012; Mohr et al., 2012). Caseworkers are responsible for determining the validity of child abuse and neglect reports, assessing safety and making placement decisions, determining eligibility for benefits and sanctions, making timely and appropriate referrals for treatment and other supports, and implementing prevention and crisis services (Ryan et al., 2006; Westbrook et al., 2012). Effectiveness in this role requires the expert use of specialized knowledge and skills in complex, dynamic, and emotion-laden situations (Glisson & James, 2002). New caseworkers typically lack the specialized expertise, training, and experience (i.e., human capital) necessary to perform this job well and as a result, service effectiveness decreases when more experienced employees leave (GAO, 2003). Effective casework also requires well-developed social networks with care providers from other sectors, community resources, and colleagues within child welfare and other state bureaucracies (i.e., social capital). High turnover disrupts these social ties and may contribute to less effective casework as a result (Ryan et al., 2006).

A growing number of studies support the prediction, based on human and social capital theories (Shaw, 2011), of a negative linear association between employee turnover rates and organizational performance. A recent meta-analysis (Park & Shaw, 2013) indicated that the population average correlation between turnover rates and organizational performance was significant and negative (ρ = -.15). However, this meta-analysis indicated that moderators played an important role in the strength of the relationship. For example, the turnover-performance correlation varied from ρ = -.02 to ρ = -.29 depending on the type of performance criteria (e.g., financial performance vs. service quality), type of employee (e.g., manual labor vs. skilled professional), and industry (e.g., manufacturing vs. healthcare). Organizational culture has also recently been identified as a significant moderator of the relationship between turnover rates and outcomes, particularly in organizations that rely on knowledge-based professions such as hospitals and human services agencies (Mohr et al., 2012). These studies indicate that simplistic linear models may be inadequate to describe the complex and contingent relationship between turnover and organizational performance in child welfare systems. In particular, organizational culture is likely to play an important role in the strength of the turnover-outcomes relationship in child welfare systems (Glisson & Green, 2006; Glisson et al., 2012; Glisson & James, 2002).

1.2. Organizational Culture and Youth Outcomes in Child Welfare

Organizational culture refers to the shared behavioral norms and expectations that characterize a work environment and determine the way employees prioritize, approach, and complete their work (Glisson et al., 2012; Verbeke, Volgering, & Hessels, 1998). Organizational culture signals to employees the types of work activities and strategies that are valued, rewarded, and supported by the organization and forms the basis for a shared understanding and enactment of meaningful responses to the work environment (Cooke & Szumal, 1993; Schein, 2010; Verbeke et al., 1998). Decades of empirical studies, summarized by a recent meta-analysis of 84 studies (Hartnell, Ou, & Kinicki, 2011) and comprehensive narrative review of 55 studies (Sackmann, 2011), support the link between organizational culture and organizational performance across a wide range of settings and outcome criteria including studies in child welfare and youth mental health (Glisson & Green, 2006; Glisson et al., 2012; Glisson, Hemmelgarn, Green, & Williams, 2013; Glisson & James, 2002; Olin et al., 2013; Yoo & Brooks, 2005). These studies confirm that the shared behavioral norms and expectations in a work environment are associated with meaningful differences in individual employee behavior and organizational effectiveness.

Applied to child welfare settings, organizational culture theory posits that shared behavioral norms and expectations impact youth outcomes by directing caseworkers in prioritizing their work, guiding their selection of assessment methods and intervention models, determining the availability, responsiveness, and continuity of services to youth throughout the care process, and contributing to the development and expression of human and social capital among child welfare caseworkers (Glisson et al., 2012; Glisson & James, 2002). Within this conceptualization, organizational culture is a multi-faceted construct consisting of specific domains of shared norms and expectations that influence targeted caseworker behaviors. For example, one prominent theory of organizational culture for child welfare settings, proposed byGlisson et al. (2012), hypothesizes that three dimensions of organizational culture—proficiency, rigidity, and resistance—impact three domains of workforce behaviors which ultimately relate to youth outcomes. Rigidity describes behavioral norms and expectations related to caseworkers’ discretion and flexibility in decision-making. Resistance describes norms related to openness to new innovations. Proficiency assesses agency expectations and norms for caseworker competence, up-to-date knowledge, and prioritization of client well-being (Glisson et al., 2012).

Of these three dimensions, proficiency is most relevant to the development of human and social capital among caseworkers and therefore is most likely to moderate the effect of turnover on child outcomes in child welfare systems. Although rigidity and resistance may have implications for youth outcomes in child welfare settings, these relationships may be orthogonal to the development of human and social capital (e.g., resistance to new innovations does not imply that an agency’s current operations, training procedures, and services are ineffective), may represent distal influences unrelated to current workforce turnover (e.g., failure to adopt newer, more effective practices may reduce effectiveness over the long-term but does not necessarily imply that current services are ineffective), or may be contingent on other factors (e.g., high rigidity may be beneficial when paired with best practices). In contrast, cultural norms and expectations related to proficiency bear directly on the extent to which agencies focus on achieving positive client outcomes through the development and expression of human and social capital within their current workforce. Proficient organizational culture represents a critical social context that sustains caseworkers’ focus on client well-being, encourages the development of competencies they need to successfully serve youth, and sustains their motivation to effectively meet the challenges of achieving safety, permanency, and well-being of children who have been neglected or abused (Grant, 2007).

1.3. Study Theoretical Model and Hypothesis

Building on human capital, social capital, and organizational culture theories, this study tests the interaction between caseworker turnover rates and proficient organizational culture in their association with youth outcomes in a national sample of child welfare systems. Drawing on these theories we argue that optimal youth outcomes will occur in child welfare systems with low caseworker turnover and high proficiency cultures. In agencies with high proficiency cultures, norms and expectations that focus on client well-being, up-to-date knowledge and skills, and availability and responsiveness to clients’ needs contribute to effective service provision and positive youth outcomes. Proficient organizational culture contributes to the development and effective use of human and social capital as caseworkers are rewarded, supported, and recognized for developing competencies that contribute to positive outcomes for youth. Because caseworkers in these systems accrue human and social capital that enhance youth outcomes, and because the services provided by these caseworkers are more likely to focus on and achieve child well-being, turnover will be negatively and linearly associated with youth outcomes (i.e., lower turnover, better outcomes) in agencies with high proficiency cultures.

Conversely, in child welfare systems with low proficiency cultures, minimal agency-level expectations for improvements in client well-being and caseworker responsiveness generate poor youth outcomes. In these systems lower turnover is not associated with better youth outcomes because the development and optimal use of human and social capital is stunted and the loss of more experienced employees does not represent a significant reduction in the already limited operative capacity of the service system. Agencies with low-proficiency cultures lack the shared behavioral norms and expectations that evoke and support effective casework and thus fail to develop the human and social capital of their workforces necessary for optimal work with clients. Thus, higher turnover in these systems will not result in worse youth outcomes.

Hypothesis 1: Organizational culture moderates the relationship between caseworker turnover rates and youth outcomes such that high turnover is negatively associated with youth outcomes in child welfare agencies with high proficiency cultures and unrelated to youth outcomes in agencies with low proficiency cultures.

This hypothesis implies that increased caseworker retention will be associated with improved outcomes for youths in agencies with high-proficiency cultures but will not be associated with improved outcomes in agencies with low-proficiency cultures. If confirmed, it suggests that reducing turnover alone is insufficient to improve youth outcomes in child welfare systems with low proficiency cultures.

2. Method

2.1. Setting and Sample

Data for this study are from the second and most recent National Survey of Child and Adolescent Well-being (NSCAW II), a national probability survey of youth, ages 0- to 18-years-old who were investigated and monitored by the U.S. child welfare system due to allegations of abuse or neglect. Details of the NSCAW II study design are presented elsewhere (Dowd et al., 2012) and summarized here.

The target population for NSCAW II includes “all children in the US who were subjects of child abuse or neglect investigations (or assessments) conducted by CPS [child protective services] and who live in states not requiring an agency first contact of the sample members” (Dowd et al., 2012, p.22). Youth were selected for the study using a national, two-stage, stratified random sampling design that built on the sampling frame from NSCAW I (Webb et al., 2010). In the first sampling stage investigators divided the US into nine strata corresponding to the eight states with the largest child welfare caseloads and a single stratum including the remaining 38 states and the District of Columbia. In the second stage, primary sampling units (PSUs) were formed within each stratum based on the geographic areas served by CPS agencies. In most cases PSUs represented single counties or continuous areas of two or more counties; however, in large metropolitan areas PSUs corresponded to the catchment areas of urban CPS agencies. The within-PSU sampling process relied on simple random selection of eligible youth with varying service characteristics who did not have a sibling in the study, had not been previously sampled, and were not being investigated as perpetrators of abuse.

The two-stage sampling design in the first NSCAW study resulted in 92 PSUs in 97 counties across the United States (Webb et al., 2010). Agencies in these counties were recruited to participate in NSCAW II and 76% were retained. Replacement agencies were sampled for NSCAW II resulting in a total of 86 PSUs, representing 81 counties in 30 states. The most common reason for agency refusal to participate was passage or reinterpretation of state legislation requiring the CPS agency to make first contact with families suspected of neglect or abuse (Dowd et al., 2012).

The population for the present study included youth, ages 18 months to 18 years old, who were investigated or assessed by the U.S. child welfare system due to allegations of neglect or abuse. Because the accuracy of risk assessment and case dispositions cannot be determined a priori, the study sample included all youths who were investigated and assessed. Youths who were younger than 18 months old at baseline were excluded from the study because caregiver report measures of youth psychosocial functioning were not available for children less than 18-months-old.

2.2. Procedure

NSCAW II (Dowd et al., 2012) is a longitudinal survey with planned data collection extending over multiple waves, across multiple levels (e.g., system level, youth level), and relying on multiple informants (e.g., child welfare administrators, caseworkers, caregivers, and youths). The present study includes data collected from youth caregivers, front-line caseworkers, and child welfare administrators during waves 1 and 2. Wave 1 data collection occurred from February 2008 to April 2009. Wave 2 data collection occurred for each youth 18 months after the close of the investigation that brought the youth into the study.

Primary data collection interviews with key informants occurred in person using standardized protocols (Dowd et al., 2012). Key informant interviews with child welfare administrators occurred during wave 1. Organizational culture data were collected as confidential responses from front-line caseworkers to a standardized measure using established protocols (e.g., no supervisors present) during wave 2. Caregiver-reported data were collected at both waves via in-person interviews. Permission was obtained from the appropriate Institutional Review Board for using the NSCAW II data files in this study.

2.3. Measures

2.3.1. Child behavior checklist

The outcome of interest was youths’ psychosocial functioning at 18 month follow-up, controlling for their level of functioning at intake, as measured by the total problems T-score of the Child Behavior Checklist (CBCL) (Achenbach & Rescorla, 2001). The CBCL is a 118-item checklist (the preschool version has 99 items) that uses likert-type items to assess youths’ behavior problems along two broadband dimensions— externalizing problems (e.g., aggression, stealing, disruptive behavior) and internalizing problems (e.g., depression, anxiety, withdrawal, somatic complaints). These dimensions are combined to form a total problem behavior score that indicates the overall level of emotional and behavior problems present. Higher scores indicate more problems. The CBCL assessment system includes forms for two age groups: youths aged 1.5- to 5-years-old and youths aged 6- to 18-years-old (Achenbach & Rescorla, 2001). The CBCL forms are widely used in research and practice applications and have demonstrated excellent score reliability and validity in numerous large and diverse samples of youth (Aschenbrand, Angelosante, & Kendall, 2005; Hudziak, Copeland, Stanger, & Wadsworth, 2004; Nakamura, Ebesutani, Bernstein, & Chorpita, 2009).

2.3.2. Organizational culture

Proficient organizational culture was assessed by the agency-level aggregated responses of front-line caseworkers’ to the proficiency subscale of the Organizational Social Context (OSC) measure (Glisson et al., 2012). The OSC is an employee survey designed to assess the work environments of child welfare, social service, and mental health agencies. The 15-item measure of proficient culture asks caseworkers to indicate the extent to which their agency is characterized by behavioral expectations and norms that prioritize client well-being and expect caseworkers to be competent and have up-to-date knowledge. The internal consistency reliability of the proficiency subscale in this sample was α = .89. Previous research has supported the association of proficiency as measured by the OSC with service quality, attitudes toward evidence-based practices, and youth outcomes in child welfare and mental health systems (Aarons et al., 2012; Glisson & Green, 2006; Glisson et al, 2013; Olin et al., 2013).

The present study used aggregate, agency-level proficiency T-scores based on the national sample of child welfare agencies (Glisson et al., 2012). Agency-level T-scores were derived for 81 of 86 agencies in the NSCAW II sample based on surveys completed by 1,740 caseworkers in the 81 agencies. T scores were not available for five agencies because they had fewer than three caseworkers who completed the OSC (k=3), they did not provide OSC data (k=1), or they had unacceptably low caseworker agreement on their OSC responses based on the rwg statistic (k=1) (James, Demaree, & Wolf, 1993). On average, 21 caseworkers per agency provided responses to the OSC with a range of 3 to 97 caseworkers per agency.Glisson et al. (2012) provided evidence of adequate within-agency agreement among caseworkers’ on their responses to the proficiency subscale to justify aggregation, based on the rwg statistic (James et al., 1993), as well as evidence of significant between-system variance.

2.3.3. Annual turnover rates

Annual child welfare agency turnover rates were defined as the proportion of caseworkers who left their jobs each year as reported by child welfare administrators in key informant interviews during the baseline assessment of the NSCAW II national survey. Child welfare administrators reported “the annual turnover rate or percent leaving” in their agencies during the most recent fiscal year by indicating the total percentage of voluntary and involuntary annual caseworker turnover. Although the NSCAW II survey did not request information about specific types of turnover (e.g., voluntary, involuntary, reduction in force), studies of turnover in child welfare settings indicate that from two-thirds to three-fourths of caseworker turnover is voluntary and preventable (Cyphers, 2001). Meta-analytic findings indicate that type of informant (i.e., key informant interview versus agency records) does not significantly moderate the association between turnover rates and outcomes (Park & Shaw, 2013) and thus key informant data reported by administrators were considered appropriate to address the research question.

2.3.4. Model covariates

Covariates were included in the analysis in order to adjust youths’ 18 month CBCL scores for their baseline level of functioning and in order to obtain estimates of the effects of proficient organizational culture and turnover above and beyond youth and agency characteristics. Youths’ personal characteristics were reported by the child’s caregiver or caseworker at baseline and included: child age in months, child gender, child race (coded as African American, Native American/Alaskan Native, or Asian/ Hawaiian/ Pacific Islander due to small cell sizes), and child ethnicity (Latino vs. non-Latino). One agency characteristic, location, was also included to control for the differences in urban versus non-urban child welfare settings.

2.4. Data Analysis

The hypothesis tested in this study required the use of hierarchical linear modeling (Raudenbush, Bryk, 2002), also known as random effects regression and multilevel modeling (Hox, 2010), to allow for specification of the cross-level effects of agency-level turnover and proficient organizational culture on individual-level youth outcomes. The hypothesis was tested using a two-level random intercept and random slopes hierarchical linear model (HLM), with youth covariates, including baseline CBCL T scores, modeled at level 1 and child welfare agency covariates modeled at level 2. The dependent variable was 18-month follow-up CBCL T-scores. The model estimated the effect of the interaction between proficient organizational culture and annual caseworker turnover rates on the covariate-adjusted scores in youth psychosocial functioning at 18-month follow-up. Analyses were implemented using HLM software, version 6.08 (Raudenbush, Bryk, Cheong, & Congdon, 2004).

Following the test of the main hypothesis, the significant interaction between turnover and proficiency was assessed further by calculating the region of significance for the conditional relationship between turnover and youth outcomes across values of proficiency (Aiken & West, 1991; Bauer & Curran, 2005; Preacher, Curran, & Bauer, 2006). The region of significance is derived using the Johnson-Neyman technique (Johnson & Neyman, 1936) as applied and extended to multilevel models (Bauer & Curran, 2005). The Johnson-Neyman technique calculates values of the moderator (i.e., proficiency) beyond which the conditional relationship between the focal predictor (i.e., agency turnover) and the dependent variable (i.e., youth outcomes) is statistically significant (Preacher et al., 2006).

3. Results

3.1. Descriptive Statistics

The sample included N = 2,346 youth who received services from k = 73 child welfare agencies. Organizational culture (i.e., proficiency) scores were based on responses from 1,544 caseworkers employed in the 73 agencies. Rates of missing data on demographic and risk variables for youth were generally low and ranged from 0% to 2.3% with the exception of 18-month CBCL score which was missing for 17.7% of youth. This rate of attrition is typical of longitudinal studies of at-risk youth and of youth who are aging out of the foster care system. Youth with missing data were excluded from the study sample. Youth for whom an 18-month CBCL was not available were older (M = 110.96 months, SD = 62.52 vs. M = 94.59, SD = 52.85), t(753.77) = 5.70, p < .001, and less impaired at baseline (M = 53.78, SD = 12.46 vs. M = 55.05, SD = 12.37), t(833.77) = −2.166, p = .031, than youth for whom an 18-month follow-up was available, but did not differ on gender, race, or ethnicity (all p’s > .200).

On average, youth in the sample were 7.88-years-old at baseline (SD = 4.40), and slightly more male (52%) and White (54%), but also included African American (33%), American Indian/ Alaskan Native (9%), and Asian American (4%). The average CBCL T-score was 55.05 (SD = 12.37) at baseline and 53.99 (SD = 12.40) at 18-month follow-up.

The majority of child welfare agencies in the sample were located in urban locations (81%) and the average organizational culture proficiency T-score was 50 (SD = 10, min = 26.59, max = 75.78). On average, child welfare administrators reported caseworker turnover rates of 14.64% (SD = 15.04%, min = 1%, max = 80%) which falls within the range of annual turnover reported by previous studies. For example, Burns & Christie (2013) reported annual caseworker turnover rates in two child welfare samples of 8% in 2006 and 11% in 2010 and the 2004 Child Welfare Workforce Survey reported a national average annual turnover rate of 22.1% with a range of 0 to 67% (Cyphers, 2005).

3.2. Correlations

Examination of the correlation matrix for agency-level covariates (N = 73) confirmed that annual caseworker turnover was not significantly correlated with proficient organizational culture (r = .097, p = .414) or urban location (r = -.126, p = .289) nor was proficient culture significantly correlated with urban location (r = .098, p = .386).

3.3. Interaction Between Turnover and Proficiency

Because the hypothesis focused on the interaction between two level-2 predictors, all level-1 and level-2 predictors were grand mean centered in order to facilitate interpretation of the results. Results from the full model supported the study hypothesis (see Table 1). After controlling for youth and agency covariates, annual caseworker turnover interacted significantly with proficient organizational culture to predict baseline-adjusted youth outcomes at 18-month follow-up. The interaction indicated that decreases in caseworker turnover were associated with improved youth outcomes in child welfare systems with proficient organizational cultures but not in systems with low-proficiency cultures. Figure 1 shows the relationship between annual caseworker turnover and youth outcomes (baseline adjusted 18-month follow-up CBCL scores) for agencies with low proficiency (10th percentile) and high proficiency (90th percentile) organizational cultures.

Table 1.

HLM analysis of the interaction between proficient organizational culture and turnover rates on youths’ 18-month CBCL total problems scores.

Variable Coefficient SE p
System-level
  Constant 53.456 .264 .000
  Urban location −.675 .585 .253
  Annual turnover −.074 .040 .068
  Proficient organizational culture −.060 .029 .039
  Turnover x Proficient culture .002 .001 .049
Youth-level
  Baseline CBCL .627 .021 .000
  Age .013 .004 .004
  Female −.277 .379 .465
  Substantiated abuse or neglect .221 .405 .585
  African American −.503 .559 .371
  Native American/ Alaskan Native −1.390 .772 .071
  Other race −1.074 1.008 .287
  Latino ethnicity −1.089 .435 .013
Random Effects
  Intercept variance 1.027 .002
  Baseline CBCL slope variance .009 .004
  African American slope variance 4.191 .005
  Residual variance 87.311
Open in a new tab

Note: Proportion of organizational variance explained = 77.41%; proportion of residual variance explained = 35.19%.

*

p < .05

**

p < .01

***

p < .001

Figure 1.

Figure 1

Open in a new tab

Association of caseworker turnover with 18-month youth CBCL scores in child welfare systems with low- and high-proficiency organizational cultures.

Note: CBCL = Child Behavior Checklist. 18-month CBCL T-scores are adjusted for youths’ baseline CBCL T-score and youth and agency covariates. Low and high proficiency cultures correspond to the 10th and 90th percentiles, respectively.

3.4. Region of Significance on Proficiency for the Turnover Effect

The region of significance indicates values of the moderator (i.e., proficiency) beyond which the association between the focal predictor (i.e., annual turnover) and the dependent variable (i.e., 18-month CBCL scores) is significant (Preacher et al., 2006). Decreased turnover was significantly associated with improvements in youth outcomes only in child welfare systems with proficient organizational cultures above the 47th percentile (T score > 49.51) on national norms for proficiency. The region of significance indicated the relationship between turnover and youth outcomes was not statistically significant in child welfare agencies with a proficiency T score below 49.51. These findings therefore indicate that decreases in turnover are not associated with positive youth outcomes in child welfare systems with a proficiency T score below the national mean (T = 50) but are associated with positive youth outcomes in agencies with proficiency T scores above the national mean.

4. Discussion

Findings from this study indicate that decreased caseworker turnover is associated with improved outcomes for youth only in child welfare agencies with proficient organizational cultures that are above the national average in their expectations for caseworkers to have up-to-date knowledge and skills and an emphasis on improving youth well-being as their top priority. Reductions in caseworker turnover are not associated with improvements in youth outcomes in child welfare systems with organizational cultures below the national average on proficiency. Based on these results, a proficient organizational culture appears to be a necessary pre-condition for caseworker retention to associate positively with outcomes for youth. Efforts to increase caseworker retention are unlikely to be associated with improvements in youth outcomes in child welfare systems below the U.S. national average for proficiency.

Previous research has examined the relationship between organizational culture and youth outcomes in child welfare systems but has paid less attention to the relationship between caseworker turnover and youth outcomes or to the interaction between turnover and culture (Yoo & Brooks, 2005). The present study suggests that specific dimensions of these system-level characteristics interact in their association with youth outcomes. Reduced caseworker turnover is associated with youth outcomes, but only in child welfare systems with proficient organizational cultures that support positive youth outcomes and the development of human and social capital among caseworkers. These findings underscore the importance of developing child welfare work environments that focus on improving client well-being as the top priority, build caseworker competence and up-to-date knowledge, and engender high levels of service availability, responsiveness, and continuity.

4.1. Implications for Practice

Findings from this study have implications for the initiation of strategies to reduce caseworker turnover and improve youth outcomes in child welfare systems. Results suggest that prior to initiating turnover reduction strategies child welfare administrators should assess their organizational cultures with tools such as the Organizational Social Context measure which has national norms for child welfare and mental health systems (Glisson et al., 2012; Glisson et al., 2008). Such assessments will aid administrators in determining the best course of action for enacting system-level changes and will help prevent the opportunity costs incurred by implementing turnover reduction strategies in the absence of a proficient organizational culture.

Child welfare administrators interested in reducing caseworker turnover have a growing number of organizational improvement strategies from which to choose (e.g., Strolin-Goltzman, 2010). Many of these strategies are crafted specifically to address the needs of child welfare systems; however, most do not focus explicitly on improving organizational culture nor do they have evidence of effectiveness from randomized controlled trials. An important exception is the ARC (for Availability, Responsiveness, and Continuity) organizational improvement intervention which has demonstrated positive effects on caseworker turnover, organizational culture and climate, and youth outcomes in child welfare and youth mental health systems in randomized controlled trials (Glisson, Dukes, & Green, 2006; Glisson, Hemmelgarn, Green, Dukes, Atkinson, & Williams, 2012; Glisson et al., 2013; Glisson et al., 2010). Studies of ARC have also shown that improvements in organizational culture and climate associated with ARC are associated with improved outcomes for youth (Glisson et al., 2013).

4.2. Implications for Research

Results from this study highlight the need for research on the effects of caseworker turnover and organizational social context on youth outcomes in child welfare systems. Recognition of the importance of social context to caseworker behavior and youth outcomes is increasing among child welfare researchers (e.g., Collins-Camargo et al., 2012; Strolin et al., 2007); however, much remains to be learned about the most effective strategies for child welfare administrators to create work environments that support positive youth outcomes and how these effects are transmitted (e.g., turnover, service processes, etc.). Studies are needed to identify the intermediate linkages between proficient organizational culture, low caseworker turnover, and youth outcomes. Several recent initiatives focused on improving caseworker training and retention have developed an infrastructure that may support these types of studies (Strand, Spath, & Bosco-Ruggiero, 2010).

4.3. Limitations

Strengths of this study include its reliance on separate informants for each of the constructs (i.e., turnover, proficient organizational culture, youth outcomes), the use of a national sample of child welfare systems, and the use of a longitudinal design that met two (temporal precedence, association) of three conditions necessary to establish causal effects. Despite these strengths, readers are cautioned against interpreting the results from this study in a causal fashion (i.e., low caseworker turnover and high proficiency cause improvements in youths’ well-being) because of the lack of random assignment of youth to conditions and the lack of manipulation of the independent variables. Findings from this study represent correlational relationships that must be replicated and experimentally tested in further research.

A second limitation pertains to the measurement of caseworker turnover. Although a recent meta-analysis (Park & Shaw, 2013) indicated that type of informant (e.g., key informant vs. agency records) did not moderate the turnover-performance relationship (ρ = -.14 vs. ρ = -.15, respectively), administrative records may provide a more precise measure of employee turnover and as such may result in more precise estimates of the magnitude of effects. Assuming agency administrators’ reports of turnover incorporate more error than agency records, the true population correlation between turnover and youth outcomes may be stronger in proficient organizational cultures than that reported here. Additional studies using alternative measurement strategies are needed to confirm and extend these results.

A related issue pertains to the potential need to discriminate between types of caseworker turnover when assessing the turnover-outcomes relationship. Some caseworkers are terminated from employment involuntarily because of poor performance and the loss of these caseworkers is likely to have a different relationship with youth outcomes than the voluntary turnover of highly-skilled caseworkers. Again, meta-analytic evidence suggests that type of turnover (e.g., voluntary vs. involuntary) does not significantly moderate the turnover-outcomes relationship; however, the correlation between involuntary turnover and outcomes was much smaller (ρ = -.01) than that of voluntary (ρ = -.15) or reduction in force (ρ = -.17) turnover (Park & Shaw, 2013). Because the present study included all types of turnover, it is possible that these results underestimate the magnitude of the relationship between turnover and youth outcomes. However, previous research indicates that the majority of caseworker turnover in child welfare settings is voluntary (Cyphers, 2001) thus mitigating this concern to a large extent.

Future studies should address remaining knowledge gaps regarding the relationship between turnover, organizational culture, and youth outcomes by conducting experiments in which caseworker turnover and organizational culture are manipulated and the consequences for youth outcomes are assessed (e.g., Glisson et al, 2013).These types of studies have the potential to inform organizational intervention efforts to increase system effectiveness for children who are abused and neglected.

Highlights.

  • Organizational culture interacts with turnover to predict child welfare outcomes

  • Reduced turnover predicts improved outcomes in proficient child welfare systems

  • Reduced turnover is not associated with outcomes in non-proficient systems

  • Efforts to improve child welfare services by lowering turnover must also create proficient cultures

Acknowledgments

Funding: This research was supported in part by a grant from the National Institute of Mental Health to NJW (F31MH099846). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Aarons GA, Glisson C, Green PD, Hoagwood K, Kelleher KJ, Landsverk JA The Research Network on Youth Mental Health. The organizational social context of mental health services and clinician attitudes toward evidence-based practice: A United States national study. Implementation Science. 2012;7:56. doi: 10.1186/1748-5908-7-56. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Achenbach TM, Rescorla LA. Manual for the ASEBA preschool forms & profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families; 2001. [Google Scholar]
  3. Aiken LS, West SG. Multiple regression: Testing and interpreting interactions. Newbury Park, CA: Sage; 1991. [Google Scholar]
  4. Aschenbrand SG, Angelosante AG, Kendall PC. Discriminant validity and clinical utility of the CBCL with anxiety-disordered youth. Journal of Clinical Child and Adolescent Psychology. 2005;34(4):735–746. doi: 10.1207/s15374424jccp3404_15. [DOI] [PubMed] [Google Scholar]
  5. Bauer DJ, Curran PJ. Probing interactions in fixed and multilevel regression: Inferential and graphical techniques. Multivariate Behavioral Research. 2005;40:373–400. doi: 10.1207/s15327906mbr4003_5. [DOI] [PubMed] [Google Scholar]
  6. Burns K, Christie A. Employment mobility or turnover? An analysis of child welfare and protection employee retention. Children and Youth Services Review. 2013;35:340–346. [Google Scholar]
  7. Collins-Camargo C, Ellett CD, Lester C. Measuring organizational effectiveness to develop strategies to promote retention in public child welfare. Children and Youth Services Review. 2012;34:289–295. [Google Scholar]
  8. Cooke RA, Szumal JL. Measuring normative beliefs and shared behavioral expectations in organizations: The reliability and validity of the Organizational Culture Inventory. Psychological Reports. 1993;72:1299–1330. [Google Scholar]
  9. Crook TR, Todd SY, Combs JG, Woehr DJ, Ketchen DJ. Does human capital matter? A meta-analysis of the relationship between human capital and firm performance. Journal of Applied Psychology. 2011;96:443–456. doi: 10.1037/a0022147. [DOI] [PubMed] [Google Scholar]
  10. Cyphers G. Report from the child welfare workforce survey: State and county data and findings. Washington, DC: American Public Human Services Association; 2001. [Google Scholar]
  11. Cyphers G. Report from the 2004 child welfare workforce survey: State agency findings. Washington, DC: American Public Human Services Association; 2005. [Google Scholar]
  12. DePanfilis D, Zlotnik JJ. Retention of front-line staff in child welfare: A systematic review of research. Children and Youth Services Review. 2008;30:995–1008. [Google Scholar]
  13. Dess GG, Shaw JD. Voluntary turnover, social capital and organizational performance. Academy of Management Review. 2001;26:446–456. [Google Scholar]
  14. Dowd K, Dolan M, Wallin J, Miller K, Biemer J, Aragon-Logan E, Smith K. National survey of child and adolescent well-being (NSACW): NSCAW II Combined waves 1-2 data file user’s manual, restricted release version. Ithaca, NY: National Data Archive on Child Abuse and Neglect, Cornell University; 2012. [Google Scholar]
  15. Drake B, Yadama G. A structural equation model of burnout and job exit among child protective services workers. Social Work Research. 1996;20:179–187. [Google Scholar]
  16. Glisson C, Dukes D, Green P. The effects of the ARC organizational intervention on caseworker, turnover, climate and culture in children’s service systems. Child Abuse & Neglect. 2006;30:855–880. doi: 10.1016/j.chiabu.2005.12.010. [DOI] [PubMed] [Google Scholar]
  17. Glisson C, Green P. The effects of organizational culture and climate on the access to mental health care in child welfare and juvenile justice systems. Administration and Policy in Mental Health and Mental Health Services Research. 2006;33:433–448. doi: 10.1007/s10488-005-0016-0. [DOI] [PubMed] [Google Scholar]
  18. Glisson C, Green P, Williams NJ. Assessing the organizational social context (OSC) of child welfare systems: Implications for research and practice. Child Abuse and Neglect. 2012;36:621–632. doi: 10.1016/j.chiabu.2012.06.002. [DOI] [PubMed] [Google Scholar]
  19. Glisson C, Hemmelgarn A, Green P, Dukes D, Atkinson S, Williams NJ. Randomized trial of the availability, responsiveness, and continuity (ARC) organizational intervention with community-based mental health programs and clinicians serving youth. Journal of the American Academy of Child and Adolescent Psychiatry. 2012;51:780–787. doi: 10.1016/j.jaac.2012.05.010. [DOI] [PubMed] [Google Scholar]
  20. Glisson C, Hemmelgarn A, Green P, Williams NJ. Randomized trial of the availability, responsiveness and continuity (ARC) organizational intervention for improving youth outcomes in community mental health programs. Journal of the American Academy of Child and Adolescent Psychiatry. 2013;52:493–500. doi: 10.1016/j.jaac.2013.02.005. [DOI] [PubMed] [Google Scholar]
  21. Glisson C, James LR. The cross-level effects of culture and climate in human service teams. Journal of Organizational Behavior. 2002;23:767–794. [Google Scholar]
  22. Glisson C, Landsverk J, Schoenwald S, Kelleher K, Hoagwood KE, Mayberg S, Green P The Research Network on Youth Mental Health. Assessing the organizational social context (OSC) of mental health services: Implications for research and practice. Administration and Policy in Mental Health. 2008;35:98–113. doi: 10.1007/s10488-007-0148-5. [DOI] [PubMed] [Google Scholar]
  23. Glisson C, Schoenwald SK, Hemmelgarn A, Green P, Dukes D, Armstrong KS, Chapman JE. Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. Journal of Consulting and Clinical Psychology. 2010;78:537–550. doi: 10.1037/a0019160. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Grant AM. Relational job design and the motivation to make a prosocial difference. Academy of Management Review. 2007;32:393–417. [Google Scholar]
  25. Griffeth RW, Hom PW, Gaertner S. A meta-analysis of antecedents and correlates of employee turnover: update, moderator tests, and research implications for the next millennium. Journal of Management. 2000;26:463–488. [Google Scholar]
  26. Hartnell CA, Ou AY, Kinicki A. Organizational culture and organizational effectiveness: A meta-analytic investigation of the competing values framework’s theoretical suppositions. Journal of Applied Psychology. 2011;96:677–694. doi: 10.1037/a0021987. [DOI] [PubMed] [Google Scholar]
  27. Hox J. Multilevel analysis: Techniques and applications. 2nd ed. New York: Routledge; 2010. [Google Scholar]
  28. Hudziak JJ, Copeland W, Stanger C, Wadsworth M. Screening for DSM-IV externalizing disorders with the Child Behavior Checklist: A receiver-operating characteristic analysis. Journal of Child Psychology and Psychiatry. 2004;45(7):1299–307. doi: 10.1111/j.1469-7610.2004.00314.x. [DOI] [PubMed] [Google Scholar]
  29. James LR, Demaree RG, Wolf G. rwg: An assessment of within-group interrater agreement. Journal of Applied Psychology. 1993;75:306–309. [Google Scholar]
  30. Johnson PO, Neyman J. Tests of certain linear hypotheses and their applications to some educational problems. Statistical Research Memoirs. 1936;1:57–93. [Google Scholar]
  31. Kacmar KM, Andrews MC, Van Rooy DL, Steilberg RC, Cerrone S. Sure everyone can be replaced … but at what cost? Turnover as a predictor of unit-level performance. Academy of Management Journal. 2006;49:133–144. [Google Scholar]
  32. Mohr DC, Young GJ, Burgess JF. Employee turnover and operational performance: the moderating effect of group-oriented organizational culture. Human Resource Management Journal. 2012;22:216–233. [Google Scholar]
  33. Nakamura BJ, Ebesutani C, Bernstein A, Chorpita BF. A psychometric analysis of the Child Behavior Checklist DSM-Oriented Scales. Journal of Psychopathology and Behavioral Assessment. 2009;31:178–189. doi: 10.1007/s10862-009-9174-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Olin SS, Williams N, Pollock M, Armusewicz K, Kutash K, Glisson C, Hoagwood KE. Quality indicators for family support services and their relationship to organizational social context. Administration and Policy in Mental Health and Mental Health Services Research. 2013 doi: 10.1007/s10488-013-0499-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Park T, Shaw JD. Turnover rates and organizational performance: A meta-analysis. Journal of Applied Psychology. 2013;98:268–309. doi: 10.1037/a0030723. [DOI] [PubMed] [Google Scholar]
  36. Preacher KJ, Curran PJ, Bauer DJ. Computational tools for probing interaction effects in multiple linear regression, multilevel modeling, and latent curve analysis. Journal of Educational and Behavioral Statistics. 2006;31:437–448. [Google Scholar]
  37. Raudenbush SW, Bryk AS. Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage Publications; 2002. [Google Scholar]
  38. Raudenbush SW, Bryk AS, Cheong YF, Congdon RT. HLM 6: Hierarchical linear and nonlinear modeling. Loncolnwood, IL: Scientific Software International; 2004. [Google Scholar]
  39. Ryan JP, Garnier P, Zyphur M, Zhai F. Investigating the effects of caseworker characteristics in child welfare. Children and Youth Services Review. 2006;28:993–1006. [Google Scholar]
  40. Sackmann SA. Culture and performance. In: Ashkanasy NM, Wilderom CPM, Peterson MF, editors. The handbook of organizational culture and climate. 2nd ed. Thousand Oaks, CA: Sage; 2011. pp. 188–224. [Google Scholar]
  41. Schein EH. Organizational culture and leadership. 4th ed. San Francisco, CA: Jossey-Bass; 2010. [Google Scholar]
  42. Shaw JD. Turnover rates and organizational performance: Review, critique, and research agenda. Organizational Psychology Review. 2011;1:187–213. [Google Scholar]
  43. Shaw JD, Duffy MK, Johnson JL, Lockhart DE. Turnover, social capital losses, and performance. Academy of Management Journal. 2005;48:594–606. [Google Scholar]
  44. Strand VC, Spath R, Bosco-Ruggiero S. So you have a stable child welfare workforce—what’s next? Children and Youth Services Review. 2010;32:338–345. [Google Scholar]
  45. Strober MH. Human capital theory: Implications for HR managers. Industrial Relations. 1990;29:214–239. [Google Scholar]
  46. Strolin JS, McCarthy M, Caringi J. Causes and effects of child welfare workforce turnover: Current state of knowledge and future directions. Journal of Public Child Welfare. 2007;1(2):29–52. [Google Scholar]
  47. Strolin-Goltzman J. Improving turnover in public child welfare: Outcomes from an organizational intervention. Children and Youth Services Review. 2010;32:1388–1395. [Google Scholar]
  48. U.S. Department of Health and Human Services. Child welfare outcomes 2007–2010: Report to Congress. Washington, DC: Author; 2011. [Google Scholar]
  49. U.S. General Accounting Office. Child welfare: HHS could play a greater role in helping child welfare agencies to recruit and retain staff. Washington, DC: Author; 2003. [Google Scholar]
  50. Verbeke W, Volgering M, Hessels M. Exploring the conceptual expansion within the field of organizational behavior: Organizational climate and organizational culture. Journal of Management Studies. 1998;35:303–329. [Google Scholar]
  51. Webb MB, Dowd K, Harden BJ, Landsverk J, Testa MF. Child welfare and child well-being: New perspectives from the national survey of child and adolescent wellbeing. New York: Oxford University Press; 2010. [Google Scholar]
  52. Westbrook TM, Ellett AJ, Asberg K. Predicting public child welfare employees’ intentions to remain employed with the child welfare organizational culture inventory. Children and Youth Services Review. 2012;34:1214–1221. [Google Scholar]
  53. Yoo J, Brooks D. The role of organizational variables in predicting service effectiveness: An analysis of a multilevel model. Research on Social Work Practice. 2005;15:267–277. [Google Scholar]

RESOURCES