Agency Culture and Climate in Child Welfare: Do Perceptions Vary by Exposure to the Child Welfare System?

Abstract

Organizational culture and climate play a critical role in worker retention and outcomes, yet little is known about whether perceptions of culture and climate vary depending on the demands of particular roles. In this study, 113 staff from a child welfare agency completed Organizational Social Context profiles. Staff were divided into three groups according to their proximity to child welfare tasks to assess whether involvement in higher stress child welfare tasks is related to perceptions of the social context. Findings suggest possible differences across groups, with those involved in core child welfare tasks appearing to perceive higher resistance to new ways of providing services and those with the least involvement in traditional child welfare perceiving a more positive social context overall.

Introduction

Variations in an agency’s culture and climate potentially affect organizational support, staff receptivity, and ultimately uptake and sustained use of new practices (Aarons & Sawitzky, 2006; Crampton, Crea, Abramson-Madden, & Usher, 2008). In child welfare settings, in particular, climate and culture might have a pronounced impact on workers’ ability to adopt new practices. Child welfare systems are typically described as chaotic environments in which caseworkers are responsible for helping families with a high level of need and responding to frequent crises, often with limited training, support and resources (Children’s Bureau, 2012; Golden, 2009). Furthermore, institutional demands limit flexibility in workers’ roles (Parada, Barnoff, & Coleman, 2007; Smith & Donavon, 2003). Many staff report being ill-prepared to effectively handle the challenges of their jobs, and child welfare is known for its high turnover rates and difficulties in retaining workers (Barak, Nissly, & Levin, 2001; DePanfilis & Zlotnik, 2008).

In 2003, the United States General Accounting Office identified poor organizational climate and caseworker turnover as key factors explaining inadequate care provided by child welfare systems nationwide. Despite identification of culture and climate as important factors impacting outcomes for families served by the child welfare system (Glisson & Green, 2011; Spath, Strand, & Bosco-Ruggiero, 2013), relatively little is known about what contributes to the more difficult organizational climates within these systems. A number of recent articles cite organizational social context as a key reason that workers either remain at or leave a child welfare agency. Supportive coworker environments, supervisor support, and recognition have been suggested as factors that retain workers despite the challenges of the job (Cahalane & Sites, 2008; Jacquet, Clark, Morazes, Withers, 2007). Supervisory support in the workplace has also been found to buffer the stress of work-family conflict on wellbeing and burnout in child welfare worker (Lizano, Hsiao, Mor Barak & Casper, 2014). Social contexts that allow workers to have a voice in organizational decisions, that help workers draw meaning from their work, and provide opportunities for learning are also associated with worker retention (Clark, Smith, & Uota, 2013; Johnco, Salloum, Olson, & Edwards, 2014). However, it is not clear whether the work in and of itself may impact one’s perception of the organizational social context. Understanding the extent to which exposure to core child welfare tasks such as investigations, child placement and placement supervision, crisis intervention, and court demands affect perceptions of culture and climate is a critical question for administrative practice and implementation efforts.

In this study, we explore variation in reports of agency culture and climate across groups within an agency to consider whether one’s role may impact experience of the organizational social context. Although caseworkers, therapists, and prevention staff are all housed in the same agency, it is possible that, because they have differing responsibilities, their perception of the organizational context will differ. Having a better understanding of whether role might be related to perception of organizational social context is important for understanding variation within measures of culture and climate, and ultimately how culture and climate in child welfare is understood.

In this study, we draw upon research from organizational literature, which suggests that subcultures, or occupational groups within an organization, may report the culture of the organization differently (Johnson, 2000; Martin, 2002; Schein, 2010). Schein (2010) suggests that organizational subcultures are essential for understanding how innovations are implemented, and that they should be considered and perhaps reconciled in order to enhance worker development and enhance opportunities for continual learning. Organizational subculture has been shown to have a greater influence on learning motivation than the overall organizational culture in health care settings (Egan, 2008). Subculture is also a better predictor of organizational commitment than organizational culture in other professions, such as nursing (Lok & Crawford, 1999). Thus, it is important to consider whether subcultures in child welfare agencies have different perceptions of their organizational environments, as variation in perceptions may have implications for practice, implementation, and service outcomes across different subcultures.

In this study, we examine three groups within one agency that differ in their exposure to core child welfare tasks in order to determine whether perceptions of social context appear to vary by role. While the groups all experienced the same higher-level administrative structure and policies, differences in culture and climate are hypothesized based on level of exposure to child welfare tasks. The first group includes those directly engaged in core child welfare tasks, including case managers, case aides, and their supervisors. The second includes those in a secondary role of supporting cases, such as child welfare clinicians, post-adoption services providers, their supervisors and program directors, as well as others in the agency who work directly with foster children but are not responsible for investigations, permanency planning, or court appearances. Within the secondary group we single out a third group, child welfare therapists, who provide mental health services to youth and families in child welfare and are responsible for providing treatment and writing court reports about therapeutic progress. The final group includes workers focused on prevention programming, which includes staff from community-based programs who do not work directly with child welfare-involved youth and families (e.g., a school-based counseling program).

Each of these groups is compared to two different sets of national norms established for dimensions of climate and culture as measured by the Organizational Social Context (OSC; Glisson, 2008): norms established for child welfare agencies and norms established for mental health agencies. We intentionally used both norms in order to better understand how therapists and other providers working in the child welfare system may be distinct in their experience of organizational social context. The national norms for child welfare workers were based off a large sample of caseworkers, while the national norms for mental health providers were based off a large sample child mental health providers who were not working in child welfare settings. Since the mental health norms include therapists, they provide a comparison for the therapists employed in child welfare versus mental health settings. Therapists and community prevention providers employed in child welfare agencies could have more positive perceptions of the organizational context than caseworkers in the same agency, but may perceive a more difficult social context than those employed in mental health agencies.

Because, to our knowledge, no previous research has examined culture and climate across subgroups within a child welfare agency, our hypotheses were based on the assumption that greater exposure to core welfare tasks will be associated with more negative perceptions of culture and climate. In addition, given the intensity of child welfare work, the core and secondary groups at this child welfare agency are expected to experience more difficult organizational contexts when compared with mental health norms. Thus, the core child welfare group was expected to profile similarly to national child welfare norms (Glisson, Green, & Williams, 2012), but more negatively than mental health norms. In contrast, the prevention group was expected to have more positive perceptions of culture and climate when compared to child welfare norms, but were considered likely to be similar to mental health norms in their perceptions of organizational context. Because the prevention workers in this study do not interact with child welfare-involved families, it is possible that the work they do is more similar to the work of a therapist in a more traditional setting who does not work with child welfare clients. The secondary group, which is involved in some core child welfare tasks, but has much less direct responsibility for these tasks, was expected to profile in between child welfare and mental health norms. Although they are not responsible for managing the daily needs of child welfare-involved families, they are still actively involved in work with families in the child welfare system, and therefore are likely to experience additional stressors than those working in more traditional mental health agencies due to the severity of the needs of their clients. Additionally, therapists within the secondary group were expected to perceive of their work culture and climate more negatively than mental health norms, due to the challenges related to their responsibility for treatment children placed in the foster care system.

Literature Review

Culture and Climate in Child Welfare

Although historically culture and climate originated from two different domains of research, with culture generally utilizing qualitative interpretive inquiry, and climate using quantitative methods, over the years the two constructs have been used interchangeably and there has been some debate as to their differences. Currently, climate is typically defined as the shared perceptions of the emotional or psychological impact of the work environment on workers’ wellbeing, while culture addresses the norms, expectations, and the “ways things are done” in an organization. Climate includes dimensions such as engagement, functionality, and stress, while culture includes constructs such as proficiency, rigidity, and resistance (Glisson et al., 2008).

Although most of the literature on organizational climate and culture was conducted within business and industrial settings, researchers in the past twenty years began to apply the study of organizational social context to human service agencies beyond hospitals, including mental health systems and child welfare agencies (Austin & Claasen, 2006; Hemmelgarn et al., 2006). Child welfare is a particularly unique context to understand culture and climate, as there are significant challenges to providing quality services within a child welfare system. Families served by child welfare systems are among the nation’s most vulnerable. Most families who enter the child welfare system face a host of challenges prior to involvement including poverty, homelessness, substance abuse, unemployment, mental illness, and incarceration (Marcenko, Lyons, & Courtney, 2011). Maltreated youth frequently have serious emotional issues related to maltreatment and trauma, yet the resources available to families are often limited (Spath et al., 2013).

In addition to having the clinical skills necessary to communicate with and assist families, child welfare workers simultaneously are asked to find foster homes for children, monitor child safety and well-being, attend frequent court hearings, and complete large quantities of paperwork. They are paid relatively little for their efforts, and many do not receive adequate levels of supervision to perform the challenging job required of them (Strand & Badger 2005). Secondary trauma related to working with child welfare families is also a noted issue for caseworkers (Pryce, Shackelford, & Pryce, 2007), and it is not surprising that child welfare professionals often become overloaded and emotionally exhausted, and that high levels of turnover among the child welfare workforce has posed a challenge for child welfare systems for decades (Clark, 2012).

Furthermore, the effectiveness of the child welfare system is continually in question. Media coverage of the failures of the child welfare system, as well as high profile cases such as child deaths can have important implications for an agency’s reputation or other options available to them. Budgetary limitations often contribute to administrators’ stress, which may in turn affect staff working with clients. In addition, threats of additional budget cuts from government or external funding sources often create particularly tense work environments (Ellett, Ellis, & Westbrook, 2007; Smith & Donovon, 2003). States vary widely on performance and outcomes, and many struggle to meet the needs of families, provide children with safe living arrangements, and achieve timely permanency (Children’s Bureau, 2012; Golden, 2009). Given the challenges inherent in child welfare work, it would seem surprising if those responsible for maintaining child safety, permanence, and wellbeing did not report stressful work environments and a negative impact of their jobs on their psychological wellbeing.

At the same time, studies also suggest that constructive social contexts characterized by more positive climates and cultures can buffer some of the challenges of effectively serving families in the child welfare system. Child welfare organizations may vary in terms of the supports and resources provided to staff to counter the impact of child welfare work, which in turn may affect services provided and client outcomes. This is supported by findings indicating significant associations between aspects of culture and climate and the types of services that families receive (Glisson & Green, 2011), child outcomes (Glisson & Hemmelgarn, 1998), effectiveness of services (Glisson & Green, 2006; Yoo & Brooks, 2005), worker attitudes (Glisson, Green & Williams, 2012), job satisfaction (Glisson et al., 2012), worker turnover (Glisson et al., 2008; Zeitlin, Augsberger, Auerbach, & McGowan, 2014), and receptivity to new practices (Aarons & Sawitzky, 2006; Patterson, Dulmus, & Maguin, 2013).

However, the mechanisms through which climate and culture affect outcomes are not yet clear. For example, although Glisson and Green (2005) found that clients served by agencies with constructive cultures were 11 times more likely to receive needed mental health services than those with the least constructive cultures, a later study did not find an effect for climate and culture through the quantity and quality of services (Glisson & Green, 2011). Consistent with earlier findings, youth served by child welfare systems characterized by more “engaged” climates in which employees perceive that they make meaningful accomplishments in their work and remain personally concerned about the wellbeing of clients had significantly better psychosocial functioning over time (Glisson & Green, 2011). Yet the quantity and quality of services did not mediate or interact with youth’s long-term outcomes. Thus, while climate is linked to improved outcomes, these findings suggest that it is not necessarily because clients receive better services. Aspects of climate such as level of staff engagement may more directly affect outcomes.

Although the effect of climate and culture on implementation of new practices in child welfare agencies has not been directly studied, research conducted outside of child welfare also indicates an association between organizational social context and openness to innovation, and, in particular, attitudes about evidence-based practices. For example, Aarons and Sawitzky (2006) found that more constructive cultures are associated with more positive attitudes toward EBP, and defensive cultures with more negative attitudes about EBPs. Similarly, Patterson and colleagues (2013) found that work groups that were more open to using evidence-supported treatments were housed in organizations with cultures that were higher on proficiency and less resistant to change. Wells and colleagues (2015) found that organizational climate was associated with staff’s endorsement of an agency’s newly implemented solution-based casework practice model, as well as congruence in worker’s beliefs with agency’s beliefs inherent in the new practice model. Research also suggests that implementation efforts that include intervention to improve agency climate and culture are associated with greater likelihood of sustainability (Novins, Green, Legha, & Aarons, 2013).

Clearly, efforts to improve service delivery within child welfare settings must attend to climate and culture to create meaningful change. While the specific mechanisms through which culture and climate impact services and outcomes are not yet completely clear, the social context has a demonstrated impact on key worker and child-level outcomes as well as attitudes toward innovation. To date, research has not examined whether different groups within the same agency differ in their perceptions of climate and culture, or how exposure to core child welfare work affects perceptions of the work environment. Understanding whether these differences exist is a critical first step, as discrepancies in experiences of the organizational context related to the type of work being done could indicate that child welfare settings will require unique implementation strategies for lasting changes in service quality and outcomes.

Study Hypotheses

Using the OSC survey, we examine how perceptions of culture and climate vary by role and proximity to child welfare tasks at the agency (core child welfare staff, secondary or prevention staff). Greater exposure to core child welfare tasks is expected to appear to be related with more negative perceptions of organizational culture and climate. The recent availability of OSC norms and standard deviations for both child welfare and children’s mental health agencies provides the opportunity to compare one large agency’s profile to both child welfare and mental health norms. We hypothesize that:

1. The core child welfare group will appear to profile similarly to child welfare national norms, but will appear to have more difficult culture and climate scores as compared to mental health norms.

2. The secondary group will appear to have perceptions that are mixed, and will profile in between child welfare and mental health norms.

3. Within the secondary group, child welfare therapists’ perceptions will appear more negative than mental health norms.

4. The prevention group will appear to experience the organizational social context as more positive than child welfare norms.

Methods

Data in this study was collected as part of a larger longitudinal study testing the dissemination of a parent management training intervention among case managers and therapists in one large private child welfare agency in Chicago (, 2015). The agency in which the research was conducted offers multiple programs including foster care, foster parent licensing, mental health treatment, intact family supports, post-adoption support services, and community prevention services, such as school counseling. All staff at the agency was invited via their agency email to complete a 20-minute survey during a pizza lunch provided by the research team. Supervisors were also asked to remind their staff when the research would occur. Research staff followed up individually with any staff who did not attend the data collection events at the designated times. Participants were given an OSC instrument to complete themselves, and received $25 after handing it in to research staff. Although the OSC was administered as a part of a longitudinal study, the OSC data was only collected at one time point. Each of the three groups had a response rate that was over 90%.

Study Sample

A total of 119 employees at two sites of one agency completed the survey. Forty-seven percent were African American, 36% were white, 3% were Asian, 1% were American Indian, and 13% identified as other. In addition, 17% identified as Hispanic. Eighty-two percent were female. These demographics are reflective of the urban child welfare workforce. Upon completion, the surveys were divided into three groups depending on the participant’s role at the agency and the extent of their exposure to core child welfare task. The groups included the following:

1. The core child welfare group consists of those involved with direct child welfare services including placement and foster care oversight. Most employees in this group are case managers (n = 24) and case management supervisors (n = 5), but it also includes adoption workers and their supervisor (n = 5), licensing workers and their supervisor (n = 5), case aides (n = 6), a placement coordinator (n = 1) and an education coordinator (n = 1). It also includes higher-level administrators who oversee these groups (n = 2). The core child welfare group consists of a total of 49 staff. Table 1 shows sample demographics by grouping.

2. The secondary group provides support to the core group, but is not responsible for placement, investigations, or court proceedings. It consists of staff who provide supportive and clinical services to foster children and their families (n = 12) and their supervisors (n = 2). It also includes staff who provide post-adoption clinical and case management services to recently adopted foster children (n = 4), intact family service caseworkers and their supervisor (n = 6), staff of an afterschool program for foster youth (n = 3), permanency specialists (n = 2), and a volunteer coordinator (n = 1). Administrators of these programs are also included (n = 2). The secondary group consists of a total of 32 staff.

3. Child welfare therapists (n = 14) are a subgroup of the 32 staff in the secondary group that provide office-based clinical services to foster children. They are not responsible for placement or attending court, but because they have responsibility for treatment of foster children and write reports for court proceedings, they could be a distinct group within the secondary group due to their greater exposure to child welfare tasks. Thus, we examine their reports both separately and within the larger group of secondary staff.

4. The prevention group works in the community with high risk groups and does not work directly with families or children with formal child welfare involvement. This group includes those providing services within the community (n =18), staff providing counseling services to schools and the director of the community school program (n = 9), community outreach workers (n = 4), staff from a parenting program for Hispanic families including their supervisor (n = 7), and the Director of the Latino Consortium (n = 1). The prevention group consists of a total of 39 staff.

Table 1.

Sample Description

Characteristic	Core (%)	Secondary (%)	Secondary/Treatment (%)	Prevention (%)
Gender
Female	77.6	93.9	92.9	78.4
Race
Native American	0	0	0	2.7
Asian	2	6.1	14.3	0
African American	59.2	36.4	21.4	40.5
Caucasian	34.7	51.5	64.3	24.3
Other	4.1	6.1	13.4	32.4
Ethnicity
Hispanic or Latino	0	12.1	0	43.2
n	49	32	14	39

Measures

The OSC (Glisson, 2008) measurement tool contains 105 items that address culture, climate, and work attitudes using Likert scale responses from 1 = not at all to 5 = to a very great extent. It was developed through an iterative, empirical process over thirty years of research in mental health and social service agencies. The OSC was first confirmed in a national sample of 100 mental health service organizations in 75 cities (N = 1,154) participating in the Clinical Systems Project (CSP) of the Research Network on Children’s Mental Health. A second norm for the OSC was created with case managers from 81 child welfare systems who participated in the National Survey of Child and Adolescent Well-Being II (NSCAW II) (N = 1,740) (Glisson et al., 2008; Glisson, Green & Williams, 2012). The existence of two national norms allows us to compare child welfare workers in our sample to other child welfare workers, and also allows us to consider whether secondary staff in one child welfare agency might differ from child clinicians in other mental health settings. Each second order scale (rigidity, proficiency, and resistance for culture and engagement, functionality, and stress for climate) related to important outcomes such as individual worker attitudes.

Organizational culture measures “the way things are done in an organization” and is operationalized on three dimensions: rigidity, proficiency, and resistance. First, the level of rigidity indicates how much discretion or flexibility workers have in completing their job responsibilities, how much input workers have in key management decisions, as well as whether workers are expected to follow bureaucratic rules and regulations (e.g., “I have to ask a supervisor or coordinator before I do almost anything” and “The same steps must be followed in processing every piece of work”). Second, proficiency is a measure of how much workers place the wellbeing of clients first in their decisions, as well as whether workers are expected to have up-to-date knowledge about best practices. The two subscales of proficiency are responsiveness and competence (e.g., “Members of my organizational unit are expected to be responsive to the needs of each client” and “Members of my organizational unit are expected to have up-to-date knowledge.”) Finally, resistance indicates whether staff is open to change or new ways of providing services. The subscales of resistance are apathy and suppression of change (e.g., “Members of my organizational unit are expected to not make waves” and “Members of my organizational unit are expected to be critical”).

Organizational climate is also operationalized on three dimensions: engagement, functionality, and stress. Engagement is measured by employee perceptions that their actions are important and worthwhile, that they are emotionally invested in their work, and that they are concerned about their clients. The two engagement subscales are personalization and personal accomplishment (e.g. “I feel I treat some of the clients I serve as impersonal objects” and “I have accomplished many worthwhile things in this job.”). Functionality is a measure of whether employees perceive that they receive the support from coworkers and supervisors that they need to be effective in their positions. Functionality is measured by three subscales of growth and advancement, role clarity, and cooperation (e.g., “The agency provides numerous opportunities to advance if you work for it,” “My job responsibilities are clearly defined,” and “There is a feeling of cooperation among my coworkers”). Finally, stress describes whether employees are emotionally exhausted from their work, are overloaded, or unable to get the necessary things done. The three subscales for stress are emotional exhaustion, role conflict, and role overload (e.g., “Interests of the clients are often replaced by bureaucratic concerns,” “I feel like I am at the end of my rope,” and “The amount of work that I have to do keeps me from doing a good job.”

Analysis

Per requirement for use of the OSC, all of the analyses from the surveys collected in this study were completed by the University of Tennessee’s Children’s Mental Health Services Research Center (CMHRC). A total of 118 completed measures were sent to CMHRC, which first conducted analyses to identify cases with response inconsistency, irregular response patterns, and excessive missing data. Two cases were deleted from the analysis due to missing data (>10%) and three cases were deleted due to highly irregular responses suggesting random responses, resulting in a final sample of 113. No cases were deleted for other reasons, including divergent responses that significantly decreased estimates of intergroup reliability (i.e., outliers). CMHRC then computed within group agreement using r_wg (Demaree & Wolf, 1984) for each subscale of the OSC for each of the three groups. Within group agreement was very high, ranging from .92–.97 for the core group, .95–.98 for the secondary group, and .91–.98 for the prevention group across the subscales. R_wg scores range from 0–1 and are viewed as acceptable if above .70; these statistics support aggregation of individual responses within each subgroup. In contrast, the scores for internal consistency across the three groups were high but had a greater range, indicating less correspondence of scores in the entire sample than within subgroups. The Cronbach’s alpha for each of the subscales were .90, .75, .70, .80, .87, and .90 for proficiency, rigidity, resistance, engagement, functionality, and stress, respectively.

After confirming that within group agreement supported computation of subgroup scores, CMHRC computed t-scores based on national norms for both child welfare agencies and children’s mental health providers. T-scores standardize scores similarly to z-scores, but use the sample variation from previous studies to estimate the population variance and set the estimated population mean at 50 with a standard deviation of 10 (Hall & Geher, 2014). Because CMHRC does not directly compare groups, provide standard deviations of submitted OSC data, or test the significance of differences between an agency’s scores with national norms, the statistical significance of the differences observed between the different groups and the national norms cannot be provided using traditional t-test statistics. Instead, we use z-test statistics to provide estimates of the significance of differences between the normed t-score means for each of the three groups examined in our study as compared to CMHRC’s normed estimates of population means for both child welfare and mental health staff. The z-test includes a one-sample test that allows for the comparison of a sample mean with a population mean. It does not require the sample’s standard deviation but instead relies on the population standard deviation (Hall & Geher, 2014). Our z-test results are tentative, since these comparisons and estimates use the CMHRC’s estimate of the population standard deviation rather than an established population standard deviation. While use of the CMHR estimates for this purpose is supported by the very high within-group agreement in our sample (suggesting at least a comparable or smaller standard deviation relative to previous studies), we present all statistical results as “tentative z-statistics” to emphasize that they should not be overgeneralized, due to this limitation.

The t-score’s correspondence to percentiles also provides highly useful information that facilitates interpretation of possible differences. For example, a t-score of 60 indicates a percentile of 82%, suggesting a score above that of 82% of others in the normative sample. Scores that differed one standard deviation or less from the norm (i.e., >40 and <60, or less than 10 points from a t-score of 50) were viewed as similar to the norm. Practically significant differences were identified if scores were a standard deviation or more from the norm. In general, when a score is outside of one standard deviation from the mean, it may be considered outside the range of what would be expected (Boyle, Saklofske, & Matthews, 2015; Geher & Hall, 2014). While the selection of a standard deviation as indicating a possible difference is somewhat arbitrary, a difference in the direction hypothesized corresponds to a higher or lower score than the majority of other child welfare or mental health agencies. Our estimates of statistical significance using the z-test results suggest most differences of this magnitude would be replicated in future studies.

Results

Hypothesis 1. The core child welfare group will profile similarly to child welfare national norms, but will appear to have more difficult culture and climate scores as compared to mental health norms

As seen in Table 2, when using the criteria of within a standard deviation of the population norm, the core group appeared similar to the national child welfare norms in proficiency, rigidity, engagement, functionality, and stress, as expected, with some indication of higher functioning than an average child welfare in proficiency, functionality and stress. Resistance was the only dimension in which they scored higher (88^th percentile). Similarly, compared to mental health norms, the core group appeared to have a more resistant culture (97^th percentile), ranking nearly two standard deviations higher on resistance. They also ranked higher on rigidity (77^th percentile), lower on engagement (16^th percentile) and higher on stress (77^th percentile) relative to mental health norms, although these differences were not a full standard deviation different. Tentative z-test results suggest these differences are likely to be replicated in future studies, with z-statistics of 5.48, 13.03, −6.17, and 5.48 for rigidity, resistance, engagement, and stress (in comparison to normed mean of 50, p < .001 for all dimensions). In contrast, functionality appeared higher in the sample’s core group as compared to mental health norms (tentative z-statistic = 8.91, p < .01).

Table 2.

Percentile (and t-score) relative to national child welfare sample

Group	Proficiency	Rigidity	Resistance	Engagement	Functionality	Stress	N
	% (t)	% (t)	% (t)	% (t)	% (t)	% (t)
Core	77(58)	55(51)	88(63)	42(48)	79(58)	21(43)	47
Secondary	73(56)	45(50)	50(51)	50(50)	58(53)	12(39)	33
Prevention	94(65)	77(57)	84(60)	92(65)	94(66)	8(36)	33

These findings indicate modest support for perceptions of a more difficult social context by core child welfare staff when compared to mental health norms, with several dimensions with scores in the direction hypothesized, but just one dimension rated a full standard deviation more difficult.

Hypothesis 2. The secondary group will have perceptions that are mixed, with a profile in between child welfare and mental health norms

As seen in Table 2, the secondary group scored within one standard deviation on all dimensions of culture and climate when compared to a child welfare sample, with the exception of stress, which was one standard deviation lower, suggesting a better perception of social context. As seen in Table 3, when compared with mental health norms, however, resistance in the secondary group was one standard deviation higher (82^nd percentile). The secondary group also appeared to have higher rigidity (73^rd percentile) and lower engagement (21^st percentile), but higher functionality (82^nd percentile). Although these differences were all within one standard deviation from the norm, z-statistics suggest they are likely to be replicated (tentative z-statistics of 4.02, 5.74, 6.17, and 5.17 for rigidity, resistance, engagement, and functionality, respectively, p < .001 for all).

Table 3.

Percentile (and t-score) relative to national child mental health sample

Group	Proficiency	Rigidity	Resistance	Engagement	Functionality	Stress	N
	% (t)	% (t)	% (t)	% (t)	% (t)	% (t)
Core	55(52)	77 (58)	97(69)	16(41)	91(63)	77(58)	47
Secondary	50(51)	73(57)	82(60)	21(42)	82(59)	70(55)	33
Therapists	39(47)	79(58)	82(60)	8(37)	73(57)	73(57)	14
Prevention	77 (57)	87(62)	96 (67)	58(52)	96(69)	61(54)	33

Thus, the hypothesis that the secondary group would profile in between child welfare and mental health norms was moderately supported, with this group profiling a full standard deviation less stressed than child welfare caseworker norms and more resistant than mental health professional norms and other dimensions showing a less strong but mixed pattern.

Hypothesis 3. Child welfare therapists’ perceptions will appear to be more negative than mental health norms

As shown in Table 3, child welfare therapists were one standard deviation higher in resistance (82^nd percentile), and more than one standard deviation lower in engagement (8^th percentile) compared to national mental health norms. Rigidity (79^th percentile) and stress (73^rd percentile) were slightly higher compared to mental health norms, but these differences were within a standard deviation from the norm. In contrast, functionality was in the 73^rd percentile as compared to mental health norms. Tentative z-statistics were 3.74, −4.86, 2.99, 2.30, 2.62 for resistance, engagement, rigidity, stress, and functionality respectively (p < .05 for all). These findings provide moderate support for the hypothesis that therapists working within this child welfare agency could experience the social context as more negative than providers in a mental health agency.

Hypothesis 4. The prevention group will appear to experience the organizational context as more positive than child welfare norms

As compared to child welfare norms (Table 2), the prevention group appeared to have more positive perceptions of both culture and climate dimensions. Proficiency (94^th percentile), engagement (92^nd percentile), and functionality (94^th percentile) were all a full standard deviation higher than the child welfare average. Stress was a more than a standard deviation lower (8^th percentile), supporting more positive perceptions for the prevention group as compared to child welfare norms. Tentative z-statistics were 8.62, 8.62, 9.19, and −8.04 for proficiency, engagement, functionality, and stress respectively (p < .001 for all).

In contrast, however, rigidity (77th percentile) and resistance (84th percentile) appeared higher when compared to child welfare norms (tentative z-statistics 6.89 and 5.74 (p < .001 for both). Although these differences are within a standard deviation of the norms, they are notable as a similar pattern is found in the other two groups, suggesting apparent consistency in these two dimensions across groups.

Discussion

This exploratory study considers differences between work groups at one child welfare agency, and examines one agency’s profile as compared to national child welfare and national mental health norms of organizational social context. A greater degree of involvement in core child welfare tasks such as case management, foster home monitoring, and court proceedings was hypothesized to be associated with more negative perceptions of the culture and climate of an organization. The agency’s OSC profile was examined and analyzed by group (core child welfare, secondary, secondary-child welfare therapist, or prevention) to understand how each group’s report of culture and climate compared to national norms. Using norms established for both child welfare and mental health agencies, we examined the extent to which core child welfare staff, secondary staff (those supporting child welfare cases, but not directly involved in child welfare case management, foster home monitoring, or court proceedings), and prevention staff (those not working with child welfare-involved families) appear similar or different from these national norms. Although we were not able to test statistical differences using t-tests, apparent differences would suggest that exposure to core child welfare tasks might affect perceptions of culture and climate, and that subgroups are important constructs to examine in future research using organizational assessments.

As we expected, the core group profiled similarly to child welfare norms, with the exception of higher resistance, suggesting that they appeared less open to new ways of performing their work than norms for child welfare caseworkers. Support for the hypothesis that the core child welfare workers would perceive the culture and climate as more difficult than mental health norms was mixed. When compared to mental health norms, the core child welfare group appeared to have much higher resistance than the average mental health agency, but they also had higher functionality, meaning that they were more likely to perceive that they received needed support from coworkers and supervisors than mental health norms. The core group’s reports of engagement were also lower than mental health norms, with a t-score of 41 and a percentile of 16%, indicating that the core group reported engagement as lower than 84% of mental health agencies. On all other dimensions, the core group was within a standard deviation of mental health norms, and overall all groups at the agency appeared to rate stress as lower than national child welfare norms.

We also hypothesized that the secondary group, which provided support to child welfare-involved families, would have perceptions that fall in between child welfare and mental health norms. As compared to child welfare norms, the perceptions of the secondary group appeared consistent with national child welfare norms for the majority of the dimensions of culture and climate. The one difference was stress, which was rated as lower among the secondary staff. This finding was consistent with the hypothesis that staff in the secondary group may perceive less stress than those involved in core child welfare tasks, such as case managers.

The secondary group was also fairly similar to mental health norms with the exception of resistance, where they ranked higher than the norms. In a sample of 1,237 clinicians, workers perceptions of greater proficiency and less resistance were associated with greater openness to using evidence-supported treatments (Patterson et al., 2013). Thus, high resistance among staff could suggest less openness to EBPs, which could have important implications for implementation efforts if these findings are replicated among other units working in child welfare agencies. In addition, the secondary group appeared more similar to the child welfare norms than the mental health norms in terms of rigidity and engagement, both which were within a standard deviation from the mental health mean but appeared to be perceived more negatively.

When we examined the scores of the child welfare therapists separately from the secondary group, this group appeared to have some deviations from mental health norms, although the small number (n = 14) of therapists means that these descriptive results may be unreliable. In particular, similar to the secondary group overall, the child welfare therapists profiled with greater resistance and lower engagement than mental health norms, potentially supporting the idea that perceptions of clinical staff within child welfare settings may differ from those of other child therapists working outside of a child welfare context. Another study using OSC data from the original mental health sample found that more proficient organizational cultures and more engaged and less stressful organizational climates were related to more positive clinician attitudes about the evidence-based practices (Aarons et al., 2012). Because this sample was less engaged and more stressed than therapists working in a mental health agency, it is important to think more about how the therapists within a child welfare agency experience the social context, and how this, in turn, impacts both the work that they do with clients and the supports they may need when new interventions are introduced.

Limited research examines how mental health treatment work differs depending on the context in which it occurs. Some research suggests that therapists may face challenges providing clinical services within a system that at times appears to work in opposition to the needs of families (Jager et al., 2009). In addition, the nature of the problems that child welfare therapists attempt to treat may be more complex due to the fact that children are separated from their parents at the time of therapy and birth parents tend to experience a host of other social problems (Marcenko et al., 2011) that make treatment more challenging than with other high-risk groups. Further, given knowledge about secondary trauma among child welfare caseworkers, as well as therapists working with traumatized youth, it is likely that those working to support child welfare cases will also have higher rates of secondary trauma than child clinicians working outside of a child welfare context (Pryce et al., 2007).

Thus, it is important to consider whether treatment staff or staff who more generally support child welfare cases have unique challenges in their roles. For example, future research could examine perceptions of organizational context among clinicians working in child welfare as well as systematic differences in providing services to children within a child welfare context as compared to providing services within mental health service systems. A better understanding of organizational dynamics and their impact on treatment or supportive work in child welfare could be helpful for adapting training to child welfare mental health service providers.

In addition, if differences among staff on levels of openness to new practices depending on position do exist, implementation efforts may require additional support such as more persuasion or troubleshooting to facilitate use of new practices. Some studies, for example, test the use of motivational interviewing as part of the implementation process (Hettema, Ernst, Williams, & Miller, 2014; Snyder, Lawrence, Weatherholt, & Nagy, 2012), or work directly with the agency to improve culture and climate in order to improve implementation efforts (Glisson, Hemmelgarn, Green, & Williams, 2013; Strolin-Goltzman, 2010). Because the organizational environment can serve as an important buffer for stress related to child welfare work, it is important to consider how to improve the organizational context, particularly for front line staff, in order to understand how to best support their efforts to meet the needs of their clients.

On the other hand, greater levels of apparent functionality among the core child welfare group when compared to mental health norms suggest that this group may be more likely to receive the support they need and have greater clarity about their roles than mental health staff. This finding was unexpected, and highlights potential differences in the organizational processes that could make a caseworker’s responsibilities more clear than other staff working to support child welfare cases. It is not clear why the secondary group had lower ratings for functionality (sense of support and role clarity) among agency staff. If these findings are replicated, qualitative interviews with staff in different role categorizations could clarify how organizational factors can better support the role of the clinician in a child welfare context.

Finally, as hypothesized, the prevention group appeared the most distinct from child welfare norms. They had ratings of proficiency, engagement, and functionality that were at least one standard deviation higher than child welfare norms, and ratings of stress that were more than a standard deviation lower than national child welfare norms. Thus, although all staff was housed in the same agency, those who were not working with child welfare-involved clients but provided prevention services in the community reported more positive perceptions of the organizational culture and climate. Because those involved in prevention work reported lower levels of stress but had greater proficiency and engagement than the normed sample of child welfare caseworkers, our findings suggest that the nature of child welfare work itself could contribute to how workers experience the agency, but this hypothesis should be statistically tested in future research.

A main contribution of this paper is an introduction to the idea that differences may exist in perceptions of organizational social context depending on one’s role at the agency, and in particular, one’s proximity to core child welfare tasks. Thus, one agency’s organizational profile may have more variation across groups than would otherwise be detected by a single profile for the whole agency. Furthermore, because the secondary group at this agency appeared to profile more similarly to child welfare workers than therapists in other systems, it seems particularly important to consider the unique context of child welfare when implementing evidence-based treatments to clinical and supportive staff working with child welfare-involved families.

Limitations

While we are able to introduce new and important considerations in the discussion about organizational social context in child welfare, our research has several limitations. Administering the OSC profile within a single agency reduced variation due to agency policies, but impacts the extent we can generalize our findings, as study results may be idiosyncratic to the agency we studied. Furthermore, given that the OSC analysis group only provides individual agency profiles to researchers using their instrument, we were not able to conduct statistical tests of differences within this one agency. We were able to make comparisons to national norms using one-sample z-tests, but use of this test requires the assumption that the mean and standard deviation used by CMHRC to norm data from the OSC represent true population means and standard deviations. Although these values are derived from large, national samples, this assumption cannot be tested. Although a low level of variation within groups is indicated by the very high R_wg scores in our data, a higher than expected level of variation would reduce the statistical significance of the differences presented if traditional t-tests were used. In future research, it will be important to use a much larger sample to examine systematic differences between subgroups of staff at multiple agencies, as well as conduct statistical tests of differences between groups.

Additional research should also link any detected differences to outcomes such as effectiveness of services and child wellbeing to understand if these differences are practically significant. While we attempted to identify practically significant findings by using a standard deviation difference to identify differences, whether the apparent differences across groups has an impact on services, turnover, or client outcomes is unknown. Additionally, the OSC has not been validated with agency members in administrative roles such as program directors, but this study included program directors and agency administrators in the analysis. Future analysis should test whether the OSC is reliable for a broader range of positions in the child welfare system.

Finally, there were differences in the composition of the groups but we were not able to test differences between staff based on gender and ethnicity. For example, while the core group was about 60% African American, the secondary group was composed of only 36% African American (21% for the clinical staff within the secondary group). In addition, the secondary group had a greater percentage of female staff than the other groups (94% vs. 77.6 and 78% for the core and prevention groups respectively). Previous research suggests that gender and ethnicity may impact staff’s perceptions of the organizational environment (Chenot, Boutakidis, & Benton, 2014; Zeitlin, 2014), and therefore these factors are important to consider in future research, as it is possible that reporting of organizational context may be driven by racial differences more than subculture differences.

Conclusion

While agency and systemic factors are important to consider for implementation and sustainability of evidence-based practices in child welfare (Barth, 2008), to date no studies have examined how different groups in a single child welfare agency compare with child welfare and mental health norms on perceptions of agency culture and climate. Findings from this study suggest that core child welfare staff, those working most directly to address the immediate needs of children in foster care, may have organizational profiles that are more negative than perceptions of staff working in mental health agencies. However, in this study, staff within the secondary group (who had less responsibility for core child welfare tasks, such as therapists) appeared to profile more similarly to child welfare workers than other mental health workers.

Future research should consider whether the unique qualities of child welfare may cause those staff supporting child welfare cases to have perceptions about their organization that are more similar to those responsible for core child welfare tasks than clinicians in other mental health systems. Those involved in prevention work that had the least involvement in core child welfare tasks had the most positive perceptions. Although these findings are based on a small sample, they provide the basis for future research, as more positive perceptions of culture and climate are potentially linked to worker satisfaction and client outcomes (Glisson & Green, 2006; Glisson et al., 2012). Knowing more about variation in perceptions of the organizational context within child welfare agencies may be critical for supporting staff working with families in the child welfare system.