Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Jan 24.
Published in final edited form as: Adm Policy Ment Health. 2020 Sep;47(5):764–778. doi: 10.1007/s10488-020-01037-y

A Systematic Review of Organizational and Workforce Interventions to Improve the Culture and Climate of Youth-Service Settings

Rachel R Ouellette 1, Allison C Goodman 1, Frances Martinez-Pedraza 1, Jacqueline O Moses 1, Kelly Cromer 1, Xin Zhao 1, Jeffrey Pierre 1, Stacy L Frazier 1
PMCID: PMC9872793  NIHMSID: NIHMS1585804  PMID: 32303926

Abstract

Both organizational culture and climate are associated with service quality and outcomes across youth-service settings. Increasing evidence indicates capacity of organizational interventions to promote a positive and effective culture and climate. Less is known about common intervention components across studies and service settings. The current systematic review reviewed 9,223 citations and identified 31 studies, across six youth-service settings, measuring changes over time in organizational culture and climate following implementation of an organizational or workforce support intervention. Results highlight the promise of organizational interventions, a need for more comparison and randomized designs, and future directions for maximizing capacity of organizations to promote health for frontline providers and the children they serve.

Keywords: Youth-service settings, organizational interventions, organizational culture, organizational climate, systematic review

An organization’s social context, most often characterized by culture and climate, can influence services for youth across education, medical, and mental health settings through multiple pathways. In addition to associations with service quality (e.g., Olin et al., 2014), engagement (e.g., Kim et al., 2015), and outcomes (e.g., Glisson et al., 2013), organizational culture and climate can impact service delivery via provider retention (e.g., Glisson et al., 2008), attitudes toward (e.g., Aarons et al., 2012) and implementation of (e.g., Williams et al., 2019) evidence-based practice. An organization’s culture and climate have long been understood to impact its likelihood or readiness for change, including the ability of the organization to grow and develop over time and effectively initiate new practices, policies, or procedures (Schneider et al., 1996). By influencing an organization’s readiness to change, the social context can either facilitate or obstruct efforts to improve the quality of services in youth-service settings (e.g., Taxman et al., 2014). In consideration of this emerging literature documenting the import of the organizational social context, a growing number of investigative teams are examining methods for improving culture, climate and readiness for change.

Organizational Culture and Climate

Variable definitions and measures of organizational culture and climate are utilized across disciplines, contexts, and investigators (e.g., Denison, 1996; Schein, 2000; Schneider, 2000). Definitions of organizational culture tend to be more consistent, and typically focus on shared and established norms, assumptions, and values of an organization, which communicate behavioral expectations to employees of a work unit (Cooke & Szumal, 1993; Sorensen, 2002). Organizational culture can be measured via surveys, observations, or interviews to gather both an insider and outsider perspective regarding an organization’s health and functioning (Peterson & Fischer, 2004). Culture is commonly described along dimensions regarding specific organizational values or profiles (e.g., innovation, openness, rigidity, proficiency).

Greater variation is reflected in definitions of organizational climate, which most often refers to employees’ descriptions and collective perceptions of their work environment. Definitions common to medical settings relate to perceived goals and priorities of an organization (e.g., a climate for trust, safety climate; e.g., Peterson & Fischer, 2004). Definitions more common to school-based settings focus on the health of its interpersonal relationships and overall functioning (e.g., Hoy et al., 1995; Hoy & Feldman, 1987). Still others, commonly across mental health and social service settings, derive organizational climate from aggregated individual reports of psychological climate (Glisson & James, 2002; Jones & James, 1979; Joyce & Slocum, 1984), which reflects the perceived impact of the work environment on an individual’s own well-being (James & James, 1989). Organizational climate is commonly measured via surveys or interviews and combined into a global assessment of healthiness or effectiveness.

Organizational Interventions

Increasing evidence illustrates potential for organizational interventions to improve organizational culture and climate. Examples include Availability, Responsiveness, and Continuity (ARC; Glisson et al., 2006), Design Teams (e.g., Anderson-Butcher et al., 2003), Plan Do Study Act cycles (Institute for Healthcare Improvement: http://www.ihi.org/), and Strategic Planning (e.g. Bryson, 1995). Organizational interventions vary in complexity, but often involve multiple stages of implementation and change, and require varying lengths of time and levels of resource and support from external expert consultants. While some organizational interventions, such as strategic planning, have been used in community organizations for many years, multiple researchers have highlighted the need for more rigorous empirical examination of organizational interventions in youth-service settings (Glisson et al., 2006; Glisson et al., 2012; Parmelli et al., 2011).

Science has demonstrated the importance of associations among perceived resources, self-efficacy, and organizational change (Weiner, 2009). Specifically, low or variable resources can directly impact an organization’s readiness to change by limiting its structural capacity to implement recommendations. Individuals also consider the resources available in their environment when forming their change efficacy judgments about implementing a new innovation. These findings are especially important for low-resource organizations that may struggle with high demands, frequent turnover, inconsistent funding, or shifting priorities accompanying changes in leadership. These organizations may be at greater risk for experiencing low readiness to change, reflecting perceptions that available resources are insufficient to meet demands required to implement a complex or multi-stage organizational change intervention.

The complexity of many organizational interventions may also make them susceptible to barriers similar to those encountered in the dissemination of manualized treatments (Chorpita et al., 2007), including compatibility with current priorities or practice, lack of access to manuals or materials, and de-adoption of beneficial strategies. Mental health researchers have made steps towards addressing the complexity and proliferation of manualized treatments through the identification of evidence-based kernels (Embry & Biglan, 2008) and common elements (Chorpita et al., 2005). Distillation of practice elements can point to active ingredients across interventions and facilitate a modularized approach whereby treatment can be matched to each patient’s individual symptom profile and therapeutic needs. Extending this to organizational interventions, then, distillation of practice elements may similarly contribute to more efficient and tailored interventions, selected to align with and respond to each organization’s unique mission, available resources, and goals for change.

Like children and families seeking medical and mental health services, each organization presents with unique needs, strengths, and constraints. It is therefore possible that not every organization would require or benefit from every component comprising more extensive organizational interventions; instead, certain components (or sequencing or combination of components) may hold greater promise towards improving different aspects of an organization’s social context. Greater feasibility and impact for some organizations may depend on identifying a menu of distinct and concrete organizational intervention components, while other organizations may benefit from a more comprehensive intervention package. Similarly, organizational climate has been found to change more quickly than culture in response to organizational interventions (Glisson et al., 2006), indicating potential variability in the ability for organizational interventions to improve different culture and climate outcomes.

Youth Serving Organizations

Youth serving organizations provide behavioral, health, and prevention-focused supports to youth across contexts, including schools, after-school, mental health, pediatrics, juvenile justice, and child welfare. Collectively, these organizations play a large role in promoting short- and long-term youth development across domains of adjustment, and contribute to mental health and wellness in particular both directly, via targeted services, and indirectly, via more universal prevention-focused skill development (e.g., social skills and emotional resilience) and physical health education (e.g., nutrition, exercise) (Das et al., 2016).

Youth serving organizations across settings experience multiple constraints and fluctuations that can impact their organizational culture and climate. Most youth serving organizations are funded by nonprofit or government sources, with youth-service settings reporting particularly large drops and fluctuations in funding over time (Boris et al., 2010; Twombly, 2005). Previous research has found that youth-service providers (e.g., social workers) leave their organizations at higher rates than other providers (e.g., Cyphers, 2001), and report higher levels of burnout (Baldschun et al., 2019; Hussein, 2018). These unique considerations could result in additional burdens on organizational culture and climate outcomes and influence the success of different organizational interventions. With these considerations in mind, we examine culture and climate outcomes and use of organizational interventions in youth serving organizations specifically.

Current Study

The current study is a systematic review of organizational interventions and workforce support efforts measuring change over time in organizational culture and/or climate outcomes across community youth-service settings. The purpose of this review is to: 1. Assess use of different study designs to examine the effects of organizational interventions; 2. Summarize and compare definitions and measures of organizational culture and climate across service settings; 3. Identify common intervention components utilized across organizational interventions and workforce support efforts, and 4. Identify future directions in organizational intervention research to promote organizational culture and climate.

Methods

Search Strategy

We completed a systematic search of the literature in March 2019 utilizing a search string tailored to match our inclusion criteria (see Table 1). We searched titles, abstracts, and keyword terms across four electronic databases (PsycINFO, ERIC, PubMed, and Web of Science), with four databases commonly found to provide good coverage when conducting reviews (Lam & McDiarmid, 2016). For PubMed, Medical Subject Headings (MeSH) terms were also included. We selected these databases because they are inclusive of multiple disciplines and contexts (i.e. Web of Science) and comprehensive of contexts where youth commonly receive mental health (i.e., PsycINFO), education (i.e., ERIC), and medical (i.e., PubMed) services. There were no limitations placed on publication year. We limited search results to peer-reviewed articles and dissertations. We restricted our search to only youth-service settings due to their unique characteristics (e.g., particularly high turnover rates; Cyphers, 2001) and to increase likelihood that review results could be effectively and cohesively summarized in one review.

Table 1.

Search string organized by inclusion criteria.

Search criteria Search string
Complete search string (infant* OR child OR children OR “child’s” OR adolescen* OR “school-age*” OR youth* OR juvenile) AND (school* OR education OR welfare OR “mental health” OR community OR “afterschool” OR “after-school” OR camp OR summer OR “justice system” OR “detention center” OR “rehabilitation center” OR “social service” OR “child care” OR “after care” OR aftercare OR childcare OR preschool OR “pre-school” OR “day care” OR daycare OR headstart OR “head start” OR “prekindergarten” OR “pre-kindergarten” OR kindergarten OR “primary care” OR “emergency room” OR pediatric) AND (“organization* culture” OR “organization* climate” OR “organisation* culture” OR “organisation* climate” OR “psychological climate” OR “school climate” OR “organization* social context” OR “organisation* social context” OR “organization* health” OR “organisation* health” OR “work environment” OR “work climate” OR “work culture” OR “work attitude*” OR burnout OR “job demand*” OR “job resource*” OR “organization* satisfaction” OR “organisation* satisfaction” OR “job satisfaction” OR “job stress” OR “organization* commitment” OR “organisation* commitment” OR “job commitment” or “organization* readiness” OR “organisation* readiness” OR “readiness for change” OR “organization* innovation” OR “organisation* innovation” OR “organization* change” OR “organisation* change” OR “organization* structure” OR “organisation* structure” OR “organization* safety” OR “organisation* safety” OR “role conflict” OR “role overload” OR “role ambiguity” OR “role clarity” OR “personal accomplishment”) AND (change OR support OR intervention* OR training OR implementation OR program OR “capacity building” OR “professional development”)
a. Youth-serving (infant* OR child OR children OR “child’s” OR adolescen* OR “school-age*” OR youth* OR juvenile)
b. Setting (school* OR education OR welfare OR “mental health” OR community OR “afterschool” OR “after-school” OR camp OR summer OR “justice system” OR “detention center” OR “rehabilitation center” OR “social service” OR “child care” OR “after care” OR aftercare OR childcare OR preschool OR “pre-school” OR “day care” OR daycare OR headstart OR “head start” OR “prekindergarten” OR “pre-kindergarten” OR kindergarten OR “primary care” OR “emergency room” OR pediatric)
c. Outcome (“organization* culture” OR “organization* climate” OR “organisation* culture” OR “organisation* climate” OR “psychological climate” OR “school climate” OR “organization* social context” OR “organisation* social context” OR “organization* health” OR “organisation* health” OR “work environment” OR “work climate” OR “work culture” OR “work attitude*” OR burnout OR “job demand*” OR “job resource*” OR “organization* satisfaction” OR “organisation* satisfaction” OR “job satisfaction” OR “job stress” OR “organization* commitment” OR “organisation* commitment” OR “job commitment” or “organization* readiness” OR “organisation* readiness” OR “readiness for change” OR “organization* innovation” OR “organisation* innovation” OR “organization* change” OR “organisation* change” OR “organization* structure” OR “organisation* structure” OR “organization* safety” OR “organisation* safety” OR “role conflict” OR “role overload” OR “role ambiguity” OR “role clarity” OR “personal accomplishment”)
d. Intervention (change OR support OR intervention* OR training OR implementation OR program OR “capacity building” OR “professional development”)
Open in a new tab

Inclusion Criteria

Inclusion criteria were as follows:

  1. Article describes an original, peer-reviewed empirical research study or dissertation.

  2. Article is written in English or Spanish (corresponding to language proficiencies among authors).

  3. Study was conducted in a community youth-service setting (i.e., schools, after-school, child welfare, juvenile justice system, medical systems, community mental health centers).

  4. Support was provided to workforce or organization in the form of skill/knowledge development, team development, or organizational-level change.

  5. At least one baseline and at least one outcome measure, quantitative or qualitative, of provider or leadership-reported organizational culture, climate or social context was collected and analyzed.

Definitions and Measures of Organizational Culture and Climate

Due to aforementioned variability in definitions across disciplines, contexts, and investigators, we consolidated to identify the following core aspects of social context commonly reflected across measures of organizational culture and climate: (1) Interactions between people; (2) Values and norms within the organization; and (3) Perceived behavioral expectations. As this review focuses on organizational-level constructs, outcome measures met criteria only if: (1) Survey items asked about organizational characteristics (e.g., To what extent does your organization value collaboration?) or collective perceptions of organizational characteristics (e.g., “To what extent do your coworkers value collaboration?”), rather than individual perceptions (e.g., “To what extent do you value coworker input when solving problems on the job?”), or; (2) Group means were estimated at the organizational- rather than individual-level, as recommended for examining organizational outcomes such as culture and climate (Glisson & Green, 2006). A subsequent review is being prepared that reports on individual psychological climate factors (e.g., burnout, stress) and work attitudes (e.g., job satisfaction and organizational commitment).

Study Selection Process

A PRISMA (Preferred Reporting Items for Systematic Review and Meta-Analyses) flow-chart of study selection is included for reference (see Figure 1). Notably, we modified the process by selecting articles in two phases. Phase one included a review of all study titles, abstracts, and keywords to identify studies that may meet inclusion criteria. Studies were screened out if they were conceptual rather than empirical, conducted in a setting that does not serve youth, or examining outcomes not relevant to this review. Seven authors (removed for masked review) contributed to phase one, first by reviewing one sub-sample (n = 20) of abstracts to refine and reach consensus on inclusion criteria. We then distributed remaining abstracts randomly across authors, who independently screened abstracts. An initial screening session was conducted as a group followed by regular check-ins to discuss instances of uncertainty across authors to maintain consensus.

Figure 1.

Figure 1.

Open in a new tab

Preferred reporting items for systematic reviews and meta-analyses (PRISMA) flow diagram.

Phase two included a full article review to identify studies meeting all inclusion criteria. Four authors (removed for masked review) contributed, first by reviewing one set of full-length manuscripts (n = 5), documenting for each which inclusion criteria were met or not met, to ensure consensus. The remaining studies were then randomly assigned across authors. The first author reviewed all articles identified for inclusion to confirm they met criteria. Senior author provided guidance in refining the inclusion criteria and served as an additional reviewer on papers where there was discrepancy regarding inclusion or exclusion.

Data Extraction and Analysis

Data Extraction

Four authors [removed for masked review] contributed to data extraction. Each author received a set of full-length manuscripts that met criteria following phase two and independently extracted demographic and outcome data (listed in Table 2). Articles were also coded for organizational intervention and workforce support components and organizational culture/climate outcome-types assessed. As new intervention components emerged across studies, they were added to the list of codes and recoded across all articles. Ultimately, every article was coded for the use or non-use of each support component (n = 11 components total) and the analysis of each type of culture/climate measure (n = 5 categories total) (summarized in Table 3). Intervention components and culture/climate outcomes were coded independently by two coders with discrepancies resolved through consensus. The first author (removed for masked review) reviewed all data extractions and codes for completeness and accuracy.

Table 2.

Outcome, demographic, and study design data extracted from included studies.

Data type Specific variables collected
Article Information – List of authors included on the article
– Year of publication
– Brief summary of study’s objectives
Setting and Workforce – Country study was conducted in
– Service setting (i.e., schools, welfare, medical, after-school, juvenile justice, mental health)
– Job titles of individuals participating
Study Design – Sample size
– % of staff in organization receiving support
– Study design (e.g., Group RCT, before-after)
– Use of quantitative or qualitative methods
– Assignment to groups (i.e., number of intervention groups, use of randomization, presence of a control/comparison group)
– Time between pre and post measurement
Outcomes – Reference level (i.e., perceptions of coworkers’ collective perceptions vs. individual perceptions)
– Culture/climate and patient outcomes collected
– Descriptive summary of results
– Coding of organizational intervention components
Open in a new tab
Table 3.

Summary of organizational intervention components and organizational culture and measures coded across 31 included studies, ordered from highest to lowest frequency.

Intervention Component Description Frequency
1. In-person training(s) Skills development via group-based trainings n = 23
2. Multi-level feedback Feedback collected across organizational levels n = 22
3. External coaching/ consultative support Ongoing individual or team-level consultative support provided by an external partner n = 18
4. Goal alignment Identification of shared goals and mission n = 17
5. Data-informed team-based decision-making Small and repeated tests of progress utilizing ongoing data collection or progress monitoring n = 17
6. Peer-to-peer support Opportunities for providers to share knowledge with and provide support for each other n = 16
7. Organizational restructuring Change to higher-order organizational functioning, including organizational procedures and job roles n = 14
8. Formation of an internal committee Individuals from the organization responsible for continuing quality improvement efforts over time n = 14
9. Implementation of evidence-based services/intervention Dissemination/implementation of an evidence-based practice via at least one of the other organizational intervention components n = 13
10. Mindfulness / counseling services Interventions or support provided to frontline providers targeting their social / emotional health n = 5
11. Leadership training Targeted training for team or organizational leaders to promote leadership skills n = 2
Culture/Climate Construct(s) Measured
1. Organizational values and norms Collective beliefs/norms about value of specific behaviors, ways of conducting work, and/or responding to change. n = 20
2. Interpersonal or social interactions Presence and value given to positive and supportive interactions among providers, supervisors, and families n = 20
3. Job demands Physical, social, or organizational aspects of job that require sustained physical or mental effort n = 7
4. Collective perceptions of stress/burnout Perceived impact of work environment on physical and emotional well-being n = 11
5. Global metrics of readiness or effectiveness Composite measures of overall organizational effectiveness or readiness for change n = 9
Open in a new tab

Assessment of Risk of Bias in Included Studies

To capture emerging literature, we decided not to limit our inclusion criteria by study design (e.g., RCT) beyond requiring at least one pre and at least one post measure of organizational culture or climate. Instead, we coded each study based on a set of factors related to study design taken from an assessment tool developed by the Task Force on Community Preventative Services (Zaza et al., 2000) to examine the quality or reliability of reported results.

These factors included sample size, assignment to groups (i.e., number of groups, presence of a control group, randomization), and overall study design (e.g., before-after, time series, group randomized trial). In addition to assessing for heterogeneity in results by service context, we also assessed for heterogeneity in study quality and design. Data regarding study design can be found in Supplemental Table 1.

Synthesis of Results

A complete table of extracted data can also be found in Supplemental Table 1. Due to substantial heterogeneity in study design, outcomes, and service contexts, we will not report study results as the pooled effect estimate in a meta-analysis. We instead list measures collected, reported results, and organizational intervention components used in each study, and through preliminary analysis and synthesis of results describe: 1. Use of different study designs and methods; 2. Definitions and measures of organizational culture and climate used across service settings, and; 3. Common intervention components utilized across organizational interventions and workforce support efforts. We also extracted data regarding youth-level outcomes reported in each paper to assess for changes in youth outcomes following organizational interventions.

Results

Search Results

Overall, 9,223 non-duplicated citations were identified through the database search. Search results for each database are reported in Figure 1. There were 8,736 articles excluded following initial screening of abstracts and titles. Full manuscript review was completed for 464 articles, of which 433 were excluded as not meeting criteria for the following reasons: no baseline measure (n = 222) or outcome measure (n = 182) of organizational culture or climate; measures of psychological climate or work attitudes were not aggregated to the organizational-level (n = 59); no intervention or change effort implemented (n = 93); and service setting did not explicitly serve youth (n = 47). Reasons for exclusion are reported in detail in Figure 1. Overall, 31 articles met inclusion criteria and were included in the data extraction phase.

Settings

Of the 31 studies included, 29% (n = 9) took place in school or early child care settings, 19.4% (n = 6) in hospital settings, 16.1% (n = 5) in child welfare settings, 12.9% (n = 4) in community mental health settings, 6.5% (n = 2) in after-school settings, 6.5% (n = 2) in juvenile justice settings, 3.2% (n = 1) in primary care medical settings, and 6.5% (n = 2) across multiple settings (i.e., juvenile justice and social services). Studies were conducted across seven countries.

Study Design

Of the 31 studies, 100% included at least one quantitative outcome in the form of observational or survey data. Seven studies also included qualitative data, primarily from focus groups or semi-structured interviews. Sample sizes varied from 14 to over 2,000 providers. Five studies did not conduct any significance tests or calculate effect sizes to examine change over time, instead qualitatively described changes in mean values from pre to post. Only 15 studies included a comparison group; among these, eight utilized random assignment, two compared two intervention groups against one another and a “treatment as usual” condition, and 13 compared one intervention to a control or comparison group. Across the 16 studies with no comparison group, 38% reported improvements across all measured organizational culture and climate outcomes, with 62% reporting mixed findings with some outcomes improving, others deteriorating, and some remaining the same. Across the 15 studies with a comparison group, 27% reported improvements across all outcomes and 73% reported mixed findings. These results indicate that studies with no comparison group were slightly more likely to report improvements across all outcomes. No studies from either study type reported deterioration or nonsignificant change across all outcomes, indicating that publication bias may be inflating reported results across study designs.

Measurement of Organizational Culture and Climate

A wide variety of organizational culture and climate outcomes were measured across the 31 studies and fell broadly into five categories: 1. Organizational values and norms; 2. Interactions between people in the workplace; 3. Collective perceptions of job demands; 4. Perceptions of collective emotional healthiness of organization, and; 5. Global metrics of an organization’s perceived readiness for change and/or effectiveness. We include summarized results by outcome type. Results are summarized across all study designs, both with and without control groups and randomization, to identify emerging trends. These results do not assert causality and should be approached with caution until confirmed via additional empirical studies utilizing comparison and randomized designs.

Organizational Values and Norms

Overall, 20 studies reported one or more outcomes related to organizational values and norms. Of these studies, 45% reported improvements across all outcomes, 45% reported mixed results, and 10% reported all nonsignificant changes. The measurement of organizational values is the outcome category that varied the most across disciplines and service settings. Studies conducted in medical settings were more likely to measure values and/or norms around specific behaviors in the workplace, such as the prioritization of infant wellness through breastfeeding best practice (Henry et al., 2017) and safety during patient transitions between hospital departments (Sheth et al., 2016). Specific values and norms such as these were measured either via self-report surveys or observations of frequency and quality of services (e.g., Henry et al., 2017). Across the 31 studies, only four utilized observations or organizational records of service provision.

Studies conducted in non-medical settings were more likely to measure organizational values and/or norms across broader dimensions. For instance, Glisson and colleagues’ (Glisson et al., 2008) Organizational Social Context Measurement System (OSC) has been used across studies conducted in juvenile justice, child welfare, after-school and community mental health settings. The OSC is a self-report survey with individual responses aggregated at the organization or agency level to assess an organization’s culture across three dimensions (proficiency, rigidity, and resistance). Resistant cultures are characterized as those that do not incorporate new practices quickly and often push against change efforts, while proficient cultures are those that prioritize having the most up-to-date resources and knowledge to serve children and families. Studies also used other measures to assess similar dimensions examining values around innovation, justice, and goal-centered behaviors (e.g., Lawrence et al., 2015; Taxman et al., 2014; Orthner et al., 2006).

Interpersonal or Social Interactions in the Workplace

Studies also included measures focused on the presence and value given to positive and supportive interactions among frontline providers, supervisors, and families served, including perceived levels of support from and cooperation between colleagues and leadership (e.g., Potter et al., 2009; Rhodes et al., 2009; Slater et al., 2018) or reported levels of teamwork (e.g., Barnett et al., 2015). Overall, 20 studies reported one or more outcomes related to social interactions; with 35% reporting improvements across all outcomes, 55% reporting mixed results, and 10% reporting all nonsignificant changes. One study measured communication via social networks before and after participating in a learning collaborative (Bunger & Lengnick-Hall, 2018). Other outcome measures focused more specifically on interactions between leadership and providers, including measures of supervisor competence or effectiveness (e.g., Potter et al., 2009; Renner et al., 2009), collegial leadership (e.g., Bradshaw et al., 2008), and participation in decision-making (e.g., Hickey, 1994). These constructs were all measured using either self-report surveys, focus groups, or semi-structured interviews. Social interactions could be measured across peers at a similar organizational level or between supervisors/leadership and frontline providers.

Job Demands

Job demands are “those physical, social, or organizational aspects of the job that require sustained physical or mental effort, and are therefore associated with certain physiological and psychological costs” (Demerouti et al., 2001). There are two primary types of job demands: challenges (e.g., role overload) and hindrances (e.g., role conflict and role ambiguity) (Crawford et al., 2010). Overall, seven studies reported one or more outcomes related to job demands; with 29% reporting improvements across all outcomes and 71% reporting mixed results.

Collective Perceptions of Organizational Stress or Burnout

Collective perceptions of psychological health in the organization are often measured as work-related stress or burnout. The most commonly used definition of organizational climate in mental health research derives organizational climate from aggregated individual reports of psychological climate, or the perceived impact of the work environment on an individual’s own well-being (Glisson & James, 2002). Overall, 11 studies reported one or more outcomes related to stress or burnout; with 27% reporting improvements across all outcomes, 64% reporting mixed results, and 9% reporting all nonsignificant changes. Measures of psychological health, burnout, or stress had to be collected or analyzed at the organizational or team level to meet criteria as a measure of organizational climate.

Global Metrics of Organizational Readiness or Effectiveness

Most studies also included a composite measure of overall organizational effectiveness or readiness for change. For instance, widely used in the education literature (and represented here in Bradshaw’s 2008 and 2009 studies) is the Organizational Health Inventory (OHI; Hoy & Feldman, 1987) with five subscales (e.g., collegial leadership, staff affiliation, and academic emphasis among students) that altogether yield a total score that represents the overall perceived effectiveness across interactions within the school environment. Related is a measure of organizational readiness (Taxman et al., 2014) that combines staff perceptions of organizational resources and support, workplace climate, and interactions with other agencies; factor analysis revealed a single underlying factor they referred to as “readiness for change”. Overall, nine studies reported one or more outcomes representing a global metric of readiness or effectiveness; with 78% reporting improvements across all outcomes and 22% reporting mixed results.

Organizational Intervention Components

Frequencies of included intervention components are reported in Table 3, with organizational culture/climate outcomes reported by component type reported in Supplemental Table 2. More than half of the studies (n = 20; 65%) did not include enough specification to confidently code for presence or absence of each component; therefore, frequency of use across studies may be under-estimated. Intervention components fall broadly into four groups, including: (1) Skill development; (2) continuous quality improvement; (3) organizational restructuring; and (3) provider social and emotional support.

Skill Development

The most commonly used support strategies across studies focused on workforce skill development, in particular in-person trainings (n = 23; 74%) and coaching or consultative support from an external partner or facilitator (n = 18; 58%). In-person trainings and consultative support typically focused on the implementation and dissemination of evidence-based interventions or services for youth (n = 13; 42%). A smaller number of in-person trainings focused more explicitly on team dynamics, and often included team-based role plays (e.g., Kennedy et al., 2013). Implementation and dissemination of evidence-based interventions and services were also supported by peer-to-peer support methods (n = 16; 52%), which included the use of learning collaboratives (e.g., Bunger & Lengnick-Hall 2018), train-the-trainer approaches (e.g., Shoushtarian et al., 2014), and networking interventions (e.g., Taxman et al., 2014). While numerous studies mentioned leadership involvement in the implementation of other support components (e.g., team-based decision-making), only two studies mentioned the use of explicit leadership training targeting development of leadership skills (Kennedy et al., 2013).

Continuous Quality Improvement

Over half of studies (n = 17; 55%) utilized data-informed team-based decision-making. An example of this type of strategy is Plan-Do-Study-Act cycles (e.g., Bunger & Lengnick-Hall, 2018; Sheth et al., 2016), which include small and repeated tests of effectiveness around change efforts utilizing ongoing data collection or progress monitoring. Common across data-informed decision-making efforts is the ongoing collection and use of data to inform organizational change (e.g., regarding organizational structure or provision of services). A large number of studies (n = 14; 45%) also formed internal committees that consisted of a group of individuals belonging to the organization who were responsible for continuing quality improvement efforts over time.

Data-informed team-based decision-making is often preceded by a period of goal alignment or organizational-level goal setting (n = 17; 55%). Interventions such as ARC (Glisson et al., 2006), strategic planning (e.g. Bryson, 1995), and Design Teams (e.g., Lawrence et al., 2016) emphasize that organizational change should be “mission-driven” and that this mission should be shared across individuals in the organization. To ensure that organization-level goals are shared, these interventions also highlight the importance of gathering feedback from multiple stakeholders (n = 22; 71%) across levels of the organization, including both leadership and frontline providers. Feedback from multiple stakeholders can be helpful towards identifying organizational-level goals to drive individual and collaborative change efforts as well as planning action steps working towards identified goals.

Organizational Restructuring

Also common to nearly half of the studies was organizational restructuring (n = 14, 45%), including procedural or role changes. For instance, the design team intervention (e.g., Lawrence et al., 2016), focuses on change to higher-order organizational functioning, such as development and implementation of new employee appraisal tools, rather than efforts to influence frontline service delivery.

Provider Social and Emotional Support

Finally, a small number of studies (n = 5; 16%) focused less on organizational change and more on organizational support, including the provision of mindfulness, counseling, or social-emotional health and wellbeing interventions to providers. An example can be found in the study by Sottimano and colleagues (2018), where they provided individual counseling and a worksite intervention to analyze and improve group dynamics; both interventions improved vertical trust and coworker social support when compared to a control group.

Change in Organizational Culture and Climate Over Time

All 31 studies reported positive changes over time on at least one measure of organizational culture or climate; however, most studies reported mixed findings across different aspects of the organizational environment, with some subdomains demonstrating improvement, other domains remaining constant, and some deteriorating over time. Among included studies with a comparison group (n = 15), seven studies examined outcomes two or fewer years following baseline, with 86% (n = 6) reporting mixed findings and 14% (n = 1) reporting positive and significant results across outcomes. Five studies collected data between three-to-five years following baseline, with 60% (n = 3) reporting mixed findings and 40% (n = 2) reporting positive and significant results across outcomes. Three studies did not report the length of time between pre and post data collection.

Consideration of results across studies highlights a few notable trends. First, impacts of organizational interventions often require time. For example, Glisson and colleagues (2006, 2012) have found across studies that organizational culture typically takes longer to change than organizational climate, but that it can still be malleable over time. Bradshaw and colleagues (2009) also found that schools receiving intervention took several years before organizational health began to improve sufficiently to differ statistically from comparison schools, and that certain aspects of organizational health (i.e., resource influence and academic emphasis) took even longer to change or never differed significantly from comparison schools (i.e., institutional integrity). Taxman and colleagues (2014) also found that, among intervention groups, perceptions of organizational readiness declined, before ultimately improving to levels exceeding those of the control condition, indicating that organizational change interventions may actually have negative impacts on organizational culture and climate at first, followed by an accelerated increase. More research is needed to understand what it takes to maintain these gains over more extended periods of time.

Second, the pattern of results for role overload was mixed. Specifically, while role overload improved over time in a small number of studies (e.g., Glisson et al., 2006), it actually deteriorated across most others (e.g., Glisson et al., 2012; Lawrence et al., 2016), even though other role-related factors (e.g., role conflict and ambiguity) and other culture or climate factors improved. Third, leadership factors demonstrated fewer improvements than other social factors across providers. While a larger number of studies found significant improvements in perceived coworker support, leadership factors often did not change as much over time (e.g., Bradshaw et al., 2008; Green et al., 2012), despite leadership feedback and participation being incorporated during the intervention process. Last is a notable finding by Bunger and Lengnick-Hall (2018), who found that changes in communication following a learning collaborative depended on team size, highlighting team size as a potential moderator between organizational intervention implementation and resulting culture and climate outcomes.

Youth Outcomes

All studies were coded for the presence of youth-level service, behavior, and/or health outcomes. Six studies (n = 3 studies with a comparison group; n = 3 without) collected relevant measures, with outcomes including student grades, behavior indicators, hospital perinatal outcomes, student climate, and patient satisfaction. Of these, two studies (n = 1 with comparison group; n = 1 without) reported mixed findings, including statistically significant improvements in some outcomes and nonsignificant changes in others, with the remaining four studies (n = 2 with comparison group; n = 2 without) reporting all outcomes significantly improving over time. No studies reported mental health specific outcomes.

Discussion

This review describes the use of organizational interventions in youth-service settings and their impacts on organizational culture and climate. A total of 31 articles describing 30 unique research studies were identified out of 9,223 total search results. Studies were conducted across seven countries and a range of youth-serving community settings, including after-school, child welfare, juvenile justice, community mental health, early child care, school, and medical settings. Most studies reported mixed findings across different aspects of the organizational environment, with some subdomains demonstrating improvement, other domains remaining constant, and some deteriorating over time. Global metrics (e.g., organizational culture profiles and overall perceptions of organizational health) were more likely to improve over time, whereas findings were more equivocal for individual components (e.g., coworker support, leadership effectiveness, role overload), such that some factors improved, others deteriorated, and still others demonstrated no meaningful change. The most common components of organizational intervention or workforce support included: data-informed team-based decision-making, goal alignment, formation of an internal committee to direct change, in-person trainings and consultative support, peer-to-peer provider support and knowledge sharing, and organizational restructuring.

Articles as recent as 2012 call for more randomized controlled trials (RCTs) and rigorous quasi-experimental designs examining influences of organizational interventions on culture and climate in youth-service settings (e.g., child welfare: Glisson et al., 2006; mental health: Glisson et al., 2012; and healthcare: Parmelli et al., 2011). We identified seven group RCTs and five quasi-experimental studies, highlighting the increasing quality of evidence for the influence of organizational interventions on organizational culture and climate. We found a higher percentage of positive findings among studies with no comparison group, indicating the importance of study design on outcomes, and the continued need for more comparison and randomized designs to better understand impacts of organizational interventions compared to typical organizational fluctuations over time.

Future Directions of Organizational Interventions

In light of increasing evidence demonstrating the malleability of organizational culture and climate, the current review was designed to advance our understanding of which interventions, and their individual components, hold the greatest promise towards meeting different organizational needs. While the studies described herein provide a vital stepping stone for understanding the effects of organizational interventions on culture and climate, there are a number of methodological recommendations that we believe will further our understanding of best practices for improving organizational culture and climate. These recommendations are reflected in the following future directions.

Future Direction 1: Improve Replicability and Adoption with Clear and Detailed Reporting of Organizational Interventions.

Individual organizations or community consultants may be interested in adopting organizational interventions (or components), as seen in the abundant use of strategic planning across government and non-profit organizations (Bryson, 2010). In the spirit of moving science to service, and closing the research-to-practice gap, results should be available to – and intervention components comprehensively described for – youth-service settings that stand to benefit most from accumulating science. Collaboration with consultants and researchers from other disciplines, such as community and organizational psychology, may also speed dissemination of findings, and development of new organizational and community-informed consultative methods for promoting a positive and effective organizational culture and climate in community-serving organizations.

Future Direction 2: Assess Resources and Demands of Organizational Interventions

Organizational interventions require significant time, expertise, and resources; thus, more transparency in reporting is necessary around both human and material resources, and both fiscal and time costs, associated with effective implementation of intervention components. Research highlights the importance of balancing demands and resources (Demerouti et al., 2001). Findings revealed increases in role overload among frontline providers across multiple studies, reflecting that organizational interventions may be accompanied by increased job demands that, especially under conditions of limited resources, may result in too many responsibilities for providers to effectively manage. Changes in job demands may improve or interfere with providers’ own psychological health or effectiveness working with youth; thus, it is important to examine changes in both demands and resources that may accompany organizational intervention or workforce support to ensure that a healthy balance is maintained.

Future Direction 3: Measure Global and Specific Domains of Organizational Culture and Climate over Multiple Time Points

Factor analyses conducted in studies such as Taxman et al. (2014) support the presence of global dimensions of organizational culture, climate, and readiness for change that can elucidate the broader impacts of organizational interventions on providers’ workplace experiences. Global measures, however, are less informative regarding the unique benefits or barriers associated with different interventions and their discrete components. The mixed results related to specific dimensions or subdomains of organizational culture and climate highlight that interventions may have differential effects on different aspects of the organizational environment. An example of this can be seen in the minimal changes found across subdomains related to leadership. This finding points to the potential benefit of adding leadership training (e.g., Aarons, et al., 2015) for organizations struggling in these subdomains. Therefore, understanding more nuanced effects of organizational interventions on specific culture and climate dimensions may be an important step towards tailoring support to each organization’s individual needs.

The importance of measuring culture and climate across multiple timepoints is highlighted by the trends illustrating that improvements in culture and climate often followed an initial period of temporary decline. Variability in change trajectories, and differences by intervention components and outcomes, suggests value in going beyond the typical pre-post designs by examining multiple time points and long-term impacts of organizational interventions. For example, it is possible that the use of particular intervention components, such as the development of an internal committee to facilitate change, is associated with more sustained improvements in culture or climate over time compared to interventions that rely on external facilitators to direct or support change. Long-term follow-up studies may help to inform the potential for organizational interventions to continue influencing organizational social context even after implementation has ended.

Future Direction 4: Examine Moderators

Findings hint at opportunities to explore potential moderators that may mitigate or promote the influence of organizational interventions on culture and climate across studies and service settings. For instance, Bunger & Lengnick-Hall (2018) showed that team size impacts changes in communication networks after participating in a learning collaborative, with larger teams demonstrating greater increases in team-level communication. Other potential moderators to explore include level and consistency of funding, staff turnover, geographic area (i.e., rural vs urban vs. suburban), available resources such as space and equipment, and service design (e.g., individual providers versus multi-person teams).

Future Direction 5: Compare and Contrast Interventions and Intervention Components via Rigorous Study Designs

Approximately only one-third of studies compared an organizational intervention to a control group. Only two studies compared multiple interventions to one another. For example, Taxman and colleagues (2014) examined three groups: a social network group, a skill/knowledge building group, and a usual practice group consisting of management directives. They found that by 12-month follow-up, only the social network intervention improved organizational readiness over time; there were no differences in outcome between the knowledge building group and the usual practice condition. Relatedly, multiple studies utilized team-based role-plays during in-person trainings, though little is known regarding the differential influence of team-based versus individual role plays on organizational factors such as perceived coworker support and cooperation. Hence, separating and comparing intervention components may elucidate their unique, incremental, additive or interactive impacts for improving organizational culture and climate. Ultimately, adopting a small number of carefully selected components may be more feasible and effective for some organizations, while others may be resourced and prepared to implement a larger, more comprehensive intervention package. Increased comparisons between groups, particularly if groups can be randomly assigned, will also increase our knowledge and confidence in the causal impacts of organizational interventions on changes in culture and climate.

Future Direction 6: Distinguish Organizational Change from Organizational Support

Mindfulness, counseling, and social-emotional health interventions represent efforts to support providers rather than change organizational functioning or service delivery. This highlights an interesting distinction between organizational support – focused on meeting providers’ self-identified or self-care needs – and organizational change – focused on altering environment, operations, relationships, or job performance to improve service quality. While mindfulness and counseling provide clear examples of organizational support, other intervention components may influence both. For example, peer-to-peer support such as learning collaboratives, which often focus group discussions towards the implementation of a particular evidence-based practice, can also activate social networks and promote social support, knowledge-sharing, and problem-solving around a combination of patient-care and self-care needs. Future research may explore the extent to which organizational support interventions are more or less feasible and effective than organizational change interventions at promoting a positive and effective culture and climate.

Limitations

Due to the wide variability in definitions and measures of organizational culture and climate across researchers and service settings, our search terms may have missed some studies that would otherwise have met criteria. We excluded six articles due to language constraints or lack of access. As such, our results may not reflect all organizational intervention efforts measuring organizational culture and climate in youth-service settings. We took multiple steps however to help increase the comprehensiveness of our search, by utilizing an expansive set of search terms across multiple service settings, conducting the search in four databases, and not imposing limitations on country or year (studies ultimately ranged from 1992 to 2018). A review of organizational interventions for adult-service settings may point to a similar set of common organizational intervention components across settings.

Implications for Mental Health

Including service settings beyond mental health can be viewed as both a limitation and a strength. Examination across settings necessitated use of more general search terms, perhaps causing us to miss more nuanced intervention components associated with mental health settings alone. The fact that similar intervention components and results were revealed across settings, though, highlights an important opportunity for mental health settings to benefit from consolidated learning and reflection related to interventions applied in other service settings. More work is needed, however, to better understand the impacts of non-specialty interventions on patient mental health outcomes. Increased measurement of service and patient outcomes over time, such as patient satisfaction, skill development, and mental health metrics, combined with measurement of culture and climate over time, may advance our understanding of how changes in culture and climate correspond to changes in service and patient outcomes.

Conclusion

There is increasing evidence indicating the promise of organizational interventions towards promoting a positive and effective organizational culture and climate. Our review summarizes the use of organizational intervention components and measurement of organizational climate and culture outcomes across community youth-service settings. Most of the examined organizational interventions demonstrated mixed effects across measures, with fewer than half of the studies including a comparison group. In regard to future directions for organizational intervention research, we reflect on lessons from Gordon Paul (1967), directing our research questions towards what will bring us closer to identifying what combination of components, for which organizations, over what period of time, and under which conditions are most likely to influence and sustain improved care for youth and the organizations that serve them.

Supplementary Material

Supplementary Table 1
Supplementary Table 2

Footnotes

Compliance with Ethical Standards:

Ethical approval: All procedures performed involving human participants were in accordance with the ethical standards of the institutional and national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent: Not applicable. No participant data was collected for this study

Conflict of interest: All authors declare that he/she has no conflict of interest

Publisher's Disclaimer: This Author Accepted Manuscript is a PDF file of an unedited peer-reviewed manuscript that has been accepted for publication but has not been copyedited or corrected. The official version of record that is published in the journal is kept up to date and so may therefore differ from this version.

References

  1. Aarons GA, Ehrhart MG, Farahnak LR, & Hurlburt MS (2015). Leadership and organizational change for implementation (LOCI): A randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science, 10, 11. 10.1186/s13012-014-0192-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Aarons GA, Glisson C, Green PD, Hoagwood K, Kelleher KJ, Landsverk JA, & The Research Network on Youth Mental Health. (2012). The organizational social context of mental health services and clinician attitudes toward evidence-based practice: a United States national study. Implementation Science, 7, 56. 10.1186/1748-5908-7-56 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Anderson-Butcher D, Lawson HA, & Barkdull C. (2002). An evaluation of child welfare design teams in four states. Journal of Health & Social Policy, 15(3–4), 131–161. 10.1300/J045v15n03_10 [DOI] [PubMed] [Google Scholar]
  4. Anderson-Butcher D, Paluta L, Sterling K, & Anderson C. (2017). Ensuring healthy youth development through community schools: a case study. Children & Schools, 40(1), 7–16. 10.1093/cs/cdx026 [DOI] [Google Scholar]
  5. Baldschun A, Hamalainen J, Totto P, Rantonen O, & Salo P. (2019). Job-strain and well-being among Finnish social workers: exploring the differences in occupational well-being between child protection social workers and social workers without duties in child protection. EUROPEAN JOURNAL OF SOCIAL WORK, 22(1), 43–58. 10.1080/13691457.2017.1357025 [DOI] [Google Scholar]
  6. Barnett T, Hoang H, Cross M, & Bridgman H. (2015). Interprofessional practice and learning in a youth mental health service: A case study using network analysis. Journal of Interprofessional Care, 29(5), 512–514. 10.3109/13561820.2015.1004042 [DOI] [PubMed] [Google Scholar]
  7. Boris ET, de Leon E, Roeger KL, & Nikolova M. (2010). Human Service Nonprofits and Government Collaboration Findings from the 2010 National Survey of Nonprofit Government Contracting and Grants. Retrieved from Center on Nonprofits and Philanthropy at the Urban Institute: //www.councilofnonprofits.org/sites/default/files/documents/Full%20Report.pdf [Google Scholar]
  8. Bradshaw CP, Koth CW, Bevans KB, Ialongo N, & Leaf PJ (2008). The impact of school-wide positive behavioral interventions and supports (PBIS) on the organizational health of elementary schools. School Psychology Quarterly, 23(4), 462. 10.1037/a0012883 [DOI] [Google Scholar]
  9. Bradshaw CP, Koth CW, Thornton LA, & Leaf PJ (2009). Altering school climate through school-wide Positive Behavioral Interventions and Supports: findings from a group-randomized effectiveness trial. Prevention Science : The Official Journal of the Society for Prevention Research, 10(2), 100–115. 10.1007/s11121-008-0114-9 [DOI] [PubMed] [Google Scholar]
  10. Bryson JM 1. (1995). Strategic planning for public and nonprofit organizations: a guide to strengthening and sustaining organizational achievement. Rev. ed. San Francisco: Jossey-Bass Publishers. [Google Scholar]
  11. Bryson JM (2010). The future of public and nonprofit strategic planning in the United States. Public Administration Review, 70(s1), s255–s267. 10.1111/j.1540-6210.2010.02285.x [DOI] [Google Scholar]
  12. Bunger AC, & Lengnick-Hall R. (2018). Do learning collaboratives strengthen communication? A comparison of organizational team communication networks over time. Health Care Management Review, 43(1), 50–60. 10.1097/HMR.0000000000000120 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Chorpita BF, Becker KD, & Daleiden EL (2007). Understanding the common elements of evidence-based practice: misconceptions and clinical examples. Journal of the American Academy of Child and Adolescent Psychiatry, 46(5), 647–652. 10.1097/chi.0b013e318033ff71 [DOI] [PubMed] [Google Scholar]
  14. Chorpita BC, Daleiden E, & Weisz JR (2005). Identifying and selecting the common elements of evidence based interventions: a distillation and matching model. Mental Health Services Research, 7, 5–20. 10.1007/s11020-005-1962-6 [DOI] [PubMed] [Google Scholar]
  15. Colarossi L, Betancourt GS, Perez A, Weidl M, & Morales H. (2014). An organizational capacity-building program to enhance adolescent sexual and reproductive health. Health Promotion Practice, 15(4), 538–547. 10.1177/1524839913478420 [DOI] [PubMed] [Google Scholar]
  16. Cooke RA, & Szumal JL (1993). Measuring normative beliefs and shared behavioral expectations in organizations: The reliability and validity of the organizational culture inventory. Psychological Reports, 72(3_suppl), 1299–1330. 10.2466/pr0.1993.72.3c.1299 [DOI] [Google Scholar]
  17. Crawford ER, LePine JA, & Rich BL (2010). Linking job demands and resources to employee engagement and burnout: a theoretical extension and meta-analytic test. Journal of Applied Psychology, 95, 834–848. 10.1037/a0019364 [DOI] [PubMed] [Google Scholar]
  18. Cross AF (1992). Development and Implementation of a Program of Evaluation to Support Improvement of Organizational Components in a Child Care Center. Retrieved from Education Resources Information Center (ERIC) Archive. (Accession No. ERIC_ED343671) [Google Scholar]
  19. Cyphers G. (2001). APHSA Report from child welfare workforce survey: State and county data and findings. Washington DC: American Public Human Services Association. [Google Scholar]
  20. Damian AJ, Gallo J, Leaf P, & Mendelson T. (2017). Organizational and provider level factors in implementation of trauma-informed care after a city-wide training: an explanatory mixed methods assessment. BMC Health Services Research, 17(1), 750. 10.1186/s12913-017-2695-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Das JK, Salam RA, Lassi ZS, Khan MN, Mahmood W, Patel V, & Bhutta ZA (2016). Interventions for Adolescent Mental Health: An Overview of Systematic Reviews. The Journal of adolescent health : official publication of the Society for Adolescent Medicine, 59(4S), S49–S60. 10.1016/j.jadohealth.2016.06.020 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Demerouti E, Bakker AB, Nachreiner F, & Schaufeli WB (2001). The job demands resources model of burnout. Journal of Applied Psychology, 86, 499–512. 10.1037/0021-9010.86.3.499 [DOI] [PubMed] [Google Scholar]
  23. Denison DR (1996). What is the difference between organizational culture and organizational climate? A native’s point of view on a decade of paradigm wars. Academy of Management Review, 21, 619–654. 10.5465/AMR.1996.9702100310 [DOI] [Google Scholar]
  24. Embry DD, & Biglan A. (2008). Evidence-based kernels: fundamental units of behavioral influence. Clinical child and family psychology review, 11(3), 75–113. 10.1007/s10567-008-0036-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Elwyn LJ, Esaki N, & Smith CA (2015). Safety at a girls secure juvenile justice facility. Therapeutic Communities: The International Journal of Therapeutic Communities, 36(4), 209–218. 10.1108/TC-11-2014-0038 [DOI] [Google Scholar]
  26. Glisson C, Dukes D, & Green P. (2006). The effects of the ARC organizational intervention on caseworker turnover, climate, and culture in children’s service systems. Child Abuse & Neglect, 30(8), 854–855. 10.1016/j.chiabu.2005.12.010 [DOI] [PubMed] [Google Scholar]
  27. Glisson C, & Green P. (2006). The effects of organizational culture and climate on the access to mental health care in child welfare and juvenile justice systems. Administration and Policy in Mental Health and Mental Health Services Research, 33(4), 433. 10.1007/s10488-005-0016-0 [DOI] [PubMed] [Google Scholar]
  28. Glisson C, Hemmelgarn A, Green P, Dukes D, Atkinson S, & Williams NJ (2012). Randomized trial of the Availability, Responsiveness, and Continuity (ARC) organizational intervention with community-based mental health programs and clinicians serving youth. Journal of the American Academy of Child & Adolescent Psychiatry, 51(8), 780–787. 10.1016/j.jaac.2012.05.010 [DOI] [PubMed] [Google Scholar]
  29. Glisson C, Hemmelgarn A, Green P, Williams NJ (2013). Randomized trial of the availability, responsiveness and continuity (ARC) organizational intervention for improving youth outcomes in community mental health programs. Journal of the American Academy of Child and Adolescent Psychiatry, 52, 493–500. 10.1016/j.jaac.2013.02.005 [DOI] [PubMed] [Google Scholar]
  30. Glisson C, & James LR (2002). The cross-level effects of culture and climate in human service teams. Journal of Organizational Behavior, 23(6), 767–794. 10.1002/job.162 [DOI] [Google Scholar]
  31. Glisson C, Schoenwald SK, Kelleher K, Landsverk J, Hoagwood KE, Mayberg S, Green P, & The Research Network on Youth Mental Health. (2008). Therapist turnover and new program sustainability in mental health clinics as a function of organizational culture, climate, and service structure. Administration and Policy in Mental Health, 35, 124–133. 10.1007/s10488-007-0152-9 [DOI] [PubMed] [Google Scholar]
  32. Glisson C, Williams NJ, Hemmelgarn A, Proctor E, & Green P. (2016). Increasing clinicians’ EBT exploration and preparation behavior in youth mental health services by changing organizational culture with ARC. Behaviour Research and Therapy, 76, 40–46. 10.1016/j.brat.2015.11.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Green BL, Malsch AM, Kothari BH, Busse J, & Brennan E. (2012). An intervention to increase early childhood staff capacity for promoting children’s social-emotional development in preschool settings. Early Childhood Education Journal, 40(2), 123–132. 10.1007/s10643-011-0497-2 [DOI] [Google Scholar]
  34. Henry LS, Christine Hansson M, Haughton VC, Waite AL, Bowers M, Siegrist V, & Thompson EJ (2017). Application of Kotter’s Theory of Change to achieve baby-friendly designation. Nursing for Women’s Health, 21(5), 372–382. 10.1016/j.nwh.2017.07.007 [DOI] [PubMed] [Google Scholar]
  35. Hickey LM (1994). Building Staff Morale in a Season of Change through Supportive, Growth-Oriented Supervision. Retrieved from Education Resources Information Center (ERIC) Archive. (Accession No. ERIC_ ED380725) [Google Scholar]
  36. Hussein S. (2018). Work engagement, burnout and personal accomplishments among social workers: A comparison between those working in children and adults’ services in england. Administration and Policy in Mental Health and Mental Health Services Research. 10.1007/s10488-018-0872-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Hoy WK, Barnes K, & Sabo D. (1995). The organizational health of middle schools: the concept and its measure. Unpublished manuscript. [Google Scholar]
  38. Hoy WK, & Feldman JA (1987). Organizational health: the concept and its measure. Journal of Research and Development in Education, 20, 30–38. [Google Scholar]
  39. James LA, & James LR (1989). Integrating work environment perceptions: explorations into the measurement of meaning. Journal of Applied Psychology, 74, 739–751. 10.1037/0021-9010.74.5.739 [DOI] [Google Scholar]
  40. Jones AP, & James LR (1979). Psychological climate: Dimensions and relationships of individual and aggregated work environment perceptions. Organizational Behavior and Human Performance, 23, 201–250. 10.1016/0030-5073(79)90056-4 [DOI] [Google Scholar]
  41. Joosten TCM, Bongers IMB, & Janssen RTJM (2014). Redesigning mental healthcare delivery: is there an effect on organizational climate? International Journal for Quality in Health Care : Journal of the International Society for Quality in Health Care, 26(1), 58–63. 10.1093/intqhc/mzt082 [DOI] [PubMed] [Google Scholar]
  42. Joyce WF, & Slocum JW (1984). Collective climate: Agreement as a basis for defining aggregate climates in organizations. Academy of Management Journal, 24, 721–742. 10.2307/255875 [DOI] [Google Scholar]
  43. Kayiwa J, Clarke K, Knight L, Allen E, Walakira E, Namy S, … Devries K. (2017). Effect of the good school toolkit on school staff mental health, sense of job satisfaction and perceptions of school climate: Secondary analysis of a cluster randomised trial. Preventive Medicine: An International Journal Devoted to Practice and Theory, 101, 84–90. 10.1016/j.ypmed.2017.05.022 [DOI] [PubMed] [Google Scholar]
  44. Kennedy JL, Jones SM, Porter N, White ML, Gephardt G, Hill T, … Thompson TM (2013). High-fidelity hybrid simulation of allergic emergencies demonstrates improved preparedness for office emergencies in pediatric allergy clinics. Journal of Allergy and Clinical Immunology- In Practice, 1(6), 608–U111. 10.1016/j.jaip.2013.07.006 [DOI] [PubMed] [Google Scholar]
  45. Kim H, Tracy EM, Biegel DE, Min MO, & Munson MR (2015). The effects of organizational culture on mental health service engagement of transition age youth. The Journal of Behavioral Health Services & Research, 42(4), 466–485. 10.1007/s11414-014-9406-y [DOI] [PubMed] [Google Scholar]
  46. King MA, Wissow LS, & Baum RA (2018). The role of organizational context in the implementation of a statewide initiative to integrate mental health services into pediatric primary care. Health Care Management Review, 43(3), 206–217. 10.1097/HMR.0000000000000169 [DOI] [PubMed] [Google Scholar]
  47. Lam MT, & McDiarmid M. (2016). Increasing number of databases searched in systematic reviews and meta-analyses between 1994 and 2014. Journal of the Medical Library Association, 104(4), 284. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Lawrence CK, Zeitlin WS, Auerbach C, & Claiborne N. (2015). Climate change in private child welfare organizations. Human Service Organizations Management Leadership & Governance, 39(4), 290–305. 10.1080/23303131.2015.1045108 [DOI] [Google Scholar]
  49. Lawrence C, Claiborne N, Zeitlin W, & Auerbach C. (2016). Finish what you start: A study of Design Team change initiatives’ impact on agency climate. Children and Youth Services Review, 63, 40–46. 10.1016/j.childyouth.2016.02.009 [DOI] [Google Scholar]
  50. Olin SS, Williams N, Pollock M, Armusewicz K, Kutash K, Glisson C, & Hoagwood KE (2014). Quality indicators for family support services and their relationship to organizational social context. Administration and Policy in Mental Health and Mental Health Services Research, 41, 43–54. 10.1007/s10488-013-0499-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Orthner DK, Cook P, Sabah Y, & Rosenfeld J. (2006). Organizational learning: A cross-national pilot-test of effectiveness in children’s services. Evaluation and Program Planning, 29(1), 70–78. 10.1016/j.evalprogplan.2005.08.002 [DOI] [Google Scholar]
  52. Parmelli E, Flodgren G, Beyer F, Baillie N, Schaafsma ME, & Eccles MP (2011). The effectiveness of strategies to change organisational culture to improve healthcare performance: a systematic review. Implementation Science, 6, 33. 10.1186/1748-5908-6-33 [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Paul GL (1967). Strategy of outcome research in psychotherapy. Journal of Consulting Psychology. US: American Psychological Association. 10.1037/h0024436 [DOI] [PubMed] [Google Scholar]
  54. Pérez-Escoda N, Guiu GF, Benet AS, & Fondevila(2013). Evaluación de un programa de educación emocional para profesorado de primaria. Facultad de Educación, 16(1). 10.5944/educxx1.16.1.725 [DOI] [Google Scholar]
  55. Peterson MF, & Fischer R. (2004). Organizational culture and climate. In Spielberger CD (Ed.), Encyclopedia of applied psychology. Oxford, UK: Elsevier Science & Technology. [Google Scholar]
  56. Potter CC, Comstock A, Brittain C, & Hanna M. (2009). Intervening in multiple states: Findings from the Western Regional Recruitment Project. Child Welfare: Journal of Policy, Practice, and Program, 88(5), 169–185. [PubMed] [Google Scholar]
  57. Renner LM, Porter RL, & Preister S. (2009). Improving the retention of child welfare workers by strengthening skills and increasing support for supervisors. Child Welfare, 88(5, SI), 109–127. [PubMed] [Google Scholar]
  58. Rhodes JE, Camic PM, Milburn M, & Lowe SR (2009). Improving middle school climate through teacher-centered change. Journal of Community Psychology, 37(6), 711–724. 10.1002/jcop.20326 [DOI] [Google Scholar]
  59. Schein EH (2000). Sense and nonsense about culture and climate. In Ashkanasy NM, Wilderom CPM, & Peterson MF (Eds.), Handbook of organizational culture and climate (pp. xxiii–xxx). Thousand Oaks, CA: Sage Publications, Inc. [Google Scholar]
  60. Schneider B, Brief AP, & Guzzo RA (1996). Creating a climate and culture for sustainable organizational change. Organizational Dynamics, 24(4), 7–19. 10.1016/S0090-2616(96)90010-8 [DOI] [Google Scholar]
  61. Schneider B. (2000). The psychological life of organizations. In Ashkanasy NM, Wilderom CPM, & Peterson MF (Eds.), Handbook of organizational culture and climate (pp. xvii–xxi). Thousand Oaks, CA: Sage Publications, Inc. [Google Scholar]
  62. Sheth S, McCarthy E, Kipps AK, Wood M, Roth SJ, Sharek PJ, & Shin AY (2016). Changes in efficiency and safety culture after integration of an I-PASS-supported handoff process. Pediatrics, 137(2), e20150166. 10.1542/peds.2015-0166 [DOI] [PubMed] [Google Scholar]
  63. Shoushtarian M, Barnett M, McMahon F, & Ferris J. (2014). Impact of introducing practical obstetric multi-professional training (PROMPT) into maternity units in Victoria, Australia. BJOG : An International Journal of Obstetrics and Gynaecology, 121(13), 1710–1718. 10.1111/1471-0528.12767 [DOI] [PubMed] [Google Scholar]
  64. Slater PJ, Edwards RM, & Badat AA (2018). Evaluation of a staff well-being program in a pediatric oncology, hematology, and palliative care services group. Journal of Healthcare Leadership, 10, 67–85. 10.2147/JHL.S176848 [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Sorensen JB (2002). The strength of corporate culture and the reliability of firm performance. Administrative Science Quarterly, 47, 70–91. 10.2307/3094891 [DOI] [Google Scholar]
  66. Sottimano I, Guidetti G, Converso D, & Viotti S. (2018). We cannot be “forever young,” but our children are: a multilevel intervention to sustain nursery school teachers’ resources and well-being during their long work life cycle. PloS One, 13(11), e0206627. 10.1371/journal.pone.0206627 [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Taxman FS, Henderson C, Young D, & Farrell J. (2014). The impact of training interventions on organizational readiness to support innovations in juvenile justice offices. Administration and Policy in Mental Health and Mental Health Services Research, 41(2), 177–188. 10.1007/s10488-012-0445-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. Twombly EC (2005). Nonprofit Child and Youth Service Providers Showed Signs of Fiscal Stress before 9/11 (Policy brief No. 1). Retrieved from Urban Institute: //www.urban.org/sites/default/files/publication/51496/311129-Nonprofit-Child-and-Youth-Service-Providers-Showed-Signs-of-Fiscal-Stress-before--.PDF
  69. Weiner BJ (2009). A theory of organizational readiness for change. Implementation Science, 9, 1–9. 10.1186/1748-5908-4-67 [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Williams NJ, Frank HE, Frederick L, Beidas RS, Mandell DS, Aarons GA, … Locke J. (2019). Organizational culture and climate profiles: relationships with fidelity to three evidence-based practices for autism in elementary schools. IMPLEMENTATION SCIENCE, 14. 10.1186/s13012-019-0863-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  71. Zaza S, Wright-De Agüero LK, Briss PA, Truman BI, Hopkins DP, Hennessy MH, & Pappaioanou M. (2000). Data collection instrument and procedure for systematic reviews in the guide to community preventive services. Task force on community preventive services. American Journal of Preventive Medicine, 18(1), 44–74. 10.1016/S0749-3797(99)00122-1 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Table 1
NIHMS1585804-supplement-10488_2020_1037_MOESM2_ESM.docx (15.2KB, docx)
Supplementary Table 2
NIHMS1585804-supplement-10488_2020_1037_MOESM1_ESM.xlsx (27.9KB, xlsx)

RESOURCES