Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Dec 4.
Published in final edited form as: J Prev Interv Community. 2014;42(4):264–281. doi: 10.1080/10852352.2014.943639

Targeting Classrooms' Emotional Climate and Preschoolers' Socioemotional Adjustment: Implementation of the Chicago School Readiness Project

Christine P Li-Grining 1, C Cybele Raver 2, Darlene Jones-Lewis 3, Sybil Madison-Boyd 4, Jaclyn Lennon 5
PMCID: PMC4670024  NIHMSID: NIHMS633809  PMID: 25321641

Abstract

Children living in low-income families are more likely to experience less self-regulation, greater behavior problems, and lower academic achievement than higher income children. To help prevent children's later socioemotional and academic difficulties, the Chicago School Readiness Project (CSRP) team implemented a clustered, randomized controlled trial (RCT) in early childhood programs with Head Start funding. Head Start sites were randomly assigned to receive CSRP services, which were offered as part of a multi-component, classroom-based mental health intervention. Here, we provide an overview of the CSRP model, its components, and a descriptive portrait of its implementation. In so doing, we address various aspects of the implementation of three of its components: 1) the training of teachers, 2) MHCs' coaching of teachers, and 3) teachers' behavior management of children. We conclude with a discussion of factors potentially related to the implementation of CSRP and directions for future research.

Keywords: implementation, intervention, preschool, school readiness

During the preschool years, young children undergo important developmental shifts in the socioemotional domain, such as improving self-regulatory strategies. Self-regulation processes involve the voluntary, adaptive control of one's behavior, emotions, and attention (Eisenberg & Spinrad, 2004). Improvement in such abilities helps lay the groundwork for lower levels of children's behavior problems and greater levels of academic achievement, which are central to their school readiness (Blair & Razza, 2007; Lengua, Honorado, & Bush, 2007). As such, it is important to focus early intervention efforts on children in low-income families, who tend to face multiple ecological stressors, and are thus at greater risk for displaying lower self-regulation and greater behavior problems (Evans & Rosenbaum, 2008; Gardner et al., 2009; McGilloway et al., 2012; Sektnan, McClelland, Acock, & Morrison, 2010).

The Chicago School Readiness Project (CSRP) aimed to enhance low-income children's school readiness, using a clustered, randomized controlled trial (RCT) design. Previous research has documented positive impacts of CSRP on classrooms' emotional climate (Raver et al., 2008). Such improvements were in turn expected to help support the development of children's self-regulatory competence. Past non-experimental studies have detected associations between more sensitivity, warmth, and limit setting and preschoolers' greater self-regulation (Bernier et al., 2010; Lengua, Honorado, & Bush, 2007). Positive interactions such as these serve as a source of external regulation for children (Bernier et al., 2010). In so doing, emotional support helps foster preschoolers' self-regulation. Indeed, past CSRP studies have found treatment effects on children's better self-regulation, as well as lower behavior problems and higher academic skills (Raver et al., 2009; 2011). To deepen our understanding of the conditions under which these intervention impacts occurred, the current paper provides a descriptive portrait of CSRP's implementation, which refers to the ways in which CSRP was carried out in the field (Durlak, 2010).

Multiple indicators of implementation exist, including fidelity, dosage, and quality (Durlak & Dupre, 1998; Fixsen et al., 2005; Greenhalgh et al., 2005; Perry et al., 2010; Stith et al., 2006). Fidelity refers to the extent to which an intervention program adhered to its protocol. Dosage involves the amount of intervention services that participants received, and quality includes how well intervention services met the needs of participants. Implementation research on early childhood intervention programs has begun to examine such factors (Domitrovich et al,, 2010; Hamre et al., 2010; Knoche et al., 2010; Odom et al., 2010). Yet, implementation of such programs has not been studied extensively (Durlak, 2010). In terms of CSRP, prior implementation studies have investigated links between dosage and teachers' psychosocial stressors (Li-Grining et al., 2010), and examined the degree to which intervention impacts on children's adjustment depended on dosage levels (Zhai et al., 2010). Here, we provide a more comprehensive descriptive portrait of the implementation of CSRP, by addressing not only dosage, but also fidelity and quality. Furthermore, given the importance of implementation to intervention outcomes, we present data on teachers' reports of factors that may be related to the implementation of CSRP (Durlak & Dupre, 2008).

To set the stage, we begin by outlining CSRP's conceptual framework and components of the CSRP model. Next, we describe various aspects of the implementation of multiple components of the model. We then conclude by discussing this descriptive portrait, with a focus on possible factors related to the implementation of CSRP and directions for future research on replications and expansions of CSRP.

Overview of the CSRP Model

Overarching Conceptual Framework

The CSRP intervention model was developed on a clear theoretical foundation emphasizing children's emotional and behavioral self-regulation. Following Izard (2002), Raver (2004), and Aber and colleagues (1998), our perspective is that children face specific stage-salient challenges that place children's regulation of attention, emotions, and impulsive behavior at “center stage” in preschool. Following Arnold, McWilliams, and Arnold (1998), our model is grounded on the hypothesis that both adults and children become increasingly dysregulated during times of conflict and child disruptiveness. Because of this, both caregiving adults (e.g, teachers) and children were targets of the CSRP intervention. Thus, the CSRP model involved a multi-tiered system, where: 1) children were served by teachers and mental health consultants (MHCs), 2) teachers were coached and trained by licensed, clinical social workers, and 3) MHCs were supervised by senior staff members who included clinical and developmental psychologists.

Using this system, the CSRP team targeted the improvement of children's self-regulation in 3 ways, with MHCs making efforts 1) to promote teachers' provision of emotional support, 2) to decrease teachers' stressors, and 3) to enhance children's self-regulation directly. First, CSRP aimed to bolster the emotional support offered to children by teachers, as a way to foster children's self-regulation (Bernier et al., 2010; Lengua, Honorado, & Bush, 2007). In particular, preschoolers may be more responsive to behavioral strategies that involve guidance and support provided by adults (Izard, 2002), and teachers with relatively little time and a large number of educational demands may prefer training that is concrete and behaviorally-focused. As such, the CSRP model emphasized the use of emotionally positive, proactive behavioral strategies that promote children's development of positive relationships and more effective self-control. (Webster-Stratton, Reid & Hammond, 2004; for more details, see below).

A second objective of CSRP MHCs was to lower teachers' stress, which was expected to facilitate teachers' provision of emotional support to children, and in turn, enhance children's self-regulation (Raver, Blair, & Li-Grining, 2012). Teachers working in early childhood education settings in low-income communities may encounter poverty-related stressors themselves, which may jeopardize the quality of emotional climate in their classrooms (Li-Grining et al., 2010). However, extensive research has not been conducted on early childhood teachers' stressors in the context of randomized early childhood intervention programs (Li-Grining et al., 2010; Raver et al., 2012; Zhai et al., 2011).

Third, mental health consultation models in early childhood settings may involve both indirect services that target the enhancement of teachers' practices and well-being, as well as direct services that serve children on an individualized basis (Perry et al., 2010). Preschoolers' self-regulatory competence may not have been enhanced via CSRP's targeting of improvements in classroom quality and reductions in teachers' stressors alone. Children with relatively high levels of dysregulation may benefit more from direct, individualized services offered by MHCs.

Components of the CSRP Model

The CSRP team strived to meet these three objectives through four specific components. These parts included: 1) the training of teachers in classroom behavior management, 2) MHCs' coaching of teachers, 3) MHCs' leading of a stress reduction workshop for teachers, and 4) MHCs' offering of direct services to children. When classrooms received higher dosage levels of mental health consultation, measured in terms of training hours, the hours MHCs devoted to classroom visits, and the provision of individualized child-focused services, the CSRP intervention yielded greater impacts on children's school readiness (Zhai et al., 2010). These findings are congruent with results from other early childhood intervention studies that have detected linkages between mental health consultation and children's lower behavior problems (Perry et al., 2010). Below, we briefly describe these aspects of the CSRP model, which was carried out across the academic year, in the Fall, Winter, and Spring.

Training of Teachers

Successful implementation relies in part on the targeted training of participants (Greenhalgh et al., 2005; Stith et al., 2006). To help improve classroom quality, teachers participated in behavior management training. The training was adapted from the Incredible Years Teacher Training Program, which is an evidence-based intervention that targets the improvement of young children's socioemotional adjustment (Webster-Stratton et al., 2004, 2008). Applying behavioral principles, the Incredible Years Teacher Training Program aims to reduce children's disruptive behaviors via the promotion of positive relationships between teachers and children, teachers' motivation of children's positive behavior through praise, teachers' problem-solving with children, and teachers' implementation of strategies to lower children's acting out behavior.

As part of CSRP, teachers were offered compensation to attend 5 training sessions, which were held on Saturdays from October through January (for a total of 30 training hours). An individual who was a licensed clinical social worker and experienced trainer led the training sessions. The training covered 5 specific types of behavior management strategies: 1) promotion of positive behavior (e.g., giving praise and encouragement), 2) management of misbehavior (e.g., ignoring attention-seeking behavior, giving warnings), 3) redirection and setting of limits (e.g., gentle reminders such as turning the lights on and off to get children's attention), 4) development of positive relationship between teachers and children, and 5) engagement in problem-solving with children (e.g., encouraging children to use words not hands; Li-Grining et al., 2010).

Coaching of teachers

Teacher training alone may not ensure that teachers transfer what they learn in training sessions to what they do in classrooms, given the everyday challenges of running a classroom smoothly (e.g., Gorman-Smith, 2003; Gross et al., 2003; Jones, Brown, Aber, & Thomas, 2006). Moreover, both young children and providers in early childhood programs in low-income neighborhoods tend to face a high number of ecological risks (Li-Grining et al., 2010), and preschoolers appear to be substantially underserved by community mental health services (Warner and Pottick, 2006). Given these risks and this level of unmet need, it may be challenging for teachers to address the multiple, cumulative disadvantages and stressors that many low-income, preschool-aged children encounter, without support and consultation. Furthermore, both initial training and on-going coaching have consistently been identified as predictors of better implementation (Durlak & Dupre, 2008).

Thus, in line with other early childhood intervention programs (Baker et al., 2010; Domitrovich et al., 2010; Odom et al.., 2010), the CSRP model included both training and coaching. Specifically, the CSRP teacher training model was combined with a “mental health consultation” model, which involved clinically trained consultants' provision of direct services to teachers. The CSRP MHCs had Master's degrees in social work, and were licensed clinical social workers. Hired by CSRP, MHCs responsibilities included coaching teachers throughout the school year and attending the behavior management training sessions with teachers.

The MHCs sought to provide direct services to teachers via weekly classroom visits, starting in the Fall and through the Spring (Donahue et al., 2000). During the Fall, the MHCs set out to follow a set of manualized coaching steps (Madison-Boyd et al., 2006), which included sharing and discussing feedback with teachers regarding their ability to foster children's behavioral and emotional development. Specifically, MHCs supported the ongoing teacher training by helping teachers implement the behavior management strategies in the classroom.

Stress reduction workshop for teachers

In addition, MHCs offered direct services to teachers in the following ways. During the Winter, MHCs at each site led a one-day stress reduction workshop for teachers, where they discussed strategies to relieve stress (e.g., co-teachers allowing each other time to take breaks). Also, MHCs began devoting time during classroom visits to discuss whether teachers were experiencing stress and ways to alleviate stress. Though this has not been investigated, the provision of these particular services may have played a role in the impact of the CSRP intervention by lowering teachers' stress levels (Zhai et al., 2010, 2011).

Mental health consultants' direct services for children

Though stress reduction, coaching, and teacher training may help facilitate teachers' efforts to promote most children's socioemotional development, these approaches may fall short of reducing children's serious behavior problems. Some children may be demonstrating emotional and behavioral problems that teachers cannot reasonably be expected to solve, given their training and other job demands. Rather, child-focused services, delivered by MHCs in classroom settings and outpatient clinic settings, may be required (Donahue et al., 2000). Therefore, in the Spring, MHCs began to focus less on coaching teachers and reducing stress among teachers, and started to center more on providing direct intervention services to children.

Mental health consultants sought to identify about 3 to 4 children per classroom for these services. In total, 137 children received direct services from MHCs (Zhai et al., 2010). These children were chosen by MHCs based on their clinical judgment, consultation with teachers, and teacher-rated behavior problems in the Fall. Mental health consultants worked with parents, teachers, and social workers at each site to formulate plans for each of these children. As part of these plans, MHCs conducted individual and group therapies to help improve socioemotional adjustment among this subgroup of children.

Control classrooms

Finally, in the context of a randomized controlled trial design, it is important to describe and to monitor comparison conditions (Durlak & Dupre, 2008). During the intervention year, teaching assistants (TAs), who held Associate degrees, visited control classrooms in order to maintain similar teacher-child ratios across treatment and control classrooms. Teacher's aides were scheduled to visit each of the control classrooms for the same amount of time as MHCs visited treatment classrooms on a weekly basis. During their time in the classroom, TAs were instructed to help teachers carry out plans that teachers developed solely on their own for a given day. Notably, during the year after the intervention, teachers in control classrooms were invited to attend the same teacher training sessions in order to equalize opportunity across teachers in the treatment and control groups.

Descriptive Data on the Implementation of CSRP

Participants

At the beginning of the preschool year, 543 children participated in the CSRP intervention program, and by the end of the academic year, 34 children left their Head Start program (Raver et al., 2011). Children were about 4-years old, and half of them were girls. Two-thirds of children were African American, over a quarter were Latino, and nearly one-tenth belonged to another race/ethnic group.

Participants also included 90 teachers (Li-Grining et al., 2010; Zhai et al, 2011). In the treatment group, there were 9 sites, 18 intervention classrooms, and 48 teachers. In the control group, there were 9 control sites, 17 control classrooms, and 42 teachers. Teachers averaged approximately 40-years-old, and nearly all of them were female. Over two-thirds of teachers were African American, one-fifth were Latina, and one-tenth were European American.

The CSRP intervention team included 1 licensed clinical social worker who led the teacher training, and 3 MHCs who were also licensed clinical social workers and acted as coaches to the teachers. The MHCs' supervisors included a Ph.D. level clinical psychologist, Ph.D. level developmental psychologist, and a senior level licensed clinical social worker with a Master's degree. Next, we turn to a descriptive portrait of the ways in which CSRP intervention services were carried out, with a focus on three aspects of the model: 1) the training of teachers, 2) MHCs' coaching of teachers, and 3) teachers' behavior management of children.

Teacher Training

Measures of the implementation of the teacher training sessions focused on fidelity, dosage, and quality. First, teacher reports on the degree to which various training activities were completed at each session provided a global measure of the fidelity of the training sessions. Teachers were asked 7 questions (e.g., regarding topics intended to be discussed at the trainings), based on a 1 (didn't come up) to 5 (completed) metric. A mean across the 7 items was calculated for each teacher, and an average rating across teachers was computed for each session. Ratings were relatively high, with the averages ratings being 4.46, 3.66, 4.34, 4.47, and 4.33 for sessions 1, 2, 3, 4, and 5, respectively. This suggests that most training activities planned for each session were completed.

Second, the dosage of teacher training sessions was captured in terms of the number of training sessions teachers attended. The mean number of training sessions attended by teachers was 3.08 (Li-Grining et al., 2010). This suggests that teachers attended most of the 5 training sessions, on average. Furthermore, 63% of teachers attended at least 3 of the training sessions, which is the same or more than what has occurred in other intervention programs (Raver et al., 2008).

Third, teachers were asked to rate each training session, which serves as a proxy for a measure of the quality of the training. Teachers answered 6 items (e.g., how much they liked the session overall), using the following scale: 1 (not at all), 2 (a little bit), 3 (somewhat), 4 (quite a bit), and 5 (very much). For each teacher, the items were averaged to create an overall rating. A set of ratings was calculated for each of the five sessions, and a mean rating was calculated across teachers for each session. The average teacher rating for each session was high, with the means being 4.75, 4.64, 4.75, 4.88, and 4.49 for sessions 1, 2, 3, 4, and 5, respectively.

Coaching

Data on the implementation of MHCs' coaching of teachers addresses dosage and quality. First, the dosage of coaching was measured in terms of the number of hours MHCs devoted to classroom visits. The average number of hours MHCs spent in classrooms per week was 4.54 (Raver et al., 2008), and totaled 128.31 across the school year (Li-Grining et al., 2010). This level of dosage is much higher than that of other early childhood intervention programs involving mental health consultation (Perry, Allen, Brennan, & Bradley, 2010). In addition, teacher assistants spent an average of 5.18 hours in control classrooms each week (Raver et al., 2008).

Second, teachers completed a survey with questions that serve as proxies for the quality of the coaching. Following work by Hengeller on Multi-Systemic Treatment (MST), the program “clients” (i.e., teachers) were given the opportunity, at the end of the intervention, to rate their satisfaction with the program and with the kinds of support that MHCs provided (Henggeler, Schoenwald, Pickrel, Rowland, & Santos, 1994). The questionnaire included the following types of items: 1) MHCs' help with teachers' implementation of behavior management strategies, 2) MHCs' assistance with lowering teachers' stress and ability to deal with personal issues at home, and 3) MHCs' help with teachers' work with children. These questions were based on ratings from 1 (not helpful at all) to 4 (very helpful). In terms of help with behavior management strategies, teachers completed 9 items (e.g., using specific praise to encourage positive behavior), and means were created for each teacher. Across teachers, the average rating was 3.67. When asked a question about MHCs' assistance with their management of stress the mean score was 3.67, and when asked a question about MHCs' help with personal issues at home, the mean rating was 3.32. Regarding help with children, teachers answered 4 questions about MHCs' assistance (e.g., working with specific children who have behavioral difficulty). A mean score was calculated for each teacher, and the average rating across teachers was 3.59. In sum, teachers reported that MHCs were somewhat to very helpful across these three kinds of assistance.

The questionnaire also asked teachers to report on MHCs' help with teachers' professional development and their relationships with co-workers. Teachers noted that MHCs were somewhat to very helpful in terms of teachers' professional development and work with other staff members. Using the same 1 to 4 metric, teachers gave MHCs an average rating of 3.49 when asked a question regarding assistance with professional development, and a mean score of 3.49 when asked a question regarding assistance with work with other staff members.

In addition, teachers were asked two general questions: 1) overall, has the MHC helped you, and 2) overall, has the MHC helped an individual child. These questions were answered on a scale where 0 = not been helpful, 1 = a little helpful, 2 = somewhat helpful, and 3 = very helpful. In response to these two questions, MHCs had mean score of 2.82 and 2.67, respectively. Put differently, teachers tended to view MHCs as somewhat to very helpful in terms of general assistance to teachers and children. Lastly, the survey asked teachers to indicate whether MHCs were easy or difficult to work with, on a metric of 0 (very difficult) to 4 (very easy). Mental health consultants received a mean rating of 3.73, meaning that teachers found working with MHCs to be somewhat to very easy.

Child Behavior Management by Teachers

We now shift from how intervention services were directly provided to teachers in the form of coaching, and turn toward measures of the dosage and quality of teachers' implementation of behavior management strategies in the classroom. In terms of dosage, past CSRP implementation research has noted the number of visits during which various behavior management strategies were used (Li-Grining et al., 2010). Across treatment sites, MHCs visited classrooms 28.50 times on average during the school year. The most commonly implemented strategies during these visits were expressing praise to motivate children's positive behavior and redirecting, consistent limit setting, and giving clear commands. These strategies were used during a mean number of 11.33 visits and 10.78 visits, respectively.

After each weekly classroom visit, MHCs completed a service provision form, which included MHCs' general ratings of the quality of teachers' implementation of behavior management strategies in the classroom. The MHCs rated teachers' success at using the behavior management techniques on a metric of 0 (made things worse), 1 (no impact, but tried), 2 (a little successful), 3 (somewhat successful), and 4 (very successful). Across visits, classrooms on averaged received a score of 3.20, which meant that teachers were viewed as somewhat to very successful at using the behavior management techniques presented in the training sessions. Given the importance of using multiple sources of data in implementation research (Durlak, 2010), it is notable that in prior CSRP research (Raver et al., 2008), independent, trained observers rated classroom behavior management as having a mean of 4.65 on a scale of 1 (inadequate) to 7 (excellent; LaParo, Pianta, & Stuhlman, 2004). This average falls in the medium range of quality on this measure.

Discussion

Based on the implementation measures that were collected, it appears that the CSRP model was generally carried out as planned, across numerous aspects of the intervention program and several dimensions of implementation. Measurement of implementation is not only important for understanding why treatment impacts occurred, but treatment effects may also vary as a function of implementation factors (Durlak & Dupre, 2008; Fixsen et al., 2005; Greenhalgh et al., 2005; Stith et al., 2006). Indeed, the CSRP team detected greater intervention impacts on children's self-regulation, math, and literacy skills with higher levels of dosage, measured in terms of teacher training attendance, classroom visits, and individualized mental health consultation services (Zhai et al., 2010). Furthermore, CSRP intervention effects may depend not only on dosage, but on fidelity and quality as well. Also, intervention outcomes may be predicted by various aspects of implementation (Durlak, 2010), which has been found in implementation research on other early childhood interventions (Domitrovich et al., 2010; Hamre et al., 2010; Odom et al., 2010).

Possible Factors Related to Implementation of CSRP

Doses of training, coaching, and behavior management strategies

In terms of dosage, a majority of teachers attended most of the training sessions, and the amount of mental health consultation provided across the school year was quite high. At the outset of the intervention, we were concerned that teachers experienced stressors like the children and families they served because teachers were likely to live in the same low-income communities. Moreover, we were concerned that teachers' experience with elevated stress might be an obstacle to our intervention efforts, with highly stressed teachers attending fewer training sessions and implementing fewer behavior management techniques. However, teachers with varying levels of stress were, on the whole, equally likely to attend the trainings and to implement the behavior management strategies introduced in the trainings (Li-Grining et al., 2010).

Still, these levels of dosage may be attributed in part to the presence of regular communication between and within the CSRP intervention program and sites (Durlak & Dupre, 2008; Fixsen et al., 2005; Greenhalgh et al., 2005; Knoche, Sheridan, Edwards, & Osborn, 2010). Preschool teachers' participation in other intervention programs for low-income children have been linked to teachers' perceptions of their workplaces as efficient, fair, collegial, and supportive (Baker, Kupersmidt, Voegler-Lee, Arnold, & Willoughby, 2010). Though we cannot test whether or not this is the case, it may be that the frequency and duration of coaching facilitated the development of long-term, communicative, and trusting relationships among intervention staff and sites' teachers and administrators. In terms of the relationship between CSRP and the sites, the CSRP program coordinator persuaded teachers to participate in the trainings, confirmed teachers' intentions to attend each training session, coordinated child care for teachers' children during the training sessions, and ensured that teachers received timely reimbursement for training attendance. Future research should collect longitudinal data on such communication and on the quality of relationships among intervention staff, teachers, and site directors (e.g., to what degree is their open communication mutual respect). Such information may help us understand the benefits of providing such large doses of mental health consultation and how the closeness of such relationships may evolve over time. Moreover, such relationships may help enhance other dimensions of the implementation process (Greenhalgh et al., 2005).

Furthermore, factors that may affect the implementation process may be related to the organization of the intervention delivery system (Durlak & Dupre, 2008). For instance, as a way of monitoring their classroom visits, teacher assistants were required to sign in at the beginning of each visit and to sign out at the conclusion of each visit. The lead teacher confirmed each visit with her signature and then faxed the log to the program coordinator at the end of each week. Although this was not tested, having to complete the sign in sheet may have helped to increase levels of dosage. Indeed, the weekly number of hours that MHCs spent in treatment classrooms and that teacher assistants spent in control classrooms were similar. Future CSRP-related research might compare different modes of monitoring teachers' aides in control classrooms, in order to determine what types of monitoring are more efficient and effective at maintaining comparison conditions.

Quality of training and coaching

On average, teachers reported that they liked the training sessions “quite a bit” to “very much,” and MHCs were rated by teachers as somewhat to very helpful on numerous dimensions, and working with MHCs was viewed by teachers as somewhat to very easy. Given that having clear roles and responsibilities tends to be related to implementation processes (Durlak & Dupre, 2008), use of the CSRP Mental Health Consultation Manual (Madison-Boyd et al., 2006) may have helped to increase the possibility that the CSRP model was carried out according to plan and at a higher level of quality. The CSRP Mental Health Consultation Manual covered topics such as a breakdown of how time and resources were to be allocated, an overview of the styles of interaction and communication to be adhered to, and benchmarks of success. Future implementation research on replications and extensions of the CSRP model should include detailed, observational measures of fidelity and quality in regards to use of the CSRP Manual (Dorner, 2010; Knoche et al., 2010).

Though the full range of factors that predict higher levels of implementation of the CSRP intervention is not clear, teachers were asked to indicate which attributes of MHCs made a difference in working with them. Teachers were given a list of characteristics and rated whether each factor made it difficult (0), didn't make a difference (1), or made it easy (2) to work with MHCs. Teachers rated the following as factors that made working with MHCs easy: a) MHCs' personality (M = 1.95), b) their skills and knowledge (M = 1.91), and c) their understanding of the challenges teachers face (M = 1.91).

In addition, teachers tended to rate the following factors as making it easier to work with MHCs: a) topics covered during their classroom visits (M = 1.87), b) their experience as mental health providers (M = 1.82), and c) their experience working with young children (M = 1.80). Although these data are descriptive, these statistics are consistent with the literature, which has noted the importance of providers' skills, knowledge, experience, and personalities to implementation processes (Fixsen et al., 2005; Greenhalgh et al., 2005).

Moreover, the training of MHCs may be related to the quality of the intervention services they offer (Durlak & Dupre, 2008; Perry et al., 2010). Before entering the field, MHCs participated in a day-long training, reviewing components of the CSRP model, discussing protocols for possible difficulties that might arise with sites, and spelling out schedules for staff meetings, clinical supervision, and submission of paperwork. In addition, MHCs received in-service training that addressed mandated reporting and coordination with school systems to obtain services for children with special needs. Mental health consultants also attended a conference on positive behavioral support in Florida. Future CSRP-related research should compare how various configurations of MHCs' training relate to the quality of implementation.

Future Directions

New investigations on replications and expansions of the CSRP model should involve: 1) more comprehensive, detailed implementation measures, and 2) a wider, in-depth portrait of factors that are predictive of multiple implementation indicators. First, building on the implementation research conducted here, new CSRP-related studies should include more comprehensive implementation indices across all types of participants and aspects of the intervention program (Domitrovich, Gest, Jones, Gill, & DeRousie, 2010; Durlak, 2010). For instance, more specific, observational measures of the fidelity and quality of the training of teachers and MHCs' coaching of teachers should be collected (Dorner, 2010).

Second, measures in future CSRP-related projects should be extended to capture: 1) additional provider characteristics (e.g., teachers' and site directors' perceived need for intervention services and perceived benefits of intervention services), 2) compatibility between the intervention program and sites, and the intervention program's adaptability to sites' needs across the academic year, and 3) factors relevant to the delivery of intervention services (e.g., general characteristics of sites, specific site practices and processes, specific staffing considerations; Baker et al., 2010; Durlak & Dupre, 2008; Stith et al., 2006). It is important to recognize pre-existing issues and ongoing dynamics at host sites that may interfere with the provision of intervention services (Durlak & Dupre, 2008). Such factors include teachers' absences, and the stressors of site directors who experience the challenges of serving children and families living in low-resourced neighborhoods.

Furthermore, intervention programs could evaluate the extent to which its own intervention team experienced a positive work climate among each other, and with their own supervisors. This climate might be assessed not only terms of the substantive aspects of administering the intervention program, but also in terms of managing the logistics of a large, multi-component project conducted at numerous sites. These issues may be compounded by carrying out such a project within a structured timeline like that of CSRP, and within the context of a randomized trial that requires control classroom conditions to be maintained appropriately.

Evaluation of such factors would be helpful in future research as we could then test whether and how such conditions played a role in MHCs' provision of services to teachers, and in MHCs' own well being, which in turn may shape their service provision to teachers. Past research has underscored the importance of understanding predictors of implementation (Durlak, 2010). With its multi-tiered design, CSRP's study of such predictors should involve the supervision of MHCs, MHCs provision of services to teachers, and teachers' provision of services to children. For instance, existing CSRP research has highlighted the importance of recognizing teachers' psychosocial stressors (Li-Grining et al., 2010; Raver et al., 2012; Zhai et al.., 2011). Future investigations of CSRP-related projects should extend these models to the stressors of MHCs who serve teachers (Perry et al., 2010). For efforts to scale up the CSRP intervention program, it may be helpful to examine the stressors experienced by MHCs and the strategies they use to provide high quality services to teachers and children despite facing risk factors.

Conclusion

CSRP's aim was to promote low-income, young children's school readiness by creating emotionally supportive classrooms and by fostering preschoolers' self-regulatory competence. Data from teachers and MHCs reveals some triumphs with implementing the CSRP model, and new research on the implementation of future replications and expansions of CSRP can build on the current study in a variety of ways. By deepening our understanding of the implementation of CSRP and similar programs, we may increase the efficiency and effectiveness of early childhood education intervention programs. Notably, the CSRP model is only one program among many interventions emerging from a new exciting area focused on addressing teachers' and children's clinical and developmental needs by providing mental health services as well as advancing basic developmental science (e.g., Gilliam & Shahar, 2006; Perry et al., 2010). Future studies are needed to evaluate how the implementation of this new generation of more comprehensive intervention models can fulfill its potential to better serve teachers' and children's mental health needs, in the short- and long-run.

Acknowledgments

We gratefully acknowledge the support of the National Institute of Child Health and Human Development (R01 HD046160-01) and of the McCormick Tribune Foundation. This research was also supported by a postdoctoral fellowship awarded to the first author from the American Psychological Association Minority Fellowship Program. Lastly, we wish to express a heartfelt thanks to senior policy leaders and program staff at Chicago's Department of Children and Youth Services, including Vanessa Rich, Anthony Raden, and Mary Ellen Caron. Thanks also to research assistants Annelise Cunningham, Kathryn Smagur, and Maria Limlingan, and to Lisa Dorner for feedback. We are especially indebted to the intervention team of the Chicago School Readiness Project, as well as the children, families, teachers, and site directors who made this work possible.

Contributor Information

Christine P. Li-Grining, Loyola University Chicago

C. Cybele Raver, New York University.

Darlene Jones-Lewis, Office of Children and Family Services, City of Chicago.

Sybil Madison-Boyd, University of Chicago.

Jaclyn Lennon, Loyola University Chicago.

References

  1. Aber JL, Jones SM, Brown JL, Chaudry N, Samples R. Resolving conflict creatively: Evaluating the developmental effects of a school-based violence prevention program in neighbourhood and classroom context. Development & Psychopathology. 1998;10:187–213. doi: 10.1017/S0954579498001576. [DOI] [PubMed] [Google Scholar]
  2. Arnold DH, McWilliams L, Arnold EH. Teacher discipline and child misbehavior in day care: Untangling causality with correlational data. Developmental Psychology. 1998;34:276–287. doi: 10.1037/0012-1649.34.2.276. [DOI] [PubMed] [Google Scholar]
  3. Baker CN, Kupersmidt J, Voegler-Lee ME, Arnold DH, Willoughby MT. Predicting teacher participation in a classroom-based, integrated preventive intervention for preschoolers. Early Childhood Research Quarterly. 2010;25:270–283. doi: 10.1016/j.ecresq.2009.09.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bernier A, Carlson SM, Whipple N. From external regulation to self-regulation: Early parenting precursors of young children's executive functioning. Child Development. 2010;81(1):326–339. doi: 10.1111/j.1467-8624.2009.01397. [DOI] [PubMed] [Google Scholar]
  5. Blair C, Razza RP. Relating effortful control, executive function, and false belief understanding to emerging math and literacy ability in kindergarten. Child Development. 2007;78(2):647–663. doi: 10.1111/j.1467-8624.2007.01019.x. [DOI] [PubMed] [Google Scholar]
  6. Domitrovich CE, Gest SD, Jones D, Gill S, DeRousie RMS. Implementation quality: Lessons learned in the context of the Head Start REDI trial. Early Childhood Research Quarterly. 2010;25:284–298. doi: 10.1016/j.ecresq.2010.04.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Donahue P, Falk B, Provet AG. Mental health consultation in early childhood. Baltimore, MD: Paul H. Brookes; 2000. [Google Scholar]
  8. Dorner LM. English and Spanish ‘para un futuro’ - or just English? Immigrant family perspectives on two-way immersion. International Journal of Bilingual Education and Bilingualism. 2010;13:303–323. doi: 10.1080/13670050903229851. [DOI] [Google Scholar]
  9. Durlak JA. The importance of doing well in whatever you do: A commentary on the special section, “implementation research in early childhood education”. Early Childhood Research Quarterly. 2010;25:348–357. doi: 10.1016/j.ecresq.2010.03.003. [DOI] [Google Scholar]
  10. Durlak JA, DuPre EP. Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology. 2008;41:327–350. doi: 10.1007/s10464-008-9165-0. [DOI] [PubMed] [Google Scholar]
  11. Eisenberg N, Spinrad TL. Emotion-related regulation: Sharpening the definition. Child Development. 2004;75(2):334–339. doi: 10.1111/j.1467-8624.2004.00674. [DOI] [PubMed] [Google Scholar]
  12. Evans GW, Rosenbaum J. Self-regulation and the income-achievement gap. Early Childhood Research Quarterly. 2008;23(4):504–514. doi: 10.1016/j.ecresq.2008.07.002. [DOI] [Google Scholar]
  13. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network; 2005. (FMHI Pub#231). Retrieved January 10, 2011 from http://www.fpg.unc.edu/∼nirn/resources/publications/Monograph/index.cfm. [Google Scholar]
  14. Gardner F, Connell A, Trentacosta CJ, Shaw DS, Dishion TJ, Wilson MN. Moderators of outcome in a brief family-centered intervention for preventing early problem behavior. Journal of Consulting and Clinical Psychology. 2009;77:543–553. doi: 10.1037/a0015622. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Gilliam WS, Shahar G. Prekindergarten expulsion and suspension: Rates and predictors in one state. Infants and Young Children. 2006;19:228–245. doi: 10.1097/00001163-200607000-00007. [DOI] [Google Scholar]
  16. Gorman-Smith D. Effects of teacher training and consultation on teacher behavior toward students at high risk for aggression. Behavior Therapy. 2003;34:437–452. doi: 10.1016/S0005-7894(03)80029-1. [DOI] [Google Scholar]
  17. Greenhalgh T, Robert G, Bate P, Macfarlane F, Kyriakidou O, editors. Implementation and Institutionalisation In Diffusion of Innovations in Health Service Organisations: A Systematic Literature Review (175-198) Oxford, UK: Blackwell Publishing Ltd; 2007. [DOI] [Google Scholar]
  18. Gross D, Fogg L, Webster-Stratton C, Garvey C, Julion W, Grady J. Parent training of toddlers in day care in low-income urban communities. Journal of Consulting & Clinical Psychology. 2003;71:261–278. doi: 10.1037/0022-006X.71.2.261. [DOI] [PubMed] [Google Scholar]
  19. Hamre BK, Justice LM, Pianta RC, Kilday C, Sweeney B, Downer JT, Leach A. Implementation fidelity of MyTeachingPartner literacy and language activities: Association with preschoolers' language and literacy growth. Early Childhood Research Quarterly. 2010;25:329–347. doi: 10.1016/j.ecresq.2009.07.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Henggeler SW, Schoenwald SK, Pickrel SG, Rowland MG, Santos SB. The contribution of treatment outcome research to the reform of children's mental health services: Multisystemic therapy as an example. Journal of Behavioral Health Services and Research. 1994;21:229–239. doi: 10.1007/BF02521330. [DOI] [PubMed] [Google Scholar]
  21. Izard CE. Translating emotion theory and research into preventive interventions. Psychological Bulletin. 2002;128:796–824. doi: 10.1037/0033-2909.128.5.796. [DOI] [PubMed] [Google Scholar]
  22. Jones SM, Brown J, Aber JL, Thomas G. Quantity and quality of implementation of the 4Rs program: Links to teacher and classroom characteristics. Paper presented at the Annual Meeting of the Society for Prevention Research; San Antonio, TX. 2006. [Google Scholar]
  23. Knoche LL, Sheridan SM, Edwards CP, Osborn AQ. Implementation of a relationship-based school readiness intervention: A multidimensional approach to fidelity measurement for early childhood. Early Childhood Research Quarterly. 2010;25:299–313. doi: 10.1016/j.ecresq.2009.05.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Lengua LJ, Honorado E, Bush NR. Contextual risk and parenting as predictors of effortful control and social competence in preschool children. Journal of Applied Developmental Psychology. 2007;28(1):40–55. doi: 10.1016/j.appdev.2006.10.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Li-Grining CP, Raver CC, Smallwood KM, Sardin L, Metzger MW, Jones SM. Understanding and improving classroom emotional climate in the “real world”: The role of Head Start teachers' psychosocial stressors. Early Education and Development. 2010;21:65–94. doi: 10.1080/10409280902783509. [DOI] [Google Scholar]
  26. Madison-Boyd S, Raver CC, Aufmuth E, Jones-Lewis D, Barden K, Williams M. The Chicago School Readiness Project mental health consultation manual. Chicago, IL: University of Chicago; 2006. [Google Scholar]
  27. McGilloway S, Mhaille GN, Bywater T, Furlong M, Leckey Y, Kelly P, Comiskey C, Donnelly M. A parenting intervention for childhood behavioral problems: A randomized controlled trial in disadvantaged community-based settings. Journal of Consulting and Clinical Psychology. 2012;80:116–127. doi: 10.1037/a0026304. [DOI] [PubMed] [Google Scholar]
  28. Odom SL, Fleming K, Diamond K, Lieber J, Hanson M, Butera G, Horn E, Palmer S, Marquis J. Examining different forms of implementation and in early childhood curriculum research. Early Childhood Research Quarterly. 2010;25:314–328. doi: 10.1016/j.ecresq.2010.03.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Perry DF, Allen MD, Brennan EM, Bradley JR. The evidence base for mental health consultation in early childhood settings: A research synthesis addressing children's behavioral outcomes. Early Education and Development. 2010;21:795–824. [Google Scholar]
  30. Raver CC, Jones SM, Li-Grining CP, Metzger M, Champion KM, Sardin L. Improving preschool classroom processes: Preliminary findings from a randomized trial implemented in Head Start settings. Early Childhood Research Quarterly. 2008;23:10–26. doi: 10.1016/j.ecresq.2007.09.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Raver CC. Placing emotional self-regulation in sociocultural and socioeconomic contexts. Child Development. 2004;75:346–353. doi: 10.1111/j.1467-8624.2004.00676.x. [DOI] [PubMed] [Google Scholar]
  32. Raver CC, Blair C, Li-Grining CP. Extending models of emotional self-regulation to classroom settings: Implications for professional development. In: Howes C, Hamre BK, Pianta RC, editors. Effective early childhood professional development: Improving teacher practice and child outcomes. Baltimore, MD; Brookes; 2012. [Google Scholar]
  33. Raver CC, Jones SM, Li-Grining CP, Zhai F, Metzger M, Solomon B. A randomized controlled trial of multi-component intervention implemented in Head Start-funded classrooms. Journal of Consulting and Clinical Psychology. 2009;77:302–316. doi: 10.1037/a0015302. [DOI] [PubMed] [Google Scholar]
  34. Raver CC, Jones SM, Li-Grining CP, Zhai F, Bub K, Pressler E. CSRP's impact on low-income preschoolers' preacademic skills: Self-regulation as a mediating mechanism. Child Development. 2011;82:362–378. doi: 10.1111/j.1467-8624.2010.01561.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Sektnan M, McClelland MM, Acock A, Morrison FJ. Relations between early family risk, children's behavioral regulation, and academic achievement. Early Childhood Research Quarterly. 2010;25(4):464–479. doi: 10.1016/j.ecresq.2010.02.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Stith S, Pruitt I, Dees J, Fronce M, Green N, Som A, Linkh D. Implementing community-based prevention programming: A review of the literature. The Journal of Primary Prevention. 2006;27(6):599–617. doi: 10.1007/s10935-006-0062-8. [DOI] [PubMed] [Google Scholar]
  37. Warner LA, Pottick KJ. Functional impairment among preschoolers in mental health services. Children & Youth Services Review. 2006;28:473–486. doi: 10.1007/s11414-006-9022-6. [DOI] [Google Scholar]
  38. Webster-Stratton C, Reid MJ, Hammond M. Treating children with early-onset conduct problems: Intervention outcomes for parent, child, and teacher training. Journal of Clinical Child & Adolescent Psychology. 2004;33:105–124. doi: 10.1207/S15374424JCCP3301_11. [DOI] [PubMed] [Google Scholar]
  39. Webster-Stratton C, Reid MJ, Stoolmiller M. Preventing conduct problems and improving school readiness: Evaluation of the Incredible Years teacher and child training programs in high-risk schools. Journal of Child Psychology and Psychiatry. 2008;49:471–488. doi: 10.1111/j.1469-7610.2007.01861.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Zhai F, Raver CC, Jones SM, Li-Grining CP. Dosage effects on school readiness: Evidence from a randomized classroom-based intervention. Social Service Review. 2010;84:615–655. doi: 10.1086/657988. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Zhai F, Raver CC, Li-Grining C. Classroom-based interventions and teachers' perceived job stressors and confidence: Evidence from a randomized trial in Head Start settings. Early Childhood Research Quarterly. 2011;26(4):442–452. doi: 10.1016/j.ecresq.2011.03.003. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES