Classifying trajectories of social-emotional difficulties through elementary school: Impacts of the Chicago School Readiness Project

Abstract

Although research has shown fade-out of the cognitive benefits of classroom-based preschool interventions, less is known regarding the durability of social-emotional impacts. This study examines the extent to which the multi-component Chicago School Readiness Project (CSRP) intervention lowered risk of internalizing, externalizing, attention, and social difficulties from Head Start through elementary school for 602 low-income children. Results suggest that most children in this sample showed few social-emotional difficulties over time. However, one quarter of the sample exhibited profiles of transitory or building difficulties over six years. Random assignment to the CSRP preschool intervention significantly reduced children’s odds of transitory attention and social difficulties in middle childhood, with preliminary evidence suggesting stronger impacts for children attending elementary schools characterized by low academic rigor and high neighborhood crime. CSRP was not found to be effective in preventing more robust, increasing forms of difficulty in the externalizing and attention domains. Implications for early childhood intervention and policy are discussed.

Over the past several decades, a large body of research has highlighted the extent to which poverty and income inequality in the United States yield major disparities in children’s psychological and behavioral health. Whereas many children in poverty develop resilient profiles of social and emotional competence, the stresses associated with poverty nonetheless increase children’s risk of sadness, anxiety, and worry, as well as behavioral and attentional dysregulation (Evans & English, 2002; Masten, 2001). Importantly, families, educators, and institutions in low-income communities actively work to support the social-emotional development of children through a range of prevention and early intervention services, including publicly-funded preschools programs such as Head Start. The current study examines the long-run impacts of one multi-component intervention implemented within Head Start settings in seven of the most economically hard-hit neighborhoods of Chicago. This classroom-based prevention approach – entitled the Chicago School Readiness Project (CSRP) – aimed to bolster children’s opportunities for success in school by supporting their social and emotional wellbeing in preschool (Author names redacted).

In the present study, we aim to address several specific questions relevant to both developmental and intervention research. First, we describe children’s profiles of resilient versus risky social-emotional development during the period between early and middle childhood. To do so, we use a statistical procedure called parallel process growth mixture modeling to classify children into groups marked by similar trajectories of internalizing, externalizing, attention, and social difficulties between Head Start and fifth grade. This approach provides a deeper understanding of the proportion of children in one low-income sample who develop healthy social-emotional trajectories, as well as the ways that a smaller group of children might develop different types of social-emotional challenges over time. Building on the results of these growth mixture models, we next consider the ways that the CSRP intervention may bolster some children’s chances of healthy social and emotional development, substantively altering children’s odds of membership in these different classes of social-emotional trajectories. Finally, we draw on recent advances in the modeling of social-emotional development among low-income children in the context of larger institutional and neighborhood contexts (e.g., Author names redacted) by exploring the role of children’s elementary school environments in the sustainability of these impacts. Specifically, we test whether the effects of CSRP are larger or smaller for children who end up in elementary schools characterized by varying levels of academic performance, perceived support and rigor, and crime. In so doing, we aim to shed light on questions of intervention “fade-out” that have arguably beleaguered early intervention research and to illustrate new ways of testing early intervention’s longer-term benefits.

Social-Emotional Development Over Time

By the time children enter preschool, they have already begun to develop the foundational social-emotional skills that play a pivotal role in their academic and psychosocial functioning throughout childhood and into adulthood. Although the majority of children face few social-emotional challenges, a small number develop difficulties with internalizing and externalizing behaviors, inattention, and poor social adjustment that have been linked to various dimensions of adult functioning, including lower educational attainment (Vitaro, Brendgen, Larose, & Tremblay, 2005), lower occupational status (Alatupa et al., 2013), and worsening mental health (Kubzansky, Martin, & Buka, 2004; Kosterman et al., 2010; Mason et al. 2004). Moreover, developmental cascade theories have highlighted the ways in which social-emotional difficulties have cumulative effects over time, where early challenges beget later challenges both within and across domains (Herrenkohl et al., 2010; Masten & Cicchetti, 2010; Masten et al., 2005; Obradovic, Burt, & Masten, 2010).

Although it is clear that early challenges predict later functioning and that social-emotional difficulties co-occur within time and dynamically influence each other over time, less is known about the nature and form of co-occurring trajectories of different social-emotional strengths and difficulties during early and middle childhood. With the increased use of person-centered methodological approaches that identify multiple patterns of growth and change that may otherwise be “masked” within average trajectories, researchers have begun to better understand how social-emotional skills and difficulties emerge over time for different groups of children. These approaches have revealed unique and often non-linear trajectories of externalizing behavior, internalizing behavior, and attention problems, with the majority of children (~50–70% in the studies cited) showing a resilient profile marked by low and stable levels of difficulties, and a smaller but variable number showing high but stable, increasing, or decreasing difficulties with time (Bongers, Koot, van der Ende, & Verhulst, 2008; Broidy et al. 2003; Feng, Shaw, & Silk, 2008; Larsson, Dilshad, Lichtenstein, & Barker, 2011; Nix et al., 2016; Thompson et al., 2010; Zhou et al., 2007).

Theories of Intervention Fade-out

Over the past several decades, a number of classroom-based interventions have been developed to support young children’s cognitive, social-emotional, and behavioral development in multiple domains. Despite the fact that many early childhood education programs show positive impacts immediately following program implementation, evidence is mixed regarding their long-term efficacy. Although several seminal programs (e.g., Perry Preschool, Abecedarian) have shown substantial effects on adult outcomes such as high school graduation, reduced criminal behavior, and employment, the majority of evaluated preschool interventions have shown evidence for “fade-out” of cognitive impacts quickly after children’s transition to elementary school (Barnett, 2011; Camilli et al., 2010; Campbell et al., 2012; Puma et al., 2012; Schweinhart et al., 2005).

What might explain this seemingly contradictory pattern of effects for early intervention? One popular hypothesis is that preschool program impacts on adult outcomes operate not through changes in cognitive skills, but rather via their sustained but often unmeasured influence on social-emotional competencies (Blair, 2002; Blair, Berry, & Friedman, 2012; Campbell, Pungello, Miller-Johnson, Burchinal, & Ramey, 2001). Recent work by Bailey, Duncan, Odgers, & Yu (2017) suggests that for interventions to generate sustainable impacts, they must target skills that are: (1) malleable, (2) fundamental, and (3) not addressed in everyday practice (outside of specific interventions). Although cognitive skills in math, language, and literacy are both malleable and fundamental, they are also the primary target of school experiences post-preschool intervention. As such, researchers have hypothesized that children who did not receive early intervention services may “catch up” to their peers in elementary school, thereby reducing any observed impacts of programming in the cognitive and academic domains (Barnett, 2011; Magnuson, Ruhm, & Waldfogel, 2007). Social-emotional skills, on the other hand, may be less salient targets of elementary education, both due to competing academic priorities and less neuroplasticity in brain regions associated with attention and regulation during middle childhood (Anderson, 2002). As a result, social, emotional, and behavioral functioning may be a more sustainable target of preschool intervention relative to cognitive skills. Indeed, several studies have found long-term program impacts on social-emotional skills despite early fade-out in cognitive domains (Chetty, Friedman, & Rockoff, 2013; Deming, 2009). In particular, Heckman and colleagues have shown that a large portion of the Perry Preschool program’s impacts on adult outcomes can be explained via reductions in aggressive and disruptive externalizing behaviors (Heckman, Pinto, & Savelyev, 2012).

Despite emerging evidence demonstrating the influence of early intervention programs on children’s social-emotional competencies at discrete moments in time, relatively little is known about programs’ effects on social-emotional trajectories. Few studies of early intervention programs include longitudinal measures of children’s social-emotional skills and difficulties, and when they do, they are rarely measured in comparable ways. Furthermore, relatively little research has explored the impact of early childhood programs during the transition from early to middle childhood, when temporary stressors (e.g., transitioning to a new school) may elicit brief periods of social-emotional difficulty that can have lasting impacts on other outcome domains, particularly for children exposed to other forms of adversity (Attar, Guerra, & Tolan, 1994; Roy, McCoy, & Raver, 2014). One exception is a recent study by Nix et al. (2016), which showed positive impacts of the Head Start REDI intervention on children’s trajectories of social competence, aggressive-oppositional behavior, learning engagement, attention problems, student-teacher closeness, and peer rejection. In the present paper, we build on this work to explore the ways in which the CSRP intervention might alter children’s social-emotional trajectories across a number of sub-domains during the preschool and elementary school years.

Trajectories in Context

In theory, many early intervention programs aim to provide additional supports to children facing life stressors. However, there is significant variability in the risks (e.g., poverty, violence) and protective factors (e.g., responsive caregiving, positive school climate) that children experience outside of early intervention programing. As a result, some argue that the degree to which impacts of early intervention fade out over time is dependent, in part, on the quality of the broader educational contexts to which children ultimately transition (Zhai, Raver, & Jones, 2012; Raver, McCoy, Lowenstein, & Pess, 2013). Several studies have shown that children who move into low-quality school environments after participating in early intervention may not be able to sustain the cognitive benefits of these programs due to high levels of stress and low levels of academic support available in these settings (Currie & Thomas, 1998; Lee & Loeb, 1995; Magnuson, Ruhm, & Waldfogel, 2007). Alternatively, it is possible that early intervention may provide the greatest benefit for children who end up in low-quality environments, as children in these settings would not otherwise receive access to the supports they need to develop these skills over time (Bailey et al., 2017; Bierman et al., 2014).

As a whole, this work demonstrates the importance of considering broader psychosocial and environmental characteristics when considering long-term program impacts on social-emotional outcomes. To our knowledge, no study has explored these questions using experimental data. To begin to address this gap, the present study considers several specific dimensions of children’s average elementary school experiences as moderators of the impact of early intervention on their social-emotional trajectories. Although this work is limited in several ways (including the fact that it is non-causal in nature and does not fully address all aspects of children’s school environments), we view these analyses as hypothesis-generating and an important first step in understanding the validity of theories of intervention fade-out.

The Present Study

In the present study, we use data from the Chicago School Readiness Project (CSRP) to explore children’s social-emotional trajectories over time. Previous work has shown CSRP’s combination of professional development, coaching, and individualized behavioral support to be effective in improving classroom climate, reducing teacher stress, improving children’s self-regulation and pre-academic skills, and reducing children’s behavior problems in the short-term using difference-in-difference designs (Author names redacted). In the current paper, we build on this early work to address three primary research aims. First, we use a person-centered analytic approach to describe patterns of growth and change in internalizing, externalizing, attention, and social difficulties from Head Start through the end of elementary school within our sample of low-income, primarily Black and Latino children. Second, we capitalize on the experimental design of the CSRP study to determine whether children’s probability of membership within particular “classes” of social-emotional trajectories differs based on their participation in the CSRP intervention during Head Start. Finally, to generate hypotheses about variation in classroom-based early childhood intervention impacts based on subsequent school experiences, we explore (in a non-causal and non-representative way) whether CSRP’s impacts on trajectory class membership are larger or smaller based on children’s average experiences of elementary school academic performance, perceived academic rigor and support, and school neighborhood crime.

In exploring these aims, the present study addresses several methodological and conceptual gaps in the developmental literature. First, we aim to characterize profiles of resilience and risk within the early to middle childhood transition for a sample of children living in high poverty and high crime areas of Chicago. In particular, our use of parallel process growth mixture modeling provides a more detailed picture of the heterogeneous and potentially non-linear social-emotional trajectories exhibited by these low-income children than is evident using variable-oriented growth curve methods. Second, in testing whether CSRP increases children’s probability of ending up in a more resilient social-emotional trajectory, as well as whether children’s subsequent school experiences moderate this impact, we explore the persistence of early intervention’s impacts on social-emotional functioning over time.

Although relatively little work has explored children’s social-emotional development in this way, we hypothesize that children will show several different patterns over time. In particular, we hypothesize that although most children will show resilient patterns of low and stable social-emotional difficulties, a smaller proportion will show elevated and/or growing problems over the elementary school period. Building on recent work by Nix et al. (2016), we hypothesize that children who were randomized to the CSRP intervention – and particularly those who received the individualized one-on-one behavioral supports – will show a higher probability of membership in resilient classes relative to their peers who were in the control group, helping to reveal heterogeneity in the way that early intervention may set some children on more a positive course of development than others. Finally, we hypothesize that the effects of the CSRP intervention on class membership will be largest – i.e., most easily sustained – for children who are enrolled in safe, high-performing, high-quality elementary schools.

Methods

Procedure & Sample

Data for the present study were taken from the Chicago School Readiness Project (CSRP), a cluster randomized controlled trial of a teacher training, coaching, and individual behavioral support intervention conducted in 18 Chicago Head Start centers in 2004–2005 (for cohort 1) and 2005–2006 (for cohort 2). Head Start centers were recruited for participation in the CSRP study based on their receipt of Head Start funding, their inclusion of at least two “full-day” classrooms, and their location in high-poverty, high-crime neighborhoods in the South and West Sides of Chicago. Prior to the start of the study, the 18 centers were matched pairwise based on a set of teacher-, child-, and center-level characteristics. Within these pairs, one center was randomly assigned to the treatment condition that included the full CSRP intervention, and the other center was assigned to the control condition.

The CSRP intervention is a bundled, multi-tiered program adapted from the Incredible Years training module (Webster-Stratton, Reid, & Hammond, 2004). The universal component of CSRP included five six-hour training sessions for lead and assistant teachers led by an experienced trainer with Licensed Clinical Social Worker qualifications, as well as weekly, in-class coaching sessions from a master’s level mental health consultant (MHC). On average, CSRP-assigned teachers spent about 18 hours (SD = 12 hours) in training sessions, and MHCs spent about 82 total hours (SD = 12 hours) or 4.54 hours per week (SD = 0.45 hours) in each classroom from September to March. The content of both the training and coaching sessions focused on building teachers’ skills in classroom management and emotional support. In addition to providing coaching sessions to teachers, in March, April, and May, MHCs were also free to deliver one-on-one support to a selected group of individual children identified by teachers as showing particular behavioral difficulties. During this three-month period, a total of 137 children in the intervention group (44%) received this one-on-one consultation at least one time. Of these, 46 (34%) received one-on-one consultation once, 41 (30%) received consultation two to five times, and the remainder were seen more than five times (max = 23). Although our team did not conduct a detailed benefit-cost analysis of CSRP, costs of the CSRP intervention are estimated to be about $22,000 per classroom, which includes approximately $2,000 for teacher training, $10,000 for MHCs, and $10,000 for additional costs (supervision, program incentives, etc.). (See Morris et al., 2010 for additional cost-benefit figures based on a subsequent replication study.) Centers assigned to the control condition did not receive training, coaching, or one-on-one behavioral supports, but did receive a weekly classroom aide to ensure balance in the student-adult ratio in both treatment and control classrooms. For additional details on the randomization protocol, intervention, and implementation fidelity, please see Author names redacted.

All children attending the treatment and control centers were eligible for participation in the CSRP study, with approximately 91 percent of children providing consent for participation for a total sample of 602 children (treatment n = 308; control n = 294). Data for the present study were collected from multiple sources across several time points. Child social-emotional data were reported by teachers at five time points: Fall of Head Start (HS Fall; n = 532), Spring of Head Start (HS Spring; n = 547), one year after Head Start (Kindergarten; n = 442), four years after Head Start (3^rd grade; n = 536), and six years after Head Start (5^th grade; n = 360). School characteristics were collected from Chicago Public Schools and the Chicago Police Department for available years in which children were in Kindergarten to 5^th grade. Basic demographic information was collected from primary caregivers at HS Fall.

Of the 602 children in the sample, 53 percent were female and 66 percent were identified as Black, 27 percent as Latino, 3 percent as non-Latino White, and 4 percent as “Other” or Biracial. At HS Fall, children were an average of 49.16 months old (SD = 7.38 months). See Table 1 for additional demographic information. Although no children were missing baseline demographic information, a total of 392 (65%) were missing social-emotional data at one or more time points. When comparing all child- and family-level characteristics, children with any missing social-emotional data were significantly (p < .10) more likely than their peers with complete social-emotional data to be girls, English-speaking, and to come from single-parent homes. See Analytic Plan section for details on how we addressed these missing data, as well as the Limitation section for implications of this missingness.

Table 1.

Sample descriptive characteristics: Mean (SD) or N (%)

	Overall Sample	Treatment	Control
Child and family characteristics	n = 602	n = 308	n = 294
Child Age (months)	49.16 (7.38)	49.22 (7.36)	49.10 (7.42)
Gender
Boy	281 (47%)	156 (51%)	125 (43%)
Girl	321 (53%)	152 (49%)	169 (57%)
Child Race/Ethnicity
Latino	162 (27%)	84 (27%)	78 (27%)
Black	396 (66%)	205 (66%)	191 (65%)
Non-Latino White	18 (3%)	3 (1%)	15 (5%)
Other	26 (4%)	16 (5%)	10 (3%)
4+ children in household	0.25 (0.43)	0.24 (0.43)	0.26 (0.44)
Spanish speaking household	0.23 (0.42)	0.21 (0.41)	0.25 (0.43)
Single parent household	0.71 (0.45)	0.72 (0.45)	0.69 (0.46)
Household poverty-related risksa	0.37 (0.33)	0.39 (0.33)	0.36 (0.32)
Child age 3 at HS entry	0.44 (0.50)	0.45 (0.50)	0.43 (0.50)
Teacher and classroom characteristics	n = 35	n = 17	n = 18
Teacher BA	21 (60%)	10 (59%)	11 (61%)
Teacher age (years)	39.33 (10.10)	37.81 (9.38)	40.94 (10.78)
Teacher depression	2.60 (2.10)	3.22 (1.65)	1.94 (2.31)
Teacher job demand	2.68 (0.59)	2.85 (0.64)	2.50 (0.50)
Teacher job control	3.29 (0.67)	3.38 (0.68)	3.20 (0.66)
Behavior management (CLASS)	4.86 (1.07)	4.56 (1.11)	5.18 (0.96)
Teacher sensitivity (CLASS)	4.82 (1.06)	4.56 (0.98)	5.11 (1.10)
Class negative climate (CLASS)	2.05 (1.00)	2.18 (1.13)	1.90 (0.86)
Classroom overall quality (ECERS)	4.72 (0.81)	4.49 (0.79)	4.97 (0.79)
N students in classroom	16.03 (2.92)	16.06 (3.08)	16.00 (2.83)
N adults in classroom	2.36 (0.69)	2.41 (0.78)	2.31 (0.59)
Center characteristics	n = 18	n = 9	n = 9
Family support worker on staff	1.39 (2.57)	0.44 (0.53)	2.33 (3.43)
N children ages 3–5	111.67 (126.08)	95.44 (55.39)	127.89 (173.54)
Proportion Black	0.73 (0.36)	0.69 (0.39)	0.78 (0.35)
Proportion teachers with BA	0.45 (0.40)	0.49 (0.36)	0.40 (0.46)
Proportion TAs with college	0.47 (0.38)	0.39 (0.34)	0.56 (0.42)
Proportion single parent families	0.85 (0.17)	0.84 (0.17)	0.87 (0.19)
Proportion families employed	0.71 (0.29)	0.81 (0.22)	0.62 (0.34)
Proportion families using TANF	0.39 (0.36)	0.35 (0.37)	0.42 (0.36)

Note:

^aFamily poverty-related risk is the proportion of risks related to mother-reported income, education, and employment experienced by the family (possible range = 0 to 1)

Measures

Child social-emotional difficulties

Three complementary measures were used to represent social-emotional difficulties at various points in the study, with items selected from each to ensure comparability over time. In HS Fall, lead and assistant teachers reported on each child’s internalizing, externalizing, attention, and social difficulties using a slightly modified version of the Behavior Problems Index (BPI; Zill, 1990), which was originally designed as a parent report scale. In the CSRP version of the BPI, respondents were asked to rate child social-emotional difficulties on a 3-point scale (0 = not true, 1 = sometimes true, and 2 = very/often true). Items were averaged into four subscales. Internalizing difficulties were represented by 6 items from the BPI’s Anxious/Depressed Mood subscale (e.g., “is unhappy, sad, or depressed”). Externalizing difficulties were represented by 6 items from the BPI’s Antisocial Problems subscale (e.g., “bullies or is cruel or mean to others”). Attention difficulties were represented by 5 items from the BPI’s Hyperactive Behavior subscale (e.g., “restless or overly active, cannot sit still”). Finally, social difficulties were represented by 4 items from the BPI’s Peer Conflict/Social Withdrawal subscale (e.g., “trouble getting along with other children”). Internal consistency for these domains ranged from α = .65 to α = .87 at HS Fall.

In the spring of Head Start and Kindergarten, lead and assistant teachers reported on social-emotional difficulties using the Caregiver-Teacher Report Form for ages 1.5 to 5 years (C-TRF; Achenbach & Rescorla, 2001). In the spring of 3^rd and 5^th grade, teachers used the Teacher Report Form for ages 6 to 18 years (TRF; Achenbach, 1991). Similar to the BPI, both the C-TRF and TRF include rating scales ranging from 0 to 2 (0 = not true, 1 = somewhat or sometimes true, and 2 = very true or often true). For the present study, a set of 61 items from the C-TRF and TRF was selected to measure social-emotional difficulties in ways that were comparable across scale and time. This smaller subset of items was averaged into domains following the structure of the TRF. Specifically, internalizing difficulties were captured using 21 items from the TRF’s Withdrawn/Depressed, Anxious/Depressed, and Somatic Complaints subscales (e.g., “cries a lot”). Externalizing difficulties were represented by 20 items from the TRF’s Rule Breaking and Aggressive Behavior subscales (e.g., “lying or cheating”). Attention difficulties were represented by a composite of 12 items from the TRF’s Inattention and Hyperactivity/Impulsivity subscales (e.g., “can’t sit still, restless, or hyperactive). Finally, social difficulties were represented by 8 items from the TRF’s Social Problems subscale (e.g., “doesn’t get along with other pupils”).

After the HS year, teachers were blind to children’s participation in the CSRP intervention. Internal consistency for all subscales from HS Spring to 5^th grade ranged from α = .66 to α = .94, with only social difficulties in K and 5^th grade falling below .70 (.66 and .69, respectively). At HS Spring, standardized t-score values using the full set of C-TRF items indicated that approximately 5.5 percent of the study sample showed elevated internalizing scores and 9.1 percent showed elevated externalizing scores (defined by a t-score of ≥ 60). Means and standard deviations for subscales at each time point can be found in Appendix Table 1.

Elementary school experiences

Children’s elementary school experiences were represented with four distinct variables: school academic performance, school climate of academic rigor, school climate of student support, and school neighborhood crime. Children’s school identifiers were collected for each year from Kindergarten through 5^th grade from Chicago Public Schools’ (CPS) student records. Given that high rates of dispersion and mobility in the present sample led children to attend more than 400 different elementary schools from 1^st to 5^th grade, we conceptualized elementary school experiences as child-level variables that represent individual children’s average experiences of elementary school characteristics over the duration of the study period (2006–2011 for cohort 1, and 2007–2012 for cohort 2).

School-level performance and climate data were captured annually between 2006 and 2011 from publicly-available CPS records. School academic performance was operationalized using the percentage of 3^rd to 8^th grade students performing at or above grade level in math and reading based on the Illinois Standards Achievement Test (ISAT; Illinois State Board of Education, 2006–2012). The ISAT is a standardized assessment that is administered annually and used as a measure of schools’ adequate yearly progress.

Schools’ climates of academic rigor and student support were captured using the Student Connection Survey (SCS), which was administered annually from 2007 to 2009 to CPS students in grades 6 to 8. Because 90 percent of elementary schools in the CPS system serve students up to 8^th grade, the SCS has been considered in past work to be reflective of the climates experienced by elementary school students (Raver et al., 2013; McCoy, Roy, & Sirkman, 2013). At the same time, it is important to note that school climate in this study is reported by students other than those included in our primary analyses. Students in each school responded to a total of 48 items using a Likert scale with four response options ranging from “Strongly Agree” to “Strongly Disagree.” Response rates on the SCS for CPS schools were high on average (M = 83%, SD = 11% in 2007). Item response theory and tests of predictive validity were used by the survey developers to create final aggregate scores representing the percentage of students in each school indicating an “excellent” or “adequate” view of each climate domain (Osher, Kendziora, & Chinen, 2008). The 17 items representing academic rigor reflect the degree to which students feel that adults in the school encourage them to think, work hard, do their best, and connect academic material to life outside of school (e.g., “My teachers often require me to explain my answers” and “The topics we are studying are interesting and challenging”). The 14 items in the student support domain represent the degree to which students feel listened to, cared about, and helped by teachers and other adults in their school (e.g., “My teachers really care about me” and “An adult at this school has helped me to plan for my life after high school”). Given that the SCS was only administered from 2007 to 2009 and that school-level SCS scores tended to be quite stable over time, school climate for 2007 was used to represent school climate in 2006, and school climate for 2009 was used to represent school climate for 2010 to 2012.

Finally, school neighborhood crime data were collected from the Chicago Police Department (CPD). To operationalize school neighborhood crime, school addresses were geocoded using ArcGIS software (ESRI, 2011) along with the precise location (i.e., latitude and longitude) of all crimes reported to the CPD. The total number of violent and non-violent crimes occurring within the school’s census tract during each school year was used to represent annual school neighborhood crime for 2007, 2008, and 2009. As with school climate, school crime data from 2007 were used to represent crime in 2006, and crime data from 2009 were used to represent crime for 2010 to 2012.

Table 2 shows the average of children’s elementary school experiences in each grade, as well as over the kindergarten to 5^th grade time period. Also shown in Table 2, approximately 3.5 percent of the sample was missing data on school performance and crime across all time points due to their movement out of the CPS system. An additional 4.8 percent of students attended CPS schools for which no school climate data were published. Children with missing data on school experiences were included in the primary analyses evaluating Study Aims 1 and 2, but were excluded from analyses for Aim 3. Children with missing school experience data were found be significantly (p < .10) younger, less likely to be Black, more likely to be Spanish speaking, less likely to come from a single parent household, and less likely to be socio-demographically at risk relative to those with complete data.

Table 2.

Children’s average elementary school experiences over time

	School Performance (%)		School Climate of Academic Rigor (%)		School Climate of Student Support (%)		School Neighborhood Crime (N)
Descriptive Characteristics	M	(SD)	M	(SD)	M	(SD)	M	(SD)
Kindergarten	52.5	(18.1)	72.3	(8.5)	77.1	(8.3)	670.9	(544.1)
1st Grade	57.2	(18.1)	75.8	(8.9)	79.3	(8.1)	708.1	(561.8)
2nd Grade	59.6	(16.1)	78.7	(8.6)	88.7	(9.0)	670.7	(512.7)
3rd Grade	62.6	(14.9)	76.6	(9.3)	95.6	(3.4)	616.3	(482.5)
4th Grade	66.4	(13.9)	76.6	(9.2)	95.6	(3.2)	658.1	(510.1)
5th Grade	69.0	(14.5)	76.9	(9.5)	95.7	(3.3)	621.0	(491.5)
Average (K-5th)	60.4	(12.6)	75.8	(7.5)	88.1	(6.4)	654.7	(438.7)
Valid N (K-5th)	581		552		552		581
Correlations
Sch Academic Rigor	.36**
Sch Student Support	.23**		.57**
Sch Neigh Crime	−.41**		−.19**		−.17**

Notes:

^**p < .01; Kindergarten = 2006 for cohort 1, 2007 for cohort 2; 1^st grade = 2007 for cohort 1, 2008 for cohort 2; 2^nd grade = 2008 for cohort 1, 2009 for cohort 2; 3^rd grade = 2009 for cohort 1, 2010 for cohort 2; 4^th grade = 2010 for cohort 1, 2011 for cohort 2; 5^th grade = 2011 for cohort 1, 2012 for cohort 2

Covariates

Covariates for the present study were chosen to be as similar as possible to those used in previous CSRP impact evaluations (see Author names redacted). All covariates were collected at HS Fall.

Child-level covariates

At the child level, covariates included (a) child age, (b) child gender, (c) child race/ethnic status of Black or Latino (reference group = White or Other), (d) household average of exposure to three poverty-related risks (mothers’ educational attainment of less than a high school degree, family income-to-needs ratio for the previous year being less than half the federal poverty threshold, mothers’ engagement in 10 hours or fewer of employment per week; Author names redacted), (e) single-headed household, (f) large family size (with 4 or more children), (g) Spanish-speaking household, and (h) whether the child was 3 years old (versus 4 years old) at Head Start entry.

Classroom/teacher-level covariates

Classroom-level covariates included teachers’ reports of (a) whether they had a BA, (b) their age, (c) their depressive symptoms using the K6 (from the U.S. National Health Interview Survey; Kessler et al., 2002), and their subscale scores on the (d) job demands and (e) job control subscales of the Child Care and Early Education Job Inventory (Curbow, Spratt, Ungaretti, McDonnell, & Breckler, 2000). Classroom quality in (f) behavior management, (g) teacher sensitivity, and (h) negative climate were assessed using the Classroom Assessment Scoring System (CLASS; La Paro et al., 2004). Overall quality was also used from (i) the Early Childhood Environment Rating Scale—Revised (ECERS-R; Harms et al., 2003). Finally, (j) the number of children and (k) the number of teachers observed in each classroom were included to account for possible differences in teacher-student ratios.

Center-level covariates

Center-level covariates included (a) the presence of a full-time family worker at the Head Start center, (b) the number of children ages 3–5 at each center, (c) the percentage of children who were Black, (d) the percentage of teachers with BA degrees, (e) the percentage of teacher assistants with some college experience, and the proportion of families served by the center that (f) were headed by a single parent, (g) included a household member who was employed, and (h) received Temporary Assistance for Needy Families.

Analytic Approach

We relied on several complementary approaches to address the study’s three primary aims. First, to classify children’s different social-emotional trajectories descriptively (Aim 1), we used a parallel process growth mixture modeling (GMM) approach. Unlike standard growth modeling, GMM allows for the simultaneous evaluation of both intra- and inter-individual differences in social-emotional development by grouping children into classes based on similar patterns of growth in social-emotional difficulties over time (Jung & Wickrama, 2008; Muthén & Muthén, 2000). We applied GMM within a parallel process framework, which assigns class membership to each child while considering all four social-emotional sub-domains simultaneously. We based the specification of our GMM functional form on previous results of an unconditional growth curve analysis within the same sample, which indicated linear growth in internalizing and externalizing behavior problems and cubic growth in attention and social behavior problems over the same time period (Author names redacted). All GMM analyses were conducted in MPlus (version 7.11; Muthén & Muthén, 2013) using a maximum likelihood estimation with robust standard errors, which accounts for missing data assuming a missing at random data pattern. Such an approach is appropriate when modeling growth under the assumption that missing data for the dependent variable (in this case, social-emotional challenges) are related to observed values of the dependent variable for that individual at other time points (Hedeker & Gibbons, 1997). Intercepts were centered around Head Start fall. Four criteria were used to evaluate model fit: (a) the sample-size adjusted Bayesian Information Criterion¹ (BIC; lower BIC indicates greater parsimony and model fit), (b) model entropy (higher entropy indicates better classification), (c) the Lo-Mendell-Rubin Likelihood Ratio Test (LMR-LRT; a significant LMR-LTR indicates better fit than a model with one fewer class), (d) number of individuals in each class, and (e) the theoretical utility and relevance of the identified classes (Jung & Wickrama, 2008; Lo, Mendell, Rubin, 2001; Nylund, Asparouhov, & Muthén, 2007; Yang, 2006).

To explore whether CSRP treatment assignment impacted the odds of children’s membership in a particular social-emotional trajectory class (Aim 2) and to determine whether this treatment impact varied based on children’s elementary school experiences (Aim 3), we estimated a three-level multinomial logistic regression model using HLM software (version 7; Raudenbush, Bryk, Cheong, Congdon, & du Toit, 2011). Multi-level multinomial logistic regression estimates the odds of particular outcomes of a categorical dependent variable while accounting for the nested structure of the data (i.e., children at level 1, Head Start classrooms at level 2, and Head Start centers at level 3). Specifically, we used the following set of equations for our primary analysis for Aim 2:

• Level 1: Child

  $$\log [{\mathrm{\phi }}_{\text{xijk}}/{\mathrm{\phi }}_{4\text{ijk}}]={\pi }_{0\text{jk}(\mathrm{x})}+{\displaystyle \sum {\pi }_{\text{njk}(\mathrm{x})}\ast ({\text{CHILDCOVAR}}_{\text{ijk}})}$$
  Where:

  $${\mathrm{\phi }}_{\text{xijk}}=\text{Prob}[\text{CLASS}(\mathrm{x})=1|{\pi }_{\text{jk}}]$$

  $${\mathrm{\phi }}_{4\text{ijk}}=1-{\mathrm{\phi }}_{\text{xijk}}=\text{Prob}[\text{CLASS}(4)=1|{\pi }_{\text{jk}}]$$
• Level 2: Classroom

  $${\pi }_{0\text{jk}(\mathrm{x})}={\beta }_{00\mathrm{k}(\mathrm{x})}+{\displaystyle \sum {\beta }_{0\text{nk}(\mathrm{x})}\ast }({\text{CLASSCOVAR}}_{\text{jk}})+r_{0\text{jk}(\mathrm{x})}$$
• Level 3: Center

  $${\beta }_{00\mathrm{k}(\mathrm{x})}={\gamma }_{000(\mathrm{x})}+{\gamma }_{001(\mathrm{x})}\ast ({\text{CSRP}}_{\mathrm{k}})+{\displaystyle \sum {\gamma }_{00\mathrm{n}(\mathrm{x})}}\ast ({\text{CENTERCOVAR}}_{\mathrm{k}})+{\mathrm{u}}_{00\mathrm{k}(\mathrm{x})}$$

Using these models, we estimated the odds of belonging to a particular social-emotional trajectory class (labeled CLASS(x) above) versus the reference “low and stable” social-emotional problems class (labeled CLASS(4) above) based on random assignment to the CSRP treatment at level three (CSRP_k). In this analysis, our primary coefficient of interest is γ_001(x), which represents the effect of random assignment to the overall CSRP treatment on the odds of membership in class × versus the resilient low and stable class (i.e., an intent-to-treat approach). To improve the precision of our estimates, we also included a core set of pre-treatment covariates at all levels (CHILDCOVAR_ijk, CLASSCOVAR_jk, and CENTERCOVAR_k) based on previous work evaluating short-term and long-term impacts of CSRP (Author names redacted).

In addition to this primary set of models for estimating the impact of random assignment to the overall CSRP intervention, we also conducted a set of exploratory analyses probing the unique contribution of the one-on-one behavioral supports offered to 44 percent of the treatment group. To do so, we re-ran the above-described primary model to include a level-one (child-level) variable indicating how many times the child had received one-on-one behavioral supports. The coefficient for this variable can be interpreted as the “added” benefit of each consultation session, above and beyond random assignment to the universal CSRP intervention. We note that although the results of the primary analyses for Aim 2 may be considered causal, this additional analysis focused on one-on-one behavioral supports is strictly descriptive, as these extra services were not randomly assigned to children. Indeed, as noted above, children were selected for these individualized supports based on their high baseline levels of behavioral risk, and therefore are not directly comparable to their peers in the treatment group who did not receive these additional services.

To address our questions of moderation by elementary school characteristics (Aim 3), we ran separate models that included a cross-level interaction between CSRP treatment assignment (at level 3) and children’s average experiences of each school characteristic across time (at level 1) within the subsample of children for whom school experience data were available. We note once again that the tests of moderation are considered exploratory, non-representative, and non-causal due to missing data, as well as the fact that the moderators – elementary school characteristics – were measured after CSRP treatment was randomly assigned.² Nevertheless, we feel that these exploratory analyses are useful for generating hypotheses about the possible roles that subsequent elementary school environments play in either sustaining, increasing, or mitigating initial benefits of early childhood intervention. For all analyses in HLM, we grand mean centered all predictor variables with the exception of treatment assignment.

Results

Aim 1: Description of Social-Emotional Trajectory Classes

Model fit indices for the GMM results are shown in Table 3. Based on model fit, the number of individuals in each class, and theoretical relevance, we selected a four-class solution for subsequent analyses. Visual representations of these classes are shown in Figure 1 and descriptive statistics for all classes can be found in Appendix Table 1. In particular, we found that a small number of children (n = 41, 7% of the total sample) fell within an “early recovery” class (Class 1), which shows linear reductions in internalizing and externalizing difficulties over time, as well as “recovery” from early increases in social and attention difficulties after the kindergarten transition. Another small group of children (n = 40, 7% of the total sample) was observed in a “late recovery” class (Class 2) characterized by increases in internalizing and externalizing difficulties over time and “recovery” from an increase in attention and social difficulties at third grade. A slightly larger group of children (n = 78, 13% of the total sample) was assigned to an “increasing attention and externalizing difficulties” class (Class 3) characterized by consistent increases in social-emotional difficulties across all outcomes, with particularly steep gains for attention and externalizing challenges. Finally, as hypothesized, the majority of children (n = 442, 74% of the total sample) were assigned to a reference “low and stable” class (Class 4) that showed low starting levels of all difficulties and no observable growth over time, with the exception of minimal gains in attention difficulties.

Table 3.

Model fit indices for parallel process growth mixture model results

	Adjusted BIC	Entropy	LMR, p-value	Class Ns
2 Classes	−4030.35	0.875	718.304, p = .007	108, 493
3 Classes	−4214.93	0.886	223.796, p = .724	49, 103, 449
4 Classes	−4362.87	0.886	192.473, p = .633	40, 41, 78, 442
5 Classes	−4471.71	0.867	193.611, p = .606	25, 39, 50, 76, 411

Figure 1. Social-emotional trajectory classes from parallel process growth mixture models.

Note: HSF = Head Start Fall, HSS = Head Start Spring, K = Kindergarten, 3^rd = 3^rd Grade, 5^th = 5^th Grade

Aim 2: Impacts of CSRP Treatment on Class Membership

Full results of our primary multi-level multinomial logistic regression analyses examining the main effect of CSRP treatment assignment (at the Head Start center level) on odds of social-emotional trajectory class membership are shown in Table 4. Children from Head Start centers that were randomly assigned to receive the CSRP intervention were significantly less likely to be in the late recovery class (Class 2) relative to the low and stable reference class (Class 4), OR = 0.11 (95% CI: 0.02, 0.79), p < .05. CSRP treatment assignment did not predict membership in the early recovery (Class 1) or increasing attention/externalizing difficulties (Class 3) classes relative to the low and stable class (Class 4).

Table 4.

Results of multi-level regression predicting behavior problem class membership (relative to the low and stable reference class) from CSRP intervention status

	Class 1: Early Recovery		Class 2: Late Recovery		Class 3: Incr. Att. & Extern.
	OR	95% CI	OR	95% CI	OR	95% CI
Level 1 - Child
Child Gender (boy)	2.32*	(1.09, 4.96)	1.71	(0.86, 3.38)	2.70**	(1.56, 4.67)
Child Age	0.90*	(0.82, 0.99)	1.02	(0.93, 1.12)	0.94+	(0.88, 1.01)
Child race/ethnicity (Black)	4.19*	(1.23, 14.27)	2.49	(0.64, 9.68)	7.98**	(1.83, 34.87)
Child race/ethnicity (Other)	2.95	(0.46, 19.05)	3.18	(0.63, 16.06)	6.24*	(1.12, 34.74)
4+ children in household	0.91	(0.37, 2.23)	0.75	(0.31, 1.78)	0.55+	(0.28, 1.09)
Spanish speaking household	1.39	(0.51, 3.79)	0.61	(0.12, 1.87)	0.51	(0.18, 1.45)
Single parent household	1.50	(0.63, 3.57)	0.70	(0.33, 1.51)	1.11	(0.57, 2.17)
Household poverty risksa	0.71	(0.20, 2.48)	0.93	(0.23, 2.90)	2.14+	(0.94, 4.88)
Child age 3 at HS entry	0.28+	(0.06, 1.21)	1.36	(0.36, 5.20)	0.59	(0.21, 1.64)
Level 2 - Classroom
Teacher BA	2.24	(0.18, 28.60)	4.37	(0.19, 102.12)	1.65	(0.25, 10.79)
Teacher age	0.99	(0.82, 1.20)	0.97	(0.84, 1.11)	1.00	(0.90, 1.10)
Teacher depression	1.06	(0.37, 3.03)	0.87	(0.35, 2.19)	0.94	(0.47, 1.90)
Teacher job demand	1.31	(0.08, 22.06)	3.41	(0.29, 40.73)	1.28	(0.20, 8.20)
Teacher job control	0.60	(0.03, 11.38)	1.35	(0.21, 8.77)	0.85	(0.25, 2.83)
Behavior management (CLASS)	0.23	(0.01, 5.34)	0.83	(0.05, 12.77)	0.77	(0.11, 5.17)
Teacher sensitivity (CLASS)	4.91	(0.15, 158.18)	2.85	(0.26, 30.84)	1.22	(0.13, 11.64)
Classroom negative climate (CLASS)	0.95	(0.09, 10.32)	0.92	(0.15, 5.66)	1.16	(0.26, 5.17)
Classroom overall quality (ECERS)	0.81	(0.08, 8.82)	0.33	(0.02, 5.67)	1.68	(0.33, 8.55)
N children in classroom	0.56+	(0.26, 1.22)	0.73	(0.44, 1.21)	0.88	(0.59, 1.29)
N adults in classroom	1.21	(0.14, 10.84)	1.07	(0.17, 6.73)	1.19	(0.35, 4.04)
Level 3 - Center
CSRP intervention group	0.35	(0.03, 3.99)	0.11*	(0.02, 0.79)	1.02	(0.18, 5.62)
Family support worker	0.39	(0.11, 1.45)	0.48	(0.18, 1.29)	0.80	(0.36, 1.76)
N children ages 3–5	1.02	(1.00, 1.04)	1.01	(0.99, 1.03)	1.01	(1.00, 1.02)
% African Americans	0.48	(0.01, 27.98)	3.62	(0.10, 129.71)	3.08	(0.20, 47.74)
% teachers with BA	9.11	(0.03, 2944.73)	3.86	(0.34, 44.03)	3.01	(0.52, 17.63)
% TAs with college	0.10	(0.00, 4.53)	1.14	(0.10, 13.81)	0.83	(0.14, 4.95)
% single parent families	2.27	(0.00, 44,120.12)	10.16	(0.04, 2975.57)	0.60	(0.01, 43.18)
% families employed	8.60	(0.01, 15,885.96)	24.07	(0.40, 1434.32)	5.95	(0.29, 120.33)
% families using TANF	0.88	(0.04, 17.57)	2.57	(0.35, 19.05)	2.15	(0.43, 10.77)
Intercept	0.02**	(0.00, 0.17)	0.10*	(0.02, 0.58)	0.02**	(0.00, 0.12)

Notes: BA = Bachelor’s Degree; TAs = Teacher Aides; TANF = Temporary Assistance for Needy Families;

^aFamily poverty-related risk is a mean value of mother-reported income, education, and employment;

^**p < .01,

^*p < .05,

⁺p < .10

In addition to these primary analyses, we also explored whether the receipt of one-on-one behavioral support sessions for children in the treatment group was related to differences in children’s probability of social-emotional class membership. These results suggest that, controlling for overall treatment status (i.e., coming from a Head Start center that was or was not randomly assigned to CSRP), individuals’ receipt of each additional behavioral support session was associated with higher odds of ending up in each of the “high risk” classes (Classes 1–3) relative to the low and stable class (Class 4). Specifically, odds ratios were 1.24 (95% CI: 1.13, 1.37), p < .01, for the early recovery class (Class 1), 1.24 (95% CI: 1.13, 1.38), p < .01, for the late recovery class (Class 2), and 1.20 (95% CI: 1.10, 1.30), p < .01, for the increasing attention/externalizing difficulties class (Class 3) relative to Class 4.

Aim 3: Moderation of CSRP Treatment Impact by School Characteristics

Of the 12 moderation analyses tested (including four moderators across three class contrasts), four were found to have a p-value of less than .05. Two significant interactions were observed for the impact of CSRP on children’s membership in the late recovery class versus the reference class (see Figures 2 and 3). Specifically, the effect of CSRP on reducing odds of membership in the late recovery class (Class 2) versus the low and stable class (Class 4) was significantly different for children who were in schools characterized by different levels of academic rigor, OR = 1.13 (95% CI: 1.00, 1.28), p < .05, and neighborhood crime, OR = 0.998 (95% CI: 0.995, 1.000), p < .05. In particular, follow-up analyses indicated that the impact of the CSRP treatment on reducing the probability of membership in Class 2 was stronger – and statistically significant – for children attending schools with lower than average (−1SD) levels of academic rigor, OR = 0.05 (95% CI: 0.01, 0.49), p < .05, and those with higher than average (+1SD) crime, OR = 0.04 (95% CI: 0.00, 0.49), p < .05. The impact of CSRP treatment was not, however, statistically significant for children attending schools characterized by high (+1SD) academic rigor or low (−1SD) neighborhood crime.

Figure 2.

Odds ratios for social-emotional trajectory class membership by treatment condition and school academic rigor

Figure 3.

Odds ratios for social-emotional trajectory class membership by treatment condition and school neighborhood crime

Although no main effects were observed for CSRP treatment on children’s likelihood of membership in the increasing attention/externalizing difficulties class (Class 3) versus the reference class (Class 4), two statistically significant interactions were observed for this contrast (see Figures 2 and 4). In particular, children in the CSRP intervention group (relative to controls) showed different odds of being in the increasing attention/externalizing difficulties class (Class 3) based on their elementary schools’ levels of academic rigor, OR = 1.11 (95% CI: 1.01, 1.22), p < .05 and student support, OR = 1.11 (95% CI: 1.00, 1.23), p < .05. Additional analyses showed that although the effect of CSRP for reducing odds of being in the increasing attention/externalizing difficulties class was stronger for children attending schools with low (−1SD) academic rigor and student support, the simple slopes were not statistically significant within these contexts, or within those characterized by high (+1SD) rigor or support.

Figure 4.

Odds ratios for social-emotional trajectory class membership by treatment condition and school student support

No differences in CSRP’s treatment impact were observed based on elementary school academic performance on standardized math and reading assessments.

Discussion

The primary aims of the present study were to (1) identify distinct patterns of social-emotional development for low-income children between entry into Head Start and the end of elementary school, (2) determine whether exposure to the CSRP intervention significantly altered children’s probability of membership within these social-emotional trajectory classes, and (3) explore (non-causally) the degree to which elementary school characteristics might moderate this treatment impact. Overall, the results of our analyses suggest several important conclusions regarding the development of resilience and the prevention of social-emotional difficulties in a sample of children facing high levels of poverty.

First, in exploring children’s social-emotional trajectories across time, we found that nearly 75 percent of our sample showed relatively low and stable levels of difficulties between Head Start and the end of elementary school. In other words, despite high levels of socioeconomic adversity, the majority of children in our sample showed no evidence of elevated difficulty in internalizing, externalizing, attention, or social domains. These results reinforce the notion that resilience is, in fact, a process of “ordinary magic”, where the majority of children facing poverty and its related stressors experience positive developmental outcomes (Masten, 2001). They are also consistent with prior research showing low and stable trajectories of challenges within individual social-emotional domains for the majority of children (e.g., Feng, Shaw, & Silk, 2008; Nix et al., 2016; Thompson et al., 2010; Larsson et al., 2011).

At the same time, the results of our growth mixture modeling analyses did reveal substantial heterogeneity in the patterns of social-emotional development for the remaining quarter of our sample. In particular, approximately 13 percent of our sample showed evidence of temporary or transitory increases in attention and social challenges either around school transition points (the “early recovery” Class 1) or in the middle of elementary school (the “late recovery” Class 2). These results reinforce the non-linear nature of children’s social-emotional development, as well as the somewhat normative – although potentially costly – nature of transitory elevations in social-emotional difficulties during high-stress periods (e.g., transitions to a new school, high-stakes testing; Diamond & Spillane, 2004). This finding offers an important reminder that cross-sectional studies of children’s behavioral difficulty in the context of environmental stressors may over-estimate long-term risk, given that some of those children’s attentional and emotional problems may decrease in subsequent years of development.

In addition to these children facing transitory challenges, an equally sized group of children showed evidence for somewhat linear increases in social-emotional difficulties over time, particularly in the domains of attention and externalizing symptoms (the “increasing externalizing/attention” Class 3). These findings are in keeping with prior work showing that a small but significant fraction of children facing high levels of poverty-related risk follow an “early onset” trajectory of acting out, aggressive, and inattentive behavior that persists and increases through adolescence and early adulthood (Moffitt et al., 1996; Shaw, Bell, & Gilliom, 2000). Given the centrality of inattention, impulsiveness, and aggression within many common clinical disorders (e.g., ADHD, oppositional defiant disorder) and previous research showing early increases in these challenges to be associated with later mental health problems, it is possible that membership within this class might serve as an early marker of risk for psychopathology (American Psychiatric Association, 2013; Broidy et al., 2003; Thompson et al., 2010). Future research is needed to understand the degree to which these early trajectories predict later-life difficulties, as well as to develop methods for early detection of particularly risky patterns.

Second, the results of this study provide new evidence that classroom-based preschool intervention can have lasting impacts on low-income children’s social-emotional development. In particular, we found that children who attended a Head Start center that was randomly assigned to the CSRP intervention were 89 percent less likely than their control group peers to show a pattern of development characterized by slow increases in internalizing and externalizing difficulties and steep – but transitory – increases in attention and social challenges around 3^rd grade. Instead, CSRP-assigned children were more likely to show resilient, stable and low patterns of social-emotional difficulty than were their control group counterparts. These results are well-aligned with recent evaluations of other universal interventions targeting preschool classroom management, including the Head Start REDI curriculum and the Good Behavior Game (Nix et al., 2016; Petras et al., 2008).

These results provide preliminary support for the hypothesis that classroom-based preschool education interventions can have lasting – although subtle – impacts on children’s social and emotional development. In particular, our findings suggest that the overall CSRP intervention may protect children from transitory elevations in attentional and social difficulties in the middle of elementary school that may be costly to children’s academic and social standing. Although this study was unable to explore the mechanisms underlying these effects, it is possible that the early supports provided in CSRP classrooms enhance children’s foundations in social-emotional competence, giving them an edge relative to their control group peers in regulating their attention and getting along with others during periods of mild stress within middle childhood (Author names redacted). At the same time, it is important to note that CSRP was not found to affect the probability of ending up in the class of difficulties characterized by rising levels of externalizing and attention difficulties, which – as noted above – may be the most clinically concerning group identified in the present study.

In sum, these results suggest that although the CSRP package of both universal and selected school-based supports may be useful in preventing elevated, transitory cases of social-emotional difficulty, more intensive, multi-setting interventions like Triple P (the Positive Parenting Program) are likely necessary to help children facing growing comorbid difficulties over time (August, Realmuto, MacDonald, Nugent, & Crosby, 1996; Bor, Sanders, & Markie-Dadds, 2002; McGoey, Eckert, & Dupaul, 2002). One challenge associated with tiered programs is that they are often too costly to implement at scale. As such, careful research is needed to disaggregate the unique effects of program components (e.g., teacher training, classroom consultation, and one-on-one support for children with high levels of behavioral risk) for supporting specific social-emotional needs (Metropolitan Area Child Study Research Group, 2002; Zhai et al., 2010). Although we find descriptive evidence in this study that the receipt of one-on-one behavioral supports in Head Start was negatively associated with children’s behavioral trajectories over time, these results are not trustworthy due to the fact that these services were not randomly assigned to children. On the one hand, it is possible that these exploratory findings reflect a lack of clinical efficacy of the relatively “light touch” one-one-one supports provided through the CSRP intervention, suggesting the need for more intensive, evidence-based, selected supports implemented by trained professionals in the future (Cowen et al., 1996; Wilson & Lipsey, 2007). On the other hand, it is also possible that these findings simply reflect the high levels of behavioral risk faced by the children selected for these supports. Moving forward, research using more nuanced experimental and quasi-experimental designs is needed to further unpack the unique benefits of universal and selected components of multi-pronged interventions like CSRP. This information will be useful for determining the most cost-, time-, and resource-effective mechanisms for program scale-up in resource-limited settings.

Future research is also needed to understand why effects of CSRP were observed only within middle childhood, and whether, for example, different types of interventions (e.g., those explicitly designed to support the pre-k to kindergarten transition, those with greater intensity or longer duration) might be more effective in enhancing resilience at various points in development. For example, prior research from model programs such as Abecedarian has shown lasting impacts of programs beginning in infancy on more severe forms of social-emotional difficulty (e.g., depression) through adulthood, suggesting the possibility that earlier intervention may be more beneficial for mitigating these challenges over time (McLaughlin, Campbell, Pungello, & Skinner, 2007). Evidence is also needed to understand the degree to which observed impacts on social-emotional development translate to similar long-term gains in other outcomes, including cognitive and academic skills.

Third, in exploring the degree to which the impacts of the CSRP intervention differed based on children’s subsequent elementary school experiences, we found that the above-mentioned positive impact of CSRP on children’s social-emotional trajectories was actually stronger for children who attended schools characterized by low levels of academic rigor and high levels of neighborhood crime. In addition, we observed differences in CSRP’s effectiveness based on elementary school characteristics, where CSRP was found to reduce children’s odds of increased attention and externalizing difficulties only in schools characterized by low levels of rigor and/or support. Contrary to previous evidence suggesting that low-resourced environments may be responsible for intervention fade-out (Currie & Thomas, 1998; Lee & Loeb, 1995; Magnuson et al., 2007), these results suggest that the CSRP intervention may actually provide the greatest benefit for children who end up in high-risk school environments. It is important to note that previous research showing reduced benefits of early intervention for children transitioning into low-quality schools was primarily focused on children’s academic and cognitive capacities, which are subject to different biological, developmental, and contextual influences than children’s social-emotional functioning. Although the specific mechanisms underlying these differences are difficult to identify, in keeping with Bailey et al.’s (2017) framework for impact sustainability, it is possible that the effects of CSRP on children’s social-emotional trajectories in lower-quality environments may have been maintained due to the fact that these low-quality schools would not otherwise have been able to develop children’s skills in this domain, whereas high-quality schools would have. Regardless, the effect sizes of these differences across schools are small. Furthermore, the validity of these analyses is limited given that school measures were collected post-intervention, selection into high- versus low-quality settings was non-random, and the contextual measures themselves were operationalized as individual-level constructs. Additional research using causal approaches is needed to confirm this finding, as well as to explore whether similar results are observed for a variety of interventions and populations. To complement this work on contextual risk, additional work is also needed to explore individual-level exposures (e.g., home environment, family SES) as additional predictors of longer-term program impact variation (Miller, Farkas, Vandell, & Duncan, 2014; Watamura, Phillips, Morrissey, McCartney, & Bub, 2011).

Overall, the results of this study highlight several important considerations for future work in the developmental and prevention sciences. From a methodological perspective, these results highlight the degree to which simple growth modeling or difference-in-difference designs mask developmental sub-patterns characterized by varying and non-linear changes over time. For example, the different patterns of effects observed in the present study suggest that although CSRP may be effective in preventing transitory social-emotional difficulties in the middle of elementary school, its dose may not have been sufficient for alleviating stressful school transitions or more pervasive patterns of steady symptom growth over time. Understanding these more nuanced aspects of treatment impact variation is particularly critical for explaining previously observed long-term effects of early childhood programs, as well as for providing information on how to design future programs for targeting specific sub-patterns of social-emotional challenges.

Limitations

Although there are numerous strengths of this analysis, the present study also has several important limitations. First, children in the CSRP study were exposed to particularly high levels of risk in their home, school, and community environments. As a result, these findings cannot be generalized outside of this particular context to other groups of children who may benefit from early intervention programming.

Second, although the social-emotional measures used in this study provided a unique opportunity to study a set of nearly identical constructs over a relatively long developmental period, they may have provided biased or incomplete estimates of children’s true skills and difficulties. For example, the measures’ brevity and coarseness may have compromised the sensitivity with which they were able to detect the types of difficult-to-observe but meaningful challenges faced by children in these settings. Furthermore, previous research has shown that teachers whose demographic backgrounds differ from their students’ may overestimate students’ difficulties, particularly within low-income settings (Dee, 2005; Zimmerman, Khoury, Vega, Gil, & Warheit, 1995). Although we did not have data on the race/ethnicity of our study teachers, broader data suggest a large mis-match between the backgrounds of students and teachers in the Chicago Public School system, where only 9.9 percent of students but exactly half of teachers are reported as White (Chicago Public Schools, 2017). As such, it is possible that the results of our study may have been biased due to reporter issues. To address these limitations and to more fully understand the multiple factors that might underlie intervention sleeper effects, additional work is needed using more precise, unbiased, and triangulated approaches to exploring a broader set of skills within the social-emotional domain (e.g., executive function, emotion regulation, motivation) in ways that are sensitive to the racial/ethnic background of study participants. Related to this, better measures are also needed for capturing contextual characteristics. In particular, the school climate reports from 6^th to 8^th graders used in the present study may not be representative of the experiences of younger children within the same schools, which may have led to biased results (McCoy, Roy, & Sirkman, 2013). Additional research that incorporates a more diverse cadre of reporters – including teachers, parents, and younger students – for operationalizing school climate is needed (Cohen, McCabe, Michelli, & Pickeral, 2009).

Third, despite the experimental nature of this study, the results of our analyses examining one-on-one behavioral supports and moderation based on elementary school characteristics are inherently non-causal due to the fact that these features were non-randomly assigned to children and may, in the case of elementary experiences, have been endogenous to the intervention. Finally, missing data may have introduced bias to our estimates. Although GMM allows us to retain the full sample when estimating patterns of social-emotional growth over time, it does so under the assumption that data are missing at random and can be explained by observed data at other time points. It is possible that this assumption does not hold in our data and, as a result, some of these estimates may be biased. Furthermore, given that full data for our school-level measures were available only for three of the six years analyzed, the estimates of school characteristics may also be inaccurate. Related to this, the fact that we do not have complete school-level data for all children in the sample reduces the generalizability of the results of Aim 3.

Conclusions

The results of the present study highlight room for cautious optimism within the field of early childhood development and intervention. In particular, the overall low prevalence of social-emotional difficulties for three-quarters of the CSRP sample suggests a longitudinal portrait of resilience within low-income communities facing high levels of environmental adversity. Additionally, evidence that the CSRP intervention appears to have modest but long-lasting impacts on children’s social-emotional trajectories suggests that early intervention makes a difference in children’s lives over time. We found evidence for the sustainability of early childhood treatment impacts beyond the preschool year for children facing the highest levels of subsequent environmental disadvantage. At the same time, CSRP was not found to be a “magic” solution for improving long-term wellbeing for all children (see Brooks-Gunn, 2003). In particular, it was not found to be effective in preventing more robust, increasing forms of social-emotional difficulty in the externalizing and attention domains. In the future, research is needed to further explore these subgroup findings in ways that can identify the “active ingredients” and dosage of early interventions that are necessary for optimizing and maintaining impacts across multiple domains for the largest number of children possible. In short, prevention science, developmental science, and community supports can and must be jointly tailored to meet the needs of our nation’s youngest learners.

Appendix Table 1.

Children’s average social-emotional difficulties over time

	Internalizing		Externalizing		Attention		Social
Head Start Fall	M	(SD)	M	(SD)	M	(SD)	M	(SD)
Full sample (n = 532)	0.09	(0.11)	0.16	(0.18)	0.21	(0.21)	0.10	(0.13)
Class 1: early recovery (n = 36)	0.26	(0.17)	0.39	(0.25)	0.46	(0.25)	0.29	(0.17)
Class 2: late recovery (n = 36)	0.12	(0.13)	0.17	(0.23)	0.29	(0.24)	0.12	(0.16)
Class 3: incr att and ext (n = 74)	0.09	(0.08)	0.23	(0.20)	0.25	(0.23)	0.11	(0.14)
Class 4: low and stable (n = 386)	0.08	(0.10)	0.13	(0.14)	0.17	(0.18)	0.07	(0.10)
Head Start Spring
Full sample (n = 547)	0.11	(0.12)	0.21	(0.28)	0.25	(0.30)	0.14	(0.19)
Class 1: early recovery (n = 35)	0.29	(0.15)	0.61	(0.36)	0.75	(0.30)	0.45	(0.17)
Class 2: late recovery (n = 37)	0.15	(0.14)	0.39	(0.39)	0.50	(0.40)	0.27	(0.32)
Class 3: incr att and ext (n = 68)	0.10	(0.11)	0.37	(0.34)	0.34	(0.31)	0.19	(0.21)
Class 4: low and stable (n = 407)	0.09	(0.10)	0.13	(0.19)	0.17	(0.22)	0.09	(0.13)
Kindergarten
Full sample (n = 442)	0.11	(0.15)	0.20	(0.33)	0.34	(0.43)	0.13	(0.22)
Class 1: early recovery (n = 28)	0.30	(0.30)	0.60	(0.43)	1.00	(0.56)	0.44	(0.34)
Class 2: late recovery (n = 33)	0.12	(0.14)	0.20	(0.32)	0.40	(0.42)	0.14	(0.22)
Class 3: incr att and ext (n = 58)	0.11	(0.11)	0.42	(0.45)	0.44	(0.47)	0.19	(0.24)
Class 4: low and stable (n = 323)	0.09	(0.13)	0.13	(0.24)	0.26	(0.35)	0.10	(0.17)
3rd Grade
Full sample (n = 536)	0.13	(0.18)	0.22	(0.35)	0.40	(0.46)	0.16	(0.24)
Class 1: early recovery (n = 39)	0.11	(0.16)	0.23	(0.41)	0.47	(0.47)	0.14	(0.21)
Class 2: late recovery (n = 38)	0.43	(0.27)	0.58	(0.34)	1.17	(0.39)	0.74	(0.26)
Class 3: incr att and ext (n = 63)	0.16	(0.15	0.72	(0.48)	0.80	(0.46)	0.33	(0.25)
Class 4: low and stable (n = 396)	0.09	(0.15)	0.10	(0.20)	0.25	(0.35)	0.07	(0.12)
5th Grade
Full sample (n = 360)	0.14	(0.16)	0.25	(0.37)	0.47	(0.46)	0.17	(0.24)
Class 1: early recovery (n = 24)	0.12	(0.09)	0.21	(0.24)	0.60	(0.43)	0.18	(0.20)
Class 2: late recovery (n = 28)	0.20	(0.17)	0.27	(0.22)	0.67	(0.46)	0.31	(0.26)
Class 3: incr att and ext (n = 47)	0.27	(0.24)	1.04	(0.37)	1.15	(0.39)	0.56	(0.26)
Class 4: low and stable (n = 261)	0.11	(0.13)	0.11	(0.16)	0.31	(0.34)	0.09	(0.15)

Note: “incr att and ext” = increasing attention and externalizing challenges