Social and Emotional Learning Services and Child Outcomes in Third Grade: Evidence from a Cohort of Head Start Participants

Abstract

A variety of universal school-based social and emotional learning (SEL) programs have been designed in the past decades to help children improve social-emotional and academic skills. Evidence on the effectiveness of SEL programs has been mixed in the literature. Using data from a longitudinal follow-up study of children (n = 414) originally enrolled in a clustered randomized controlled trial (RCT) when they were in Head Start, we examined whether universal SEL services in third grade were associated with the development of children from disadvantaged families. We took advantage of pairwise matching in the RCT design to compare children who had similar family background and preschool experiences but received different doses of SEL services in third grade. The results showed that the frequent (i.e., weekly to daily) exposure to SEL opportunities was associated with favorable social-emotional and academic development in third grade, including increased social skills, student-teacher relationship, and academic skills, as well as reduced impulsiveness.

School-based social and emotional learning (SEL) programs were introduced almost two decades ago to support children's emotional and behavioral development as well as to improve the emotional climate of schools (Collaborative for Academic, Social, and Emotional Learning [CASEL], 2003, 2012; Elbertson, Brackett, & Weissberg, 2010). SEL is the process of acquiring and effectively applying knowledge, attitudes, and skills in five major areas of social-emotional competence, including self-awareness (e.g., identifying emotions and recognizing strengths), social awareness (e.g., perspective taking and respecting others), self-management (e.g., managing emotions and goal setting), relationship skills (e.g., communication and cooperation), and responsible decision-making (e.g., evaluation and reflection) (CASEL, 2003, 2012; Elbertson et al., 2010; Zins & Elias, 2006). Some examples of typical SEL activities include those targeting students’ development of respect and responsibility, acceptance of others, engagement, anger management, verbal and physical aggression, bullying, conflict resolution, and healthy life choices (Jones, Brown, & Aber, 2008).

In the past decade, a variety of universal SEL programs that are available to all children in a school or classroom regardless of risk status have been designed to help improve children's social-emotional and academic skills. Recently there has been rapid expansion in the implementation of SEL services in schools to support social-emotional, along with academic, domains of children's development (CASEL, 2003, 2012; Elbertson et al., 2010). This effort has been driven by the increasing recognition that, to maximize their effectiveness, school-based, universal interventions that take an integrative, holistic approach to provide more coordinated, sustained, and systematic services may have better chances of greater impact than those programs that are academically-focused alone (Elbertson et al., 2010; Jones, Brown, & Aber, 2011; Social and Character Development [SACD] Research Consortium, 2010; Zins, Bloodworth, Weissberg, & Walberg, 2004).

As detailed below, evidence on the effectiveness of SEL programs has been mixed in the literature. Although the effect sizes of universal SEL programs available to all students in a classroom or school have been found slightly higher than those targeting high-risk students (Jones & Bouffard, 2012), few studies have directly examined the effectiveness of universal SEL services on disadvantaged students who attended high-quality preschool programs such as Head Start. In this study, we used longitudinal follow-up data in a cohort of Head Start participants in a preschool intervention study, the Chicago School Readiness Project (CSRP), to examine how classroom-based promotion of SEL in third grade was associated with children's social-emotional and academic outcomes. As detailed below, the CSRP used a clustered randomized controlled trial (RCT) design and provided multifaceted intervention services to a group of Head Start teachers and children in disadvantaged neighborhoods in Chicago. By taking advantage of the pairwise matching procedure employed in the CSRP design, we compared the outcomes of children who had similar family background and preschool experiences but received varying doses of SEL services in third grade. In addition, we investigated whether SEL services in third grade played a moderating role as a “booster” of the initial effects of CSRP intervention (e.g., Raver et al., 2009, 2011).

SEL and Child Development

School- and classroom-based SEL programs are expected to improve the social-emotional and academic skills of children, especially those in low-income families. Prior research has shown that the benefits of many high-quality early interventions targeting economically disadvantaged children, such as Head Start, tend to dissipate in early years of elementary school (Barnett, 1995; Currie & Thomas, 1995; Magnuson, Ruhm, & Waldfogel, 2007). As the perspective of life cycle skill formation and human capital accumulation suggests, enrichment programs such as SEL services can boost the skill acquisition and school achievement of children with skill advantages gained from high-quality early interventions and also compensate for the skill deficits experienced by at-risk children at school entry (Cunha, Heckman, Lochner, & Masterov, 2006; Hamre & Pianta, 2005; Magnuson et al., 2007). Similarly, improvement in a school's social-emotional climate has been argued to benefit learning, more broadly, and may make important contributions not only to children's social skills but also academic performance. For example, students’ perceptions of teacher support and mutual respect are linked to positive changes in academic motivation and engagement (Ryan & Patrick, 2001). A review of risk prevention programs also suggests that promoting mastery of social-emotional competence is associated with positive youth development (Guerra & Bradshaw, 2008).

Neuropsychological models of learning in the context of adversity provide more theoretical and empirical foundations for SEL programs. These models, as supported by empirical research, identify a neurological basis for the links between children's emotional experience within stressful versus supportive environments and their social and academic performance in those contexts (Blair & Raver, 2012; Greenberg, Kusché, & Riggs, 2004; Riggs, Greenberg, Kusché, & Pentz, 2006). For example, models that combine affective and academic processes have been used in the development of multiple universal prevention curricula for promoting children's social-emotional development in preschools and elementary schools, such as the broadly implemented Incredible Years and the Promoting Alternative Thinking Strategies (PATHS) (Greenberg, 2006; Riggs et al., 2006; Webster-Stratton, Reid, & Hammond, 2004). These affective-cognitive models suggest that young children's regulation of their own emotions (as well as their responses to others’ emotions) may alternately support or disrupt attention, working memory, and other prefrontal cortical processes central to learning in the classroom (Ursache, Blair, & Raver, 2011). The elementary school years have been hypothesized to serve as a developmental sensitive period, when children's behavior problems become evident and may be exacerbated by exposure to stressful experiences with teachers and peers (SACD Research Consortium, 2010; Thornberry, Huizinga, & Loeber 2004). Empirical evidence from recent field experiments of SEL interventions is aligned with this hypothesis. For example, research shows that the PATHS curriculum increases inhibitory control and verbal fluency (i.e., two mediators of social competence), which, in turn, are related to the reduction in behavior problems (Riggs et al., 2006).

Despite their promise, the findings on the effectiveness of SEL programs have been mixed in the literature. Many empirical studies conducted in the past decade on a variety of SEL programs found positive associations with children's developmental outcomes in preschool through high school years (CASEL, 2012; Greenberg et al., 2003; Jones et al., 2011; Payton et al., 2008; Zins & Elias, 2006). In a meta-analysis of 213 school-based, universal SEL programs that included 270,034 students from kindergarten to high school during 1970-2007, SEL participation was found to be associated with improved social-emotional skills, attitudes about self, others, and school, prosocial behaviors, and academic performance, as well as reduced conduct and internalizing problems (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011). An earlier meta-analysis of 165 studies of school-based prevention of conduct problems from kindergarten to high school found that self-control or social competence promotion instruction as well as non-instructional cognitive and behavioral methods effectively reduced alcohol and drug use, dropout and nonattendance, and other conduct problems (Wilson, Gottfredson, & Najaka, 2001). In contrast, other studies found little evidence that SEL programs improved children's development. For example, a study of seven social and character development programs implemented in 84 schools from third grade (in 2004) to fifth grade (in 2007) found little impact of these programs on student outcomes (SACD Research Consortium, 2010). These findings were consistent with two previous meta-analyses of a wide range of SEL and anti-bullying interventions implemented in kindergarten through 12^th grade, which did not find meaningful change in children's outcomes to be associated with social-emotional curricula (Merrell, Gueldner, Ross, & Isava, 2008; Park-Higgerson, Perumean-Chaney, Bartolucci, Grimley, & Singh, 2008).

The mixed findings on the impact of SEL programs on children's outcomes are mainly due to the variations in program definitions, designs, and fidelity of implementations as well as the measures of SEL activities across studies. One inherent challenge in SEL research is that SEL has been widely defined or characterized, varying from conflict resolution, anti-bullying, and civic engagement, to a host of other important but quite different topics (CASEL, 2012; Jones & Bouffard, 2012). Universal SEL programs available to all students in a classroom or school have slightly smaller effect sizes (approximately 0.2 to 0.3 of a standard deviation) than those of more targeted programs for high-risk students who are identified by teachers as having substantial problems (effect sizes of approximately 0.5) (Durlak et al., 2011; Jones & Bouffard, 2012). On balance, high-quality evaluations of individual programs that base their work on carefully articulated theories of action that are closely aligned with SEL theory and practice (e.g., PATHS) tend to reveal positive effects on children (e.g., Aber, Brown, Jones, Berg, & Torrente, 2011; Durlak et al., 2011). In contrast, in some other evaluations, the underlying theories or the combinations of SEL activities might be inadequate to alter students’ overall social-emotional outcomes because a subset of students who had developmental deficits may require more targeted and intensive interventions than school-wide programs (CASEL, 2012; SACD Research Consortium, 2010). In addition to the definition and design of SEL programs, implementation fidelity and quality are also critically important factors contributing to SEL effectiveness. Well-implemented SEL programs are usually associated with positive outcomes, while poorly implemented programs have been found to have small or no effects no matter how well they may have been initially designed (CASEL, 2012; Durlak et al., 2011; Jones & Bouffard, 2012; SACD Research Consortium, 2010; Wilson, Lipsey, & Derzon, 2003). Moreover, many states have legislation that requires or promotes SEL activities in school and thus, to estimate the impact of SEL, it would be impractical to identify a comparable control group that does not have any SEL activities. Therefore, it has been suggested that SEL practices be measured as continuous variables, rather than as all-or-none phenomena, to assess the extent or the comprehensiveness of using active learning techniques (Durlak et al., 2011).

The Present Study

Using longitudinal data from the CSRP, we examined how the number of SEL activities that teachers used in class, a continuous measure of SEL practices as one aspect of implementation, were associated with children's social-emotional and academic outcomes in third grade. The initial CSRP program was a preschool intervention targeting self-regulation in early childhood and was implemented in Chicago during 2004-2006 when children were in Head Start. This present study represented an opportunity to test whether SEL services that were delivered in third grade conferred additional advantage, serving as a “booster” set of services to CSRP-enrolled children as they grew older. For the present analyses, all children who participated in the third grade data collection lived in Chicago or other areas in Illinois. Illinois was the first state that passed legislation in 2003 (i.e., Children's Mental Health Act, Public Act 93-0495) to require every school district develop a plan for implementing SEL programs and incorporate SEL skills as part of student learning standards (CASEL, 2003; Durlak et al., 2011; Elbertson et al., 2010; Zins & Elias, 2006). As detailed below, in the third grade survey, the CSRP collected data on different types of SEL activities reported by teachers as a rough proxy for children's greater versus lower exposure to SEL curricula in their classroom. Using these data, we were able to directly investigate the links between the comprehensiveness in SEL services (measured by the extent of using six SEL activities) and children's outcomes.

To capture the associations between the frequency of SEL activities and children's outcomes across multiple developmental domains, we adopted several sets of measures that have been used widely in large-scale surveys and small clinical trials. These measures included social-emotional outcomes (i.e., social skills, child-teacher relationship, and impulsiveness), behavior problems, and academic outcomes (i.e., language, literacy and mathematic skills). In the analysis, we took advantage of the pairwise matching employed as part of the RCT design of the CSRP and first located children who were in the matched Head Start sites that were randomly assigned to the CSRP intervention or control group the Head Start year, and who received varying doses of SEL services in third grade. Overall, children were comparable in the matched Head Start sites that were located in the same neighborhoods. We then compared their outcomes based on varying receipt of SEL services in third grade, controlling for original CSRP intervention status as well as child, family, and classroom covariates at baseline. We also examined the possible “booster” role of SEL services in third grade where early elementary intervention might magnify or extend the initial benefits of CSRP intervention (Raver et al., 2009, 2011). We thus further conducted moderation analyses to investigate whether SEL services played an empirically detectable role as a booster intervention working in combination with preschool intervention to benefit children's development over the longer term.

Method

Data and Analysis Sample

Data for this study were from the CSRP, which originally used a clustered RCT design. As shown in the flow chart in Figure 1, in an effort to balance generalizability and feasibility, 18 Head Start sites were selected within seven of Chicago's economically disadvantaged neighborhoods and then were matched into 9 pairs based on 14 site-level demographic characteristics (e.g., teachers’ average annual salary, the number of teaching staff and family support workers, and the percentages of African American children, slots subsidized by child care subsidy, and teachers with bachelor's degrees), using a pair-wise matching procedure recommended in education research (Bloom, 2005). In each matched pair, one Head Start site was randomly assigned to the intervention group and the other site to the control group. Two classrooms from each site were initially included, with 35 out of the 36 classrooms retained in the study after randomization. The project was conducted within two cohorts, including Cohort 1 (from 10 Head Start sites) in 2004–05 and Cohort 2 (from 8 Head Start sites) in 2005–06.

Figure 1.

CSRP and SEL participant flow chart

As illustrated in Figure 1, overall, a total of 602 children enrolled in the initial classrooms agreed to participate in the CSRP in the Head Start year (308 in the intervention group and 294 in the control group) and were contacted for each follow-up data collection. Due to attrition and the availability of data on their focal measures, prior studies on the CSRP have included a various sample size of children in their analyses (e.g., Raver et al., 2009, 2011; Zhai, Raver, & Jones, 2012). In the collection of data in third grade that were used in the present study, parents and teachers were contacted by mail, phone, or school visit to obtain their consent to participate in the study. Overall, 524 children were successfully recruited and 414 of them (212 in the intervention group and 202 in the control group) from 22 schools had non-missing information on classroom-based SEL activities as well as social-emotional and academic outcome measures. An attrition analysis shows that, compared to children who were excluded from the analysis sample of the present study due to missing data, children who were included were more likely to be non-Hispanic black and less likely to speak Spanish at home.

The CSRP provided multifaceted services in the intervention group in both cohorts from fall to spring in the Head Start year. The interventions included teacher training on behavior management strategies adapted from the Incredible Years teacher training module (Webster-Stratton et al., 2004), teacher-directed coaching on the implementation of behavior management strategies, and mental health consultation for a small number of children who had high behavioral and emotional problems. To ensure that the child-staff ratio was similar across intervention and control classrooms, a teacher's aide was placed in classrooms in the control group who only provided additional staffing support (see Raver et al., 2009, 2011, and Zhai, Raver, & Li-Grining, 2011, for more detail on CSRP intervention services).

Outcome Measures

Children's social-emotional and academic outcomes in this study were from the CSRP third grade survey conducted in the spring semester (from April to June), including teacher reports of children's social skills, child-teacher relationships, impulsiveness, behavior problems, as well as language, literacy and mathematic skills. To reduce word counts and enhance understanding, the discussion below briefly summarizes the specific measures of these outcomes, while a detailed description, including their items, subscales, and psychometric information (i.e., Cronbach's alpha) calculated using data of the analysis sample, is presented in Appendix Table 1.

Social skills were measured using the Social Skills Rating System (SSRS), which provides a broad assessment of student social behaviors and focuses on positive and pro-social skills (Gresham & Elliot, 1990). The analysis included the teacher-reported total scores of social skills as well as three subscales, including cooperation, assertion, and self-control. Child-teacher relationship was measured using the Student-Teacher Relationship Scale (STRS), which was developed to measure student-teacher relationship patterns in terms of closeness and conflict, as well as the overall quality of the relationship (Hamre & Pianta, 2001; Pianta, 2001). The items in the CSRP were summed up into a total STRS score and two subscales of closeness and conflict. Children's impulsiveness was assessed using the Barratt Impulsiveness Scale, Version 11 (BIS-11; Patton, Stanford, & Barratt, 1995). The BIS-11 was designed to assess an individual's general impulsiveness, taking into account the multi-factorial nature of the construct. The CSRP adapted the scale for teachers to report on children in the classroom context, including a total BIS score and three subscales of attentional impulsiveness (to measure attention and cognitive instability), non-planning impulsiveness (to assess self-control and cognitive complexity), and motor impulsiveness (to measure motor control and perseverance). Children's behavior problems were measured by teachers’ reports using the Caregiver-Teacher Report Form (C-TRF; Achenbach & Rescorla, 2001). Raw scores were summed into a total C-TRF score as well as Internalizing and Externalizing subscales. Finally, children's academic skills were measured using a modified version of the Academic Rating Scale (ARS; Rock, Pollack, & Hausken, 2002). Teacher reported items were summed up into a total ARS score and two subscales of language and literacy as well as mathematical thinking.

Measures of SEL Services

Data on teachers’ classroom-based promotion of SEL were collected in the third grade survey. Teachers were asked about the specific SEL activities or programs that they had used with their class since the beginning of the school year of third grade. The survey used the measure of Classroom SEL Services, adapted from the Teacher Report on Classroom and School (TRCS) employed in the seven programs evaluated by the SACD Research Consortium (2010), including the New York City Study of Social and Literacy Development (Jones et al., 2008). These services and activities have been provided by many school-based SEL programs and include the recommended SAFE (i.e., sequenced, active, focused, and explicit) practices, which use coordinated activities and active forms of learning to help students develop personal, social, and other specific SEL skills (see reviews by Durlak et al., 2011; Elbertson et al., 2010; Zins & Elias, 2006). Moreover, these activities are also in accordance with the SEL standards and benchmarks developed and implemented by the Illinois State Board of Education, as required in the Illinois Children's Mental Health Act (i.e., Public Act 93-0495), to help educators select and design curricula, classroom activities and instruction, and performance-based assessments.

Specifically, there were six types of SEL activities in the CSRP third grade survey, including (1) behavior or effective classroom management (e.g., activities designed to reduce student disruptive behavior, increase student attention and engagement); (2) violence prevention or peace promotion (e.g., activities designed to address verbal and physical aggression, weapons, bullying, conflict resolution, or peacemaking); (3) social and emotional development (e.g., activities that help students with anger management, recognizing emotions, empathy, and friendships); (4) character education (e.g., activities that help students develop respect, responsibility, honesty, fairness, caring); (5) tolerance or diversity (e.g., activities that target acceptance of others, celebrating cultural and ethnic differences, or reduction in prejudice); and (6) risk prevention or health promotion (e.g., activities that target alcohol, tobacco and drug use, or promote healthy life choices).

Teachers were asked to rate how often they used each of the above six types of SEL activities or programs with their class since the beginning of the school year on a scale of 1 (not using), 2 (as needed), and 3 (weekly or daily). Few teachers (ranging from 0 to 7% by the type of activities) reported they did not use these activities in their classroom, which may be due to the legislative requirement that SEL skills be addressed as part of student learning standards in Illinois, as discussed above. As a result, the distribution of the ratings on the individual activities as well as the total score was negatively skewed. Therefore, the ratings were dichotomized to indicate whether teachers used these activities weekly/daily or not (i.e., item scores of 1 and 2 collapsed as “no” compared to item score of 3 as “yes”). A total score on six types of activities (α = 0.71) was summed to reflect the number of different types of SEL activities that teachers used weekly or daily in their classrooms. The measure ranged from 0 to 6 types of activities, with a mean of 2.60 types of activities and a median of 2.00 types of activities. This measure reflects the dosage and comprehensiveness of SEL services provided by teachers in their classroom, with higher scores mean SEL services indicating a higher dose or more comprehensive.

Baseline Covariates

The covariates in the analyses included child and family characteristics as well as teacher, classroom, Head Start site covariates at baseline when children were 3 to 5 years old (average age of 4.09). Research has suggested that in the evaluation of preventive interventions, especially those targeting low-income children, it is important to account for the potential pre-intervention confounding variables of child, family, and school demographic characteristics (e.g., Aber et al., 2011; Currie & Thomas, 1995; Hill, Waldfogel, & Brooks-Gunn, 2002; Jones et al., 2011; Love, Chazan-Cohen, Raikes, & Brooks-Gunn, 2013; Magnuson et al., 2007; Raver et al., 2009). These variables, as detailed below, have been taken into account in previous SEL studies as well to address the heterogeneity of students that may distort the relationships between SEL and child outcome measures (CASEL, 2012; Durlak et al., 2011; Jones & Bouffard, 2012; Merrell et al., 2008; Park-Higgerson et al., 2008; SACD Research Consortium, 2010; Wilson et al., 2001).

Specifically, child covariates included child gender, race/ethnicity, and pre-intervention measures at baseline. Since the majority of children in the study were non-Hispanic black (69%), followed by Hispanic (24%) and other race/ethnicity (7%, including non-Hispanic white, biracial, and other), we used a dichotomized measure of race/ethnicity to indicate whether the child was non-Hispanic black or not. For outcomes in third grade, slightly different scales were used as pre-intervention covariates since the same measures were unavailable at baseline. For measures of social skills, student-teacher relationship, and impulsiveness in third grade, similar measures of sociability, affective competence, and impulse control at baseline were controlled in respective models (Smith-Donald, Raver, Hayes, & Richardson, 2007). Similarly, teacher-reported behavior problems at baseline, measured by the Behavior Problems Index (BPI; Zill, 1990), were controlled in the models of behavior problems in third grade, while the models of academic skills in third grade controlled for direct assessment of academic development at baseline, measured by the Peabody Picture Vocabulary Test (PPVT) and Early Math Skills (Dunn & Dunn, 1997; Zill, 2003). Family covariates included maternal education (i.e., whether mother had less than high school education) and employment (i.e., whether mother worked 10 hours or less per week), poverty (i.e., whether family had income-to-needs ratio at less than 50% of the federal poverty threshold), and whether Spanish was spoken at home (while other languages were not asked since the vast majority of the sample spoke either English or Spanish).

We also controlled for preschool quality and emotional climate at baseline measured using the Early Childhood Environment Rating Scale-R (ECERS-R; Harms, Clifford, & Cryer, 2003) and the Classroom Assessment Scoring System (CLASS; La Paro, Pianta, & Stuhlman, 2004), as well as student-staff ratio, teacher education (i.e., having a bachelor degree or not), and teacher psychological distress to control the potential confounding of differences in children's experiences of early education prior to school entry (Blair & Raver, 2012). Since these class-level measures may also confound both SEL activities in the classroom and children's social-emotional and academic outcomes in third grade, the estimates of their associations may be biased due to the lack of data on these measures in the third grade survey.

Analytic Strategy

Using data from the sample of Head Start children who participated in the CSRP, we conducted a series of models to examine whether SEL activities were associated with children's social-emotional and academic outcomes in third grade. Prior research found children had different experiences subsequent to the CSRP intervention. For example, Zhai et al. (2012) reported that children in the CSRP intervention group were more likely to enroll in high-performing schools than those in the control group. In this study, we first conducted regression analyses on the predictors of SEL services, which found no evidence that children's receipt of SEL services varied as a function of CSRP intervention, child, family and classroom covariates, or pre-intervention measures at baseline. Therefore, selection did not appear to be an issue in children's receipt of SEL services in third grade. We then conducted straightforward models, as specified below, to examine the associations between SEL activities and child outcomes.

To reiterate, the initial CSRP intervention study employed a clustered RCT design in which Head Start sites were matched and randomly assigned to the CSRP intervention or a control group. In the analysis, we included the dummy variables for Head Start site matches, which allowed us to compare outcomes of children in the same matched Head Start sites who subsequently went on to receive different SEL services in third grade. Specifically, we conducted a set of increasingly controlled models to test the robustness of the results. In Model 1, we included only the measure of SEL services in third grade as a key predictor of child outcomes. Model 2 included additional child and family covariates and pre-intervention measures at baseline. Classroom and teacher covariates were added in Model 3. In Model 4, the CSRP intervention assignment (i.e., intervention or control group) and the dummy variables of matched Head Start site pairs at baseline were included. To further test whether SEL services played a moderating role as a booster of initial CSRP impact, we included the interaction term between SEL services and CSRP intervention assignment in additional models based on Model 4.

Further, the initial CSRP study design was “clustered,” wherein children in the CSRP were nested within classrooms that, in turn, were nested in Head Start sites. To account for this hierarchical structure in the CSRP design, we adopted three-level hierarchical linear modeling (HLM) in the current analysis. In general, HLM allows for the simultaneous estimation of variance associated with individual (within-subjects) and population (between-subjects) change based on the specification of fixed- and random-effect variables in the models (Raudenbush & Bryk, 2002). In the three-level HLM analysis, Level 1 had child-level covariates, including SEL services in third grade as well as child and family demographics and pre-intervention measures at baseline; Level 2 included classroom and teacher covariates at baseline; and Level 3 had CSRP intervention assignment and the dummy variables of matched Head Start sites.

A small number of children in the analysis sample had missing data on demographics (i.e., 2% on employment and 9% on education) or pre-intervention measures at baseline (11% to 17%). We adopted a multiple imputation method to address the issue of missing data and to increase the statistical power of analyses. Multiple imputation uses multiple predictions for each missing value of certain variables, based on the valid information of other variables and cases, to account for the uncertainty in imputed values (Hill et al., 2002; Rubin, 1987; Schafer, 1997). The prediction models included child, family, and classroom covariates, pre-intervention measures at baseline, outcome variables in third grade, CSRP intervention assignment, as well as the fixed effects of paired Head Start sites at baseline to account for any heterogeneity between paired Head Start sites. We also used a bootstrap method, which estimates regression coefficients in a bootstrap sample of the non-missing observations and thus has the advantage of robustness (Royston, 2005; van Buuren, Boshuizen, & Knook, 1999). To eliminate any uncertainty in SEL services or outcome variables in third grade, we kept the original data on these variables and only imputed the missing data on covariates (i.e., child and family demographics and pre-intervention measures at baseline). We generated five sets of imputations and as a result, the expected relative efficiency for recovering missing values ranged from 98.3% to 99.8% (Rubin, 1987). The final estimates were obtained using Rubin's (1987) rules for combining results from multiple imputation.

Furthermore, a large number of statistical tests on multiple outcomes across models may increase the probability of significant findings by chance. To address this issue of false discovery, we conducted a Benjamini-Hochberg test to limit the false discovery rate of estimates to no more than 10% (Benjamini & Hochberg, 1995; U.S. Department of Health and Human Services, 2012). In this procedure, the original p-values for the individual estimates were ranked from 1 to m, where m was the total number of estimates for the sets or “families” of outcome measures (i.e., SSRS, STRS, BIS, C-TRF, and ARS) across models. If a p-value was smaller than a calculated value equal to its rank position in the ordering multiplied by 0.05 and divided by m, the estimate was declared to pass the Benjamini-Hochberg test with a 10% false discovery rate.

Results

Descriptive Statistics

Table 1 shows the descriptive statistics (i.e., means with standard deviations in parentheses) of covariates in the full analysis sample (n = 414) and subsamples of CSRP intervention (n = 212) and control (n = 202) groups. T-tests were conducted to test the mean differences between CSRP intervention and control groups, with significant levels, if statistically significant, being indicated on the results of the CSRP intervention group.

Table 1.

Descriptive statistics of covariates by CSRP intervention assignment

	Full Analysis Sample (n = 414)	CSRP Intervention (n = 212)	CSRP Control (n = 202)
Child Variables
Boy	0.46 (0.50)	0.50 (0.50)+	0.42 (0.49)
Non-Hispanic black	0.69 (0.46)	0.69 (0.46)	0.68 (0.47)
Pre-intervention measures at baseline
Sociability	4.12 (1.30)	4.12 (1.27)	4.13 (1.33)
Affective competence	23.04 (3.37)	23.15 (3.24)	22.92 (3.50)
Impulse control	14.09 (4.11)	14.27 (3.94)	13.89 (4.29)
Internalizing behavior problems	2.28 (2.52)	2.55 (2.59)*	2.01 (2.42)
Externalizing behavior problems	5.74 (5.93)	6.30 (6.33)+	5.17 (5.45)
Early math skills	0.40 (0.20)	0.42 (0.19)	0.38 (0.22)
PPVT	0.43 (0.16)	0.44 (0.15)	0.42 (0.17)
SEL services in third grade	2.60 (1.64)	2.63 (1.59)	2.56 (1.69)
Baseline Parent and Family Covariates
Spanish as home language	0.20 (0.40)	0.16 (0.37)*	0.25 (0.43)
Less than high school education	0.23 (0.42)	0.22 (0.41)	0.25 (0.44)
Working ≤ 10 hours per week	0.41 (0.49)	0.40 (0.49)	0.41 (0.49)
Income below 50% poverty line	0.45 (0.50)	0.47 (0.50)	0.44 (0.50)
Baseline Classroom Covariates
Overall quality	4.55 (0.79)	4.37 (0.71)	4.79 (0.86)
Emotional climate	15.62 (2.89)	15.20 (3.04)	16.17 (2.71)
Student-staff ratio	7.06 (1.63)	7.00 (1.46)	7.13 (1.90)
Teacher with a bachelor degree	0.64 (0.49)	0.69 (0.48)	0.58 (0.51)
Teacher psychological distress	2.46 (1.92)	3.16 (1.68)*	1.54 (1.90)

Notes: means with standard deviations in parentheses; t-tests were used to test the mean differences between CSRP intervention and control groups, with significant levels, if statistically significant, being indicated on the results of the CSRP intervention group

^*p < 0.05

⁺p < 0.10.

As presented in Table 1, in the full analysis sample, 46% of the children were boys, 69% were non-Hispanic black children, and 20% spoke Spanish at home. Children in the sample faced high-level poverty related risks at baseline: 23% of children had mothers with less than high school education, 41% had mothers who worked for 10 hours or less per week, and 45% had family income below 50% of poverty line. In Head Start programs at baseline, the student-staff ratio was 7.12 and 58% of teachers had a bachelor degree. Overall, the distribution of covariates at baseline was balanced across the CSRP intervention and control groups. A few significant differences suggest that children in the CSRP intervention group were slightly more disadvantaged at baseline (e.g., speaking Spanish at home, having more behavior problems, and having teachers with more psychological distress) than children in the control group. The analysis included all covariates to account for any differences between the CSRP intervention and control groups. On average, children received 2.60 types of SEL activities (2.63 in the CSRP intervention group and 2.56 in the control group) in third grade.

Associations between SEL Services and Child Outcomes

Table 2 presents the results on the associations between SEL activities and children's social-emotional and academic outcomes in third grade. The table shows the coefficients with standard errors in parentheses from the three-level HLM analyses after combining the estimates in five datasets of multiple imputation (similar results were found in complete case analyses). These results were obtained from separate analyses for each of the outcome variables in each model. The significant findings that passed the Benjamini-Hochberg tests at a 10% false discovery rate were highlighted in bold. The results of all variables in the full model (i.e., Model 4 in Table 2) for the total scores of the outcome measures are presented in Appendix Table 2.

Table 2.

SEL services and social-emotional, behavioral, and academic outcomes

	Model 1	Model 2	Model 3	Model 4
Social skills (SSRS)	0.89** (0.35)	1.01** (0.34)	1.04** (0.34)	0.99** (0.34)
Cooperation	0.31* (0.15)	0.31* (0.14)	0.31* (0.14)	0.30* (0.14)
Assertion	0.45** (0.13)	0.47** (0.13)	0.49** (0.13)	0.47** (0.13)
Self-Control	0.14 (0.14)	0.23+ (0.13)	0.24+ (0.13)	0.22+ (0.13)
Student-teacher relationship (STRS)	1.26** (0.31)	1.24** (0.31)	1.29** (0.31)	1.23** (0.31)
Closeness	0.99** (0.18)	0.95** (0.18)	0.96** (0.18)	0.93** (0.18)
Conflict	−0.30 (0.21)	−0.34 (0.21)	−0.35+ (0.21)	−0.33 (0.21)
Impulsiveness (BIS)	−0.77** (0.31)	−0.74** (0.30)	−0.77** (0.30)	−0.74* (0.30)
Attentional impulsiveness	−0.21* (0.10)	−0.19* (0.10)	−0.20* (0.10)	−0.19* (0.10)
Non-planning impulsiveness	−0.45** (0.15)	−0.43** (0.14)	−0.44** (0.14)	−0.42** (0.14)
Motor impulsiveness	−0.07 (0.10)	−0.07 (0.10)	−0.08 (0.10)	−0.10 (0.10)
Behavior problems (C-TRF)	−0.46 (0.78)	−0.12 (0.73)	−0.30 (0.73)	−0.21 (0.73)
Internalizing	0.01 (0.17)	0.04 (0.18)	−0.02 (0.17)	0.00 (0.17)
Externalizing	−0.07 (0.30)	−0.01 (0.28)	−0.08 (0.28)	−0.07 (0.28)
Academic skills (ARS)	0.07* (0.03)	0.06* (0.03)	0.05+ (0.03)	0.06** (0.02)
Language and literacy	0.06* (0.03)	0.06* (0.03)	0.05+ (0.03)	0.06* (0.03)
Mathematical thinking	0.06* (0.03)	0.05 (0.03)	0.04 (0.03)	0.05 (0.03)

Notes: coefficients with standard errors in parentheses combined from the estimates of five datasets generated by multiple imputation; Model 1 includes SEL activities, Model 2 has additional child and family covariates as well as pre-intervention measures at baseline, Model 3 adds classroom and teacher covariates at baseline, and Model 4 includes additional variables of CSRP intervention assignment and the matched pairs of Head Start sites at baseline

^**p < 0.01

^*p < 0.05

⁺p < 0.10

Bold results indicate they passed the Benjamini-Hochberg tests for multiple comparisons with a 10% false discovery rate.

We found that overall SEL activities in third grade were significantly associated with teachers’ reports of children's social skills, impulsiveness, student-teacher relationship, and academic skills. The results were robust across models in terms of their magnitude and statistical significance. The increasingly added control variables from Models 2 to 4 only slightly changed, if any, the results in Model 1, which suggests that these covariates may not have played significant confounding roles in the relationships between SEL activities and child outcomes. Most of the findings passed the Benjamini-Hochberg tests. In the discussion below we focused on the results that were statistically significant at p < 0.05 in Model 4, which controlled for all covariates, and indicated the findings that did not pass the Benjamini-Hochberg tests.

As presented in Table 2, SEL activities were significantly related to children's developmental skills in third grade. These included positive changes in teachers’ reports of children's social skills (including its subscales of cooperation and assertion), student-teacher relationship (including its subscale of closeness), and academic skills (including its subscale of language and literacy). Specifically, the weekly/daily use of one additional type of SEL activities in class was positively associated with overall social skills of 0.99 points (in Model 4; effect size, calculated by the coefficient divided by the standard deviation in the control group, d = 0.09) as well as cooperation scores of 0.30 points (d = 0.06) and assertion scores of 0.47 points (d = 0.11). Similarly, the weekly/daily use of one additional type of SEL activities was positively associated with student-teacher relationship of 1.23 points (d = 0.12) and its subscale of closeness of 0.93 points (d = 0.16). The weekly/daily use of one additional type of SEL activities was also positively associated with children's academic skills and its subscale of language and literacy (which did not pass the Benjamini-Hochberg test), both of 0.06 points (d = 0.06).

Table 2 also shows that SEL activities were robustly associated with teacher-reported reductions in children's impulsiveness. In particular, the weekly/daily use of one additional type of SEL activities was negatively associated with children's overall score of impulsiveness of 0.74 points (d = −0.08) and its subscales of attentional impulsiveness of 0.19 points (d = −0.06, which did not pass the Benjamini-Hochberg test) and non-planning impulsiveness of 0.42 points (d = −0.09). No statistically significant associations were found between SEL activities and behavior problems or other measures of social-emotional and academic skills, including student-teacher conflict, motor impulsiveness, and mathematical thinking.

Effects of CSRP Intervention and Interaction with SEL Services

Table 3 shows the findings on the effects of CSRP intervention (in the second column) from Model 4 in Table 2 and the results from additional models that included the interaction term between SEL activities and CSRP intervention (in the last three columns).

Table 3.

Effects of SEL services, CSRP intervention, and their interactions

	Without Interactions		With Interactions
	SEL	CSRP	SEL	CSRP	Interaction
Social skills (SSRS)	0.99** (0.34)	1.21 (1.38)	1.58** (0.47)	4.28* (2.21)	−1.18 (0.76)
Cooperation	0.30* (0.14)	0.37 (0.57)	0.55** (0.20)	1.71+ (0.92)	−0.51+ (0.28)
Assertion	0.47** (0.13)	1.12* (0.52)	0.58** (0.18)	1.67* (0.83)	−0.21 (0.25)
Self-Control	0.22+ (0.13)	−0.27 (0.54)	0.45* (0.19)	0.91 (0.87)	−0.45 (0.36)
Student-teacher relationship (STRS)	1.23** (0.31)	0.40 (1.26)	1.70** (0.43)	3.01 (2.04)	−0.98 (0.61)
Closeness	0.93** (0.18)	0.99 (0.71)	1.01** (0.24)	1.43 (1.16)	−0.17 (0.34)
Conflict	−0.33 (0.21)	0.59 (0.86)	−0.70* (0.29)	−1.44 (1.38)	0.77 (0.51)
Impulsiveness (BIS)	−0.74* (0.30)	−0.58 (1.27)	−1.15** (0.43)	−2.70 (2.02)	0.81 (0.60)
Attentional impulsiveness	−0.19* (0.10)	−0.04 (0.40)	−0.31* (0.13)	−0.67 (0.64)	0.24 (0.19)
Non-planning impulsiveness	−0.42** (0.14)	−0.60 (0.60)	−0.67** (0.20)	−1.87* (0.95)	0.49 (0.38)
Motor impulsiveness	−0.10 (0.10)	0.31 (0.39)	−0.14 (0.13)	0.06 (0.63)	0.09 (0.19)
Behavior problems (C-TRF)	−0.21 (0.73)	−1.26 (2.95)	−0.98 (1.00)	−5.37 (4.74)	1.58 (1.43)
Internalizing	0.00 (0.17)	−0.78 (0.71)	−0.16 (0.24)	−1.65 (1.14)	0.33 (0.34)
Externalizing	−0.07 (0.28)	1.22 (1.12)	−0.39 (0.38)	−0.52 (1.80)	0.67 (0.54)
Academic skills (ARS)	0.06** (0.02)	0.17+ (0.10)	0.08** (0.03)	0.30+ (0.16)	−0.05 (0.05)
Language and literacy	0.06* (0.03)	0.13 (0.10)	0.09** (0.04)	0.31+ (0.17)	−0.07 (0.05)
Mathematical thinking	0.05 (0.03)	0.20+ (0.11)	0.04 (0.04)	0.19 (0.18)	0.00 (0.05)

Notes: results from models including all covariates (i.e., Model 4 in Table 2); coefficients with standard errors in parentheses combined from the estimates of five datasets generated by multiple imputation

^**p < 0.01

^*p < 0.05

⁺p < 0.10

Bold results indicate they passed the Benjamini-Hochberg tests for multiple comparisons with a 10% false discovery rate.

As presented in Table 3, overall, the results showed little evidence that the CSRP intervention had significant effects on children's social-emotional and academic outcomes in third grade. The CSRP intervention only showed a statistically significant effect of on assertion at p < 0.05 and marginally significant effects at p < 0.10 on mathematical thinking as well as overall academic skills, but none of them passed the Benjamini-Hochberg tests. Moreover, in the results from the models with the interaction term between SEL activities and the CSRP intervention, none of the coefficients of the interaction term, as presented in the last column of Table 3, were statistically significant at p < 0.05. Therefore, there was no evidence that SEL activities in third grade were a booster of the initial impact of CSRP intervention.

Discussion

This study examined the role of universal SEL services in third grade in supporting low-income children's social-emotional and academic outcomes. To answer the question of whether SEL services benefited children, we capitalized on longitudinal data from a sample of Head Start children initially enrolled in a randomly assigned preschool intervention, in which the majority of those children were then subsequently followed into early elementary school.

We found evidence that Head Start children's exposure to more SEL services in third grade was positively associated with their social-emotional and academic skills. In particular, the weekly/daily use of each additional type of SEL activity was associated with an effect size of 0.06-0.11 on social skills, 0.12-0.16 on student-teacher relationship, 0.06 on academic skills, and −0.06 to −0.09 on impulsiveness. While these effect sizes were relatively small, it is important to note that they were associated with just one type of SEL activities, while the “dose” of SEL activities in the sample was relatively high and thus benefits could be expected to be correspondingly larger. For example, children in the study sample on average received approximately 3 types of SEL activities, which suggests that the average effect sizes of SEL were 0.18-0.33 on social skills, 0.36-0.48 on student-teacher relationship, 0.18 on academic skills, and −0.18 to −0.27 on impulsiveness. These results aligned with the findings in recent reviews and meta-analyses (e.g., CASEL, 2012; Durlak et al., 2011) and randomized evaluations of individual programs (e.g., Aber et al., 2011; Jones et al., 2011), with effect sizes falling between those of universal SEL programs (approximately 0.2 to 0.3) and more targeted programs for high-risk students (approximately 0.5) (Jones & Bouffard, 2012).

This study's findings regarding the link between SEL activities in third grade and positive social-emotional and academic outcomes of Head Start children are positive news. However, the findings on the long-term benefits of social-emotional-oriented services delivered in preschool were not at all optimistic: We found no evidence for sustained benefits of the CSRP preschool intervention by third grade. There was no evidence suggesting that the benefits of SEL activities in third grade were moderated by having experienced a high-quality early childhood environment, which may be due to the limited statistical power of the moderation analysis with a relatively small sample size in detecting significant interaction effects between SEL activities and CSRP intervention. Given that early childhood interventions such as CSRP are often implemented with the hope of long-term educational benefit, these null findings are disappointing. One implication is that short-term gains such as those found in CSRP preschool years are fragile unless later elementary school settings are also supported through more intensive efforts to support emotional and instructional quality (see for example, My Teaching Partner as a promising model; Hamre & Pianta, 2006). Similarly, it is also important to conduct a follow-up study to examine whether the effects of SEL activities in third grade are more stable over time than preschool interventions and whether these interventions are worth the public investment if their effects wear off quickly. These issues need to be addressed in future research.

Our findings regarding the benefits of SEL in elementary school are also qualified by a set of study limitations. First, our descriptive findings suggested that most teachers reported using a surprisingly high number of SEL activities, given that other studies conducted within the same school system have found that many students report not feeling safe or supported by adults in their classrooms (Steinberg, Allensworth, & Johnson, 2011). As noted above, this may be partially due to the legislative requirements of providing SEL programs in school districts in Illinois where the CSRP was conducted. Meanwhile, teachers’ reports may have been colored by the social desirability of claiming to engage in activities that support students’ social-emotional wellbeing, such as peace promotion and conflict resolution. Importantly, our results suggest that, despite this risk, there was sufficient variance in teachers’ reports of SEL activities provided to still be of analytic use in addressing our hypotheses. It remains an interesting question to be explored further in future studies regarding how teachers and students view classroom and school social climate and emotional support in different ways.

A second key limitation is that we were substantially constrained in our ability to draw causal inferences from the findings. SEL activities in third grade were not randomly assigned across the classrooms and children surveyed in the follow-up study. The SEL activities were specified as those used by teachers since the beginning of the school year of third grade, while children's social-emotional and academic measures were assessed by teachers when the survey was conducted in the spring semester of third grade from April to June. In addition, the questions on teachers’ SEL activities and their assessments of individual children's social-emotional and academic development were asked in separate questionnaires for teachers, along with other sections in the third grade survey. Therefore, teachers might not have intentionally associated their reports of SEL activities with their assessments of individual children when they were interviewed for the survey. Moreover, due to budget constraints, the CSRP third grade survey did not include assessments from other sources besides teachers, such as independent assessors, children themselves, or parents, which may cross verify child development data and thus should be considered in further research. Therefore, the nature of cross-sectional data collection and the reliance on teachers’ reports of both SEL activities and child outcomes would make it difficult to establish a casual relationship between SEL activities and children's development in third grade. Further, teachers were asked how often they used the SEL activities with their class and in particular, teachers’ ratings were dichotomized to indicate whether they used these activities weekly or daily. It was unknown for how long teachers used these activities if they used them weekly or daily, which may contribute to the evaluation of SEL effectiveness as well.

With these caveats in mind, our findings provided valuable empirical support for the benefits for SEL services, given that we carefully compared similar children through complex matching and random assignment procedures earlier in their academic trajectories, and also carefully included children's prior levels of social-emotional and academic skills in preschool as important “controls” to account for selection bias. Overall, this study provided empirical evidence on the positive associations between universal SEL services in third grade and the social-emotional and academic skills of children who attended Head Start programs. These findings were aligned with a series of other observational studies as well as randomized experiments of “character education” that suggest substantial academic and social-emotional benefits for students when teachers and administrators focus on supporting a positive emotional climate of respect, improved conflict resolution, and violence prevention in addition to focusing on academic instruction (Brown et al., 2010; Jones & Bouffard, 2012). Together with experimental evaluations, findings such as ours can provide important implications for policymakers and help them make decisions on allocating scarce public funds to continuing enrichment programs such as SEL services, especially for low-income children.

Highlights.

• A sample of Head Start participants originally enrolled in a clustered randomized controlled trial (RCT) in preschool

• Focusing on the different doses of classroom-based social and emotional learning activities in third grade

• Frequent (i.e., weekly to daily) exposure to SEL opportunities in class was associated with favorable social-emotional and academic development in third grade

Appendix

Appendix Table 1.

Description of outcome measures in third grade

Measures	Purposes	Items in CSRP	Total and Subscales
Social Skills Rating System (SSRS)	Students’ positive and pro-social skills	30 items asked teachers to report the frequency (i.e., never, sometimes, and often) of specific behaviors a student had exhibited in the past month or two	Items were summed up into a total score (30 items; α = 0.95) as well as three subscales of cooperation (10 items; α = 0.93), assertion (10 items; α = 0.87), and self-control (10 items; α = 0.91). Higher scores indicate higher levels of social skills.
Student-Teacher Relationship Scale (STRS)	Student-teacher relationship patterns, including overall quality, closeness, and conflict	A 5-point Likert rating scale (from “definitely does not apply” to “definitely applies”) to assess a teacher's perceptions of relationship with the student, beliefs about the student's feelings toward him/her, and the student's interactive behavior him/her	Items were summed up into a total score (15 items; α = 0.88) and two subscales, including closeness (8 items; α = 0.85) and conflict (7 items; α = 0.92). Higher scores on closeness indicate a closer child-teacher relationship; higher scores on conflict mean more conflict between children and teachers. Higher total scores refer to better child-teacher relationship.
Barratt Impulsiveness Scale (BIS)	General impulsiveness, taking into account the multi-factorial nature of the construct	16 items asking the teacher to report impulsiveness of the child in the classroom context using a 4-point Likert rating scale (i.e., rarely/never, occasionally, often, and almost always/always)	Items were summed up into a total score (16 items; α = 0.92) and three subscales, including attentional impulsiveness (4 items; α = 0.83) to measure attention and cognitive instability, non-planning impulsiveness (7 items; α = 0.83) to assess self-control and cognitive complexity, and motor impulsiveness (5 items; α = 0.73) to measure motor control and perseverance. Higher scores mean higher levels of impulsiveness.
Caregiver-Teacher Report Form (C-TRF)	Teacher-reported Children's behavior problems	120 items asking the teacher to rate the child for how true each item of behavior problems was currently or within the past two months using a 3-point response scale from 0 (i.e., not true) to 2 (i.e., very true or often true)	Raw scores were summed into a total C-TRF score (120 items; α = 0.97) and two subscales of Internalizing (33 items; α = 0.88) and Externalizing (32 items; α = 0.94). Higher scores indicate more behavior problems.
Academic Rating Scale (ARS)	Children's academic skills reported by teachers	Teachers were asked to compare the target child to their same age peers on a 1-5 scale (i.e., not yet, beginning, in progress, intermediate, and proficient)*	A total ARS score (20 items in Cohort 1 and 13 items in Cohort 2) and two subscales of language and literacy (12 items in Cohort 1 and 7 items in Cohort 2) and mathematical thinking (8 items in Cohort 1 and 6 items in Cohort 2). Their reliability coefficients were between 0.93 and 0.97. Higher scores mean higher levels of academic skills.

Note: the psychometric information (i.e., Cronbach's alpha) was calculated using data of the analysis sample for this present study.

^*Due to administrative error, somewhat different sets of items of the ARS measures were administered to Cohort 1 (n = 226) versus Cohort 2 (n = 188) in the third grade wave of data collection. To minimize the impact of the inconsistency in ARS measures in the two cohorts, we used the average scores of items by cohort and controlled for the matched pairs of Head Start sites at baseline in the analysis, as detailed in the Method section, so that children with high versus low SEL exposure were compared to one another within the same cohort.

Appendix Table 2.

Full regression results of SEL and covariates

	SSRS	STRS	BIS	C-TRF	ARS
Child Variables
SEL activities	0.99** (0.34)	1.23** (0.31)	−0.74* (0.30)	−0.21 (0.73)	0.06** (0.02)
CSRP intervention	1.21 (1.38)	0.40 (1.26)	−0.58 (1.27)	−1.26 (2.95)	0.17+ (0.10)
Pre-intervention	0.84+ (0.48)	−0.02 (0.17)	0.29 (2.19)	1.76** (0.26)	0.05** (0.01)
Boy	−6.56** (1.09)	−5.13** (1.01)	5.41** (1.00)	12.29** (2.38)	−0.10 (0.08)
Non-Hispanic black	−3.99+ (2.04)	−1.81 (1.88)	0.65 (1.82)	2.04 (4.40)	−0.04 (0.15)
Baseline Parent and Family Covariates
Spanish as home language	1.09 (1.64)	0.09 (1.49)	−1.03 (1.46)	−3.44 (3.47)	0.02 (0.12)
Less than high school education	−1.84 (1.43)	−0.59 (1.26)	1.05 (1.27)	3.37 (3.01)	−0.27** (0.11)
Working ≤ 10 hours per week	0.38 (1.18)	−0.73 (1.08)	−0.15 (1.07)	−1.42 (2.66)	0.14 (0.09)
Income below 50% poverty line	−2.02+ (1.18)	−2.63* (1.08)	2.15* (1.07)	5.00* (2.55)	−0.20* (0.09)
Baseline Classroom Covariates
Overall quality	0.57 (1.48)	0.84 (1.35)	1.64 (1.36)	5.63+ (3.19)	0.18+ (0.11)
Emotional climate	−0.07 (0.42)	−0.33 (0.39)	−0.42 (0.39)	−1.41 (0.90)	−0.06+ (0.03)
Student-staff ratio	0.51 (0.47)	0.32 (0.43)	−0.07 (0.43)	0.23 (1.01)	0.07* (0.03)
Teacher with a bachelor degree	−3.27+ (1.76)	−1.65 (1.60)	1.26 (1.62)	7.85* (3.80)	−0.14 (0.13)
Teacher psychological distress	−0.64+ (0.37)	−0.56 (0.35)	0.53 (0.34)	0.90 (0.80)	−0.01 (0.03)
Matched Head Start Pairs	Yes	Yes	Yes	Yes	Yes

Notes: results of all covariates from Model 4 in Table 2; SSRS = Social Skills Rating System, STRS = Student-Teacher Relationship Scale, BIS = Barratt Impulsiveness Scale, C-TRF = Caregiver-Teacher Report Form, ARS = Academic Rating Scale; coefficients with standard errors in parentheses combined from the estimates of five datasets generated by multiple imputation

^**p < 0.01

^*p < 0.05

⁺p < 0.10