Cluster (School) RCT of ParentCorps: Impact on Kindergarten Academic Achievement

Abstract

OBJECTIVE:

To evaluate the impact of an early childhood, family-centered, school-based intervention on children’s kindergarten academic achievement.

METHODS:

This was a cluster (school) randomized controlled trial with assessments from pre-kindergarten (pre-k) entry through the end of kindergarten. The setting was 10 public elementary schools with 26 pre-k classes in 2 school districts in urban disadvantaged neighborhoods serving a largely black, low-income population. Participants were 1050 black and Latino, low-income children (age 4; 88% of pre-k population) enrolled in 10 schools over 4 years. Universal intervention aimed to promote self-regulation and early learning by strengthening positive behavior support and effective behavior management at home and school, and increasing parent involvement in education. Intervention included after-school group sessions for families of pre-k students (13 2-hour sessions; co-led by pre-k teachers) and professional development for pre-k and kindergarten teachers. The outcome measures were standardized test scores of kindergarten reading, writing, and math achievement by independent evaluators masked to intervention condition (primary outcome); developmental trajectories of teacher-rated academic performance from pre-k through kindergarten (secondary outcome).

RESULTS:

Relative to children in control schools, children in intervention schools had higher kindergarten achievement test scores (Cohen’s d = 0.18, mean difference = 2.64, SE = 0.90, P = .03) and higher teacher-rated academic performance (Cohen’s d = 0.25, mean difference = 5.65, SE = 2.34, P = .01).

CONCLUSIONS:

Early childhood population-level intervention that enhances both home and school environments shows promise to advance academic achievement among minority children from disadvantaged, urban neighborhoods.

What’s Known on This Subject:

At least half of the achievement gap for low-income, minority children is present at kindergarten entry; however, there are no population-level early childhood interventions that effectively engage and support families and teachers to ameliorate the impact of adversity on achievement.

What This Study Adds:

This study evaluated ParentCorps, a family-centered, school-based intervention to promote self-regulation and learning for all children entering school in disadvantaged, urban neighborhoods. ParentCorps results in higher kindergarten achievement among low-income, minority children.

Despite major policy initiatives, minority children from disadvantaged, urban neighborhoods remain at high risk for costly public health problems, including obesity, delinquency, and high school dropout.^1–4 Students from low-income families are 10 times more likely to drop out of high school than their more affluent counterparts,¹ and only half of black and Latino students in US public schools graduate high school.⁵ Sixty percent of minority children living in poverty fail to achieve proficient reading skills.⁶

At least half of the well-documented achievement gap for low-income, minority children is present in kindergarten.^7,8 There is considerable evidence that success begets success as foundational skills in one developmental period become tools for attaining success in other areas and in the future.⁹ Accordingly, early difficulties in academic achievement will likely undermine development in other domains, including psychological well-being and physical health.¹⁰

Specific aspects of the early home environment, including the quantity and quality of parent-child interactions, are among the factors that explain education and health disparities.^7,8,11–14 Although developmental pathways have not been fully articulated, there is substantial evidence that stress associated with poverty and social disadvantage interferes with positive parent-child relationships and effective behavior management,^13–17 disrupting healthy brain development, self-regulation, and early learning.^10,18–20 Self-regulation (across social-emotional and behavioral domains) is recognized as a core component of “school readiness” and is foundational for academic achievement and economic productivity.^21–26

Behavioral intervention for early childhood teachers has been shown to foster school readiness, including self-regulation.²⁷ In addition, preventive intervention for low-income families at the transition to parenthood results in long-term benefits for their children's learning, behavior and health.^21,28–31 Despite this evidence, there has been limited attention to the potential of a population-level approach that engages and supports parents as an essential component of early childhood education initiatives to reduce the achievement gap for low-income, minority children.

ParentCorps aims to buffer the adverse effects of poverty on early child development by engaging and supporting both parents and teachers at school entry. The goal is to promote child self-regulation and early learning by increasing positive behavior support (eg, nurturing parent-child interactions, reinforcement for competencies, proactive strategies), effective behavior management (eg, limit setting, consistent consequences for misbehavior), and parent involvement in education (eg, stimulation for learning, book sharing, parent-teacher communication) in home and early childhood education settings.

An initial randomized controlled trial (RCT) of ParentCorps with pre-kindergarten (pre-k) students from ethnically and socioeconomically diverse urban neighborhoods supported the feasibility of engaging families (at various levels of risk) in a series of 13 2-hour group sessions held at the school during early evening hours.³² Relative to those in control schools, parents of pre-k students in intervention schools provided more positive behavior support (including parent-child interactions observed in the home by evaluators masked to condition) and effective behavior management. Dose-response analyses showed a linear relation between number of group sessions attended and changes in parenting, and suggested that families would need to attend at least 5 of 13 sessions to make meaningful changes at home. The study design did not allow for evaluation of ParentCorps relative to education as usual, because pre-k teachers in all schools received professional development before randomization. Still, some important observations were made. Relative to students in control schools, pre-k students in intervention schools had better behavioral regulation at school according to their teachers. Consistent with other studies of school-based innovations,^33,34 the impact on child behavior was more than twice as large in the second year of implementation relative to the first year. This was interpreted as reflecting accumulated experience among teachers. Finally, for the subgroup of black families (∼40%), there were modest benefits on teacher-rated parent involvement in education and school readiness test scores.

Building on these findings, we designed a second cluster RCT of ParentCorps to serve as an essential replication of the short-term impact on the home environment and child self-regulation, and to rigorously evaluate impact on children’s learning. The current study evaluated the impact of ParentCorps, implemented at the level of the school, relative to education as usual, on individual academic achievement (reading, writing, and math). It was hypothesized that relative to children in control schools, children in intervention schools would have higher kindergarten achievement test scores. The study also examined intervention impact on developmental trajectories of teacher-rated academic performance over 2 school years, and considered whether impact varies by level of baseline school readiness skills, year of implementation, or level of family participation.

Methods

Participants

The trial was conducted in 2 school districts in highly disadvantaged minority and immigrant-dense New York City (NYC) neighborhoods with high school graduation rates of ∼50%.³⁵ To be included, schools were required to have a pre-k program (offered 1 year before kindergarten) with at least 2 classes and a student population >80% black and >70% low-income (eligible for free lunch). Nearly half of students in elementary schools meeting criteria scored below grade level in reading and math.³⁵ Principals of all 10 schools identified agreed to participate. All schools remained in their assigned condition and participated throughout the study period (2005–2010). Early childhood teachers in the 10 study schools provided written consent to participate.

The trial aimed to enroll all pre-k students in 4 consecutive years (2005–2008); the only inclusion criterion was having an English-speaking caregiver (7% ineligible). The study was introduced at pre-k orientation and parents provided written consent (children were not asked to assent given their age). Procedures were approved by the institutional review boards of New York University Langone Medical Center and the NYC Department of Education (DOE).

Intervention Conditions

NYC DOE pre-k and kindergarten programming (education as usual) was provided in intervention and control schools. In intervention schools, ParentCorps included (1) after-school group series for families of pre-k students (13 2-hour sessions co-led by pre-k teachers), and (2) professional development for pre-k and kindergarten teachers. The intervention aimed to increase the following strategies at home and school: (1) positive behavior support; (2) effective behavior management; and (3) parent involvement in education. Delivery of the family program at school by teachers was expected to strengthen parent-teacher communication and facilitate progress toward shared goals for children. The culturally-informed approach to behavior change was collaborative and nonprescriptive; focused explicitly on cultural values, beliefs, and norms; considered multiple stressors associated with urban disadvantage; and recognized a broad spectrum of strengths in families and schools (eg, traditional cultural values, strong commitment to children’s academic success).

Professional development included group-based activities (5 days in year 1; 2 days per year in years 2–4) to introduce strategies, and individual consultation (∼6 hours per year) to facilitate adoption of strategies. Teachers who chose to co-lead the family program received additional training (2 days per year), coaching (1 hour per week for 13 weeks), and financial compensation for after-hours work (4 hours per week for 13 weeks).

All pre-k students and their families were invited to participate in the group series; concurrent sessions for parents and children (∼15 members in each group) were typically held from 5 to 7 pm. Mental health professionals led parent groups using a range of techniques to promote adoption of strategies (eg, role play, problem solving, discussion about a ParentCorps video). In groups for pre-k students, teachers used puppets, play, positive behavior support, and a systematic approach to behavior management to promote children’s self-regulation skills. As in the initial RCT,³⁰ all intervention procedures were clearly specified in manuals and multimethod, multi-informant assessments of the intervention indicated that fidelity was high.

Outcomes

The primary outcome, achievement test scores, was assessed at the end of kindergarten. Trained study staff masked to intervention condition administered the Kaufman Test of Educational Achievement (KTEA) Brief Form, second edition.³⁶ The KTEA provides reliable estimates of reading, writing, and math achievement (mean = 100, SD = 15). To obtain a baseline indicator of “school readiness” (ie, language, concepts, motor development) predictive of achievement, masked study staff administered the Speed Diagnostic Indicators for the Assessment of Learning (DIAL) at the beginning of pre-k.³⁷

The secondary outcome, developmental trajectories of academic performance from pre-k through kindergarten, was assessed at the beginning and end of the pre-k and kindergarten years (repeated 4 times) with teacher report on a global rating of academic problems from a validated scale³⁸ and a rating of academic progress (1 = well below average to 5 = superior) developed for the study to serve as a proxy for grades. The composite performance rating was correlated with KTEA (r ≥ 0.55, all P < .001).

Sample Size Determination

The study sample size was selected based on NYC DOE data on expected class size (18 students per class, up to 4 classes per school) in schools meeting inclusion criteria and the literature on educational outcomes (intraclass correlation coefficients ≤0.10 for kindergarten achievement scores in low-income, urban schools; ∼50% of variance in achievement explained by school readiness).^39–41 The number of schools was selected to ensure 80% power for a 2-sided test with α = 0.05 to detect a meaningful effect of 0.33 SD for the primary outcome.⁴²

Randomization

To ensure approximately equal numbers of children in the 2 conditions, before randomization of schools (clusters), a statistician, unfamiliar with study objectives and uninformed of school identities, matched schools on size and split them into pairs; within each pair, one school was randomly assigned to intervention and the other to control.

Statistical Models and Methods of Analysis

Analyses were conducted on student-level data and based on the intent-to-treat principle. The intervention effect on achievement test scores was evaluated with a multivariate analysis of variance–type analysis using linear mixed effects models.⁴³ Scores were modeled as a function of intervention (2-level factor) and domain (3-level factor: reading, writing, math). Residuals between domains were allowed to correlate. To account for potential correlations among outcomes of children, random effects for classes and schools were included in the model. The effect on achievement was estimated from a model that included the main effects for intervention and domain, and adjusted for school readiness and year of implementation (1–4).

The effect on trajectories of academic performance was estimated using mixed effects models for longitudinal data. The postintervention ratings were modeled as a linear function of time, intervention, and their interaction, adjusting for baseline ratings. In addition to random intercepts and slopes for individual children, the model included random effects for classes and schools.

Secondary analyses considered baseline school readiness as a moderator of intervention, and whether impact varied by year of implementation and family participation (ie, dose). Dose-response analyses considered: (1) the relation between test scores and number of family sessions attended and (2) compared scores for children from intervention schools who received “full dose” (family attended ≥5 sessions) or “partial dose” (<5 sessions) to children in control schools.

SAS MIXED procedure (SAS Institute, Inc, Cary, NC) was used to fit all models. Effect sizes were computed as mean differences (estimated from the models) divided by the normative SD (SD = 15) for KTEA and the pooled baseline SD for teacher ratings.

Results

Participant Flow

Ten schools and 1280 children were assessed for eligibility and 1192 were eligible (Fig 1). The final sample included 1050 children, representing 88% of eligible pre-k students. Rates of enrollment were comparable by condition and year.

FIGURE 1.

Flow of schools and students from enrollment, randomization and follow-up.

Of the 1050 pre-k students enrolled, 92% were retained in the study in kindergarten (92% intervention, 93% control). Attrition from the study was largely due to children transferring to schools closer to their residence for kindergarten (6% in both conditions); <2% of children withdrew from the study (2% intervention, 1% control). Children retained in the study did not differ from those not retained (transferred or withdrew) on baseline demographic characteristics, school readiness, or teacher-rated performance. Ten percent of retained children were missing KTEA because they were absent on testing dates, but teacher ratings were available; 2.5% of retained children had missing data on both KTEA and teacher ratings. Missing data patterns did not differ by condition, and children with KTEA did not differ from retained children without KTEA on baseline measures.

Intervention Dose

Most (58%) families of pre-k students attended at least 1 session and 39% attended ≥5 sessions (full dose). Nearly 100% of pre-k and kindergarten teachers participated in professional development (DOE-sanctioned, during school hours) and most (>80%) pre-k teachers chose to co-lead family sessions. Thus, 61% of children in intervention schools received partial dose attributable to teacher participation in ParentCorps.

The percentage of families attending at least 1 session increased over the 4 years of implementation (50%, 59%, 59%, 65%); the percentage of families with full dose also increased (32%, 39%, 39%, 45%). Families of children with lower baseline school readiness scores attended at rates comparable to families of children with higher school readiness (P > .10).

Baseline Characteristics and Intervention Equivalence

As shown in Table 1, study schools served primarily low-income (72%), black (91%) students. Of the 170 pre-k and kindergarten teachers in the study schools, 53% were white, 27% black, and 9% Latino. Most (61%) study families of pre-k students were low-income (≤$38 700 for 4-person household). Most (85%) self-identified as non-Latino black (AfroCaribbean and African American), 10% Latino, and 4% other. Most caregivers were mothers (88%, mean age 33.9 years) and nearly half were single. One-third of households in neighborhoods where study families lived were low-income; most adults in these neighborhoods were black (85%) and single (67%). There were no differences between intervention and control conditions on any school, family, or neighborhood characteristic measured, including census tract data and masked observations of pre-k classroom climate and teacher behavior⁴⁴ (all P > .05; Table 1).

TABLE 1.

Baseline Demographic Characteristics by Condition

	Total, 10 Schools, %, n = 1050	Intervention, 5 Schools, %, n = 561	Control, 5 Schools, %, n = 489
Students in schools
Low-income (eligible for free lunch)	71.9	67.8	76.0
Black	90.7	89.2	92.2
Latino	6.2	7.2	5.2
Children and families
Child gender, male	49.3	49.2	49.5
Single parent	44.7	45.7	43.5
Parent unemployed	36.4	35.7	37.3
Low-income	60.8	65.3	55.1
Parent education ≤ high school diploma	46.5	49.6	42.7
Black	85.4	85.6	85.3
Latino	10.2	10.9	9.4
Neighborhoods of children and families
Single adults	67.0	69.3	64.3
Unemployed adults	9.5	10.1	8.9
Low-income households	36.5	40.8	31.5
Black	85.4	83.2	88.0
Latino	7.9	9.6	6.0

Low-income for families and neighborhoods is defined as <200% of the federal poverty threshold. There were no significant differences between the intervention and control conditions at baseline (all P > .05).

Study children (49% boys) were 4.15 years old (SD = 0.28) at pre-k entry and 5.74 years old (SD = 0.28) when tested (KTEA) at the end of kindergarten. There were no differences between conditions on any child characteristic, including school readiness test scores (DIAL) obtained at baseline by evaluators masked to condition (all P > .05; Table 2).

TABLE 2.

Intervention Effect on Kindergarten Academic Achievement Test Scores

	Mean Score From Raw Data		Model-Based Inference
	Intervention Mean (SD), n = 429	Control Mean (SD), n = 384	Difference (SE)	(95% CI)	P
Baseline test score (DIAL)	98.58 (12.71)	99.52 (13.38)	−1.04 (1.38)	(−3.75−1.67)	.45
Kindergarten test score (KTEA)	108.11 (13.50)	106.03 (14.12)	2.67 (1.21)	(0.29−5.05)	.03

CI, confidence interval; DIAL, Diagnostic Indicators for the Assessment of Learning.

School-related intraclass correlation coefficient (ICC) for DIAL = 0.05; ICC for KTEA = 0.09. The mean scores for KTEA are averages of reading, writing, and math. The model-based inference for KTEA is from modeling the 3 KTEA academic domain scores (reading, writing, and math) as a function of intervention and domain, adjusting for baseline DIAL school readiness test scores and year of implementation, and reflects the average effect of the intervention on all academic domains.

Intervention Effect on Kindergarten Achievement Test Scores

Relative to children in control schools, children in intervention schools had significantly higher kindergarten achievement test scores (reading, writing, and math; Cohen’s d = 0.18; P = .03; Table 2).

Intervention Effect on Teacher-Rated Academic Performance

There was an intervention effect on trajectories of academic performance (intervention-by-time F(1745) = 4.62, P = .03). By the end of kindergarten, ratings were higher for children in intervention schools relative to controls (difference = 5.65, SE = 2.23, Cohen’s d = 0.25, P = .01; Fig 2).

FIGURE 2.

Intervention effect on teacher-rated academic performance. The slope for children in control schools was a 2.79-point increase per year (SE = 1.72, P = .105), whereas the slope for children in intervention schools was a 7.91-point increase per year (SE = 1.64, P < .001). CI, confidence interval.

Baseline School Readiness, Year of Implementation, and Family Participation

Baseline school readiness did not moderate intervention impact on achievement test scores (reading, writing, math) or performance trajectories (all P > 0.05). Impact on achievement was related to both year of implementation and level of family participation, with the clearest associations for reading. Table 3 shows the stepwise increase in impact on reading scores with each year of implementation (all years: d = 0.34, P < .01; year 4: d = 0.49, P < .05). Dose-response analyses showed that the effect for reading increased by d = 0.04 (P < .01) with each family session attended. Impact on reading for children who received partial dose was d = 0.20 (P = .15), whereas impact on reading for children who received full dose was d = 0.60 (P < .0001) overall and d = 0.88 (P < .0001) by year 4.

TABLE 3.

Intervention Effect of Kindergarten Reading Achievement by Year of Implementation

	Mean Score From Raw Data		Model-Based Inference
	Intervention Mean (SD), n = 429	Control Mean (SD), n = 384	Difference (SE)	(95% CI)	P
KTEA Reading overall	113.59 (20.16)	110.38 (20.60)	5.16 (2.04)	(1.17−9.15)	.01
Implementation year
Year 1	111.44 (20.14)	112.47 (20.56)	1.65 (3.61)	(−5.43−8.73)	.65
Year 2	113.11 (19.16)	108.29 (20.62)	3.56 (3.20)	(−2.74−9.85)	.27
Year 3	114.28 (20.65)	109.46 (22.48)	5.46 (3.16)	(−0.74−11.67)	.08
Year 4	115.77 (20.78)	111.62 (18.71)	7.37 (3.49)	(0.52−14.21)	.03

Inference is based on a model that includes intervention, year of implementation, and intervention-by-year interaction as predictors, adjusting for baseline DIAL school readiness test scores; n = 813 for KTEA Reading (Years 1−4); n = 190 Year 1; n = 214 Year 2; n = 213 Year 3; n = 196 Year 4. The intervention effect increased with each year of implementation (F[1767] = 2.97, P = .09).

Discussion

A range of societal disparities have been traced to differences that begin in early childhood. Several early family-centered preventive interventions tested in rigorous experiments have been found to improve child health and well-being.^{28–31,45–47} However, the potential of engaging and supporting families to reduce the achievement gap for low-income, minority children has not been fully realized. Through a series of culturally-informed, integrated strategies for parents and teachers, ParentCorps aims to promote self-regulation and early learning. An initial RCT demonstrated impact on behavioral regulation at school.³² This second larger RCT in more disadvantaged neighborhoods found that, relative to children in control schools, children in intervention schools had higher kindergarten achievement test scores and more positive trajectories of academic performance from pre-k through kindergarten.

This cluster (school) RCT meets rigorous design and implementation standards.⁴¹ The high rates of enrollment and participation support the feasibility of a population-level approach in minority-dense, low-income, urban neighborhoods. The primary outcome was assessed with a standardized test of reading, writing, and math achievement, by staff masked to condition, and the magnitude of intervention impact was similar for achievement test scores and teacher-rated academic performance.

Because the ultimate goal of ParentCorps is to reduce or eliminate the achievement gap for minority, low-income children, effect sizes can be compared with established performance gaps for these groups.^48,49 The black-white achievement gap has been well documented to be one-half SD on kindergarten reading and math scores.^42,50,51 Other relevant performance benchmarks include differences in achievement between poor and nonpoor children (all d = 0.66–0.75) and between average- and low-performing schools (all d = 0.20–0.40).^48,49

Rigorously evaluated school-based interventions yield effect sizes of about one-quarter SD.^42,48 A meta-analysis of widely used comprehensive school reforms found smaller effects (all d = 0.09–0.15), with larger impact after 5 years of implementation (all d = 0.25–0.39).⁵² A more recent study of a school reform program for reading in early grades found effects between 0.21 and 0.36 on reading after 3 years of implementation.³³ Thus, the magnitude of ParentCorps impact on kindergarten achievement (all d = 0.18–0.25), reading (d = 0.34), reading in the fourth year of implementation (d = 0.49), and reading in the fourth year for children with full dose (d = 0.88) suggests the potential to improve on current efforts to reduce the achievement gap for low-income, minority children.

Although this study was not designed to disaggregate components of intervention, findings from dose-response analyses are consistent with ParentCorps’ emphasis on both parents and teachers as key agents of change. Replication of impact for an entire school population requires participation by parents and teachers at levels similar to those in this trial. Therefore, school policies that motivate and engage teachers and families are viewed as essential for successful implementation. We are collaborating with schools and policymakers to evaluate innovative strategies and to identity opportunities for reallocation of resources from remediation to preventive interventions.

There are several study limitations that deserve consideration. First, this RCT included a relatively small number of randomization units (schools), which increases the potential for type 1 error.⁵³ Second, although there was no evidence that children who left study schools were different from those who remained, they may have differed on unmeasured characteristics. Third, at least half of students in study schools are expected to perform below grade level by the end of elementary school and fail to complete high school.⁵⁴ Yet, study-administered achievement tests in control schools yielded scores within the average range, potentially reflecting benefits of universal pre-k. It will be important to continue to study the academic progress of children in control schools to fully interpret the impact of intervention and generalization of findings. Fourth, although ratings of academic performance are potentially biased because of teacher involvement in intervention, repeated measures over multiple grades allow for examination of trajectories. An ongoing follow-up study that repeats the achievement test and teacher ratings in later grades, and obtains DOE records, including yearly statewide achievement tests for all study children who remain in NYC public schools, will address some remaining questions. Finally, although 10% of the sample consisted of Latino children, non-English speaking families were excluded. Further attention should be paid to evaluating ParentCorps with non-English speaking, immigrant populations.

Conclusions

ParentCorps, a population-level approach to increase self-regulation and early learning by engaging and supporting parents and teachers in their roles, can contribute to reducing the achievement gap for low-income, minority children.

Glossary

DIAL

Diagnostic Indicators for the Assessment of Learning

DOE

Department of Education

KTEA

Kaufman Test of Educational Achievement, second edition

NYC

New York City

pre-k

pre-kindergarten