Gender Differences in the Benefits of an Influential Early Childhood Program

Abstract

This paper studies the life-cycle impacts of a widely-emulated high-quality, intensive early childhood program with long-term follow up. The program starts early in life (at 8 weeks of age) and is evaluated by an RCT. There are multiple treatment effects which we summarize through interpretable aggregates. Girls have a greater number of statistically significant treatment effects than boys and effect sizes for them are generally bigger. The source of this difference is worse home environments for girls with greater scope for improvement by the program. Fathers of sons support their families more than fathers of daughters.

1. Introduction

This paper examines impacts by gender of two closely related influential early childhood programs: the Carolina Abecedarian Project (ABC) and its sister program, the Carolina Approach to Responsive Education (CARE), henceforth ABC/CARE. Both were evaluated by the method of random assignment. While specific outcomes of ABC/CARE have been studied previously, our paper is the first to aggregate and summarize all of the reported outcomes to evaluate the program.

ABC/CARE was conducted in Chapel Hill, North Carolina for a sample of children born between 1972 and 1980. This pioneering program focused on improving the early years of disadvantaged children. It is a template for many current and proposed early childhood programs.¹ It started at 8 weeks of age and continued through age 5. Treatment and control children were followed through their mid 30s, with data collected on multiple dimensions of human development with over 100 reported program outcomes.²

There are pronounced gender differences in the treatment effects of ABC/CARE. To avoid cherry-picking, we analyze aggregates of treatment effects as reported in Table 1. The aggregates are constructed by counting the proportion of outcomes by category for which male mean treatment effects for items in a category equal or exceed female treatment effects. To understand the entries to the table, consider the row for the outcome “employment.” In the control group, all mean outcomes are larger for males compared to females. This is denoted by the fraction 1.000. In the table, we also report an exact, non-parametric test of the null hypothesis that the distribution of employment outcomes in the control group is the same for males and females. The p-value associated with the test is 0.117.³ The comparable statistics for the treatment group are 0.75 and a p-value of 0.080, respectively.

Table 1:

Summary of Gender Differences in Outcome Aggregates

[1]	[2]	[3]	[4]	[5]	[6]	[7]	[8]
Category	# Outcomes	Control		Treatment		Treatment - Control	Overall
		Proportion Male > Female	H0: Male = Female p-value	Proportion Male > Female	H0: Male = Female p-value	[5] - [3]	H0: Male = Female p-value
IQ	15	0.733	0.290	0.533	0.228	−0.200	0.000
Achievement	12	0.833	0.065	0.000	0.341	−0.833	0.303
Socio-Emotional	22	0.455	0.191	0.364	0.599	−0.091	0.165
Parenting	7	0.571	0.977	0.286	0.142	−0.286	0.477
Parental Income	8	0.750	0.290	0.875	0.230	0.125	0.076
Education	9	0.667	0.191	0.111	0.142	−0.556	0.076
Employment	4	1.000	0.117	0.750	0.080	−0.250	0.030
Crime	4	0.000	0.000	0.000	0.000	0.000	0.000
Risky Behavior	5	0.400	0.000	0.200	0.080	−0.200	0.000
Health	22	0.545	0.003	0.545	0.000	0.000	0.003
Mental Health	11	0.818	0.191	0.545	0.599	−0.273	0.165

Note: Column [1] lists the outcome category. The specific outcomes in each category are listed in Appendix B. Column [2] lists the number of outcomes observed within each category. Column [3] lists the proportion of outcomes for which the average for males is greater than the average for females for the control group. 1.000 denotes that all mean outcomes across category are greater for males than for females. 0.000 denotes the opposite. A proportion is bold when it statistically significantly differs from 0.50 at the 10% level, so both 1.000 and 0.000 could be statistically significant. The relevant null hypothesis is 0.50 at which point neither males nor females outperform the other. For inference, we use the Wilcoxon signed-rank test comparing the proportions over 1,000 bootstraps. For bootstrap b ∈ [1,…, 1000] we compute the proportion of variables in each outcome category for which the male average is greater than the female average and obtain a non-parametric p-value from this procedure. Column [4] displays the p-value for an exact, non-parametric test due to Rosenbaum (2005) for the null hypothesis that the joint distribution of the outcomes within each category is the same for males and females in the control group. Column [5] is analogous to column [3] for the treatment group. Column [6] is analogous to column [4] for the treatment group. Column [7] is the difference between columns [5] and [3] and the inference procedure is analogous to that in column [3]. Column [8] is analogous to column [4] but performs an exact test across genders pooling the control and treatment groups.

Treatment reduces gaps in these aggregates between males and females. The difference between the male-female gap for treatments and the male-female gap for controls is 0.25. We decisively reject the null of equality of the pooled male and pooled female distributions. This pattern holds generally for the outcomes that we study.⁴ Females benefit more than males from treatment, which reduces the male-female gaps in the controls. However, males also benefit substantially from the program.

Differential treatment effects by gender arise because control-group girls grow up in less favorable environments compared to control-group boys. Specifically, in the homes of girls, fewer fathers are present and maternal human capital is lower. Girls in the control group who stay at home (27% of the control-group girls), were raised in more disadvantaged environments. Girls in the control group who went to preschools other than ABC/CARE (73% of the control-group girls), likely went to lower-quality preschools. Their families were more resource constrained compared to their male counterparts for whom more fathers are present.⁵ Girls benefited more from treatment because without it they would have grown up in more disadvantaged environments.⁶

Parents of the controls in ABC/CARE had the option of keeping their children at home or sending them to daycare facilities other than ABC/CARE. There is an important difference in the take-up of alternatives by the gender of controls. Among control-group girls, those from more disadvantaged families stay at home. Among control-group boys, the more advantaged stay at home instead of attending lower-quality alternative childcare. Boys benefit more from participating in ABC/CARE when compared to attending alternative preschools because of their relatively better home environments. Girls benefit more from ABC/CARE when compared to home-based care because those who stay at home are more disadvantaged.

A companion paper, García et al. (2018), presents a cost-benefit analysis of ABC/CARE that monetizes program treatment effects. While a greater number of statistically significant treatment effects is found for girls, the monetized values of these effects are greater for boys. Untreated boys can do more costly harm, which the program prevents.

This paper unfolds in the following way. Section 2 describes the experimental data we analyze. We document the take-up of alternative out-of-home childcare attended by many control-group subjects. Section 3 defines the treatment effects we estimate and how we summarize them. Section 4 reports the treatment effects overall and by gender. We show sharp gender differences for many categories of outcomes. Section 5 discusses the sources of these differences. Section 6 summarizes and places our analysis in the context of a broader literature.

2. Data

We analyze a combined sample of the two closely related programs: ABC and CARE. Table 2 summarizes their main features. Both interventions were implemented by researchers at the Frank Porter Graham Center (FPG) at the University of North Carolina Chapel Hill, and targeted children from disadvantaged families in the Chapel Hill area. ABC had four cohorts born between 1972 and 1977, and CARE had two cohorts born between 1978 and 1980. Eligibility was determined on the basis of a High Risk Index (HRI) developed for ABC and adapted for CARE (Ramey and Smith, 1977; Wasik et al., 1990). Components of the HRI include father’s presence, parental employment, and participation in welfare.⁷ Based on these eligibility requirements, García et al. (2018) calculate that 43% of African Americans were eligible during the period of intervention and that 19% of all African American children are eligible now.

Table 2:

Overview of the ABC and CARE Programs

Site	Chapel Hill, North Carolina
Cohorts	4 (ABC), 2 (CARE)
N	58 treatment, 56 control (ABC)
	17 treatment, 23 control (CARE)
Eligibility	High Risk Index (HRI) > 11
	Biologically healthy
Treatment years	1972–1981 (ABC), 1977–1983 (CARE)
Treatment duration	5 years
Home visits	2.5–2.7 per month (CARE)
Center care	50 weeks per year
	30–45 hours per week
Other treatment components	Formula until 6 months
	Diapers until 6 months
	Health check-ups
	Medical care
	Parenting instruction
	Counseling
	Transportation to center
Control-group incentives	Formula until 6 months
	Diapers until 6 months
	Health check-ups until 1 year (ABC, cohort 1)
Adult-child ratio	1:3–1:6
Teacher requirements	High school through masters
	Experience with children
Specialists	Physician, nurse, social worker

Note: Biologically healthy includes lack of serious illness, including mental retardation.

Sources: Ramey et al. (1976); Ramey and Smith (1977); Ramey et al. (1985); Wasik et al. (1990); Ramey and Campbell (1991).

Both interventions involved intensive center-based care for subjects in the treatment group starting at 8 weeks and continuing until age 5 before the children started kindergarten. In addition to free access to this center-based care, treatment-group subjects also received daily health screenings, diapers, and formula for 6 months. Control-group families received diapers and formula as well for the same period of time.⁸ Between ages 5 and 8, there was an additional component of treatment with home visits to tutor the children and to encourage families to be involved in their child’s schooling. In CARE, all the subjects who received center-based care also received this school-age component. In ABC, treatment status of this component was randomized. We do not analyze the post-5 home-based visits because previous work has found that this component of treatment has no statistically significant impact (Campbell et al., 2002, 2014).

The program focused on developing language, cognition, and social-emotional skills.⁹ Program curricula emphasized child-led learning of skills important for future learning (Ramey and Smith, 1977; Wasik et al., 1990; Ramey and Campbell, 1991). The teachers and classroom aides were trained throughout the intervention. Researchers and child development experts at FPG observed classroom interactions and gave detailed feedback to the instructors (Ramey et al., 2012).¹⁰

CARE included an additional arm of treatment. Besides the services offered in ABC, those in the CARE treatment group also received home visiting from birth to age 5. Home visiting consisted of biweekly visits focusing on improving parental problem-solving skills. To test the effectiveness of this home-visiting component, there was a third randomized group (N = 23) that received only the home visiting component at ages 0-5, but not center-based care (Wasik et al., 1990). In light of previous analyses of CARE finding no effect of the early-age home-visiting component, we drop this last group from our analysis.¹¹ These analyses justify merging the treatment groups of ABC and CARE. We henceforth analyze the samples as coming from a single ABC/CARE program.

Table 3 compares pre-program variables for experimental and gender groups. The only marginally statistically significant difference in baseline variables between boys and girls is in the HRI score, which is 1.78 points lower for males than for females, consistent with their better home environments. A higher HRI is associated with greater disadvantage.

Table 3:

Baseline Differences, ABC/CARE

Variable	Female Mean	Male Differential	p-value	Control Mean	Treatment Differential	p-value
Mother’s age	19.72	1.13	0.15	20.52	−0.51	0.51
Mother works	0.23	0.09	0.28	0.21	0.11	0.18
Mother’s IQ	84.46	1.33	0.46	84.65	0.98	0.58
Father at home	0.24	0.04	0.56	0.29	−0.05	0.51
Number of siblings	0.59	0.07	0.67	0.71	−0.18	0.29
HRI score	21.57	−1.78	0.06	21.39	−1.47	0.13
Apgar score, 1 min.	7.68	−0.07	0.80	7.60	0.09	0.76
Apgar score, 5 min.	8.94	−0.20	0.33	8.87	−0.04	0.83
Birthweight	7.18	−0.20	0.34	7.17	−0.19	0.38
Gestational age	39.85	−0.42	0.27	39.87	−0.50	0.19

Note: The variables in this table are all measured at baseline, close to when the children were born. Maternal labor supply (“Mother works”) is represented using an indicator variable. A larger HRI (High Risk Index) score indicates more disadvantage. Apgar, measured at 1 and 5 minutes after birth, is a test of the health condition of newborn babies. A score closer to 10 indicates a healthier condition (Apgar, 1966). Birthweight is in pounds and gestational age is in weeks. Control means and treatment differentials pool males and females.

2.1. The Randomization Protocol

Randomization for ABC/CARE was conducted on child pairs matched on family background. Siblings and twins were jointly randomized into either treatment or control groups. For siblings, this only occurred when two siblings were close enough in age so that both of them were eligible for the program. Pairing was based on the High Risk Index, as well as maternal education, maternal age, and gender of the subject.¹²

ABC collected an initial sample of 121 subjects. All providers of health care and social services (referral agencies) in the area of the ABC/CARE study were informed of the programs. They referred mothers whom they considered disadvantaged. Eligibility was corroborated before randomization. Encouragement from the referral agencies was such that all but one of the referred mothers agreed to participate in the initial randomization.¹³ We discuss the pattern of missing observations in Appendix A.3. In Appendix D.3, we document that our estimates are robust when we adjust for missing data using standard weighting methods described in Appendix B.2.

Twenty-two subjects in ABC did not stay in the program through age 5. The number of dropouts is evenly balanced across treatments and controls. Dropping out was primarily related to the health of the child and the mobility of families rather than as a result of dissatisfaction with the program. The 22 dropouts include four children who died, four children who left the study because their parents moved, and two children who were diagnosed as developmentally delayed.¹⁴ Details are in Table A.2. Everyone offered the program was randomized to either treatment or control. Dropping out occured after randomization and was balanced across treatment groups. We conduct the same analysis for the CARE sample, although there were far fewer dropouts and no compromised randomization.¹⁵

2.2. Control Group Substitution

In ABC/CARE, many control-group subjects (but no treatment-group subjects) attended alternative center-based care.¹⁶ The figure is 75% for ABC and 74% for CARE. This information comes from a survey administered to ABC/CARE families asking about the childcare arrangements made during each month between birth and age 5. Home care includes parental care, as well as care of a relative, neighbor, or friend. The survey also captured specifics, such as the name of the center-based institutions, allowing for a detailed understanding of alternative care environments.

Figure 1a shows the cumulative distribution of the proportion of time in the first five years that control subjects were enrolled in alternative formal childcare programs. Figure 1b shows the dynamics of enrollment. Those who enrolled generally stay enrolled over time. As control-group children aged, they were more likely to enter childcare.¹⁷

Figure 1:

Control Substitution Characteristics, ABC/CARE Control Group

Note: Panel (a) displays the cumulative distribution function of enrollment in alternatives. Panel (b) displays the fraction of ABC/CARE control-group children enrolled in alternatives, conditional on being enrolled in the previous age (at least one month).

As a group, the children in the control group who enroll in alternative early childcare programs are less economically disadvantaged at baseline compared to children who stay at home, although, as we show below, there are important differences by gender of the child. Disadvantage is measured by maternal education, maternal IQ, Apgar scores, and the High Risk Index that defines ABC/CARE eligibility. Control children who attend alternative formal care generally have fewer siblings. On average, they are children of mothers who are more likely to be working at baseline (statistically significant at 10%). Parents of girls are much more likely to use alternative center childcare if assigned to the control group.

Table A.4 tests differences across these variables between children in the control group who attended and those who did not attend alternative childcare. The only statistically significant difference in observed baseline characteristics between the controls who use formal childcare and those who stay at home is whether the mother works. The mother is more likely to work at baseline for children attending alternative care. We discuss selection into alternative care settings by gender of the child in Section 5.

While most of the alternative childcare centers received federal subsidies and were subject to the federal regulations of the era, they were relatively low-quality compared to ABC/CARE (Burchinal et al., 1989). In terms of child-staff ratios, ABC/CARE far exceeded the highest state and federal standards of the day.¹⁸ We do not have baseline information on the quality of the parent-child interaction to be able to precisely analyze the environments of those who stayed at home.

When we compare ABC/CARE treatment to that from alternative preschools, ABC/CARE produces substantial treatment effects. Parents perceived that ABC/CARE was superior to alternative preschools because all offered it chose to participate in it. The access of control-group children to alternative programs creates both a problem of substitution bias and an opportunity to learn about the benefits or harms of lower-quality childcare arrangements. Exploiting the heterogeneous experiences of the control-group subjects, we isolate the effects of developmentally enriched environments compared to home environments, and the effects of lower-quality center environments compared to home environments for boys and girls.

2.3. Data Collection

Measures of cognitive, social-emotional, and parenting skills were collected during the intervention and while the subjects were in school.¹⁹ The researchers collected information on the subjects’ academic performance including grade retention and special education. The adult surveys (at ages 21 and 30) cover items related to employment, post-secondary education, health, criminal activity, and family structure. When the subjects were in their mid 30s, the researchers collected administrative crime data and conducted a full medical survey. Appendix D describes the data that we use more completely.

3. Parameters of Interest

Random assignment to treatment does not guarantee that conventional treatment effect estimators answer policy-relevant questions. We define and estimate three parameters that address different policy questions. Let W = 1 indicate that the parents referred to the program participate in the randomization protocol. W = 0 indicates otherwise. R indicates randomization into the treatment group (R = 1) or to the control group (R = 0). D indicates participation in the program, i.e., D = R implies compliance with the initial randomization protocol.

Individuals are eligible to participate in the program if baseline background variables $\mathit{B}\in {\mathcal{B}}_0$. ${\mathcal{B}}_0$ is the set of scores on the risk index that determines program eligibility. Because all of the eligible people given the option to participate choose to do so (W = 1 and D = R), we can safely interpret the treatment effects generated by the experiment as average treatment effects for the population for which $\mathit{B}\in {\mathcal{B}}_0$ and not just treatment effects for the treated.²⁰

Let $Y_j^1$ be the outcome j for the treated, and $Y_j^0$ be the control counterfactual. $Y_j^0$ depends on the exposure to various alternative preschools while ABC/CARE was active (i.e., it depends on the degree of control substitution).²¹ The index set for the outcomes is $\mathcal{J}$, which we can partition by outcome category ( ${\mathcal{J}}_{\ell }$ with $\mathcal{J}=\bigcup _{\ell \in \mathcal{L}}{\mathcal{J}}_{\ell }$ and $\mathcal{L}$ indexes categories).

All treatment-group children had the same exposure to the ABC/CARE treatment and no exposure to alternative center-based care.²² It would be desirable to identify and estimate parameters evaluating ABC/CARE against all possible levels of exposure to alternative preschools, but our samples are too small to credibly do so. We simplify the analysis of the counterfactual to ABC/CARE by creating two categories. “H” indicates that the control-group child stays at home throughout the entire length of the program. “C” indicates that a control-group child is in alternative center childcare for any amount of time.²³ We thus compress a complex reality into two counterfactual outcome states for each outcome j:

$$\begin{array}{cc}\hfill Y_{j,H}^0& :\text{Subject received home care exclusively}\hfill \\ \hfill Y_{j,C}^0& :\text{Subject attended some alternative preschool.}\hfill \end{array}$$

One parameter of interest addresses the question: What is the effect of the program as implemented? This is the effect of the program compared to the next best alternative as perceived by the parents (or the relevant decision maker) and is defined by

$${\Delta }_j≔\mathbb{𝔼}[Y_J^1-Y_j^0\mid W=1]=\mathbb{𝔼}[Y_j^1-Y_j^0\mid \mathit{B}\in {\mathcal{B}}_0],$$ (1)

where the second equality follows because everyone who was eligible elected to participate in the program. For the sample of eligible people, this parameter addresses the effectiveness of the program relative to the perceived best (by the agent) quality of all available alternatives when the program was implemented, including staying at home. This is the Local Average Treatment Effect (LATE).²⁴

We define V as a dummy variable indicating that the control-group child attended alternative center-based childcare. V = 0 denotes that the control-group child stayed at home. The outcome when a child is in the control group is

$$Y_j^0≔(1-V)Y_{j,H}^0+(V)Y_{j,C}^0.$$ (2)

It is fruitful to assess the effectiveness of the program compared to a condition in which the child stays at home full time. The associated causal parameter is:

$${\Delta }_j(\text{Fix}V=0)≔\mathbb{𝔼}[Y_j^1-Y_j^0\mid \text{Fix}V=0,W=1]=\mathbb{𝔼}[Y_j^1-Y_j^0\mid \text{Fix}V=0,\mathit{B}\in {\mathcal{B}}_0].$$ (3)

It is also useful to assess the average effectiveness of a program relative to attending an alternative preschool with associated causal parameter:

$${\Delta }_j(\text{Fix}V=1)≔\mathbb{𝔼}[Y_j^1-Y_j^0\mid \text{Fix}V=1,W=1]=\mathbb{𝔼}[Y_j^1-Y_j^0\mid \text{Fix}V=1,\mathit{B}\in {\mathcal{B}}_0].$$ (4)

“Fix V” means V is fixed to the designated value.²⁵

Random assignment to treatment does not directly identify the parameters in Equations (3) or (4). Econometric methods are required. In this paper, we rely on matching to control for selection into home or an alternative preschool by the control group. We assume that observed characteristics are sufficient to describe the selection into alternative center-based arrangements. In Appendix D, we show the balance across the groups in the matched samples along the observed selection variables (e.g., family characteristics, Apgar scores, gender), further justifying this approach.²⁶

We report estimates from alternative empirical strategies, including instrumental variables and control functions, in Appendix E. The estimates from these alternative estimation strategies are consistent with results from matching but lack precision. Appendix D.13.1 displays results with alternative definitions of V (i.e., different thresholds define if a child attended alternative preschool). The results are robust to the various definitions. What matters is whether any center-based child care is being used (V > 0)—not the specific exposure time above a zero value.

3.1. Summarizing Multiple Treatment Effects

The extensive data for ABC/CARE generate many outcomes that we can use to evaluate the program. Summarizing these effects in an interpretable way is challenging.²⁷ We present effect sizes averaged over outcomes. We also construct combining functions that count the proportion of treatment effects that are positive for different categories of outcomes. In a similar fashion, we study the count of the proportion of treatment effects that are positive and statistically significant at the 10% level. We complement these analyses by applying an exact non-parametric test on the equality of the distributions of outcomes across treatment groups developed in Rosenbaum (2005).

Combining Functions.

Consider a block of outcomes ${\mathcal{J}}_{\ell }$, ℓ ∈ {1,…, L}, with cardinality D_ℓ, and associated treatment effects Δ₁,… Δ_Dℓ. We assume that outcomes can be ordered so that Δ_j > 0 is beneficial.²⁸ The count of positive treatment effects within block ${\mathcal{J}}_{\ell }$ is:

$$C_{\ell }=\sum _{j=1}^{D_{\ell }}1({\Delta }_j>0).$$ (5)

The proportion of beneficial outcomes, our combining function, is C_ℓ/D_ℓ. In our empirical analysis we consider the outcomes as a block. Different blocks are grouped by common categories (e.g., employment, health, crime).

Under the null hypothesis of no treatment effect for the block of outcomes indexed by ${\mathcal{J}}_{\ell }$ and assuming the validity of asymptotic approximations, the mean of C_ℓ/D_ℓ is centered at $\frac{1}{2}$.²⁹ We compute the fraction C_ℓ/D_ℓ and the corresponding bootstrapped empirical distribution to obtain a p-value. The bootstrap procedure accounts for dependence in unobservables across outcomes (within blocks) in a general way.

A test based on the number of outcomes for which the treatment effect is statistically significant at the 10% level produces similar inference. Under the null hypothesis, 10% of all outcomes should be “significant” at the 10% level even if there is no treatment effect of the program.³⁰ The combining functions avoid: (i) arbitrarily picking outcomes that have statistically significant effects—“cherry picking”; or (ii) arbitrarily selecting blocks of outcomes to correct the p-values when accounting for multiple hypothesis testing. We present p-values for these hypotheses and a number of combining functions by outcome category in Appendix D.³¹

An Exact Non-Parametric Test.

We also test for equality of treatment and control distributions by outcome using an exact test developed by Rosenbaum (2005). We provide a brief explanation of the test and refer interested readers to the original source for more details. Let $\mathcal{N}$ index the individuals in our sample and consider the block of outcomes ${\mathcal{J}}_{\ell }$. Let d_ij be the distance between the individuals i, j ∈ $\mathcal{N}$, i ≠ j, based on the outcomes in ${\mathcal{J}}_{\ell }$. In our application, this is the Mahalanobis distance.³² There is an optimal non-bipartite pairing of individuals according to d_ij.³³ This is obtained by minimizing the distance across all possible pairings i,i′ in the sample.

Under the null hypothesis of no treatment effects, pairings of treatment-group children with control-group children should be as frequent as pairings of treatment-group children with other treatment-group children and control-group children with other control-group children. If a relatively large number of pairs are matched equally across groups using this metric, we fail to reject the null hypothesis that the joint distribution of outcomes in block ${\mathcal{J}}_{\ell }$, ℓ ∈ {1,…, L} is the same across the treatment and control groups.

The number of treatment-control pairings in the optimal non-bipartite pairing within the block of outcomes ${\mathcal{J}}_{\ell }$, denoted by A_ℓ, ℓ ∈ {1,…, L}, is a summary statistic allowing us to test the null hypothesis of interest. Its exact p-value can be calculated. Asymptotically, the studentized value of A_ℓ follows a standard normal distribution. For each block, we present these asymptotic p-values to complement the information provided by the combining functions.³⁴

4. Estimates and Tests of Differences in Treatment Effects

This section presents our estimates of treatment effects by gender. We categorize outcomes and present estimates of treatment effects pooled within each category. Treatment effects for individual outcome variables are listed in Appendix D.

Table 4 aggregates treatment effects across all ages and within categories. The benefits of treatment are noticeable for both males and females. Benefits appear across the life cycle and across multiple outcomes. Participants in ABC/CARE benefit in terms of both cognitive and socio-emotional skills. They also benefit in terms of scores on achievement tests, which help measure both cognitive and non-cognitive skills.³⁵ These estimates reveal a clear female advantage in the program’s effect on skill development.

Table 4:

Combining Functions and Exact Non-Parametric Tests

	Average Effect Size	% > 0 Treatment Effect	% > 0, Significant Treatment Effect	Rosenbaum (2005) p-value
IQ
Females	0.719	100.000	100.000	0.046
Males	0.664	100.000	85.714	0.045
Achievement
Females	0.672	100.000	100.000	0.046
Males	0.235	100.000	40.000	0.086
Social-emotional
Females	0.385	92.857	71.429	0.235
Males	0.059	50.000	21.429	0.147
Parental Income
Females	0.283	100.000	37.500	0.086
Males	0.157	100.000	25.000	0.147
Parenting
Females	0.274	100.000	100.000	0.602
Males	0.060	80.000	0.000	0.147
Education
Females	0.356	83.333	66.667	0.000
Males	0.174	83.333	16.667	0.235
Employment
Females	0.200	100.000	50.000	0.151
Males	0.438	100.000	100.000	0.022
Crime
Females	0.242	100.000	100.000	0.715
Males	−0.093	33.333	0.000	0.812
Risky Behavior
Females	0.099	100.000	0.000	0.469
Males	0.011	25.000	25.000	0.086
Health
Females	0.060	68.750	6.250	0.046
Males	0.061	73.333	420.000	0.000

Note: This table displays summaries of treatment effects by outcome category and gender. Each of the panels contains statistics calculated using outcomes grouped by category. The average effect size is calculated by averaging over the effect size of the outcomes in the outcome category. The effect sizes of the individual outcomes are calculated by dividing the treatment-control mean difference by the standard deviation of the control group. We present bootstrapped p-values. For the proportion of outcomes that are positive and significant, we do a “double bootstrap” procedure. The null hypothesis for the average effect sizes is that they are 0. The null hypothesis for the proportion of outcomes that are (significantly) positive is that they are (10%) 50%. The Rosenbaum (2005) p-value tests the null of equality of pooled outcome treatment and control distributions within each category. For computational simplicity, we approximate the exact p-values using asymptotic p-values. Rosenbaum (2005) presents several simulation exercises showing that the validity of this approximation. Statistics significant at the 0.10 level are bolded.

ABC/CARE offered full-day child care for participants and thus facilitated maternal employment and education.³⁶ The program has a sizable effect on education. The effect size is 0.356 for females — about twice that for males, 0.174. The program enhanced parental income. The program enhanced parenting as measured by HOME scores for children and their mothers between ages 0 and 8.³⁷ The effect size on parenting for boys is smaller than that for girls (0.06 vs. 0.274), in part due to the fact that the HOME measurement depends on punishment and boys are more likely to be punished than girls. The families of boys scored lower in this dimension than the families of girls.

In outcomes like education, employment, crime, risky behavior (which includes, for example, drug use) there are also sizable treatment effects. A companion paper, García et al. (2018), finds that monetized versions of these treatment effects translate into a benefit/cost ratio of 7.3. This estimate accounts for the costs of implementing the program, including the welfare loss generated by taxing society in order to fund the program.³⁸

Consistent with the results in Table 1, the results in Table 4 show that females generally benefit more from treatment compared to males. In 8 out of the 10 categories that we consider, the within-category average effect size is larger for females. In 9 out of 10 categories, the proportion of positive effects is at least as large for females compared to that of males. We next drill down on gender differences.

In 7 out of 10 categories, the percentage of treatment effects that are statistically significant at the 10% level is greater for women.

Pooling outcomes within categories we test equality of treatment and control distributions for each gender using a test due to Rosenbaum (2005). We find statistically significant differences for both genders. There is evidence of treatment effects for both males and females on IQ, achievement tests, and health. Effects on parental income are only statistically significant for the parents of girls, as are the effects of education.³⁹ The opposite pattern is found for boys on employment and risky behavior.⁴⁰

5. Explaining Gender Differences

We have established that there are pronounced gender differences in the patterns of positive and statistically significant treatment effects.⁴¹ Figure 2 reports the estimated combining functions by gender and by category of care used by the control group.⁴²

Figure 2:

Positively Impacted Outcomes, ABC/CARE Males and Females

Note: Panel (a) displays the percentage of positive treatment effects in accordance with the parameter in Equation (3)—treatment vs. staying at home—by gender. Panel (b) is analogous for Equation (4)—treatment vs. alternative formal childcare. Standard errors are based on the empirical bootstrap distribution. The null hypothesis is that the proportions of positive treatment effects are greater than 50%. For a full list of the estimated combining functions, see Appendix D.

The null hypothesis of no treatment effect is equivalent to the hypothesis that the proportion of outcomes favoring men is equal to 0.500. Figure 2a displays the proportion of positive treatment effects comparing outcomes of the treated to outcomes for those who stay at home. Figure 2b is the counterpart of Figure 2a and displays analogous combining functions comparing treatment effects of those participating in ABC/CARE to those who attend alternative formal childcare. These estimates account for selection into the mode of childcare using matching.

Comparing ABC/CARE to those who stay at home (Figure 2a), a greater proportion of the treatment effects is positive for women than for men. While the female proportion is statistically significantly larger than 0.500, the male proportion is not. Figure 2b shows a different pattern when we compare outcomes from ABC/CARE with outcomes of those who attended alternative formal childcare. Close to 75% of the treatment effects are positive for men. For women, the proportion of positive treatment effects is similar to the proportion obtained from comparing ABC/CARE to those who stay at home. From this analysis, we conclude that ABC/CARE was effective for men compared to alternative formal childcare programs, but not when compared to staying at home. ABC/CARE was effective for women regardless of the mode of childcare used by the controls.⁴³

This difference is consistent with the interpretation that boys and girls faced different environments in their control group conditions, especially in their home environments.^44,45 About the same percentage of control-group girls attended alternative formal childcare (73%) as did control-group boys (76%). See Figure 3a. No girls who stayed at home had working mothers at baseline while 23% of the girls who attended alternative formal childcare had mothers working at baseline. For boys, 14% of those who stayed at home had mothers working at baseline while 29% of those who attended alternative formal childcare had working mothers.

Figure 3:

Gender and Baseline Socioeconomic Disadvantage in the Control Group

Note: Panel (a) displays the cumulative distribution function of enrollment in alternatives by gender. Panel (b) displays how girls and boys separately fit into the overall (girls and boys pooled) distribution of socioeconomic disadvantage. Panel (c) displays how girls who did not enroll and girls who enrolled in alternatives fit into the overall female distribution of socioeconomic disadvantage. Panel (d) is analogous to Panel (c) for boys. Our measure of socioeconomic disadvantage is a latent of the following variables: Maternal age, education, and IQ, as well as number of siblings and HRI score.

Girls’ families were more resource-constrained compared to their male counterparts. Girls in the control group were raised in a more disadvantaged environment and many likely went to lower-quality preschools. Thus, as documented in Section 4, they benefited more from participation in ABC/CARE than boys when compared to the next best alternative as perceived by their parents.

To formally test the differences in home-life advantage between control-group girls and boys, we create an index of socioeconomic disadvantage at baseline using mother’s age, education, IQ, marital status, and employment, as well as number of children and father’s presence at home.⁴⁶ We assess how girls and boys fit into the overall distribution of this latent measure in the control group. Boys are disproportionately more advantaged than girls (Panel 3b). In Panels 3c and 3d of Figure 3, we further assess socioeconomic disadvantage by gender of the child.

Table 5 uses our constructed measure of disadvantage to test the difference in baseline disadvantage across boys and girls. We reject the null hypothesis of a common distribution of socioeconomic disadvantage across girls and boys (at baseline).

Table 5:

Gender and Baseline Socioeconomic Disadvantage in the Control Group

	H0	Rosenbaum (2005) p-value
[1]	All Controls
	Male = female	0.007
[2]	Males
	Alternative = Stay at Home	0.006
[3]	Females
	Alternative = Stay at Home	0.110

Note: Row [1] displays an exact, non-parametric p-value for the null hypothesis that the control males and control females have the same level of disadvantage. Row [2] displays the same p-value for the null hypothesis that within males, those who attend alternative formal childcare and those who stay at home have the same level of disadvantage. Row [3] is analogous to Row [2] except for females. These tests are all based on a scalar measure of socioeconomic disadvantage (mother’s age, education, IQ, marital status, and employment, as well as number of siblings and father’s presence at home). Under the null hypotheses, the pairs with the closest distance in disadvantage would be comprised of one male and one female (for the comparison of males vs. females). Rejecting the null implies that the distributions are significantly different. Statistics significant at the 0.10 level are bolded.

Parents of more advantaged girls in the control group are more likely to send their daughters to alternative preschools. Parents of more advantaged boys in the control group are more likely to keep their sons at home. Thus, boys benefited more from treatment when compared to attending alternative formal childcare as opposed to staying at home where they faced better environments than girls. The opposite pattern holds for girls, although the differences between the treatment effects by mode of alternative childcare are smaller for girls than for boys.

As shown in Section 4, the childcare supplied by ABC/CARE increases maternal employment and family income in childhood. This effect is especially pronounced for the mothers of girls. Differentially higher employment of mothers induced by the program led to larger treatment effects on income for families of girls as a result of the childcare afforded mothers. HOME scores are also differentially enhanced for girls.

More fathers are present for boys. At baseline, this leads to more family resources for boys. Family income is higher for boys than girls after treatment despite the differential treatment effect in employment for the mothers of girls. While boys benefit from the greater presence of the father, the program does not appear to attract fathers to stay at home.

6. Summary and Conclusions

This paper examines gender differences in the percentage of treatment effects favoring men of an influential early childhood program targeted to disadvantaged children. Instead of analyzing gender differences in the number of program treatment effects one at a time and cherry-picking outcomes reporting only “significant” effects, we analyze aggregate summaries of treatment effects. We document that girls benefit more than boys in the sense that effect sizes are generally much bigger for girls than boys and more treatment effects are positive and statistically significant for girls.

We examine the source of the gender difference. They originate in differences in control conditions. Baseline family environments for girls are worse. At baseline, fathers of sons are more likely to be present at home than are fathers of daughters. This leads to more resources at baseline for boys. The more advantaged boys are more likely to stay at home. Differences in family environments explain why treatment effects are generally larger for males in comparison to alternative formal care than in comparison with staying at home.

For girls, the differences in outcomes across the two control conditions is not as stark. For both genders, treatment enhances family income through supporting maternal employment and improved HOME scores. The increments in family income are greater for girls, so are the increments in HOME scores. However, the level of family income after treatment is greater for boys despite the greater growth of family income for girls.

Our analysis has implications for the recent literature on gender differences in the consequences of childcare. Baker et al. (2008, 2015) establish harmful effects of childcare. Kottelenberg and Lehrer (2014) localize these harmful effects to boys. One interpretation of their findings is that young boys are more vulnerable to being taken away from home than are young girls.⁴⁷ A rich literature supports the greater vulnerability of boys.⁴⁸

Another interpretation, and the one emphasized in this paper, is that male home environments are generally better. This is consistent with the evidence of Dahl and Moretti (2008) who show that fathers are more likely to stay at home with the mother if a boy is born. This improves family income at baseline. Our evidence on baseline differences by gender is consistent with this interpretation. Girls benefit relatively more in terms of the gains in HOME scores and in family income.

Our data are too crude to distinguish sharply between these two interpretations. However, the weight of the evidence in this paper supports the latter interpretation. Baseline conditions need to be carefully accounted for in interpreting the sources of gender differences in treatment effects.