Understanding the Mechanisms Through Which an Influential Early Childhood Program Boosted Adult Outcomes

Abstract

A growing literature establishes that high quality early childhood interventions targeted toward disadvantaged children have substantial impacts on later life outcomes. Little is known about the mechanisms producing these impacts. This paper uses longitudinal data on cognitive and personality traits from an experimental evaluation of the influential Perry Preschool program to analyze the channels through which the program boosted both male and female participant outcomes. Experimentally induced changes in personality traits explain a sizable portion of adult treatment effects.

A growing literature establishes that early childhood environments substantially impact later life outcomes (e.g., Knudsen et al., 2006, Heckman, 2008 and Almond and Currie, 2011). Less is known about the channels through which early environments operate to produce their long term effects. This paper examines the sources of the success of the Perry Preschool program, a flagship early childhood intervention in the US.¹

The Perry program was a randomized trial that targeted disadvantaged, low IQ African American children ages 3–4. After two years, all participants left the program and entered the same public school. Data were collected for treatment and control groups through age 40.

Heckman et al. (2010a) and Conti et al. (2012) show that the Perry program significantly enhanced adult outcomes including education, employment, earnings, marriage, and participation in healthy behaviors, and reduced participation in crime.² We summarize many of these findings in Table 1. All treatment effects displayed there are statistically significant and survive adjustments for multiple hypothesis testing.³ Heckman et al. (2010b) show that the internal rate of return to the program for both boys and girls is a statistically significant 6–10 percent per year—above the historical return to equity.⁴ Positive effects of the Perry program have become a cornerstone of the argument for preschool programs (e.g., Shonkoff and Phillips, 2000). Currently, about 30 percent of all Head Start centers nationwide offer a version of the Perry curriculum (ICPSR, 2010).⁵

Table 1.

Program Treatment Effects

Variable	Treatment Effect			Control Group		Treatment Group
	Effect	Effect Size	p-value	Mean	Standard Error	Mean	Standard Error
A. Males
CAT total at age 14, end of grade 8	0.566 *	0.652	(0.060)	0.000	(0.164)	0.566	(0.204)
# of misdemeanor arrests, age 27	−1.21 **	−0.363	(0.036)	3.03	(0.533)	1.82	(0.445)
# of felony arrests, age 27	−1.12	−0.324	(0.101)	2.33	(0.554)	1.21	(0.342)
# of adult arrests (misd.+fel.), age 27	−2.33 **	−0.402	(0.024)	5.36	(0.927)	3.03	(0.734)
Monthly income, age 27	0.876 **	0.607	(0.018)	1.43	(0.231)	2.31	(0.352)
Use tobacco, age 27	−0.119 *	−0.236	(0.093)	0.538	(0.081)	0.419	(0.090)
# of misdemeanor arrests, age 40	−3.13 **	−0.372	(0.039)	8.46	(1.348)	5.33	(1.042)
# of felony arrests, age 40	−1.14 *	−0.266	(0.092)	3.26	(0.684)	2.12	(0.598)
# of adult arrests (misd.+fel.), age 40	−4.26 **	−0.373	(0.041)	11.7	(1.831)	7.46	(1.515)
# of lifetime arrests, age 40	−4.20 *	−0.346	(0.053)	12.4	(1.945)	8.21	(1.778)
Employed, age 40	0.200 **	0.394	(0.024)	0.500	(0.085)	0.700	(0.085)
Sample	72			39		33
B. Females
CAT total, age 8	0.565 *	0.614	(0.062)	0.000	(0.196)	0.565	(0.223)
CAT total, age 14	0.806 **	0.909	(0.014)	0.000	(0.209)	0.806	(0.204)
Any special education, age 14	−0.262 **	−0.514	(0.025)	0.462	(0.100)	0.200	(0.082)
Mentally impaired at least once, age 19	−0.280 **	−0.569	(0.017)	0.364	(0.105)	0.083	(0.058)
# of misdemeanor violent crimes, age 27	−0.423 **	−0.292	(0.032)	0.423	(0.284)	0.000	(0.000)
# of felony arrests, age 27	−0.269 **	−0.325	(0.021)	0.269	(0.162)	0.000	(0.000)
Jobless for more than 1 year, age 27	−0.292 *	−0.573	(0.071)	0.542	(0.104)	0.250	(0.090)
Ever tried drugs other than alcohol or weed, age 27	−0.227 **	−0.530	(0.045)	0.227	(0.091)	0.000	(0.000)
# of misdemeanor violent crimes, age 40	−0.537 **	−0.364	(0.016)	0.577	(0.289)	0.040	(0.040)
# of felony arrests, age 40	−0.383 **	−0.425	(0.028)	0.423	(0.177)	0.040	(0.040)
# of lifetime violent crimes, age 40	−0.574 **	−0.384	(0.019)	0.654	(0.293)	0.080	(0.055)
Months in all marriages, age 40	39.6 *	0.539	(0.076)	47.8	(15.015)	87.5	(18.853)
Sample	51			26		25

Note: Statistics are shown for the outcomes analyzed in this paper. There are differences in treatment effects by gender although strong effects are found for both. “CAT total” denotes the California Achievement Test total score normalized to control mean zero and variance of one. Test statistics are corrected for the effect of multiple hypothesis testing and threats to validity (see Heckman et al., 2010a, Conti et al., 2012). The reported effect is the difference in means between treatment and control groups. The effect size is the ratio of the effect to the standard deviation of the control group. Stars denote statistical significance:

^***- 1 percent level,

^**- 5 percent level,

^*- 10 percent level. Monthly income is adjusted to thousands of year-2006 dollars using annual national CPI.

Previous studies of Perry focus on estimating treatment effects and do not attempt to explain their sources.⁶ This paper identifies the psychological skills changed by the Perry program and decomposes the treatment effects on adult outcomes displayed in Table 1 into components attributable to improvements in these skills.

The literature in the economics of education assumes the primacy of cognitive ability in producing successful lifetime outcomes (e.g., Hanushek and Woessmann, 2008). From this perspective, the success of the Perry program is puzzling. Although the program initially boosted the IQs of participants, this effect soon faded. A few years after the program finished, there was no statistically significant difference in IQ between treatments and controls for males and only a borderline statistically significant difference for females (see Figure 1). Consistent with this evidence, we show negligible effects of increases in IQ in producing program treatment effects.

Figure 1. Stanford-Binet IQ Test Scores by Gender and Treatment Status.

Note: Bold lines display mean IQs. Fine lines represent standard errors for the corresponding means (one standard error above and below). For a detailed description of the cognitive measures and results for other IQ tests, see Web Appendix B. Numbers below each chart are treatment and control mean test scores. See panels (a)–(d) of Figure B.6 of Web Appendix B for comparable graphs on the Leiter and PPVT measures of IQ.

Although Perry did not produce long run gains in IQ, it did create persistent improvements in personality skills.⁷ The Perry program substantially improved Externalizing Behaviors (aggressive, antisocial, and rule-breaking behaviors), which, in turn, improved a number of labor market outcomes, health behaviors, and criminal activities (see panels (a) and (b) of Figures 2 and 3).⁸

Figure 2. Histograms of Indices of Personality Skills and CAT scores.

Note: Indices for Externalizing Behavior and Academic Motivation are unweighted averages of measures listed in Table 2. “CAT” is the California Achievement Test score expressed in percentiles of the general population distribution of the scores. See Web Appendix B.4 for description of the CAT. The one-sided p-values for difference in means between treatments and controls are 0.001, 0.043, and 0.000 for Externalizing Behavior, Academic Motivation, and CAT scores respectively. Histograms are based on the pooled sample of males and females. See Figures C.6 and C.7 of Web Appendix C and Figure B.5 of Web Appendix B for the corresponding gender-specific figures.

Figure 3. Cumulative Distribution Functions of Indices of Personality Skills and CAT scores by Gender.

Note: Indices for Externalizing Behavior and Academic Motivation are unweighted averages of measures listed in Table 2. “CAT” is the California Achievement Test score expressed in percentiles of the general population distribution of the scores. Numbers above the charts are one-sided p-values testing the equality of means of the indices for the treatment and control groups.

The program also enhanced Academic Motivation (see panels (c) and (d) of Figures 2 and 3), but the effect is primarily for girls.⁹ This differential enhancement of endowments by gender helps to explain the positive treatment effects for education-related outcomes such as achievement tests and mental impairment for girls. Academic Motivation is not significantly enhanced for boys, and plays no role in explaining their treatment effects.

While the Perry program did not boost long-term IQ, it did boost long-term achievement test scores (see panels (e) and (f) of Figures 2 and 3). The effect is stronger for girls, but also occurs for boys.¹⁰ Achievement tests measure acquired knowledge, which is enhanced for children with better cognitive and personality traits. Enhanced personality skills promote learning, which, in turn, boosts achievement test scores.¹¹ This finding is consistent with recent evidence that 30–40 percent of the explained variance in achievement test scores across students is due to personality traits and not IQ.¹².

This paper contributes to an emerging literature on the economics of personality. Our demonstration of the powerful role of personality skills is in agreement with a large body of evidence summarized in Borghans et al. (2008) and Almlund et al. (2011).¹³

Our analysis shows the benefits and limitations of social experiments. The Perry study generated experimentally determined treatment effects for outcomes and skills. However, knowledge that the program enhanced skills and improved a number of outcomes is not enough to establish that the improvement in measured skills caused the improvement in outcomes. Without further assumptions, data from the experiment do not determine the production function relating changes in skills to changes in adult outcomes. The program may also improve unmeasured skills. Changes in measured skills may simply proxy changes in unmeasured skills that affect outcomes. To address this issue, we supplement the treatment effects obtained from the experiment with an econometric model that estimates the relationship between outcomes and experimentally induced changes in measured skills. Our method accounts for measurement error and treatment-induced changes in unmeasured skills. Access to experimental data allows us to test some of its identifying assumptions. Evidence from a series of specification tests supports our econometric procedure.

The paper proceeds in the following way. Section I describes the Perry program and the experiment that evaluated it. Section II presents our econometric model. Section III discusses the variety of measures of psychological traits at our disposal and the need to create low-dimensional summaries of them. It explains how we construct summary measures and test for the validity of the constructed summaries. Section IV presents our analysis of the sources of the Perry treatment effects. Section V concludes. A Web Appendix presents supplementary material.

I. The Perry Program: Design and Background

The Perry program targeted African American children with low IQs and socioeconomic status (Schweinhart and Weikart, 1981). The experiment was conducted during the mid-1960s in the district of the Perry elementary school in Ypsilanti, Michigan. Children began the program at age three and were enrolled for two years.¹⁴ Parents were disadvantaged as measured by their income and education. Roughly 47 percent of the children in the study did not have fathers present in the household at age three.

The 123 participants were randomized into treatment and control groups.¹⁵ The Perry sample consists of 51 females (25 treatment and 26 control) and 72 males (33 treatment and 39 control). There was relatively little attrition: only 11 participants left the study by the time of the interview at age 40.¹⁶

The Perry curriculum is based on the principle of active participatory learning, in which children and adults are treated as equal partners in the learning process, and children engage with objects, people, events, and ideas.¹⁷ Abilities to plan, execute, and evaluate tasks were fostered, as were social skills, including cooperation with others and resolution of interpersonal conflicts. The Perry curriculum has been interpreted as implementing the theories of Lev Vygotsky (1986) in teaching self control and sociability.¹⁸ A widely implemented program based on these principles — Tools of the Mind — is designed to promote self control.¹⁹

Sessions lasted 2.5 hours and were held five days a week during the school year. Teachers in the program, all of whom had bachelor’s degrees (or higher) in education, made weekly 1.5-hour home visits to treatment group mothers with the aim of involving them in the socioemotional development of their children. The control group had no contact with the Perry program other than through annual testing and assessment (Weikart, Bond and McNeil, 1978).

Perry predates Head Start and had no competitors, so there was no control group contamination (see Schweinhart and Weikart, 1981). All eligible parents enrolled their children in the program, so there was no issue of bias arising from noncompliance (Weikart, Bond and McNeil, 1978).

Numerous measures were collected annually from ages 3–15 on a variety of socioeconomic outcomes for treatment and control participants. There were three additional follow-ups at ages 19, 27, and 40. The Perry sample was representative of a particularly disadvantaged cohort of the African American population. About 16 percent of all African American children in the US had family and personal attributes similar to those of Perry participants at the time when the Perry program was conducted.²⁰ The statistically significant treatment effects of the experiment for boys and girls survive rigorous adjustments for multiple hypothesis testing and compromises in the randomization protocol.²¹

II. Methodology

This paper explains the sources of the Perry treatment effects in terms of improvements in early measures of psychological traits broadly classified into cognitive and personality skills.²² We first estimate treatment effects for these skills. We then estimate the relationship between skills and later life outcomes and decompose treatment effects for adult outcomes into components due to treatment-induced changes in different skills.²³

To perform valid decompositions, we need to address two features of the Perry data. First, as previously noted, the randomized design of the Perry study allows us to identify the causal effect of the treatment on measured skills and on adult outcomes, but it does not directly allow us to identify the causal effect of increases in measured skills on outcomes. We use econometric methods to address this problem. Using experimental variation, we can be more confident in the validity of our decompositions because we can test some of the assumptions maintained in our procedure. However, it is necessary to maintain some exogeneity assumptions in order to construct valid decompositions. This section makes those assumptions explicit.

Second, the Perry study has many highly correlated measurements of psychological traits (skills) that are laden with measurement error.²⁴ Moreover, the sample size of the study is small. We would exhaust the available degrees of freedom if we use all available psychological measurements to predict outcomes. Instead, we use factor analysis to create low dimensional, interpretable and informative aggregates that summarize a range of psychological traits and account for measurement error.

Section II.A presents our model for outcomes. Section II.B presents our strategy for reducing numerous error-laden measurements to manageable summary measures and addressing the problem of measurement error. Section II.C discusses identification. We establish what features of the model are overidentified, and hence testable. Section II.D summarizes a simple and robust three-step estimation procedure that is developed more extensively in the appendix.

A. The Outcome Equation

Let D denote treatment assignment. D = 1 if an agent is treated and D = 0 otherwise. Let Y₁ and Y₀ be the counterfactual outcomes when D is fixed at “1” and “0” respectively. We use the subscript d ∈ {0, 1} to represent variables when treatment is fixed at d. Fixing corresponds to manipulating treatment status d holding everything else constant.²⁵ The observed outcome is

$$Y={DY}_1+(1-D)Y_0.$$ (1)

We assume that outcomes are independent across participants conditional on observed pre-program variables X that are assumed not to be affected by the program. We introduce the notion of skills that can be changed by the program and that they produce (in part) the treatment effect. The vector of skills when treatment is fixed at d is given by ${\mathit{\theta }}_d=({\theta }_d^j:j\in \mathcal{J})$, where is an index set for skills. We define θ in a fashion analogous to Y : θ = Dθ₁ + (1 − D) θ₀.

Our analysis is based on the following linear model:

$$Y_d={\kappa }_d+{\mathit{\alpha }}_d{\mathit{\theta }}_d+{\mathit{\beta }}_d\mathit{X}+{\stackrel{\sim }{\epsilon }}_d,d\in \{0,1\},$$ (2)

where κ_d is an intercept, α_d and β_d are, respectively, | |-dimensional and |X|- dimensional vectors of parameters where |Q| denotes the number of elements in Q. While the pre-program variables X are assumed not to be affected by the treatment, their effect on Y can be affected by the treatment. ε̃_d is a zero-mean error term assumed to be independent of regressors θ_d and X.

Perry analysts collected a rich array of measures of cognitive and personality skills. However, it is very likely that there are relevant skills that they did not measure. Notationally, let ⊆ be the index set of skills on which we have measurements. The measurements may be imperfect so even these skills may not be directly observed. We decompose the term α_dθ_d in equation (2) into components due to skills we measure and skills we do not:

$$\begin{array}{l}{Y}_d={\kappa }_d+\sum _{j\in \mathcal{J}}{\alpha }_d^j{\theta }_d^j+{\mathit{\beta }}_d\mathit{X}+{\stackrel{\sim }{\epsilon }}_d\\ ={\kappa }_d+\underset{\begin{array}{c}\text{skills}\text{on}\text{which}\text{we}\\ \text{have}\text{measurements}\end{array}}{\underbrace{\sum _{j\in {\mathcal{J}}_p}{\alpha }_d^j{\theta }_d^j}}+\underset{\begin{array}{c}\text{skills}\text{on}\text{which}\text{we}\\ \text{have}\text{no}\text{measurements}\end{array}}{\underbrace{\sum _{j\in \mathcal{J}\backslash {\mathcal{J}}_p}{\alpha }^j{\theta }_d^j}}+{\mathit{\beta }}_d\mathit{X}+{\stackrel{\sim }{\epsilon }}_d\\ ={\tau }_d+\sum _{j\in {\mathcal{J}}_p}{\alpha }_d^j{\theta }_d^j+{\mathit{\beta }}_d\mathit{X}+{\epsilon }_d,\end{array}$$ (3)

where d ∈ {0, 1}, ${\tau }_d={\kappa }_d+{\sum }_{j\in \mathcal{J}\backslash {\mathcal{J}}_p}{\alpha }_d^{j}E({\theta }_d^j)$, and ε_d is a zero-mean error term defined by ${\epsilon }_d={\stackrel{\sim }{\epsilon }}_d+{\sum }_{j\in \mathcal{J}\backslash {\mathcal{J}}_p}{\alpha }_d^j({\theta }_d^j-E({\theta }_d^j))$. Any differences in the error terms between treatment and control groups can be attributed to differences in the skills on which we have no measurements. Without loss of generality we assume that ${\stackrel{\sim }{\epsilon }}_1\stackrel{\mathit{dist}}{=}{\stackrel{\sim }{\epsilon }}_0$, where $\stackrel{\mathit{dist}}{=}$ means equality in distribution. Note that the error term ε_d is correlated with the measured skills if measured skills are correlated with unmeasured skills.

We seek to decompose treatment effects into components attributable to changes in the skills that we can measure. Assuming that changes in unmeasured skills attributable to the experiment are independent of X, treatment effects can be decomposed into components due to changes in skills E(Δθ^j) and components due to changes in parameters $\mathrm{\Delta }{\alpha }^j(={\alpha }_1^j-{\alpha }_0^j)$:

$$\begin{array}{l}E(\mathrm{\Delta }{Y}_d\mid \mathit{X})=E(Y_1-Y_0\mid \mathit{X})\\ =({\tau }_1-{\tau }_0)+E\left(\sum _{j\in {\mathcal{J}}_p}\left({\alpha }_1^j{\theta }_1^j-{\alpha }_0^j{\theta }_0^j\right)\right)+({\mathit{\beta }}_1-{\mathit{\beta }}_0)\mathit{X}\\ =({\tau }_1-{\tau }_0)+E\left(\sum _{j\in {\mathcal{J}}_p}\left(\left(\mathrm{\Delta }{\alpha }^j+{\alpha }_0^j\right)E(\mathrm{\Delta }{\theta }^j)+(\mathrm{\Delta }{\alpha }^j)E\left({\theta }_0^j\right)\right)\right)+({\mathit{\beta }}_1-{\mathit{\beta }}_0)\mathit{X}.\end{array}$$26 (4)

Equation (4) can be simplified if treatment affects skills, but not the impact of skills and background variables on outcomes, i.e. ${\alpha }_1^j={\alpha }_0^j;j\in {\mathcal{J}}_p$; j ∈ and β₁ = β₀.²⁷ Under the latter assumption, the term associated with X drops from the decomposition.

We establish below that if measured and unmeasured skills are independent in the no-treatment outcome equation, α₀ can be consistently estimated by a standard factor analysis. Under this assumption, and if α₁ = α₀, we can test if the experimentally-induced increments in unmeasured skills are independent of the experimentally induced increments in measured skills.²⁸ The intuition for this test is as follows. The skills for treated participants are the sum of the skills they would have had if they were assigned to the control group plus the increment due to treatment. If measured and unmeasured skill increments are independent, α₁ is consistently estimated by a standard factor analysis and we can test if plim α̂₁ = plim α̂₀ where (α̂₀, α̂₁) are estimates of (α₀, α₁).²⁹ Assuming the exogeneity of X, we can also test if plim β̂₁ = plim β̂₀, where (β̂₀, β̂₁) are estimates of β₀ and β₁. We test and do not reject these hypotheses.

Imposing these assumptions simplifies the notation. Equation (3) may be expressed as

$$Y_d={\tau }_d+\sum _{j\in {\mathcal{J}}_p}{\alpha }^j{\theta }_d^j+\mathit{\beta }\mathit{X}+{\epsilon }_d,d\in \{0,1\}.$$ (5)

In this notation, equation (1) becomes

$$\begin{array}{l}Y=D\underset{Y_1}{\underbrace{({\tau }_1+\sum _{j\in {\mathcal{J}}_p}{\alpha }^j{\theta }_1^j+\mathit{\beta }\mathit{X}+{\epsilon }_1)}}+(1-D)\underset{Y_0}{\underbrace{({\tau }_0+\sum _{j\in {\mathcal{J}}_p}{\alpha }^j{\theta }_0^j+\mathit{\beta }\mathit{X}+{\epsilon }_0)}}\\ ={\tau }_0+\tau D+\sum _{j\in {\mathcal{J}}_p}{\alpha }^j{\theta }^j+\mathit{\beta }\mathit{X}+\epsilon ,\end{array}$$ (6)

where τ = τ₁ − τ₀ is the contribution of unmeasured variables to mean treatment effects, ε = Dε₁ + (1 − D) ε₀ is a zero-mean error term, and ${\theta }^j=D{\theta }_1^j+(1-D){\theta }_0^j,j\in {\mathcal{J}}_p$, j ∈ denote the skills that we can measure.

If the θ^j, j ∈ , are measured without error and are independent of the error term ε, least squares estimators of the parameters of equation (6) are unbiased for α^j, j ∈ .³⁰ If, on the other hand, the unmeasured skills are correlated with both measured skills and outcomes, least squares estimators of α^j, j ∈ , are biased and capture the effect of changes in the unmeasured skills as they are projected onto the measured components of θ, in addition to the direct effects of changes in measured components of θ on Y.

Equation (6) is the basis for the decompositions reported in this paper. The treatment effect is

$$E(Y_1-Y_0)=\underset{\begin{array}{c}\text{treatment}\text{effect}\text{due}\\ \text{to}\text{unmeasured}\text{skills}\end{array}}{\underbrace{({\tau }_1-{\tau }_0)}}+\underset{\begin{array}{c}\text{treatment}\text{effect}\text{due}\\ \text{to}\text{measured}\text{skills}\end{array}}{\underbrace{\sum _{j\in {\mathcal{J}}_p}{\alpha }^{j}E({\theta }_1^j-{\theta }_0^j)}}.$$ (7)

Skill j can explain treatment effects only if it affects outcomes (α^j ≠ 0) and, on average, is affected by the experiment ( $E({\theta }_1^j-{\theta }_0^j)\ne 0$). We test both conditions.

Decomposition (7) would be straightforward to identify if the measured variables are independent of the unmeasured variables, and the measurements are accurate. The second term of (7) is easily constructed by using consistent estimates of the α^j and the effects of treatment on skills. However, psychological measurements are riddled with measurement error (Cunha and Heckman, 2008). In addition, there are a large number of highly intercorrelated psychological measures that need to be condensed. We address these problems in this paper.

B. Low-Dimensional Characterizations of Skills

One way to summarize the psychological measures is to form simple unweighted indices constructed by taking averages of interpretable groups of items. This way of proceeding is widely used in psychology.³¹ It is, however, fraught with difficulties. First, there are many ways to form aggregates. Second, the weightings of the measures used to form such aggregates are arbitrary. Third, this approach does not correct for measurement error, except through simple averaging.

This paper forms interpretable aggregates through factor analysis—a statistical method that summarizes the covariability among observed measures using low-dimensional latent variables. The method also accounts for measurement error.³² We use the early measures of skills in the Perry data to extract the latent skills (θ^j; j ∈ ) in equation (7) where the latent skills are the factors. We use a common measurement system for treated and untreated participants although ${\theta }_1^j$, j ∈ and ${\theta }_0^j$, j ∈ , may differ.

More formally, let the index set for measures associated with factor j ∈ be . Denote the measures for factor j in treatment group d by $M_{m^j,d}^j$, d ∈ {0, 1}. Henceforth, let θ_d denote the vector of factors associated with the skills that can be measured in treatment state d, i.e., ${\mathit{\theta }}_d=({\theta }_d^j:j\in {\mathcal{J}}_p),d\in \{0,1\}$.

Following the psychometric literature summarized in Gorsuch (1983, 2003) and Thompson (2004), we assume that each measure is associated with at most one factor. We assume that the same measurement equations govern treatment and control groups so that the following equation is assumed to describe the relationship between the measures associated with factor j and the factor:

$$\text{Measures}:M_{m^j,d}^j={\nu }_{m^j}^j+{\varphi }_{m^j}^j{\theta }_d^j+{\eta }_{m^j}^j,j\in {\mathcal{J}}_p,m^j\in {\mathcal{M}}^j.$$ (8)

To simplify the notation, we keep the covariates X implicit. Parameters ${\nu }_{m^j}^j$ are measure-specific intercepts. Parameters ${\varphi }_{m^j}^j$ are factor loadings. The ε_d in (5) and ${\eta }_{m^j}^j$ are mean-zero error terms assumed to be independent of θ_d, d ∈ {0, 1}, and of each other. The factor structure is characterized by the following equations:

$$\text{Factor}\text{Means}:E[{\theta }_d^j]={\mu }_d^j,j\in {\mathcal{J}}_p$$ (9)

$$\text{Factor}\text{Covariance}:\text{Var}[{\mathit{\theta }}_d]={\mathit{\sum }}_{{\mathit{\theta }}_d},d\in \{0,1\}.$$ (10)

The assumption that the parameters ${\nu }_{m^j}^j,{\varphi }_{m^j}^j,\text{Var}({\eta }_{m^j}^j):m^j\in {\mathcal{M}}^j,j\in {\mathcal{J}}_p$, do not depend on d simplifies the notation, as well as the interpretation of the estimates obtained from our procedure. It implies that the effect of treatment on the measured skills operates only through the latent skills and not through changing the measurement system for those skills. This assumption can be tested by estimating measurement systems separately for treatment and control groups and testing if measurement equation factor loadings and measurement equation intercepts differ between treatment and control groups.³³ We do not reject the hypotheses of equality of these parameters across treatment and control groups.³⁴

C. Identification

Identification of factor models requires normalizations that set the location and scale of the factors (e.g., Anderson and Rubin, 1956a).³⁵ We set the location of each factor by fixing the intercepts of one measure—designated “the first”—to zero, i.e. ${\nu }_1^j=0,j\in {\mathcal{J}}_p$. This defines the location of factor j for each counterfactual condition. We set the scale of the factor by fixing the factor loadings of the first measure of each skill to one, i.e. ${\varphi }_1^j=1,j\in {\mathcal{J}}_p$. For all measures that are related to a factor (i.e. have a non-zero loading on the factor, ${\varphi }_{m^j}^j$), the decomposition of treatment effects presented in this paper is invariant to the choice of which measure is designated as the “first measure” for each factor and to any affine transformations of the measures.³⁶

Identification is established in four steps. First, we identify the means of the factors, ${\mu }_d^j$. Second, we identify the measurement factor loadings ${\varphi }_{m^j}^j$, the variances $\text{Var}({\eta }_{m^j}^j)$ of the measurement system, and the factor covariance structure Σ_θd. Third, we use the parameters identified from the first and second steps to secure identification of the measurement intercepts ${\nu }_{m^j}^j$. Finally, we use the parameters identified in the first three steps to identify the factor loadings α = (α^j; j ∈ ) and intercept τ_d of the outcome equations. We discuss each of these steps in turn.

1. Factor Means

We identify ${\mu }_1^j$ and ${\mu }_0^j$ from the mean of the designated first measure for treatment and control groups: $E(M_{1,d}^j)={\mu }_d^j,j\in {\mathcal{J}}_p,d\in \{0,1\}$.

2. Measurement Loadings

From the covariance structure of the measurement system, we can identify: (a) the factor loadings of the measurement system ${\varphi }_{m^j}^j$; (b) the variances of the measurement error terms, $\text{Var}({\eta }_{m^j}^j)$; and (c) the factor covariance matrix, Σ_θd. Factors are freely correlated. We need at least three measures for each skill j ∈ , all with non-zero factor loadings.³⁷ The ${\varphi }_{m^j}^j$ can depend on d ∈ {0, 1}, and we can identify ${\varphi }_{m^j,d}^j$. We test if $H_0:{\varphi }_{m^j,1}^j={\varphi }_{m^j,0}^j,j\in {\mathcal{J}}_p$, and we do not reject these hypotheses.³⁸

3. Measurement Intercepts

From the means of the measurements, i.e. $E(M_{m^j,d}^j)={\nu }_{m^j}^j+{\varphi }_{m^j}^j{\mu }_d^j$, we identify ${\nu }_{m^j}^j$, m^j ∈ \{1}, j ∈ . Recall that the factor loadings ${\varphi }_{m^j}^j$ and factor means ${\mu }_d^j$ are identified. Assuming equality of the intercepts ( ${\nu }_{m^j}^j$) between treatment and control groups guarantees that treatment effects on measures, i.e. $E(M_{m^j,1}^j)-E(M_{m^j,0}^j)$, operate solely through treatment effects on factor means, i.e. ${\mu }_1^j-{\mu }_0^j$. However, identification of our decomposition requires intercept equality only for the designated first measure of each factor. We test and do not reject $H_0:{\nu }_{m^j,1}^j={\nu }_{m^j,0}^j$ for all m^j ∈ \{1}, j ∈ .³⁹

4. Outcome Equation

Adult outcome factor loadings in equation (5) can be identified using the covariances between outcomes and the designated first measure of each skill. We form the covariances of each outcome Y_d with the designated first measure of each skill j ∈ to obtain Cov(Y_d, M_1,d) = Σ_θd α where α = (α^j; j ∈ ). By the previous argument, Σ_θd is identified. Thus α is identified whenever det(Σ_θd) ≠ 0. Outcome factor loadings α can depend on d ∈ {0, 1}, as they can be identified through Cov(Y_d, M_1,d) = Σ_θd α_d, which can be separately identified for treatments and controls. We test $H_0:{\alpha }_1^j={\alpha }_0^j,j\in {\mathcal{J}}_p$, and we do not reject these hypotheses.⁴⁰ Using E(Y_d), we can identify τ_d because all of the other parameters of each outcome equation are identified.

D. Estimation Procedure

We estimate the model using a simple three stage procedure. First, we estimate the measurement system. Second, from these equations we can estimate the skills for each participant. Third, we estimate the relationship between participant skills and lifetime outcomes. Proceeding in this fashion makes identification and estimation transparent. In Section IV and Appendix L we show that a one-step procedure produces estimates very similar to those obtained from the three-step procedure. We estimate the model separately for males and females in light of the evidence that there are strong gender differences in program effects.⁴¹

We compute p-values using the bootstrap. We draw K = 1000 bootstrap samples of the original data and apply the estimation procedure to each pseudo-sample the same way we apply it to the original data. For a one-tailed test with an upper tail rejection region, the bootstrap p-value is estimated by

$$\widehat{p}(\widehat{\varrho })=\frac{1}{K}\sum _{k=1}^K\mathbf{1}({\varrho }_k^{\ast }>\widehat{\varrho }),$$ (11)

where ϱ̂ is the parameter of interest as estimated from the original data, and ${\varrho }_k^{\ast }$ is the k-th draw from the bootstrap data-generating process satisfying the null hypothesis.⁴² We describe the details of our estimation procedure in the Appendix.

III. Measures of Cognitive and Personality Skills

This section explains how we condense the data on psychological traits. Using standard psychometric methods, we establish that only three factors are required to explain the available psychological measures. The extracted factors have clear interpretations. After extracting the factors, we test the validity of the derived system.

A. Our Measure of Cognition

A large literature establishes the importance of cognition, as measured by IQ, in explaining a variety of life outcomes (see, e.g., Gottfredson, 1997, and Jensen, 1998, for surveys). We use the Stanford-Binet Intelligence Test (Terman and Merrill, 1960) as our measure of cognition. Mean differences in Stanford-Binet scores between treatment and control groups are plotted by age in panels (a) and (b) of Figure 1. A boost in the IQs of children in the treatment group is observed soon after the program starts at age three. A few years after the program ends, the effect of treatment on IQ essentially disappears for males. A small, borderline statistically significant, positive effect remains for females. In our analysis, we use IQs at ages 7, 8, and 9, since this is the period when the treatment effect on IQ becomes relatively stable for both genders, and IQ becomes rank stable after those ages.⁴³

B. Personality Skills

The most influential taxonomy of personality skills is the Big Five personality inventory (McCrae and John, 1992; John and Srivastava, 1999).⁴⁴ Unfortunately, the Big Five was developed long after the Perry experiment was conducted. We only have access to psychological measures of personality skills collected before the Big Five was codified.⁴⁵

Perry Measures of Personality Skills

There are 43 child personality measures in the Perry data. These measures belong to two separate psychological inventories of personality skills: the Pupil Behavior Inventory (PBI) and the Ypsilanti Rating Scale (YRS). These measures are displayed in Tables C.1 and D.1 in the Web Appendix, where their correspondence with Big Five traits is noted.⁴⁶ The PBI inventory was developed by Vinter et al. (1966) to measure behavioral and attitudinal factors that affect academic success. The YRS measures were developed by the Perry analysts to measure academic potential and socio-emotional skills (Weikart, Bond and McNeil, 1978). The PBI and YRS questionnaires consist of multiple questions called items. They were given to teachers in classes attended by Perry students after the program was completed to assess students in their classes. An example of one of the PBI items is “lying or cheating”, and the possible answers are (1) very frequently, (2) frequently, (3) sometimes, (4) infrequently, and (5) very infrequently. The YRS questionnaire asks questions about socio-emotional skills, such as Academic Potential and Social Development.⁴⁷

C. Approaches to Summarizing the Data

There are many ways to summarize the available psychological measures in an interpretable fashion. One way is to form indices of measures using the groupings employed by the Perry psychologists.⁴⁸ The PBI and YRS scales were designed to alert educators of behavioral and motivational problems of children in school. There is considerable overlap among items in the groupings, and the relationship of these measures to more interpretable psychological constructs is unclear.

An intuitively appealing way to construct summary measures is to ignore the groupings used by the Perry psychologists and to select measures from all of their scales based on common sense and previous research in psychology, such as their interpretability in terms of the Big Five. This procedure is called “operationalization” in psychology, and is inherently subjective.⁴⁹ There are many ways to form such indices leading to a complex model selection problem.

The approach used in this paper is to apply exploratory factor analysis (EFA) to the available measures. EFA is a standard statistical method widely used in psychometrics to allocate measures to factors (e.g., Gorsuch, 2003; Thompson, 2004). It is used to form the Big Five (See Goldberg, 1993). EFA establishes links between a small number of latent factors and the available measures. Each measure is allowed to depend on at most one factor, and the derived factors are allowed to be freely correlated.

Our application of EFA produces three interpretable factors which we interpret as Cognition, Externalizing Behavior, and Academic Motivation.⁵⁰ The derived factors are consistent with previous research in psychology on the predictive power of psychological traits. We implement EFA in two stages. We first select the number of factors. We then allocate the measures of personality traits to different factors.

D. Exploratory Factor Analysis

We use several accepted procedures to determine the number of factors: the scree test (Cattell, 1966), Onatski’s test (2009), and Horn’s (1965) parallel analysis test.⁵¹ Overall, these procedures point to a three-factor characterization for both men and women.⁵²

Exploratory Factor Analysis: Establishing Dedicated Measures

We extract factors following the criteria for EFA laid out by Gorsuch (1983).⁵³ This method is widely used although its application requires judgement on the part of the analyst (see Gorsuch, 2003, and Thompson, 2004). EFA identifies blocks of measures that are strongly correlated within the each block (i.e. satisfy convergent validity), but are weakly correlated between blocks (i.e. satisfy discriminant validity). It discards measures that load on multiple factors. Application of standard EFA methodology to the 46 cognitive and personality Perry measures gives the 13 measures displayed in Table H.2.⁵⁴

We follow Gorsuch (2003) and Thompson (2004), and derive a fully dedicated system as described by equations (8)–(10) i.e. a system in which each measure is associated with at most one factor. This procedure is called confirmatory factor analysis (CFA), which Gorsuch (2003) and Thompson (2004) advocate as the next step after conducting EFA. It produces the interpretable system displayed in Table 2, based on three factors: Cognition, Externalizing Behavior, and Academic Motivation.⁵⁵ Externalizing Behavior is proxied by measures of behavior related to lying, stealing, and swearing, as well as being aggressive and disruptive. It has been linked to crime and aggressive behavior.⁵⁶ Academic Motivation is proxied by measures of student interest, persistence, and initiative in learning and is linked to performance in schools as measured by achievement tests. The personality measures proxying Externalizing Behavior and Academic Motivation are conceptually related to Big Five factors in personality psychology.⁵⁷

Table 2.

Cognitive and Personality Factors and Their Measures

Cognition		Externalizing behavior		Academic Motivation
Measures(a)	Age	Measures(a)	Age(b)	Measures(a)	Age(b)
Stanford-Binet IQ	7	Disrupts classroom procedures	7–9	Shows initiative	7–9
Stanford-Binet IQ	8	Swears or uses obscene words	7–9	Alert and interested in school work	7–9
Stanford-Binet IQ	9	Steals	7–9	Hesitant to try, or gives up easily	7–9
		Lying or cheating	7–9
		Influences others toward troublemaking	7–9
		Aggressive toward peers	7–9
		Teases or provokes students	7–9
Cronbach’s alpha(c), males	0.838	Cronbach’s alpha, males	0.906	Cronbach’s alpha, males	0.901
Cronbach’s alpha, females	0.913	Cronbach’s alpha, females	0.916	Cronbach’s alpha, females	0.896

Note:

^(a)See Web Appendix B for a detailed description of Stanford-Binet IQ measures. Externalizing Behavior and Academic Motivation are proxied by items of the Pupil Behavior Inventory (PBI) described in Web Appendix C. PBI items are described in this table the same way they appear in the questionnaire. For example, “lying or cheating” and “steals” were the full descriptions of misbehavior that teachers were asked to evaluate.

^(b)Each personality measure is an average over non-missing observations at ages 7, 8, and 9.

^(c)Cronbach’s alpha (see Cronbach, 1951) is a statistic that captures how well a set of measures proxies a latent skill. Cronbach’s alpha is the lower bound of the internal consistency reliability of measures that are proxies for a skill. The internal consistency reliability is defined as the square of the correlation between the measured scale defined as the sum of the measures and the underlying skill θ (Allen and Yen, 2002). In our case, the correlations between the skills and the scales (equal to the square roots of alphas) range from 0.70 to 0.82 for males and from 0.80 to 0.84 for females. The Cronbach’s alphas can also be interpreted as a correlation between the observed scale and a hypothetical alternative scale measuring the same skill and based on the same number of hypothetical alternative items (Nunnally and Bernstein, 1994). For this table, the alphas (i.e., the correlations) range from 0.84 to 0.91 for males and from 0.90 to 0.92 for females.

The Predictive Power of Externalizing Behavior and Academic Motivation

The factors extracted by the EFA procedure are closely linked to traits that have been shown to be predictive of adult outcomes. This gives us greater confidence in using them to explain the Perry treatment effects.

The recent literature in economics shows that Externalizing Behavior predicts child and adult outcomes (Segal, 2008, 2009).⁵⁸. The literature in psychology shows that Externalizing Behavior is negatively associated with Academic Achievement.⁵⁹ Childhood Externalizing Behaviors have also been shown to be related to adolescent and adult delinquency (e.g., Nagin and Tremblay, 1999, and Broidy et al., 2003).

The literature in criminology and psychology demonstrates that early antisocial behaviors are highly predictive of adolescent and adult antisocial behaviors (Gersten et al., 1976; Huesmann et al., 1984; Olweus, 1979). Antisocial behaviors measured between ages 7 and 11 strongly predict criminal behaviors in adulthood (Loeber, 1982; Mofitt, 1993). Meanwhile, disobedient and aggressive behaviors measured as early as ages 3–5 predict later childhood conduct disorders and adolescent arrests (Mofitt, 1993; White et al., 1990). Most children with conduct disorders experience social difficulties in adulthood, whereas only 8 percent of children without conduct disorders experience such difficulties (Zoccolillo et al., 1992). Similarly, many children with antisocial behavior around ages 8–10 become antisocial adults (Coie et al., 1995; Olweus, 1979; Robins, 1978)⁶⁰ and chronic criminal offenders (Loeber, 1982). Almost all antisocial adults were antisocial children (Robins, 1978). Our analysis confirms previous evidence on the stability of antisocial traits into adulthood. We find stable rank correlations between Externalizing Behavior at ages 7–9 and subsequent measures of crime as late as ages 19, 27, and 40 (see Figure 4). The evidence from the literature in psychology and criminology joined with the evidence from this paper suggests that reducing early externalizing behavior reduces crime.

Figure 4. Spearman’s Rank Correlations between Externalizing Behavior at Ages 7–9 and Number of Arrests by Ages 19, 27, and 40.

Note: *** - 1 percent, ** - 5 percent, * - 10 percent significance levels (one-sided tests).

Academic Motivation, apart from its obvious link to performance in school, has been shown to be a statistically significant predictor of decreased drug use (Bryant et al., 2003, and Razzino et al., 2004) and alcohol consumption (Zimmerman and Schmeelk-Cone, 2003, Simons-Morton, 2004, and Vaughan, Corbin and Fromme, 2009). Since drinking and drug use are associated with crime (Anglin and Perrochet, 1998, and Greenfeld, 1998), youth with higher levels of Academic Motivation are less likely to engage in criminal activities. Flouri and Buchanan (2002) show that for both males and females, low academic motivation in adolescence is positively related to trouble with the police at age 16. Cymbalisty, Schuck and Dubeck (1975) show that for males who have already committed crimes, recidivism decreases with motivation for learning. Therefore, it is expected that experimentally induced enhancements in Academic Motivation would be a source of treatment effects for education and crime outcomes. We confirm such effects for education, but not for crime.

IV. The Effect of Treatment on Traits and the Sources of Treatment Effects

We first study how treatment affects the extracted factors. We then investigate how the factors affect life outcomes. Finally, we decompose adult treatment effects into components corresponding to changes in each factor. The first analysis is based on the output of step two of the three-step estimation procedure described in detail in the Appendix. The second and third analyses are based on the output of the third step of the procedure.

A. The Effect of the Perry Program on Cognitive and Personality Skills

Figure 5 graphs kernel densities of factor scores and presents one-sided p-values for testing the equality of the means for each skill between the treatment and control groups.⁶¹ The Perry program has a statistically significant treatment effect on Externalizing Behavior at the 5 percent level for males and at the 1 percent level for females. The effects on Cognition and Academic Motivation are statistically significant at the 10 percent and 5 percent levels, respectively, for females, but are not statistically significant for males. This evidence is consistent with the evidence in Table 1 of a statistically significant treatment effect on achievement test scores, which is much stronger for girls than for boys.

Figure 5. Kernel Densities of Factor Scores.

Note: Probability density functions of Bartlett (1937) factor scores are shown. Densities are computed based on a normal kernel. Numbers above the charts are one-sided p-values testing the equality of factor score means for the treatment and control groups. Higher Externalizing Behavior corresponds to more socially desirable behavior. See Web Appendix L for the empirical CDFs of the factor scores (Figure L.5). Vertical lines locate factor score means for treatment and control groups.

The kernel densities reveal different patterns of the effect of the program on the distribution of skills. The cognition of females is enhanced mostly in the right tail of the distribution (panel b). In contrast, a substantial part of the improvement in Externalizing Behavior for females operates through enhancing low levels of the skill (panel d). Externalizing Behavior in males is improved at all levels. Academic Motivation in females is improved at all levels except for the top percentiles (see panel f). There is no statistically significant difference in the distribution of Cognition for males (panel a).⁶²

B. The Effect of Cognitive and Personality Skills on Outcomes

In order to estimate the effects of factors on outcomes we estimate a model described by a system of equations (5) and (8) conditioning on background variables X. We present estimates of α = (α^j, j ∈ ) in equation (7). Table 3 shows that all three factors (Cognition, Externalizing Behavior, and Academic Motivation) have statistically significant effects on at least one outcome.

Table 3.

Factor Loadings of Outcome Equations

Outcome	Cognition		Externalizing Behavior		Academic Motivation		Sample Size
	Coefficient	p-value	Coefficient	p-value	Coefficient	p-value
Males
CAT total at age 14, end of grade 8 (+)	0.819 ***	(0.000)	−0.203	(0.845)	0.700 ***	(0.000)	45
# of misdemeanor arrests, age 27 (−)	−0.259	(0.359)	−1.226 **	(0.028)	−0.152	(0.367)	59
# of felony arrests, age 27 (−)	−0.618	(0.235)	−1.333 **	(0.023)	0.219	(0.557)	59
# of adult arrests (misd.+fel.), age 27 (−)	−0.876	(0.251)	−2.559 **	(0.014)	0.067	(0.549)	59
Monthly income, age 27 (+)	0.970 **	(0.038)	0.698 **	(0.046)	−0.257	(0.670)	55
Use tobacco, age 27 (−)	−0.179	(0.121)	−0.332 ***	(0.001)	0.159	(0.847)	57
# of misdemeanor arrests, age 40 (−)	−0.620	(0.383)	−2.424 *	(0.087)	0.196	(0.501)	59
# of felony arrests, age 40 (−)	−0.628	(0.266)	−1.755 **	(0.014)	0.293	(0.570)	59
# of adult arrests (misd.+fel.), age 40 (−)	−1.248	(0.327)	−4.180 **	(0.039)	0.489	(0.525)	59
# of lifetime arrests, age 40 (−)	−1.100	(0.359)	−4.740 **	(0.030)	0.239	(0.519)	59
Employed, age 40 (+)	0.277 **	(0.012)	0.230 **	(0.011)	−0.270	(0.991)	54
Females
CAT total, age 8 (+)	0.219 **	(0.039)	−0.134	(0.729)	0.689 ***	(0.000)	35
CAT total, age 14 (+)	0.154	(0.113)	−0.448	(0.931)	0.899 ***	(0.001)	31
Any special education, age 14 (−)	−0.041	(0.273)	0.119	(0.759)	−0.209 *	(0.064)	37
Mentally impaired at least once, age 19 (+)	−0.039	(0.283)	0.227	(0.948)	−0.308 ***	(0.008)	33
# of misdemeanor violent crimes, age 27 (−)	0.083	(0.778)	−1.080 **	(0.043)	0.150	(0.700)	37
# of felony arrests, age 27 (+)	0.021	(0.609)	−0.451 *	(0.053)	0.140	(0.808)	37
Jobless for more than 1 year, age 27 (−)	0.139	(0.920)	0.048	(0.608)	−0.465 ***	(0.003)	36
Ever tried drugs other than alcohol or weed, age 27 (−)	−0.043	(0.201)	−0.146	(0.144)	0.122	(0.854)	34
# of misdemeanor violent crimes, age 40 (−)	0.084	(0.774)	−1.078 **	(0.043)	0.081	(0.592)	37
# of felony arrests, age 40 (−)	0.047	(0.704)	−0.589 **	(0.014)	0.078	(0.643)	37
# of lifetime violent crimes, age 40 (−)	0.096	(0.807)	−1.220 **	(0.023)	0.165	(0.704)	37
Months in all marriages, age 40 (+)	21.748	(0.111)	13.591	(0.289)	10.453	(0.280)	36

Note: Regression coefficients for factor scores in Equation (5) are shown with one-sided p-values in parentheses. (+) and (−) denote the sign of the total treatment effect on the corresponding variable. Estimates are corrected based on the bias-correcting procedure described in Equation (A4). “CAT total” denotes the California Achievement Test total score normalized to control mean zero and variance of one. See Tables L.14 and L.15 of Web Appendix L for more detailed versions of this table containing coefficients for background variables. Stars denote significance levels:

^***- 1 percent,

^**- 5 percent, and

^*- 10 percent. Monthly income is adjusted to thousands of year-2006 dollars using annual national CPI.

Different factors affect different outcomes. Cognition primarily affects achievement tests and also affects certain labor market outcomes. Externalizing Behavior affects crime outcomes, labor market outcomes, and health behaviors. Academic Motivation boosts educational outcomes and reduces long-term unemployment.

Treatment effects are generated through changes in skills if (1) skills affect outcomes and (2) skills are enhanced by the intervention. Thus, even though Cognition and Academic Motivation are positively related to the CAT scores of males, the absence of a relationship for males between treatment and both Cognition and Academic Motivation makes this channel either weak or nonexistent.

C. Decomposing Treatment Effects on Outcomes by Source

Figures 6 and 7 present our estimated decompositions of treatment effects into experimentally induced improvements in Cognition, Externalizing Behavior, Academic Motivation, and other factors. By “other factors” we mean the residual treatment effect associated with unmeasured skills. We report the percentage of each treatment effect attributable to each component.⁶³ The numbers shown above each component are one-sided p-values for the test of whether the component is zero. We stress that these decompositions are invariant to the choice of normalizing measures and to affine transformations of the measures.⁶⁴

Figure 6. Decompositions of Treatment Effects on Outcomes, Males.

Note: The total treatment effects are shown in parentheses. Each bar represents the total treatment effect normalized to 100 percent. One-sided p-values are shown above each component of the decomposition. The figure is a slightly simplified visualization of Tables L.10 and L.14 : small and statistically insignificant contributions of the opposite sign are set to zero. See Web Appendix L for detailed information about the simplifications made to produce the figure. “CAT total” denotes California Achievement Test total score normalized to control mean zero and variance of one. Asterisks denote statistical significance: * – 10 percent level; ** – 5 percent level; *** – 1 percent level. Monthly income is adjusted to thousands of year-2006 dollars using annual national CPI.

Figure 7. Decompositions of Treatment Effects on Outcomes, Females.

Note: The total treatment effects are shown in parentheses. Each bar represents the total treatment effect normalized to 100 percent. One-sided p-values are shown above each component in each outcome. The figure is a slightly simplified visualization of Tables L.11 and L.15 : small and statistically insignificant contributions of the opposite sign are set to zero. See Web Appendix L for detailed information about the simplifications made to produce the figure. “CAT total” denotes California Achievement Test total score normalized to control mean zero and variance of one. Asterisks denote statistical significance: * – 10 percent level; ** – 5 percent level; *** – 1 percent level.

We decompose the treatment effect for a number of outcomes: performance on the California Achievement Test (CAT), special education at school and mental impairment, labor market outcomes such as income and employment, health behaviors such as smoking tobacco or using drugs, marriage duration, and crime outcomes. The crime outcomes are especially important since they are the dominant component of Perry program’s total benefit ( e.g., Heckman et al., 2010b). We only decompose treatment effects that have been shown to be statistically significant at the 10 percent level or below after adjusting for the effects of multiple-hypothesis testing on significance levels ( Heckman et al., 2010a, Conti et al., 2012). Proceeding in this fashion leads to somewhat different decompositions for males and females.

The effect of the intervention on life outcomes operates primarily through the program’s enhancement of Externalizing Behavior. Components attributable to changes in this factor are generally statistically significant and, in most cases, explain 20–60 percent of the treatment effects on crime for males and about 40–60 percent for females (see Figures 6 and 7).

The mediating effects of Externalizing Behavior are not only statistically significant, but also economically significant. Reported arrests and registered crimes are only a small fraction of the actual number of crimes. For instance, only one in 15 property crimes and one in five violent crimes actually leads to an arrest.⁶⁵ We find that experimentally induced reductions in Externalizing Behavior (by one standard deviation) leads to a decline in the total number of lifetime arrests by statistically significant 1.7 (p=0.077) and the number of felony arrests by 0.6 (p=0.056) for males at age 40.⁶⁶ For females, the total number of felony arrests by age 40 is reduced by 0.31 (p=0.050), and the number of registered lifetime violent crimes is reduced by 0.65 (p=0.046).⁶⁷ The reduction in actual crimes is likely several times larger than these reductions in the number of arrests and registered crimes. Since Externalizing Behavior is both malleable at early ages (see Figure 5) and strongly predictive of crime (see Table 3), it should not be surprising that crime reduction has been found to be a major benefit of the Perry program.

We also decompose the effect of the program on an achievement test (CAT) for both males and females. For females, enhancements in Academic Motivation explain about 30 percent of the treatment effect on CAT scores at age 8. This estimate is statistically significant at a 10 percent level (p = 0.057). For CAT scores at age 14, the role of Academic Motivation is not precisely determined for males or for females (p = 0.161 and 0.528).

Finally, we decompose a number of education, labor market, and health outcomes. Academic Motivation consistently explains a share of treatment effects for all education-related outcomes, which is not surprising given strong links between Academic Motivation and education outcomes presented in Table 3. However, only some components of these decompositions are precisely determined (e.g. CAT and the status of being mentally impaired for females).

For labor market outcomes, we find that about 20 percent of the treatment effect on monthly income at age 27 (p=0.089) and also about 20 percent of the treatment effect on the probability of employment at age 40 (p=0.085) are explained by early improvements in Externalizing Behavior. Additionally, Externalizing Behavior explains about 40 percent of tobacco use at age 27 (p = 0.046).

D. Decompositions Based on Indices vs. Decompositions Using Factor Scores

A simple alternative to our factor procedure for summarizing the evidence on the effects of experimentally induced changes in measurements on outcomes is to use indices of the measures in place of factor scores. Such indices, which are unweighted averages over the measures, are commonly used. In making this comparison we take as given the cluster of measures identified by application of EFA (see Table 2). Picking the clusters of the traits in this fashion avoids the serious practical problem that many groupings are possible, and many tests among competing specifications are non-nested. The comparison being made is one between using unweighted averages of measures not correcting for measurement error with a method that extracts factors by weighting the measures by the estimated factor loadings and adjusting for measurement error.

Figure 8 decomposes a selection of the treatment effects for a variety of outcomes for each gender using both indices and factor models.⁶⁸ For each outcome, we show the results of the two different estimation procedures using a pair of bars. The first bar in each pair corresponds to the estimates from the procedure used in this paper. These bars are identical to those presented in Figures 6 and 7. The second bar in each pair corresponds to the decomposition obtained from estimating equation (7) using indices.

Figure 8. Decompositions of Treatment Effects by Indices versus Factor Scores.

Note: The total treatment effect is normalized to 100 percent. One-sided p-values are shown above each component in each outcome. (1) “SCORE” denotes models where personality traits are measured by factor scores.(1) “INDEX” denotes models where personality traits are measured by indices constructed using unweighted averages over the items. The figure is a slightly simplified visualization of results from Tables L.10, L.11, K.1, and K.2 of the Web Appendix: small and statistically insignificant contributions of the opposite sign are set to zero. See Web Appendix L for detailed information about the simplifications used to produce the figure.

Comparing the first bar with the second bar reveals that even though results of the two procedures lead to similar qualitative conclusions about the role of mediating skills, the estimates of the explained treatment effect components and the associated p-values are numerically different. As is apparent from equation (7), there are two possible sources of difference in the decompositions: (1) different factor loadings α^j and (2) different estimates of the treatment effect on the factors: $E({\theta }_1^j-{\theta }_0^j)$, j ∈ . The approach using indices only partially corrects for attenuation bias by reducing measurement error through simple averaging. Our factor approach explicitly addresses measurement error. Thus the index approach likely generates downward-biased decompositions. Indeed, the shares of the treatment effects explained by Externalizing Behavior are generally smaller for the index-based procedure. The p-values from the index-based procedure are somewhat smaller for females but somewhat larger for males. Most of the comparisons presented in Figure 8 are consistent with this interpretation. ^69,70

E. More Efficient Estimates

The three step estimation procedure used in this paper is simple and intuitive. In general, it is not statistically efficient given that we do not impose cross-equation restrictions across the stages of the estimation. In Web Appendix L, we compare the decompositions obtained from our three-step estimation procedure to those obtained from a one-step maximum likelihood estimation method where the measurement system and outcome equation are estimated jointly.⁷¹ The results from both procedures are in close agreement, although p-values from the maximum likelihood procedure are generally lower.⁷²

F. Testing the Validity of the Derived System

The procedure used to create the dedicated factor system is based in part on judgements by the analyst. Such judgements are widely used in the psychometric literature.⁷³ To gain greater confidence in the system created by our EFA analysis, we test the validity of the derived factor structure.

Our application of the EFA methodology yields 13 dedicated measures out of 46 available cognitive and personality measures. The 33 unused measures do not survive the EFA testing criteria. To test the validity of this specification, we run a series of tests on the measurement and outcome equations. We first determine if, conditional on the extracted factors, the unused measures exhibit a treatment effect. If they do not, they are not candidates for explaining the treatment effect for outcomes. We also determine whether, conditional on the extracted factors, the unused measures explain outcomes. Evidence from both types of tests support the low-dimensional specification of equations (5) and (8)–(10) derived from applying EFA. Web appendix M presents a detailed discussion of these tests.

G. A Framework for Unifying Diverse Studies of Child Development

The framework developed in this paper facilitates the interpretation of diverse treatment effects within and across programs as the manifestations of program-induced changes in a low-dimensional set of skills of participants. This framework can be used to unify the interpretation of the treatment effects across different studies with different interventions applied to different populations. By focusing on the channels through which the different programs produce their effects, we gain a deeper understanding of the traits that matter and how they can be affected by various influences on the child. Systematic application of this framework will enable the intervention literature to move beyond meta analyses to understand the common mechanisms producing success in children and how different interventions boost different traits to different degrees. This framework also offers a basis for unifying observational studies of family influence with intervention studies.⁷⁴ Investments made by families boost θ as do the investments made in intervention programs. Using the framework developed and applied in this paper, we can in principle compare family investments and interventions in terms of their effects on θ.

V. Conclusions

Using experimental data from an influential early childhood program, we analyze the sources of program treatment effects. Coupling experimental variation with an econometric model, we estimate the role of enhancements in Cognition, Externalizing Behavior, and Academic Motivation in producing the Perry treatment effects. Persistent changes in personality skills play a substantial role in producing the success of the Perry program. The reduction in Externalizing Behavior, which explains the bulk of the effects of Perry program on criminal, labor market, and health behavior outcomes, is especially strong.⁷⁵

We offer a new understanding of how a few hours per day of preschool at ages three and four with a curriculum that promotes social competency, planning, and organization can significantly and beneficially affect life outcomes. The importance and malleability of these skills deserves greater emphasis in public policies designed to promote skills and alleviate poverty.⁷⁶

Appendix on the Three Step Estimation Procedure

Step 1

For a given set of dedicated measurements, and choice of the number of factors, we estimate the factor model using measurement system (8)–(10).

Step 2

We use the measures and factor loadings estimated in the first step to compute a vector of factor scores for each participant i. We form unbiased estimates of the true vector of skills ${\mathit{\theta }}_i=({\theta }_i^j;j\in {\mathcal{J}}_p)$ for agent i. The factor measure equations contain X which we suppress to simplify the expressions. Notationally, we represent the measurement system for agent i as

$$\underset{\mid \mathcal{M}\mid \times 1}{\underbrace{{\mathit{M}}_i}}=\underset{\mid \mathcal{M}\mid \times \mid {\mathcal{J}}_p\mid }{\underbrace{\mathit{\varphi }}}\underset{\mid {\mathcal{J}}_p\mid \times 1}{\underbrace{{\mathit{\theta }}_i}}+\underset{\mid \mathcal{M}\mid \times 1}{\underbrace{{\mathit{\eta }}_i}},$$ (A1)

where ϕ represents a matrix of the factor loadings estimated in first step and M_i is the vector of stacked measures for participant i subtracting the intercepts ${\nu }_{m^j}^j$ of equation (8). The dimension of each element in equation (A1) is shown beneath it, where = ∪_j∈ is the union of all the index sets of the measures , j ∈ . The error term for agent i, η_i, has zero mean and is independent of the vector of skills θ_i. Cov(η_i, η_i) = Ω. The most commonly used estimator of factor scores is based on a linear function of measures: θ_S,i = L′M_i. Unbiasedness requires that L′ϕ = I_|p|, where I_|p| is a |p|-dimensional identity matrix.⁷⁷ To achieve unbiasedness, L must satisfy L′ = (ϕ′Ω⁻¹ ϕ) ⁻¹ ϕ′Ω⁻¹. The unbiased estimator of the factor is:

$${\mathit{\theta }}_{S,i}={\mathit{L}}^{\prime }{\mathit{M}}_i={({\mathit{\varphi }}^{\prime }{\mathit{\Omega }}^{-1}\mathit{\varphi })}^{-1}{\mathit{\varphi }}^{\prime }{\mathit{\Omega }}^{-1}{\mathit{M}}_i.$$

Factor score estimates can be interpreted as the output of a GLS estimation procedure where measures are taken as dependent variables and factor loadings are treated as regressors. By the Gauss-Markov theorem, for a known ϕ the proposed estimator is the best linear unbiased estimator of the vector of skills θ_i.⁷⁸

Step 3

The use of factor scores instead of the true factors to estimate equation (5) generates biased estimates of outcome coefficients α. Even though estimates of θ_i are unbiased, there is still a discrepancy between the true and measured θ_i due to estimation error. To correct for the bias, we implement a bias-correction procedure. Because we estimate the variance of θ and the variance of the measurement errors in the first step of our procedure, we can eliminate the bias created by the measurement error.

Consider the outcome model for agent i:

$$Y_i=\mathit{\alpha }{\mathit{\theta }}_i+\mathit{\gamma }{\mathit{Z}}_i+{\mathit{\epsilon }}_i,$$ (A2)

where (θ_i, Z_i) ε_i and E(ε_i) = 0. For brevity of notation, we use Z_i to denote pre-program variables, treatment status indicators, and the intercept term of equation (5). From equation (A1), the factor scores θ_S,i can be written as the skills θ_i plus a measurement error V_i, that is,

$${\mathit{\theta }}_{S,i}={\mathit{\theta }}_i+{\mathit{V}}_i\text{such}\text{that}({\mathit{Z}}_i,{\mathit{\theta }}_i)⫫{\mathit{V}}_i\text{and}E({\mathit{V}}_i)=0.$$ (A3)

Replacing θ_i with θ_S,i yields Y_i = αθ_S,i + γZ_i + ε_i − αV_i. The linear regression estimator of α and γ is inconsistent:

$$\text{plim}\left(\begin{array}{c}\widehat{\mathit{\alpha }}\\ \widehat{\mathit{\gamma }}\end{array}\right)=\underset{\mathit{A}}{\underbrace{{\left(\begin{array}{cc}\text{Cov}({\mathit{\theta }}_S,{\mathit{\theta }}_S)& \text{Cov}({\mathit{\theta }}_S,\mathit{Z})\\ \text{Cov}(\mathit{Z},{\mathit{\theta }}_S)& \text{Cov}(\mathit{Z},\mathit{Z})\end{array}\right)}^{-1}\left(\begin{array}{cc}\text{Cov}(\mathit{\theta },\mathit{\theta })& \text{Cov}(\mathit{\theta },\mathit{Z})\\ \text{Cov}(\mathit{Z},\mathit{\theta })& \text{Cov}(\mathit{Z},\mathit{Z})\end{array}\right)}}\left(\begin{array}{c}\mathit{\alpha }\\ \mathit{\gamma }\end{array}\right).$$ (A4)

This is the multivariate version of the standard one-variable attenuation bias formula. All covariances in A can be computed directly except for the terms that involve θ. The covariance Cov(θ, θ) is estimated in step (1). Using equation (A3), we can compute Cov(Z, θ_S) = Cov(Z, θ). Thus, A is identified. Our bias-correction procedure consists of pre-multiplying the least squares estimators (α̂, γ̂) by A⁻¹, thus providing consistent estimates of (α, γ).⁷⁹ A one step maximum likelihood procedure, while less intuitive, directly estimates the parameters without constructing the factors and accounts for measurement error. It is justified in large samples under standard regularity conditions. It produces estimates very close to those obtained from the three step procedure but with smaller standard errors. See Web Appendix L.