Children’s Academic Achievement and Foster Care

Abstract

BACKGROUND AND OBJECTIVES:

Poor school outcomes for children in out-of-home placement (OHP) raise concerns about the adequacy of child welfare and educational policy for this vulnerable population. We analyzed the relation between OHP and academic achievement, focusing on reading and math achievement in grades 3 through 8.

METHODS:

Linked administrative data were used for our analytic sample comprising 529 597 child-year observations for 222 049 children who experienced OHP or were in a comparison group. Three models were estimated: a pooled ordinary least squares regression that considered placement status and test scores net of the full set of control variables; an identical model that added the previous year’s test scores as an additional control; and a final model that included child-specific fixed effects.

RESULTS:

Children in OHP settings had achievement test scores at least 0.6 SD below average. However, we found similar deficits across children with past, current, and future exposure to OHP and, in our preferred model, OHP (past, current, or future placement) had no statistically discernible relation with either reading or math achievement.

Conclusions:

OHP by itself is not significantly related to school achievement; however, evidence reveals consistently low average math and reading achievement among children involved with Child Protective Services.

What’s Known on This Subject:

There is extensive literature documenting that children experiencing foster care placement have myriad adverse developmental outcomes, including poor academic achievement. However, such children face a host of other risk factors that may jeopardize healthy development independent of foster care placement.

What This Study Adds:

Using statewide administrative data from Wisconsin, we observed children before, during, and after foster care placement and compared their educational outcomes with those of the general population, as well as with children more similar in terms of unobserved characteristics.

Children in out-of-home placement (OHP) as a result of Child Protective Services (CPS) involvement are at increased risk of adverse school outcomes relative to the general population of children. The implication of this finding depends critically on the role that OHP plays. For example, if children in care have a history of low achievement, then a continued achievement gap may simply demonstrate that placement does not compensate for past disadvantage. Alternatively, if OHP significantly compromises achievement, this unintended consequence would have important implications for policy and practice. The present study analyzed the relation between OHP and academic achievement by using statewide longitudinal administrative data from Wisconsin to produce a range of estimates of the potential effect of OHP on achievement, as captured by reading and math scores.

An estimated 6.3 million children were reported to CPS in the United States in 2012, and >460 000 children reside in some form of OHP as a result of CPS involvement.¹ Furthermore, ∼6% of all US children and 12% of black children will experience OHP by 18 years of age²; thus, CPS involvement and OHP are not uncommon experiences, particularly for black (and, more generally, disadvantaged) children. This finding, coupled with the fact that children in OHP are explicitly under the guardianship of the state, emphasizes the importance of understanding whether, how, and under what conditions OHP may be beneficial, harmful, or neutral for child development.

The first priority of CPS is to ensure that children are safe, either in their own homes or, if not possible, in an OHP. Since the Adoption and Safe Families Act of 1997, CPS agencies have also been accountable for promoting children’s educational, physical, and mental health needs.³ Individuals who experience OHP as a consequence of CPS involvement exhibit poorer average developmental outcomes in both childhood and adulthood than individuals who have never experienced OHP.^4–8 Children in OHP settings are also considerably more likely than other children to experience a host of risk factors, including child maltreatment, poverty, parental substance abuse, and mental health problems, which are likely to jeopardize their development independent of OHP. ^9–12 Thus, research has not established whether poor developmental outcomes are the result of OHP or whether the association between OHP and adverse outcomes is spuriously driven by other factors.

There are a number of pathways through which OHP may affect achievement. First, the disruption associated with placement, which involves separation from a child’s primary caregiver(s) and, often, siblings and other family members, may cause emotional and social stress and thereby reduce school achievement. In addition, OHP may have a direct effect on achievement due to associated changes in home and school environments. If a child is removed from a chaotic or abusive home and placed in a high-quality OHP setting, achievement may be improved, especially after an initial transition period. It is also possible that placement will be associated with a decline in support for educational achievement; however, child welfare policies and expectations of substitute care providers should limit this possibility. Changing schools as a result of placement may have further implications for achievement. A new school may be of higher or lower quality, and the setting may be a better or worse match for the student, both academically and socially; effects may also vary over time if a new school presents benefits (or challenges) that are realized only after a period of transition and stabilization.

We considered the relation between OHP and academic achievement as measured by standardized test scores. The fundamental challenge to identifying the effect of OHP on educational outcomes is that removal from the home is not independent of either individual or environmental factors that are also associated with educational outcomes. Children from homes that require CPS intervention, and especially OHP, can be expected (all else equal and on average) to perform less well in school, and previous research has shown that children experiencing OHP exhibit poorer school-related outcomes across a host of domains.^13–17 To the extent that these differences are related to measurable characteristics (eg, poverty status), we can adjust for them statistically. However, when comparing children in an OHP versus children in the general population, unmeasured differences are likely to bias estimates.^10,17,18 An additional limitation of many earlier studies is that they were unable to observe children before, during, and after OHP and/or relied on relatively small samples selected based on CPS involvement.

Given these concerns, we used statewide administrative data from Wisconsin, in which we observed children before, during, and after OHP, to compare outcomes for children in an OHP with those of the general population, as well as with several subsamples that we expect to be more similar: children who were the subject of a screened-in CPS call but were not removed to an OHP, and children from economically disadvantaged families, as indicated by receipt of Supplemental Nutrition Assistance Program (SNAP) benefits.

Methods

Data

The study used longitudinal linked administrative data from the Wisconsin Department of Public Instruction (DPI) and the 2011 Multi-Sample Person File database housed at the Institute for Research on Poverty at the University of Wisconsin–Madison. The DPI data include information on academic performance, attendance and behavior, and basic demographic characteristics for all children in Wisconsin public schools from the 2005–2006 through 2011–2012 school years. The Multi-Sample Person File database includes linked individual-level administrative data from a host of public social welfare programs, including Wisconsin’s CPS system.

The Wisconsin Knowledge and Concepts Examination is a statewide standardized test that measures student achievement. It is administered to students enrolled in Wisconsin public schools during the fall of each school year. Reading and math tests are administered in grades 3 through 8, as well as grade 10. We focused our primary analyses on reading and math test scores for students in grades 3 through 8 to examine these scores across consecutive years. Our analytic sample comprised 529 597 child-year observations for 222 049 children who experienced OHP or were in a comparison group, had nonmissing math or reading test score data, and for whom we had information on days suspended from school (mean imputation was used to replace missing values on all other covariates).¹⁹

Measures

Academic Achievement

Academic achievement in grades 3 through 8 were assessed by using children’s math and reading scores on the Wisconsin Knowledge and Concepts Examination, standardized according to grade and year.

Placement Status

Placement status was measured by using 5 mutually exclusive dichotomous indicators, including: (1) the child was in an OHP at the time of the test; (2) the child was in an OHP at some point in the 12 months before the test but in-home at the time of the test (the month of October in a given year); (3) the child had been investigated by CPS at some point between 1 and 5 months preceding the test and was not removed from the home before the test, but he or she was removed from the home after the test and within 6 months of the investigation; (4) the child was investigated by CPS at some point in the 12 months before the test but was not removed from the home before the test or within 6 months of the investigation (if the investigation occurred between 1 and 5 months before the test); and (5) the child received SNAP in the 12 months before the test but did not experience a CPS investigation or OHP. Category 1, in which a child was in OHP at the time of the test, was our primary condition of interest. Categories 2 through 5 represent counterfactual groups ordered from most to least comparable to the treatment condition.

Controls

We controlled for the following factors: child female; child race/ethnicity (white, Asian, black, Hispanic, and Native American); child was a member of a household that met the income eligibility guidelines for free school lunch in the month before the test; child grade retention during the previous school year; days the child was suspended in the year before the test; child’s English language proficiency; mother’s age; family structure in the year of the test; number of children in the household in the year of the test; total family wages reported to the unemployment insurance system in the year before the year of the test; parental incarceration in the past 15 years; parental incarceration in the year before the test; an indicator representing the school accountable for the student at the time of the test; an indicator for year of observation; and age of the student.²⁰

Models

We first presented bivariate comparisons among our 5 samples of children and then estimated a series of regression models of the relation between OHP and school achievement. Each model includes indicators for which OHP status the child experienced in the 12 months before a given test. The SNAP-only group is the reference group in all models. Three models were estimated: a pooled ordinary least squares (OLS) regression that considered placement status and test scores net of the full set of control variables; an identical model that added the previous year’s test score as an additional control; and a final model that included child-specific fixed effects. In the final model, each child serves as his or her own comparison, and we essentially estimated the change in achievement associated with a change in placement status for the same child. The method expresses each variable as a deviation from a child’s mean value (over time) on that measure and differences the regression equation across time periods. This eliminates all time-invariant observed and unobserved variables from the model and thereby prevents the coefficients from being biased by such factors. However, fixed effects estimates are subject to bias if unobserved variables (or their effects on the outcome) are time-varying. Our fixed effects models estimated the change in a given child’s test score (relative to his or her mean test score across all observation points) as a function of changes in placement status and changes in the control variables.

Results

According to our comparison of children’s test scores by placement status, in each case, we compared outcomes with the average outcome for all children taking the test. The first bar in each cluster in the left panel of Fig 1 shows the average reading score for children in OHP at the time of the test, according to age: children in OHP had scores that were 0.60 (at age 8 years [generally, in grade 3]) to 0.84 (at age 13 years [generally, in grade 8]) of an SD below average for the population of children taking the test at each age (normalized to have a mean of 0 and SD of 1), with the discrepancy generally increasing with age. Math scores showed a similar pattern. These deficits for children in care, relative to the general population of children taking the test, are substantively significant. However, children in an OHP are likely to have experienced disadvantages associated with lower school achievement. A comparison with scores for the other groups suggests that a substantial portion of the negative relation between OHP and test scores is accounted for by factors other than placement itself, because, for example, children with screened-in calls but no OHP also have low scores relative to the general population, as do those who participated in SNAP. Of particular note, the achievement of children in an OHP at the time of the test did not differ significantly from the achievement of those who returned home before the test, nor did it vary significantly from the achievement of those placed in an OHP after the test.

FIGURE 1.

Average of standardized test z scores according to age and OHP status. Means were calculated by using the unbalanced sample of students who took the regular test and who were aged 8 to 13 years during school years 2005–2006 through 2011–2012.

Columns 1 through 3 of Table 1 present results for the standard OLS (column 1), OLS with a lagged dependent variable (column 2), and OLS with child-specific fixed-effects regressions, for reading. Columns 4 through 6 present the same results for math. The SNAP-only group is the reference group in all models; this comparison group reduces the potential bias from unobserved heterogeneity, relative to a comparison with the full population. As discussed earlier, estimates from the standard OLS model are particularly vulnerable to bias because of unmeasured differences between children; the fixed-effects model is least vulnerable to such bias. The standard OLS results indicate that all 4 CPS-involved groups had lower reading and math scores than the SNAP-only group. Furthermore, the pattern of coefficients suggests that the largest deficits were experienced by those who were investigated but placed in OHPs later. Indeed, this group had significantly lower reading and math scores than all other groups. Scores for children who were placed at the time of the test, before but not at the time of the test, and after the test were generally similar. This finding suggests that, whereas children in OHP when taking the test exhibit lower achievement than non–CPS-involved children receiving SNAP, they fare equally to or better than other CPS-involved children with respect to math and reading. If anything, the children who fare the worst are those who were investigated and remained home at the time of the test but would subsequently be removed. Adding the lagged dependent variable (columns 2 and 5) substantially attenuates all the placement status coefficients. However, the general pattern of results does not change.

TABLE 1.

Estimations on Standardized Reading and Math Test Scores (Main Explanatory Variables)

Variable	Reading			Math
	Pooled OLS	Pooled OLS Controlling for Previous Test Score	Child Fixed Effects	Pooled OLS	Pooled OLS Controlling for Previous Test Score	Child Fixed Effects
	(1)	(2)	(3)	(4)	(5)	(6)
Base = only SNAP in the previous year
OHP during test	−0.154 [0.018]***	−0.029 [0.011]**	−0.008 [0.020]	−0.202 [0.017]***	−0.053 [0.011]***	−0.017 [0.019]
OHP before test	−0.124 [0.017]***	−0.039 [0.015]**	−0.004 [0.019]	−0.185 [0.017]***	−0.088 [0.015]***	−0.048 [0.018]***
Screened-in and placed in OHP later	−0.33a,b [0.043]***	−0.16a,b [0.038]***	−0.086 [0.043]**	−0.326a,b [0.039]***	−0.163a [0.036]***	−0.063 [0.038]
Screened-in but not placed in OHP	−0.18b,c [0.007]***	−0.06a,c [0.005]***	−0.011 [0.006]*	−0.172c [0.006]***	−0.056b,c [0.005]***	−0.012 [0.006]**
Previous test score		0.712 [0.002]***			0.736 [0.002]***
Age (as of September 1) (base = 8)
9 y	−0.075 [0.004]***	−0.084 [0.022]***	−0.019 [0.004]***	−0.033 [0.003]***	−0.07 [0.028]**	0.014 [0.004]***
10 y	−0.105 [0.004]***	−0.098 [0.022]***	−0.033 [0.006]***	−0.061 [0.004]***	−0.107 [0.028]***	0 [0.005]
11 y	−0.171 [0.006]***	−0.099 [0.022]***	−0.019 [0.008]**	−0.124 [0.005]***	−0.113 [0.028]***	0.007 [0.007]
12 y	−0.164 [0.007]***	−0.066 [0.022]***	−0.004 [0.010]	−0.132 [0.006]***	−0.095 [0.028]***	−0.002 [0.009]
13 y	−0.153 [0.007]***	−0.049 [0.022]**	0.012 [0.011]	−0.14 [0.006]***	−0.088 [0.028]***	−0.017 [0.010]
Female	0.151 [0.004]***	0.046 [0.002]***		−0.066 [0.004]***	−0.013 [0.002]***
Race/ethnicity (base = white)
Asian	−0.075 [0.013]***	0.004 [0.006]		0.039 [0.013]***	0.041 [0.006]***
Black	−0.302 [0.008]***	−0.088 [0.004]***		−0.387 [0.008]***	−0.091 [0.004]***
Hispanic	−0.091 [0.009]***	−0.028 [0.004]***		−0.133 [0.009]***	−0.035 [0.004]***
Native American	−0.161 [0.017]***	−0.053 [0.008]***		−0.186 [0.015]***	−0.05 [0.007]***
Eligible for free school lunch (base = no)
Yes	−0.105 [0.006]***	−0.031 [0.005]***	0.005 [0.007]	−0.079 [0.006]***	−0.018 [0.004]***	0.013 [0.006]**
Missing	−1.105 [0.230]***	−0.562 [0.600]	−0.104 [0.265]	−0.876 [0.184]***	−0.275 [0.292]	−0.008 [0.167]
Retained in the previous school year	−0.022 [0.014]	0.29 [0.017]***	0.203 [0.016]***	−0.07 [0.013]***	0.301 [0.015]***	0.21 [0.015]***
Days suspended in the previous school year (base = none)
1	−0.236 [0.007]***	−0.073 [0.006]***	−0.007 [0.007]	−0.202 [0.006]***	−0.073 [0.005]***	−0.003 [0.007]
2	−0.297 [0.009]***	−0.105 [0.008]***	−0.012 [0.010]	−0.247 [0.009]***	−0.091 [0.007]***	−0.006 [0.009]
3	−0.296 [0.011]***	−0.101 [0.009]***	−0.009 [0.012]	−0.256 [0.010]***	−0.099 [0.008]***	−0.007 [0.011]
≥4	−0.413 [0.009]***	−0.122 [0.006]***	−0.009 [0.010]	−0.359 [0.008]***	−0.126 [0.006]***	−0.027 [0.010]***
Days suspended, imputed	0.032 [0.008]***	−0.019 [0.062]	−0.023 [0.009]**	−0.068 [0.008]***	−0.011 [0.080]	−0.065 [0.009]***
English language proficiency (base = never an ELL)
Entering	−1.862 [0.045]***	−0.409 [0.058]***	−0.617 [0.071]***	−1.123 [0.030]***	−0.258 [0.041]***	−0.194 [0.043]***
Beginning	−1.361 [0.020]***	−0.303 [0.022]***	−0.182 [0.037]***	−0.867 [0.017]***	−0.266 [0.015]***	−0.074 [0.029]**
Developing	−0.667 [0.011]***	−0.179 [0.008]***	−0.011 [0.026]	−0.452 [0.010]***	−0.137 [0.007]***	−0.043 [0.024]*
Expanding	−0.146 [0.010]***	−0.05 [0.006]***	0.036 [0.025]	−0.047 [0.010]***	−0.02 [0.005]***	−0.002 [0.023]
Bridging	0.225 [0.011]***	0.067 [0.007]***	0.06 [0.025]**	0.278 [0.012]***	0.065 [0.007]***	0.017 [0.024]
Fully ELP	0.439 [0.014]***	0.141 [0.007]***	0.102[0.026]***	0.464 [0.015]***	0.13 [0.007]***	0.048 [0.024]**
Mother’s age	0.003 [0.000]***	0.001 [0.000]***		0.003 [0.000]***	0.001 [0.000]***
Mother’s age, imputed	0.134 [0.025]***	0.076 [0.016]***		0.103 [0.024]***	0.036 [0.015]**
Family structure (base = 2 parents)
Single mother	0.01 [0.005]*	−0.001 [0.003]	−0.003 [0.008]	0.022 [0.005]***	0.004 [0.002]*	−0.008 [0.007]
Single father	−0.002 [0.010]	−0.002 [0.005]	0.004 [0.015]	0.042 [0.010]***	0.012 [0.005]***	−0.009 [0.014]
Missing	−0.125 [0.084]	−0.039 [0.056]		−0.189 [0.082]**	−0.058 [0.054]
No. of children (base = 1)
2	−0.017 [0.007]**	0.001 [0.004]	0.015 [0.010]	0.028 [0.006]***	0.011 [0.003]***	0.005 [0.009]
3	−0.047 [0.007]***	−0.008 [0.004]**	−0.001 [0.012]	0.031 [0.007]***	0.019 [0.003]***	0.005 [0.011]
4	−0.08 [0.008]***	−0.015 [0.004]***	−0.006 [0.013]	0.018 [0.008]**	0.015 [0.004]***	0.003 [0.012]
≥5	−0.152 [0.010]***	−0.033 [0.005]***	−0.018 [0.016]	−0.023 [0.009]**	0.014 [0.004]***	0.003 [0.014]
No. of children, imputed	0.009 [0.010]	−0.007 [0.006]	0.013 [0.014]	−0.011 [0.009]	−0.01 [0.006]*	0.023 [0.013]*
Wage in previous year (base = zero)
<2000	−0.008 [0.007]	0.001 [0.005]	0.003 [0.006]	−0.011 [0.006]*	−0.002 [0.004]	0.005 [0.006]
2–6000	0.024 [0.006]***	0.01 [0.004]**	0.005 [0.006]	0.022 [0.006]***	0.005 [0.004]	0.01 [0.005]*
6–10 000	0.018 [0.007]***	0.003 [0.004]	−0.001 [0.006]	0.016 [0.006]***	0.006 [0.004]	0.005 [0.006]
10–15 000	0.034 [0.006]***	0.005 [0.004]	0.003 [0.006]	0.04 [0.006]***	0.013 [0.004]***	0.011 [0.006]*
15–20 000	0.037 [0.007]***	0.009 [0.004]**	0.008 [0.007]	0.043 [0.006]***	0.007 [0.004]*	0.006 [0.006]
20–30 000	0.054 [0.006]***	0.011 [0.004]***	0.005 [0.007]	0.064 [0.006]***	0.014 [0.003]***	0.006 [0.006]
30–40 000	0.086 [0.008]***	0.024 [0.005]***	0.009 [0.008]	0.095 [0.007]***	0.026 [0.004]***	0.011 [0.007]
≥40 000	0.123 [0.008]***	0.033 [0.004]***	0.006 [0.009]	0.139 [0.008]***	0.039 [0.004]***	0.015 [0.008]*
Parent incarcerated in the past 15 y	−0.075 [0.008]***	−0.032 [0.004]***		−0.066 [0.007]***	−0.029 [0.004]***
Parent incarcerated	0.005 [0.010]	0.008 [0.006]	0.014 [0.010]	0.01 [0.009]	0.014 [0.005]**	0.01 [0.009]
Constant	−0.239 [0.020]***	0.005 [0.067]	−0.628 [0.109]***	−0.215 [0.020]***	0.014 [0.085]	−0.627 [0.095]***
Adjusted R2	0.1749	0.6042	0.7476	0.1889	0.6281	0.7611
N	526 102	348 430	526 102	528 810	350 536	528 810
No. of clusters (mothers)	114 796	93 611	114 796	115 161	93 913	115 161

ELL, English language learners; ELP, English language proficient; SI = CPS investigation. *P < .1; **P < .05; ***P < .01. Robust and clustered SEs according to mother in brackets. All estimations include school fixed effects and year fixed effects, but they are not presented in the table due to space limit.

^aDiffers significantly from OHP/during test, at P < .05.

^bDiffers significantly from OHP/before test, at P < .05.

^cDiffers significantly from SI/placed OHP later, at P < .05.

The final model (columns 3 and 6), which includes child-specific fixed effects, identifies differences in academic achievement by using within-child variation in placement status. As such, it provides the most rigorous adjustment for social selection of the 3 models. The results indicate that children have achievement scores that are relatively similar regardless of the placement status in which they are observed. Indeed, few coefficients in each model are significant. Relative to being observed when receiving SNAP only, when children were investigated but not immediately placed (whether they would be placed later), they performed worse in reading, and when they were in an OHP before but not at the time of the test, as well as when they were investigated and not immediately or subsequently removed, they performed more poorly in math. Furthermore, none of the coefficients for the CPS-involved groups differ significantly from one another. Again, these results suggest that some forms of CPS involvement are negatively associated with academic achievement. However, only a screened-in report with no subsequent OHP is significantly associated with both poorer reading and math; in addition, being in an OHP had no statistically discernible relation to reading or math scores in our preferred model.

Conclusions

Consistent with previous research, our bivariate analyses indicate that children experiencing OHP have considerably poorer reading and math achievement scores than the general population of children.^4–8 However, in contrast to earlier analyses, we considered a more focused and relevant set of comparisons that support fundamentally different conclusions. We found that the difference in scores between children experiencing OHP and those receiving SNAP, a socioeconomically disadvantaged group that should be more similar to those experiencing OHP than are children in the general population, is considerably smaller. In addition, we found relatively few differences in achievement between children in OHP and other CPS-involved children. These findings suggest that, whereas all CPS-involved children exhibit poorer achievement than non–CPS-involved children (including SNAP participants), OHP is not particularly salient with regard to academic achievement among children investigated by CPS. This finding was confirmed by our multivariate analyses. In our most rigorous specifications (OLS regressions with child-specific fixed effects), we find no differences in achievement by level of CPS involvement; all CPS-involved children seem to perform similarly.

We also conducted additional analyses (not reported here), which provide suggestive evidence that children perform slightly worse in the early stages of placement or during short placements than after being in care for a longer period. Thus, future research should consider a longer observation period and explicitly consider changes in placement settings (ie, moves from one care provider to another) and changes in schools attended. The analyses reported here include only children who took the standard Wisconsin achievement test and not those who took alternative tests. Supplemental analyses (data not shown) suggest that the association between a child’s previous test score and whether the child subsequently took the standard achievement test does not differ by placement status; we also find little to suggest that those children who would have experienced the largest decline in achievement as a result of OHP are disproportionately likely to be missing from our achievement score analyses. Nonetheless, future research should further examine which children in an OHP are most likely to switch from standard achievement tests to alternative tests or, indeed, to forgo taking achievement tests. Test scores are an important indicator of child outcomes, especially in the context of growing reliance on high-stakes testing. Future data system improvements should support the analysis of additional outcomes, including attendance, school moves, and disciplinary outcomes. Considering a broader array of outcomes would strengthen our understanding of the relation between OHP and educational achievement. Finally, we note that our analysis was restricted to children in Wisconsin, a state that experienced relatively large declines in foster care populations²¹ and has a modestly low placement rate²²; we might expect different patterns in states with higher placement rates, especially if they result from the removal of children otherwise at less risk of maltreatment.

Our results imply that OHP, by itself, is not significantly related to school achievement. This finding suggests that a focus on reducing the negative educational consequences of OHP may be misplaced. However, whereas we found no evidence of a causal relationship between OHP and achievement, we did find consistent evidence of low average math and reading achievement among children involved with CPS. Thus, pediatricians should be concerned about the cognitive development of all children experiencing maltreatment. Likewise, practice and policy would be well served by focusing on the need to support achievement among all socioeconomically vulnerable children, as well as all CPS-involved children, rather than focusing specifically on the consequences of OHP on school achievement.