Mental health predicts better academic outcomes: A longitudinal study of elementary school students in Chile

Abstract

The world’s largest school-based mental health program, Habilidades para la Vida [Skills for Life, SFL], has been operating at a national scale in Chile for fifteen years. SFL’s activities include using standardized measures to screen elementary school students and providing preventive workshops to students at risk for mental health problems. This paper used SFL’s data on 37,397 students who were in first grade in 2009 and third grade in 2011 to ascertain whether first grade mental health predicted subsequent academic achievement and whether remission of mental health problems predicted improved academic outcomes. Results showed that mental health was a significant predictor of future academic performance and that, overall, students whose mental health improved between first and third grade made better academic progress than students whose mental health did not improve or worsened. Our findings suggest that school-based mental health programs like SFL may help improve students’ academic outcomes.

Introduction

A large and growing body of research from studies in the United States and throughout the world has demonstrated a significant relationship between mental health and academic performance in children and adolescents.^1-19 Both cross-sectional and longitudinal studies have found that children who experience mental health problems are more likely to have unfavorable educational outcomes, including poor grades,^{1-4,7,10,13-15,17,19} delays in reading,^5,6,18 grade repetition,^2,5 school drop-out ideation,² and lower educational attainment than their peers without mental health problems.^9,11,12 To provide just one example, a recent national study of more than 11,000 students in Chile found that mental health in first grade, as assessed with brief standardized teacher and parent screens, was one of the strongest predictors of performance on national achievement tests of language, mathematics, and science three years later.¹⁶

Although there have been only a few studies to date, recent research suggests that improving mental health is associated with improved academic outcomes. For example, two studies on the trajectories of ADHD found that elementary school children with remittent ADHD symptoms were less likely to receive detention,²⁰ more likely to remain in mainstream classes,²⁰ and more likely to eventually graduate from high school²¹ than students whose symptoms had not remitted.

Based on such findings, a growing number of school-based programs have included substantial mental health components in their academic curricula, including Head Start in the United States²² and Habilidades para la Vida in Chile [translated as Skills for Life in English; Guzman et al (2011)].^16,23 However, the inclusion of mental health programs in schools is not yet mainstream, possibly because the empirical evidence supporting their effectiveness is limited. Research studies supporting the link between mental health and academic performance have used relatively small samples,^15,18 lacked prospective data,^11,17-19 used only a single measure of mental health,^17,18 and/or relied on self-report data of academic functioning.¹⁷

The current study overcame these limitations by using a large, longitudinal, epidemiological sample of 37,397 elementary school children for whom standardized mental health and academic data were collected routinely as a part of an ongoing intervention (Skills for Life) run by a governmental agency (the Junta Nacional de Auxilio Escolar y Becas [JUNAEB; National Association of School Assistance & Scholarship]) in Chile. We examined the relationship between mental health and academic performance for children who were in first grade in 2009 and third grade in 2011 and hypothesized that: (1) first grade mental health would predict first and third academic performance and (2) remission from mental health problems identified in first grade would predict improved academic performance from first to third grade. An association between improving mental health and improved academic performance would suggest that participation in an effective mental health intervention, such as the Skills for Life (SFL) workshops in Chile, could improve elementary school students' academic progress and psychosocial development.

Method

Sample and Procedures

Data for this study came from a sample of Chilean students who attended schools that participated in the SFL Program in 2009 and 2011. SFL is available on a voluntary basis to selected public and government-subsidized private schools in Chile which are defined as ‘high risk,’ based in part on an algorithm created by the World Health Organization which takes factors such as family income and maternal education level into account.¹⁶ All students who attended SFL-participating schools in first grade in 2009 (n=61,807) and third grade in 2011 (n=56,476) were eligible for inclusion in the current study.

The dataset used in this study is a merged composite from four different datasets. JUNAEB created two datasets with TOCA-RR and/or PSC-CL screening data (see below for description of measures), one for students who attended SFL schools as first graders in 2009 and the second for students who attended SFL schools as third graders in 2011. The other two datasets included academic information for all students in Chile who were in first grade in 2009 and third grade in 2011. JUNAEB uses official academic information on a regular basis. The JUNAEB Program Coordinator used individual student identifiers (similar to United States social security numbers) to merge the datasets before removing all identifying information and sending the combined dataset to the United States for the analyses reported on here. All study procedures were reviewed and approved as exempt by the Partners Human Research Committee as well as the requisite authorities in Chile.

Measures

Outcomes

Academic performance in first and third grade was assessed using students’ end-of-year grade point averages (GPAs) and the percent of school days attended. These are standard fields in the national database maintained by the Ministry of Education for all students in Chile. GPAs are calculated in the same way for all students by averaging students’ performance on all academic subjects. For each subject, grades can range from 1 (worst grade possible) to 7 (best grade possible). Since this differs from the scale used in the United States, for US readers we converted these GPAs into percentile ranks among all students in our dataset who had valid GPAs in 2009 (n=51,912) and 2011 (n=54,623).

Predictors

Mental health risk was assessed in first and third grade using two standardized screening instruments: 1) the Teacher Observation of Classroom Adaptation-Revised-Chilean Version (TOCA-RR); and 2) the Pediatric Symptom Checklist-Chilean Version (PSC-CL). Both measures were originally developed and are still used in the United States.

The Teacher Observation of Classroom Adaptation-Revised (TOCA-R) is a valid and reliable measure of children’s classroom performance and behavior which has been used in studies of US youth for more than two decades.²⁴ The TOCA-RR is a modified version of the TOCA-R. It was created in the 1990s by Chilean investigators who worked with the US scale developers to translate, adapt, and validate the TOCA-R for a Chilean context.^25,26 Cronbach alpha values range from .92 to .96 on the TOCA-R’s subscales²⁴ and .74 to .95 on the TOCA-RR’s subscales.²⁵ Like the TOCA-R, the TOCA-RR is a 31-item structured interview administered to the teacher of each student by staff, in this case a mental health worker from the SFL Program. Each item is rated on a six-point Likert-type scale ranging from 1 (almost never) to 6 (almost always). The TOCA-RR is administered at least four months into the school year, after teachers have gotten to know their students. At the end of first grade, SFL’s National Leadership Team examines each child’s score pattern on the TOCA-RR’s six subscales (e.g., authority acceptance, hyperactivity) and uses standardized algorithms²⁷ to ascertain whether the child screened positive for one of three specific types of mental health risk (aggressive-hyperactive, shy-oppositional, and inhibited-passive). Students who meet the criteria for at least one of these three profiles are considered to be at overall risk for mental health problems and are referred to group-administered, school-based preventive workshops that address one of the three specific types of problems. For this study, we used case/not case categorization (overall risk on the TOCA-RR) as the primary indicator of mental health problems or lack of them; children with aggressive-hyperactive, shy-oppositional, and/or inhibited-passive TOCA-RR profile(s) were considered at risk.

The Pediatric Symptom Checklist (PSC) is a 35-item, parent-completed questionnaire that assesses children’s overall psychosocial functioning.²⁸ The PSC is one of the most widely used psychosocial screens for children in the world. In the US, the PSC has been validated in a national sample and many subpopulations.²⁹ It is a recommended screening instrument in at least a half dozen states, and it has recently been endorsed for national and statewide use by the National Quality Forum,³⁰ a group charged with selecting measures for evaluating US healthcare.³¹ On a global scale, it has been translated into more than two dozen languages and validated in countries on five continents.³² The PSC has a Cronbach alpha of .91,^33,34 a test-retest reliability of r=.84-.91, and a specificity of .68 and sensitivity of .95 for detecting psychiatric impairment as assessed through a standardized psychiatric interview.³⁵

The PSC also went through an extensive adaptation to the Chilean context by the same team that validated the TOCA-RR.^25,26,36 The PSC-CL contains 33 items, each of which is reported by the parent as 1 (never), 2 (sometimes), or 3 (often present). Similar to the US version of the PSC, a total score is obtained by summing the scores for all items, with higher scores indicating more problems. Total scores are re-coded dichotomously based on a pre-determined cutoff score; for the PSC-CL, scores of 65 or higher indicate mental health risk.^26,36 The measure has a Cronbach alpha of .85.²⁵ In most participating schools, the PSC-CL was administered near the beginning of each school year (often at ‘meet the teacher night’).

Correlations between the TOCA-RR and PSC-CL were low-to-moderate in their initial validation study, justifying the complementary administration of both instruments since parents and teachers appeared to observe different features of children’s behavior.²⁵

Covariates

All adjusted models controlled for the following socio-demographic factors, which have been shown in previous studies to relate to mental health or academic outcomes: gender (0=female; 1=male),³⁷ family participation in welfare (known as Chile Solidario; 0=no receipt of welfare; 1=receipt of welfare),³⁸ and presence of each of five additional risk factors assessed on the PSC-CL form, each rated as (0=absent; 1=present):¹⁶ (1) whether the mother was a teenager when the child was born, (2) whether the father lives with the child, (3) whether the child has a chronic illness leading to one or more school absences a month, (4) whether the child lives with a relative disabled by mental illness, and (5) whether the family participates in organized social activities (e.g., church activities, neighborhood boards, and sports clubs).³⁹ In models examining third grade academic performance, we also included first grade academic performance as a covariate (either percent of school days attended^37,39 or GPA rank,⁸ selecting the one that was not the outcome measure in each analysis). Finally, we used school identification number to adjust for school effects.

Data Analysis

We used two primary analytic samples. The first and larger sample consisted of all students with complete data on the predictors: first grade TOCA-RR and PSC-CL screens; covariates: gender, family participation in welfare, family risk factors, and school in first grade; and outcomes: GPA rank and percent attendance in first and third grade (n=37,397). This sample included 61% of the 61,807 students who attended first grade in SFL-participating schools or 66% of the 56,476 students who attended third grade in SFL-participating schools. The second and smaller analytic sample was the subset of students who had complete data on all of the above variables and also on third grade TOCA-RR and PSC-CL screens (n=18,969; 51% of the first analytic sample).

We conducted these analyses in several steps. First, we used the larger analytic sample (n=37,397) to calculate the bivariate correlations between each of the variables from first grade and GPA ranks in first and third grade (Table 2). Second, we used multiple linear regression models to examine whether first grade mental health measures predicted first and third grade academic performance (Table 3). In Model 1, we examined the predictive power of first grade TOCA-RR and PSC-CL risk, without adjusting for covariates. In Model 2, we added all covariates in order to examine whether first grade TOCA-RR and PSC-CL risk predicted first and third grade academic performance beyond other known risk factors.

Table 2.

Correlations (Pearson’s r) between risk factor variables and first and third grade academic performance (n=37,397)

Variable	1		2
	r	r 2	r	r 2
1. 1st grade GPA percentile	--	--	0.65	0.42
2. 3rd grade GPA percentile	0.65	0.42	--	--
3. Percent attendance in 1st grade	0.28	0.08	0.17	0.03
4. TOCA-RR risk in 1st grade	−0.36	0.13	−0.25	0.06
5. PSC-CL risk in 1st grade	−0.27	0.07	−0.22	0.05
6. Male gender	−0.12	0.01	−0.12	0.01
7. Family participating in welfare	−0.16	0.03	−0.14	0.02
8. Mother was a teenager when child was born	−0.05	0.003	−0.05	0.003
9. Child not living with his/her father	−0.10	0.01	−0.09	0.01
10. Child has a chronic illness	−0.09	0.01	−0.05	0.003
11. Child living with a relative disabled by mental illness	−0.08	0.01	−0.07	0.005
12. Family does not participate in organized social activities	−0.03	0.001	−0.04	0.002

All p-values are .01

Table 3.

Multiple regression analyses predicting academic performance in first and third grade (n=37,397)

Model and predictor variable	r 2	β	tc
Predicting 1st grade GPA percentile
Model 1a: Unadjusted	0.17
Mental health risk on the TOCA-RR in 1st grade		−0.31	−57.87***
Mental health risk on the PSC-CL in 1st grade		−0.21	−40.49***
Model 2a: adjusted for covariates	0.24
Percent attendance in 1st grade		0.22	26.61***
Mental health risk on the TOCA-RR in 1st grade		−0.28	−53.41***
Mental health risk on the PSC-CL in 1st grade		−0.16	−33.55***
Male gender		−0.09	−16.90***
Family participating in welfare		−0.10	−17.53***
Mother was a teenager when child was born		−0.02	−4.14***
Child not living with his/her father		−0.04	−8.20***
Child has a chronic illness		−0.01	−3.02**
Living with a relative disabled by mental illness		−0.03	−6.12***
Family does not participate in organized social  activities		−0.02	−3.56***
Predicting 3rd grade GPA percentile
Model 1b: unadjusted	0.09
Mental health risk on the TOCA-RR in 1st grade		−0.22	−38.57***
Mental health risk on the PSC-CL in 1st grade		−0.17	−32.68***
Model 2b: adjusted for covariates	0.43
1st grade GPA percentile		0.63	88.99***
Percent attendance in 1st grade		−0.02	−2.07*
Mental health risk on the TOCA-RR in 1st grade		−0.02	−2.84**
Mental health risk on the PSC-CL in 1st grade		−0.03	−7.37***
Male gender		−0.04	−6.81***
Family participating in welfare		−0.04	−8.49***
Mother was a teenager when child was born		−0.01	−2.68**
Child not living with his/her father		−0.02	−3.84***
Child has a chronic illness		0.01	1.15
Living with a relative disabled by mental illness		−0.01	−3.36**
Family does not participate in organized social  activities		−0.01	−3.44**

^*p<.05;

^**p<.01;

^***p<.001

^cdf=1371

We then conducted an additional set of analyses with our smaller analytic sample of students who also had third grade TOCA-RR and PSC-CL screens (n=18,969). These analyses were designed to assess whether the evolution (positive or negative) of psychosocial risk from first to third grade was associated with changes in academic functioning over the same years. For these analyses, we grouped students’ psychosocial evolution into four mutually exclusive categories (we did this twice, once for each mental health screen [TOCA-RR and PSC-CL]) : (1) mental health risk in first grade, but not in third grade (i.e., ‘early risk only’); (2) mental health risk in both first and third grade (i.e., ‘early and persistent risk’); (3) not at mental health risk in first grade, but at risk in third grade (i.e., ‘later risk only’); and (4) not at mental health risk in first or third grade (‘never high risk’). We used multiple linear regression models to determine (1) whether early risk only students showed better academic progress than early and persistent risk students and (2) whether later risk only students fared worse academically than never high risk students. We did this separately for the TOCA-RR and PSC-CL and controlled for covariates in all analyses.

We adjusted for clustering by schools in all analyses by estimating robust variances using the Huber-White technique in SPSS Complex Samples. The Huber-White technique is preferred when data are not highly skewed and cluster membership is not misidentified, as in this study.⁴⁰

Results

Participant Characteristics

Table 1 presents descriptive statistics for study participants in the two primary analytic samples. In both the larger (n= 37,397) and smaller subsamples (n=18,969), GPA percentile ranks were slightly above the average of 50.00 (SE=.13 in first grade and SE=.12 in third grade) for the full sample (n=61,807 in first grade and n=56,476 in third grade). Both of our analytic subsamples slightly but significantly overrepresented higher-GPA students, the smaller subsample (M=57.77, SE=.19 in first grade and M=55.64, SE=.21 in third grade) even more so than the larger subsample (M=52.40, SE=.15 in first grade and M=51.60, SE=.15 in third grade).

Table 1.

Demographic data for the first (n=37,397) and second analytic subsamples (n= 18,969)

Variable	Students with 1st grade mental health screens (n=37,397)		Students with 1st and 3rd grade mental health screens (n=18,969)
	M	SE	M	SE
GPA percentile rank
1st grade	52.4	0.15	57.77	0.19
3rd grade	51.6	0.15	55.64	0.21
Percent attendance
1st grade	91.33	0.04	92.34	0.05
3rd grade	89.52	0.09	91.56	0.09
	N	%	N	%
Teacher-reported mental health risk (TOCA-RR)
1st grade risk	6,300	16.8	2,530	13.3
3rd grade risk	3,406a	14.3	2,564	13.5
Parent reported risk mental health (PSC-CL)
1st grade risk	5,241	14	2,152	11.3
3rd grade risk	2,396b	12.4	2,356	12.4
Gender
Male	19,098	51.1	9,323	49.1
Female	18,299	48.9	9,646	50.9
Family participating in welfare	7,186	19.2	3,440	18.1
Mother was a teenager when child was born	3,855	10.3	1,846	9.7
Child not living with his/her father	12,553	33.6	6,024	31.8
Child has a chronic illness	5,648	15.1	2,706	14.3
Child living with a relative disabled by mental illness	4,161	11.1	1,944	10.2
Family does not participate in organized social activities	20,988	56.1	10,464	55.2
Schools in 2009	1,372		1,204

SE = standard error

^aOnly 23,810 students had third grade data on TOCA-RR

^bOnly 19,263 students had third grade data on PSC-CL

We also found that students in our smaller analytic subsample had better outcomes on our other indicator of academic performance, percent of school days attended. Students in our larger subsample attended 91.3% (SE=.04) of days in first grade and 89.5% (SE=.09) of days in third grade, while students in our smaller subsample attended 92.3% (SE=.05) of days in first grade and 91.6% (SE=.09) of days in third grade.

According to teachers’ mental health assessments on the TOCA-RR, 16.8% of students in the larger subsample and 13.3% of the students in the smaller subsample were at risk in first grade. In third grade, 14.3% of students in the larger subsample (data was available for 23,810 of the students) and 13.5% of students in the smaller subsample were at risk. According to parents’ mental health assessments on the PSC-CL, 14% of students in the larger subsample and 11.3% of the students in the smaller subsample were at risk in first grade. In third grade, 12.4% of the students in both the larger (data was available for 19,263 of the students) and smaller subsamples were at risk.

Using first grade data to provide a demographic snapshot of the sample, the larger subsample included slightly more males (51%), while the smaller subsample included slightly more females (51%). In keeping with the program’s mandate to serve high-risk students, the sample was highly socio-economically disadvantaged: nearly one-fifth (18%-19%) of students were from a family that was participating in welfare. Students also experienced many family-related risk factors: 10% had mothers who were teenagers when they were born; 32%-34% were not living with their father; 14%-15% had a chronic illness leading to one or more school absences per month; 10%-11% were living with a relative disabled by mental illness; and 55-56% were from families that did not participate in organized social activities.

Correlations between risk factors and academic performance

Table 2 shows the bivariate relationships between risk factor variables measured in first grade and GPA ranks calculated in first and third grade. All first grade risk factors were significantly (p<.01) associated with lower GPA ranks at both time points, although the effect sizes (r²) were small. Outside of GPA rank itself, TOCA-RR risk had the largest correlations with first (r=−.36, r²=.13) and third grade (r=−.25, r²=.06) GPA rank and PSC-CL risk had the next largest correlations (r=−.27, r²=.07 with first grade GPA rank and r=−.22, r²=.05 with third grade GPA rank). TOCA-RR and PSC-CL risk had larger correlations with poorer academic performance than any other risk factors, with only one exception (PSC-CL did not have a significantly different correlation with first grade GPA rank than percent attendance [r=.28, r²=.08]).

First grade mental health and risk factors as predictors of academic performance

Table 3 summarizes the results of multiple linear regression analyses that examined the strength of first grade mental health and risk factors as predictors of first and third grade academic performance. Model 1 shows the predictive power of the two mental health screens without adjusting for covariates. Model 2 repeats these analyses, adjusting for all other variables measured in first grade.

Predicting first grade academic performance

In Model 1a (r²=.17), teacher- and parent-reported mental health on the TOCA-RR (β=−.31, t=−57.87) and PSC-CL (β=−.21, t=−40.49) both independently predicted first grade academic performance (p<.001). As shown in Model 2a (r²=.24), the predictive power of teacher- and parent-reported mental health remained significant when controlling for other risk factors: first grade TOCA-RR risk was the best predictor of first grade GPA rank (β=−.28, t=−53.41, p<.001) and first grade PSC-CL risk was the third strongest predictor (β=−.16, t=−33.55, p<.001), with only percent attendance in first grade any stronger (β=.22, t=26.61, p<.001). Demographic risk factors were less powerful than the mental health screens as predictors of achievement but still statistically significant, with family participation in welfare (β=−.10, t=−17.53) and gender (β=−.09, t=−16.90) having the greatest relative strengths among them (p<.001).

Predicting third grade academic performance

Model 1b (r²=.09) shows that TOCA-RR (β=−.22, t=−38.57) and PSC-CL (β=−.17, t=−32.68) risk in first grade also independently predicted third grade academic performance (p<.001). Model 2b appeared much stronger than all other models (r²=.43) because it included first grade GPA rank, a very powerful predictor of third grade GPA rank (β=.63, t=88.99, p<.001). The addition of first grade GPA rank weakened the predictive power of TOCA-RR risk (β=−.02, t=−2.84, p<.01) in relation to other variables, presumably due to the collinearity between the TOCA-RR score and student GPA (possibly because both were reported by the teacher). First grade risk on the PSC-CL remained one of the strongest predictors in the model (β=−.03, t=−7.37, p<.001). The only first grade risk factor that did not significantly predict third grade academic performance was having a chronic illness causing frequent absences (p=.25).

Longitudinal relationship between changes in mental health and changes in academic performance

Tables 4a and 4b show the relationship between changes in mental health risk and changes in GPA rank for the smaller analytic sample of students who had data on mental health screens from first and third grade (n=18,969).

Table 4a.

Grade point average percentile change from first to third grade for students with early (first grade only) versus early and persistent (first and third grade) mental health risk

Mental health risk change from 1st to 3rd grade	N	%	GPA percentile in 1st grade		GPA percentile in 3rd grade		GPA percentile change from 1st to 3rd grade		Cohen’s d
			%	SE	%	SE	%	SE
TOCA-RR risk in 1st grade***a	2,530	100
Early risk only	1,655	65.4	38.83	0.57	41.79	0.64	2.96	0.53	0.14
Early and persistent risk	875	34.6	31.41	0.68	26.38	0.73	−5.03	0.67	0.25
PSC-CL risk in 1st grade***b	2,152	100
Early risk only	1,219	56.6	42.76	0.7	43.03	0.75	0.27	0.63	0.01
Early and persistent risk	933	43.4	38.35	0.75	33.66	0.81	−4.68	0.71	0.22

SE = standard error

^***p<.001

^aChange in TOCA-RR risk independently predicted academic progress when we controlled for covariates; β=−0.23, t=−12.39, p<0.001

^bChange in PSC-CL risk independently predicted academic progress when we controlled for covariates; β=−0.14, t=−6.54, p<0.001

Table 4b.

Grade point average percentile change from first to third grade for students with later (third grade only) versus never high (neither first nor third grade) mental health risk

Mental health risk change from 1st to 3rd grade	N	%	GPA percentile in 1st grade		GPA percentile in 3rd grade		GPA percentile change from 1st to 3rd grade		Cohen’s d
			%	SE	%	SE	%	SE
Non-TOCA-RR risk in 1st grade***a	16,439	100
Later risk only	1,689	10.3	47.39	0.59	36.24	0.62	−11.15	0.53	0.52
Never high risk	14750	89.7	62.65	0.20	61.15	0.22	−1.49	0.17	0.08
Non-PSC-CL risk in 1st grade***b	16,817	100
Later risk only	1,423	8.5	46.57	0.65	38.27	0.70	−8.29	0.59	0.38
Never high risk	15,394	91.5	61.17	0.20	59.58	0.22	−1.59	0.16	0.08

SE = standard error

^***p<.001

^aChange in TOCA-RR risk independently predicted academic progress when we controlled for covariates; β=−0.20, t=−22.03, p<0.001

^bChange in PSC-CL risk independently predicted academic progress when we controlled for covariates; β=−0.13, t=−15.41, p<0.001

Students who were at risk for mental health problems in first grade

Table 4a focuses on students who had screen(s) indicating mental health risk in first grade and compares change in GPA rank for students with early (first grade only) versus early and persistent (first and third grade) mental health risk. As defined by the TOCA-RR, 13% (n=2,530) of students were at risk in first grade. The first row of the table shows that, among those students, 65% (n=1,655) were no longer at risk in third grade. Academic improvement paralleled mental health improvement for students with early risk only, as their mean GPA rank improved by 2.96 percentile points (from 38.83 to 41.79) between first and third grade (t=−5.64, p<.001, d=.14). Conversely, 35% (n=875) of students had early and persistent mental health risk on the TOCA-RR. Those students’ GPAs decreased by an average of 5.03 percentile points (31.41 to 26.38) over the two years (t=7.46, p<.001, d=.25). Multiple linear regression showed that improving teacher-reported mental health predicted academic progress, even after controlling for all covariates¹ (β=−.23, t=−12.39, p<.001).

Table 4a also shows that, on the PSC-CL, 57% (n=1,219) of students who were at risk in first grade were no longer at risk in third grade. While early risk only students did experience an improvement in GPA rank (42.76 to 43.03) between first and third grade (M=+.27), this change did not reach statistical significance. However, as on the TOCA-RR, students with early and persistent risk (n=933) decreased by an average of 4.68 percentile points (38.35 to 33.66) over those years (t=6.98, p<.001, d=.22). Like teacher-reported mental health on the TOCA-RR, improving parent-reported mental health on the PSC-CL predicted academic progress, even after controlling for covariates (β=−.14, t=−6.54, p<.001).

Students who were not at risk for mental health problems in first grade

Table 4b shows longitudinal outcomes for students who were not at risk on mental health screen(s) in first grade and compares change in GPA rank for students with later (third grade only) versus never high (neither first nor third grade) mental health risk.

On the TOCA-RR, 10% (n=1,689) of students had screens indicating later mental health risk. This group experienced a large, significant decrease in GPA rank between first and third grade (M_change=−11.15, t=21.16, p<.001). While the 90% (n=14,750) of students who were never high risk also decreased in GPA rank over the two years (M_change=−1.49, t=8.95, p<.001), comparing the effect sizes of their decrease (d=.08) with later risk students’ decrease (d=.52) shows that their decrease was of much smaller magnitude. Multiple linear regression showed that developing mental health risk according to the TOCA-RR predicted declining academic performance, even after controlling for covariates (β=−.20, t=−22.03, p<.001).

Results were similar when we compared academic outcomes for students who were later risk only versus never high risk according to the parent mental health screen. On the PSC-CL, 8.5% (n=1,423) of students had screens indicating later mental health risk. Among students in this group, mean GPA rank decreased significantly between first and third grade (M_change=−8.29, t=14.11, p<.001). As on the TOCA-RR, students who were never high risk (n=15,394) also decreased in GPA rank (M_change=−1.59, t=9.73, p<.001). But once again, the difference between effect sizes showed that this reduction was of a smaller magnitude for students who were not at risk in third grade (d=.08 for never high risk students and d=.38 for later risk students). As in all previous analyses, the change in parent-reported mental health risk predicted declining academic performance, even after controlling for covariates (β=−.13, t=−15.41, p<.001).

Attendance an outcome measure for academic progress

While we chose to focus on GPA rank as our main academic outcome measure in this paper, it is important to note that our findings were similar, albeit weaker, when we used percent attendance as an academic outcome measure. For example, parent-reported mental health in first grade independently predicted first (β=−.02, t=−2.67) and third grade (β=−.02, t=−3.47) attendance (p<.01). Children who developed mental health problems between first and third grade according to their parents (β=−.04, t=−3.99) or teachers (β=−.04, t=−4.93) also had significantly decreased school attendance over those years compared to children who remained healthy (p<.001).

Accounting for the impact of treatment on academic change scores

Since our available data indicates that about 90% of the first grade TOCA-RR positive children attended preventive workshops through Skills for Life and more than 30% of TOCA-RR positive children received mental health services in the local health care system, we repeated all of the main analyses using those two variables (regular attendance at workshops and regular use of mental health services in the local health care system) as covariates. Treatment or lack of treatment did not change the relationship between changes in mental health functioning and academic progress. Analyzing the impact of the workshops and professional referrals is a complex project that will be the focus of future reports.

Discussion

Results from this study demonstrate a clinically significant relationship between mental health as assessed through brief standardized screens administered during first grade and two widely-used, real-world benchmarks of academic performance (grade point average and percent attendance) for students in first and third grade. This study allowed us to compare the relative impact of mental health and a number of established risk factors for poor academic outcomes among elementary school-aged children, including low socioeconomic status,^16,38 male gender,³⁷ and belonging to a single-parent household.¹⁶ According to bivariate correlations, mental health risk in first grade was more strongly associated with poor academic performance in first and third grade than any other risk factors. Furthermore, parent- and teacher-reported mental health still independently predicted academic performance when we controlled for risk factors, school, treatment, and earlier academic performance. Although first grade academic performance was by far the strongest predictor of third grade academic performance, first grade mental health was also a robust predictor of third grade academic performance. We also found that mental health as assessed near the start of first grade independently predicted the percent of school days children went on to attend in first and third grade.

These results suggest that mental health measures may be among the most important predictors of future academic performance for elementary school children and may therefore be useful in identifying students who might benefit from preventive interventions. While current academic performance was by far the best predictor of future academic performance, such data are typically not available until the end of the school year. School systems may use results from the PSC-CL and/or TOCA-RR, which can be administered earlier in the academic year, to identify students in need of additional classroom support.

This study made a unique contribution by using longitudinal academic and mental health data to demonstrate the relationship between changes in mental health and changes in academic performance in elementary school. Among the 18,969 students with complete data from both time points, the trajectory of mental health functioning between first and third grade predicted the trajectory of academic achievement over those years. On the whole, students whose mental health improved made significantly better academic progress than students who developed mental health problems or continued to experience them. These findings suggest that school-based preventive interventions like the Skills for Life (SFL) program in Chile could positively influence students’ outcomes in academic as well as mental health domains. Although it goes without saying that this naturalistic, correlational study cannot be interpreted as showing that improving mental health in some students was the cause of their improved academic outcomes, the current analyses do set the stage for further work on the academic impact of improving mental health.

It is important to note that we cannot rule out the possibility that academic improvement had a salutary effect on mental health or that change on a third, unknown factor led to changes in both domains. The large samples in this study also increased the statistical significance of our findings, so readers must bear in mind that most effect sizes were small to medium (see Tables 2, 4a, and 4b). An additional limitation in this study is the attrition that occurred when the parent-rated variables and longitudinal data were used. Although our larger analytic subsample included about two-thirds of students who attended SFL schools in 2009 or 2011, our smaller analytic subsample only included half of our first analytic subsample. Attrition rates are explained mainly by student mobility (changes in school attended) between first and third grade. At the time of data collection, the information and financial resources to track those students were not available.³⁶ Although our subsamples included students from the vast majority of SFL schools, our subsamples’ students were rated as functioning better on academic and risk factor variables in first grade than students who could not be included because of missing data. However, since our analyses showed similar patterns in our small (less representative) and larger (more representative) subsamples and since the findings in both types of analyses were robust, we believe that the results reported here would be replicated in the population itself. In our previous paper¹⁶ on the SFL Program, we used propensity analyses to adjust for possible bias due to missing data and found that patterns we reported remained the same after this adjustment.

It is also important to note that attrition may be an unavoidable concomitant of evaluations of programs aimed at the highest-risk students. The SFL program is meant to serve Chile’s most vulnerable schools and, in Chile as well as in the US, there is a great deal of movement between schools and school districts among the poorest families. Some of the students move from one SFL school to another, but many move to non-SFL schools where current evaluation procedures do not track them.³⁶ Other unavoidable real-world factors also impact attrition rates. To give just two additional examples, a large amount of information for the current analyses was lost when one region experienced difficulties with administration in 2011, and some schools that were participating in SFL in 2009 were closed and no longer existed in 2011.

Despite these methodological limitations, there were strengths in this naturalistic design. The mental health and academic data were collected as a part of routine program administration, not added in for research purposes. Grade point average and attendance are almost universally used and widely accepted as meaningful benchmarks of student performance. In addition, the standardized measures employed by SFL are used around the world. Our participants’ outcomes may therefore represent typical trajectories of mental health and academic performance, at least for elementary school children in high-risk populations.

Although poverty and other social ills that are strongly associated with poorer quality of life have been notoriously difficult to eradicate, mental health interventions may be more feasible. Routine mental health screening enables educators to identify students at risk for poor academic outcomes and target those children for early, preventive interventions. By teaching at-risk children better social-emotional skills, it might be possible to leave fewer children behind.

Future research should aim to replicate this study’s findings in other countries, and preferably those with demographic and cultural characteristics that are different from Chile’s. Since some SFL schools have added screening with the TOCA-RR and PSC-CL for pre-kindergarten and/or eighth grade students, we hope to add an additional assessment point from an earlier and/or later grade to the types of analyses in this paper. Such analyses may help us elucidate the relationship between mental health and academic performance over a longer period of development.

We are currently exploring the impact of preventive interventions and access to mental health services on mental health and academic outcomes by analyzing recently collected data on SFL preventive workshop participation and use of mental health services in the local health care system. So far, students’ psychosocial and academic outcomes are indicating significant improvements in those domains. If our final analyses show that participation in the SFL preventive intervention was associated with improving mental health, academic performance, or both, the United States or other countries may be encouraged to provide similar programs to their high-risk student populations.

Summary

A large and growing body of research suggests mental health may be an important determinant of academic performance in children. The current study sought to advance our understanding of this relationship by using a large, longitudinal sample of elementary school children from Chile. We hypothesized that: (1) first grade mental health would predict first and third academic performance and (2) remission from mental health problems identified in first grade would predict improved academic performance from first to third grade.

JUNAEB in Chile uses standardized parent- (PSC-CL) and teacher-completed (TOCA-RR) measures to screen students in first and third grade as a part of a national school-based mental health program, Habilidades Para la Vida [Skills for Life]. JUNAEB merged the PSC-CL and TOCA-RR data with academic and demographic information on students who were in first grade in 2009 and third grade in 2011. This information included grade point average (GPA), percent of school days attended, gender, family participation in welfare, and presence of family risk factors (e.g., whether the child was living with his/her father and whether the child’s mother was a teenager when he/she was born). Bivariate correlations between first grade variables and first and third grade GPA percentiles were calculated for 37,397 students. Multiple linear regression was used to explore whether first grade PSC-CL and TOCA-RR scores independently predicted first and third grade GPA percentiles (n=37,397) and whether changes in PSC-CL or TOCA-RR risk between first and third grade predicted changes in GPA percentiles over those years (n=18,969). All analyses were adjusted for clustering by schools using the Huber-White technique in SPSS Complex Samples.

Overall, mental health had a stronger relationship with first and third grade academic performance than other risk factors. The parent- and teacher-completed measures both independently predicted first and third grade academic performance, even when controlling for all risk factors. On the whole, students whose mental health improved between first and third grade made significantly better academic progress than students who developed mental health problems or continued to experience them by third grade. Findings were similar, albeit not as powerful, when we used attendance rather than GPA percentile as the academic outcome measure.

Our results showed that mental health was a stronger predictor of academic performance than other predictors and students whose mental health improved made better academic progress than students whose mental health did not improve or worsened. School-based mental health interventions such as Skills for Life may therefore positively influence participants’ academic and psychosocial development.