Longitudinal Impact of Two Universal Preventive Interventions in First Grade on Educational Outcomes in High School

Abstract

This study examined the longitudinal effects of 2 first-grade universal preventive interventions on academic outcomes (e.g., achievement, special education service use, graduation, postsecondary education) through age 19 in a sample of 678 urban, primarily African American children. The classroom-centered intervention combined the Good Behavior Game (H. H. Barrish, Saunders, & Wolfe, 1969) with an enhanced academic curriculum, whereas a second intervention, the Family–School Partnership, focused on promoting parental involvement in educational activities and bolstering parents’ behavior management strategies. Both programs aimed to address the proximal targets of aggressive behavior and poor academic achievement. Although the effects varied by gender, the classroom-centered intervention was associated with higher scores on standardized achievement tests, greater odds of high school graduation and college attendance, and reduced odds of special education service use. The intervention effects of the Family–School Partnership were in the expected direction; however, only 1 effect reached statistical significance. The findings of this randomized controlled trial illustrate the long-term educational impact of preventive interventions in early elementary school.

As a result of federal policies such as the No Child Left Behind Act and the Individuals With Disabilities Education Act, there is an increasing emphasis on the use of evidence-based programs in schools to prevent disruptive behavior problems and promote academic success. Implementation of evidence-based, universal preventive interventions that simultaneously teach prosocial behavior and academic skills can assist schools in promoting healthy academic and social development among all students (Greenberg et al., 2003; Gresham, 2004; Walker, Ramsay, & Gresham, 2004). Yet there are relatively few educational and social–emotional preventive interventions that have been shown through rigorously designed randomized controlled trials to produce long-term educational outcomes (Catalano, Berglund, Ryan, Lonczak, & Hawkins, 2002; Durlak & Wells, 1997).

The need for efficacious programs is particularly great in urban communities, where the risk for school failure and early school leaving is considerably increased (Institute of Education Sciences, 2007; Perie, Grigg, & Donahue, 2005). The current study examined the longitudinal impacts of two first-grade universal preventive interventions targeting the early antecedent risk behaviors of poor academic achievement and aggressive/disruptive behavior and their distal correlates in a sample of urban, primarily African American children. Although both prevention programs were aimed to promote a range of outcomes related to classroom learning, one program focused on the proximal influence of the classroom environment, whereas the other program focused on the Family–School partnership. By increasing understanding of the academic outcomes associated these types of universal preventive interventions, the current study could inform both policy and practice related to the future use of family- versus classroom-focused universal prevention programs in early elementary school.

Background and Theoretical Framework

Consistent with a three-tiered public health approach to prevention, universal preventive interventions target the general public or a whole population that has not been identified on the basis of individual risk (Mrazek & Haggerty, 1994). These programs are positive and proactive and are provided regardless of the student’s risk status. A social-emotional learning curriculum delivered to an entire classroom is an example of a universal prevention program. In contrast, selective preventive interventions target individuals or subgroups that are at elevated risk of developing disorders as a result of biological or social risk factors. Finally, indicated preventive interventions target individuals who are identified as having prodromal symptoms of severe behavioral problems or other disorders and whose symptoms are not yet serious enough to meet diagnostic criteria for a mental health or behavioral disorder (Mrazek & Haggerty, 1994). A similar tiered preventive intervention framework is utilized in positive behavior support (Sugai & Horner, 2006) and response to intervention (Hawken, Vincent, & Schumann, 2008).

A number of empirical studies support the focus on academic achievement and aggressive/disruptive behavior problems of preventive interventions for improving mental health, behavioral, and educational outcomes among urban children (Ialongo et al., 2006; Kellam, Mayer, Rebok, & Hawkins, 1998; McIntosh, Horner, Chard, Boland, & Good, 2006). For example, several studies have shown that learning problems predict mental health problems and anxiety and depressed mood in particular (Kistner, David, & White, 2003; Schwartz, Gorman, Duong, & Nakamoto, 2008). Similarly, aggressive behavior, displayed as early as first grade, has been shown to predict later substance use, antisocial behavior, and criminality (Schaeffer et al., 2006; Schaeffer, Petras, Ialongo, Poduska, & Kellam, 2003). Further complicating this association is the finding that aggressive/disruptive behavior problems often co-occur with poor academic achievement (Bradshaw, Buckley, & Ialongo, 2008; Herman & Ostrander, 2007), with some indication that conduct problems precede academic problems (Smart, Sanson, & Prior, 1996).

The life course/social field framework, originally described by Kellam, Branch, Agrawal, and Ensminger (1975) and more recently by Kellam and Rebok (1992), provided the conceptual and theoretical model for the design of the two universal preventive interventions examined in the current study. At the core of this framework is the concept that psychological well-being is reciprocally and positively related to how successfully people meet the social task demands faced at each stage of life. Success at an earlier stage of development may increase the likelihood of success at a later stage in the life course (Ialongo et al., 2006). Social task demands associated with the transition to first grade, the developmental stage on which the two preventive interventions focused, include academic achievement, compliance, attention, and participation in classroom and peer activities. The underlying theory of change (Izzo, Connell, Gambone, & Bradshaw, 2004) is that success in meeting these earlier social task demands will be associated with increased psychological well-being, academic outcomes, and overall success in meeting future task demands.

According to the theory, programs that target disruptive behavior problems and academic factors in first grade will improve academic outcomes on both the individual (student) and the collective (classroom) level (Ialongo et al., 2006; Kellam & Rebok, 1992). As a result of this collective classroom-level achievement, there should be a greater number of academically successful youths in the classroom for their classmates to model, and this would increase opportunities for learning, academic effort, and individual achievement (Bandura, Barbaranelli, Caprara, & Pastorelli, 1996). Success in the academic domain over the elementary school years would in turn reduce the child’s likelihood of experiencing developmentally incongruent events, such as school dropout (Jessor, Turbin, & Costa, 1998; Patterson, DeBaryshe, & Ramsey, 1989; Patterson, Reid, & Dishion, 1992).

The extant theoretical and empirical work suggests that universal preventive interventions implemented in first grade would have a significant and long-term impact on students’ academic performance and would reduce the risk for other untoward outcomes (Kellam et al., 1998). Therefore, a universal preventive intervention was designed for teachers and a second program was designed for parents; both interventions were to be implemented in first grade. Consistent with the life course/social field framework (Kellam & Rebok, 1992), the focus of both programs was on several outcomes specific to classroom learning and the management of disruptive behavior problems. Whereas the classroom-focused intervention was meant to enhance teachers’ instructional and behavior management practices, a separate universal family-focused program was aimed at improving parental support and involvement in the academic process, as well as management of disruptive behavior problems. We theorized that both programs would reduce the early antecedent risk behaviors of poor academic achievement and disruptive behavior problems, on the individual and collective levels, by improving teachers’ and parents’ disciplinary practices and focus on academics (Ialongo et al., 2006; Kellam & Rebok, 1992). The current study examined the impact of the two programs on the students’ educational outcomes in high school.

The Classroom-Centered and Family–School Partnership Universal Preventive Interventions

The first program, which we refer to as the classroom-centered (CC) intervention (Ialongo, Werthamer, Brown, Kellam, & Wai, 1999), is an integrated protocol designed to address both sets of early risk behaviors (poor achievement and aggressive and/or shy behavior). The CC intervention represents a combination of a behaviorally focused classroom management program called the Good Behavior Game (Barrish et al., 1969), which in previous trials demonstrated a beneficial impact on aggressive and shy behavior but not on achievement, with a set of specific enhancements to the academic curriculum, which in previous trials demonstrated a beneficial impact on achievement but only a modest crossover effect on aggression (Kellam et al., 1998). The CC intervention was designed to reduce the early risk behaviors of poor achievement and aggressive and shy behaviors through enhancements to the curriculum, improvements in teacher instructional and classroom behavior management practices, and specific strategies for children who were not performing adequately.

In addition to testing the CC intervention, we developed a second intervention, the Family–School Partnership (FSP; Canter & Canter, 1991; Ialongo et al., 1999), to improve collaboration between parents and teachers or school mental health professionals by enhancing parents’ teaching and behavior management skills. The decision to develop and test a family-based universal preventive intervention was consistent with the life course/social field framework, which highlights the significance of parenting behaviors (e.g., monitoring, communication) as regulating influences on children’s development. The FSP intervention was intended to help parents facilitate their children’s early successful social adaptation in response to developmental challenges, which should in turn promote later successful adaptation when the children are faced with new social task demands (Kellam & Rebok, 1992). Similarly, considerable empirical evidence has emerged documenting the important influences exerted by families on their children’s academic success (Gallagher, 1987; Scott-Jones, 1984) and social development (Patterson et al., 1992), along with the benefits to children’s behavior and achievement of strong parent-teacher partnerships and parental involvement (Epstein, 1983). The test of a family-based universal preventive intervention is consistent with the work of Hawkins et al. (1992) and Reid, Eddy, and Fetrow (1999), which demonstrated both the feasibility and the effectiveness of a universal family intervention implemented through the school.

Prior Research on the Impact of the CC and FSP Interventions

The two interventions were implemented during the 1993-1994 school year in first-grade classrooms in Baltimore City and were evaluated with a randomized controlled design (see Method section for additional information regarding the study design). Data were collected preintervention, and multi-informant data on behavioral and educational outcomes were collected in elementary, middle, and high school. At the end of the first and second grades, the CC intervention was shown to have significantly improved both academic achievement (reading and math) and classroom behavior (e.g., concentration problems, aggression, shyness). At the end of the sixth grade, children assigned to the CC intervention were significantly less likely to meet the criteria for a diagnosis of conduct disorder, to have been suspended from school, and to have received or been judged in need of mental health services in middle school by a teacher and/or parents (Ialongo, Poduska, Werthamer, & Kellam, 2001). At the end of the seventh grade, the CC intervention was shown to have significantly reduced the risk of initiating use of tobacco (Storr, Ialongo, Anthony, & Kellam, 2002). During Grades 6 through 8, the CC intervention was shown to have significantly reduced the risk of initiating use of hard drugs, such as heroin, crack, and cocaine powder (Furr-Holden, Ialongo, Anthony, Petras, & Kellam, 2004). Relative to the CC intervention, the FSP intervention has demonstrated a narrower range of significant impacts. In particular, none of the substance abuse or behavioral health effects of the FSP intervention proved statistically significant during middle school, with the exception of the effects for the FSP girls, who were significantly less likely to have been suspended than were control girls in sixth grade (Ialongo et al., 2001).

Overview and Rationale for the Current Study

The vast majority of the research on the CC and FSP has focused on behavioral outcomes through middle school. Less research has focused on longitudinal educational impacts of the two interventions. We aimed to add to the growing body of research documenting the positive outcomes of the CC and FSP universal preventive interventions by examining the distal impacts on academic outcomes by the end of high school. In the current study, we examined how exposure to either the CC or the FSP intervention in first grade affected standardized test performance in high school, teacher-rated academic achievement, special education service use, high school graduation, and college attendance. We theorized that the proximal impacts of the programs on reduced behavioral problems (Ialongo et al., 1999, 2001) would provide greater opportunity for learning (Scott & Barrett, 2004), which in turn would lead to improved academic performance, a reduced need for educational services (e.g., special education), and higher rates of high school graduation.

This study is significant in part because, to our knowledge, there are no randomized controlled trials of early elementary school-based, universal preventive interventions targeting early learning and behavior with follow-up from first grade through the end of high school. We are in the unique position of determining whether these early and relatively short-lived school-based interventions yielded long-term effects on a range of educational outcomes up to 12 years later. Moreover, because the study is based on an epidemiologically defined sample of urban, primarily economically disadvantaged African American children, such research could inform the use of preventive interventions in early elementary school to promote positive educational outcomes among urban African American youths.

Method

Study Design

The trial was designed to contrast the classroom-focused CC intervention with the family-based FSP intervention and a (third) control condition. This would allow us to determine which prevention program was more effective at promoting positive academic and mental health outcomes for the participating students. A randomized block design was employed, with schools serving as the blocking factor. Three first-grade classrooms in each of the nine urban elementary schools participated in the trial. Each classroom (including the teachers and students) within a school was randomly assigned either to one of the two intervention conditions (i.e., FSP or CC) or to the control condition, so that all three conditions were represented within each of the nine schools. We randomly assigned classrooms to one of the three conditions and balanced for student gender. Following an initial baseline assessment during the fall of the first-grade year, the interventions were implemented over the course of that first-grade school year. The CC and FSP interventions were implemented only in first grade, and there was no overlap in program content between the CC and FSP programs. The control classrooms followed the standard curriculum. Student participants were followed from first grade through age 19, and periodic assessments were conducted in Grades 1–3 and Grades 6–12 and at age 19 regarding a variety of mental health and academic outcomes.

Participants

Initial recruitment of first graders

In the fall of 1993, students from 27 classrooms in nine Baltimore City public elementary schools (chosen as representative of public elementary classrooms and schools in Baltimore City) were recruited to participate in a longitudinal study of the CC and FSP prevention programs. Recruitment began when the target sample was in kindergarten. Project staff attended parent–teacher meetings and distributed information regarding the project. When the target cohort entered the first grade, the project staff led an information session for parents and attended parent–teacher conferences. Two parent liaisons were hired by the project to conduct follow-up visits to the parents in order to provide additional information about the project and obtain written parental consent. No incentives for initial participation were provided. Written informed consent was obtained from parents, and verbal assent was obtained from the youths. The recruitment procedure and all aspects of the data collection were approved by the institutional review board of Johns Hopkins University.

Written parental consent was obtained for 97% of the 678 children available for assessment in the fall of first grade. Three percent of the parents or guardians either refused to allow their children to participate in the assessments or failed to respond to the consent request. Chi-square analyses and t tests failed to reveal any significant differences in terms of sociodemographic characteristics (ethnicity, age, gender, and free lunch status) between the children for whom parental consent was obtained and those for whom it was not. The resulting epidemiologically-defined sample was 53% male and 86.8% African American. At the beginning of first grade, the sample of students ranged in age from 5.3 to 7.7 years (M = 6.2 years, SD = 0.34). The majority of children (68.3%) received free or reduced-priced lunch. Additional demographic characteristics of the sample by intervention status are reported in Table 1.

Table 1. Baseline Characteristics of the Participants by Intervention Condition and Gender.

Baseline characteristic	Overall	Boys	Girls
Age, M (SD)
CC	6.20 (0.34)	6.21 (0.33)	6.18 (0.35)
FSP	6.25 (0.37)	6.29 (0.43)	6.20 (0.29)
Control	6.25 (0.36)	6.26 (0.38)	6.24 (0.33)
% receiving free lunch
CC	68.4	75.2	60.2
FSP	67.4	62.3	67.3
Control	71.0	73.6	75.2
% African American
CC	87.2	88.3	85.8
FSP	83.8	82.6	85.3
Control	83.5	80.3	86.8
Early academic readiness, M (SD)
CC	3.14 (1.35)	3.34 (1.30)	2.89 (1.38)
FSP	2.85 (1.34)	3.03 (1.34)	2.65 (1.33)
Control	2.57 (1.36)	2.80 (1.43)	2.32 (1.26)

Note. CC = classroom-centered intervention; FSP = Family–School Partnership intervention; early academic readiness = teacher’s rating in fall of first grade, with higher scores indicating less readiness.

Follow-up data collection

Data on a number of education performance indicators (e.g., standardized test performance, high school graduation) were collected for the current study in high school (Grades 6–12) and at age 19. Written consent was required from those participants who were 18 or older. Of those turning 18 during the spring fielding period, 192 (28.3%) gave written consent to participate. Written parental consent for participation was obtained for 382 (56.3%) of the 678 youths who had yet to turn 18. A total of 574 youths (84.7%) consented to a Grade 12 assessment. Parents refused consent for 39 youths (5.7%) and 9 of those participants who were 18 or older refused, for a total of 48 refusals (7.1%). Three participants were deceased (0.4%); the remaining 53 youths (7.8%) are currently being located to obtain consent for future data collections. No significant differences have been found in attrition or refusal rates between or across intervention conditions.

Interventions

The CC intervention

The CC intervention was designed to reduce the early risk behaviors of poor achievement and aggressive and shy behaviors by enhancing the classroom curriculum and teacher instructional and behavior management practices. In particular, the CC intervention featured (a) curricular enhancements, (b) improved classroom behavior management techniques, and (c) accompanying strategies for children not responding adequately to the universal intervention. To increase listening and comprehension skills, the reseachers augmented the intervention with an interactive, read-aloud component. Journal writing activities and the Reader’s Theater, a dramatic presentation of written work in a script form that includes expressive voices and gestures, were added to improve compositional and reading skills. These activities were intended to make the core curriculum more meaningful and fun for the students. To improve critical thinking skills, the researchers incorporated a new component called Critique of the Week. This directed-thinking activity was developed to help students learn strategies for analyzing perspectives. It uses the context of images and resources from the students’ daily life to teach students to examine the content, to look at the way they think, and to formulate their own position with a system of value and reasoning. The Mimosa math curriculum augmented the existing math curriculum and featured a whole-language approach to promoting math skill development. The academic component of the CC intervention divided the class into small, diverse groups, which provided the underlying structure for the curricular and behavioral components of the intervention.

The researchers augmented the existing classroom behavior management practices with the Good Behavior Game (Barrish et al., 1969). As briefly described in the Introduction, the Good Behavior Game is a whole-class strategy intended to decrease disruptive behaviors. Children are assigned to teams, and only those teams that do not exceed a specified criterion of precisely defined off-task, disruptive, and aggressive behaviors are allowed to “win.” Teachers were given additional strategies to use with children who failed to respond to the Good Behavior Game and/or the curricular enhancements. The strategies employed with respect to academic nonresponders included individual or small-group tutoring and modifications in the curriculum to address individual learning styles.

The FSP intervention

As briefly described in the Introduction, the FSP (Canter & Canter, 1991; Ialongo et al., 1999) was developed to improve collaboration between parents and teachers and school mental health professionals and to enhance parents’ teaching and behavior management skills. The major features of the FSP intervention are (a) training teachers, school mental health professionals, and other relevant school staff in parent–school communication and partnership building (Canter & Canter, 1991); (b) weekly home–school learning and communication activities; and (c) a series of nine workshops for parents led by a first-grade teacher and a school psychologist or social worker. The workshop series for parents began immediately after the pretest assessments in the fall of the first-grade school year and ran for 7 consecutive weeks (one per week) through early December. Two follow-up or booster workshops were held during the winter and spring semesters.

The initial parent workshops were intended to establish an effective and enduring partnership between parents and school staff, and they set the stage for parent–school collaboration to support children’s learning and behavior. In the first workshop, called Read Aloud, teachers shared with parents the benefits of reading aloud to their children along with strategies to enhance the experience. Parents were loaned a different book each week to read aloud to their child; the books contained sample questions and activities developed by Karweit and Bond (1993) and Handel and Goldsmith (1990) that have been found to promote development of literacy skills (Whitehurst, Epstein, Payne, Crone, & Fischel, 1994). The second workshop focused on Fun Math activities that were derived from the University of California at Berkeley’s Family Math program. These activities have been shown to be effective in stimulating children’s understanding of mathematical concepts and operations (Stenmark, Thompson, & Cossey, 1986). During this workshop, parents were given a kit of “manipulatives” to use when carrying out the weekly Fun Math activities sent home by their child’s first-grade teacher.

The next five workshops focused on effective disciplinary strategies. The Parents and Children series, a videotape modeling and group discussion program (Webster-Stratton, 1984), formed the basis for the positive discipline component of the FSP intervention. The series was led by the school psychologist or social worker and covered topics that included effective praise, play, limit setting, time-out versus spanking, and problem solving. Parents observed a series of videotapes of modeled parenting skills. After viewing each vignette, the leader paused the videotape and asked openended questions about the scenes. Parents reacted to and discussed the episodes and problem solved alternative approaches. Many situations were role-played and rehearsed by group members. Families were also asked to discuss and problem solve other problem situations that occur at home. The facilitating school psychologist/social worker also established a voicemail system with which to maintain parent involvement and provide consultation as needed with respect to learning or behavior management difficulties. To further foster Family–School communication, researchers asked parents to fill out and return comment sheets indicating whether they had completed the assigned weekly home learning activities and if they had encountered any problems in doing so.

Intervention Fidelity

The training and intervention manuals were precisely and uniformly delineated and codified, and the content of training and intervention contacts was standardized. This allowed us to monitor and sustain the integrity of the first-grade interventions. In addition, each intervener had available a number of materials designed to foster the correct execution of the interventions (e.g., detailed outlines and checklists that prescribed the necessary materials for each intervention contact, the specific themes or tasks that needed to be covered). Finally, the intervener had extensive training prior to the initiation of the interventions and received ongoing supervision, feedback, and training throughout the entire intervention period. Teachers who took part in the CC received 60 hr of training and direct supervision in its use. Parents who participated in the FSP were offered a total of nine workshop sessions, each of which was approximately 90 min in length. The monitoring of fidelity of implementation for the CC intervention involved three parts: (a) measures of setting up the classroom, (b) independent classroom observations, and (c) classroom visit record reviews. For the FSP intervention, interveners were required to provide documentation of each contact with parents (e.g., workshop attendance, level of parental participation, parental and student compliance with homework assignments).

Measures

Student demographic information

The school district provided information on the students’ gender, ethnicity, and free or reduced meals status in first grade.

Externalizing behavior problems

The Teacher Observation of Classroom Adaptation—Revised (TOCA–R; Werthamer-Larsson, Kellam, & Wheeler, 1991) was administered in the fall of the first grade to the students’ teachers. The TOCA–R is a brief measure of the adequacy of each child’s performance on core tasks in the classroom. Using a structured interview conducted by a trained member of the project staff, the teachers rated each child’s behavior on a scale measuring frequency of occurrence from 1 (almost never) to 6 (almost always). Academic readiness ratings in the fall of first grade was a covariate in the analyses in the current study. This subscale comprises 9 items (e.g., “concentrates,” “pays attention,” “is eager to learn,” “learns up to ability,” “works hard,” “stays on task”). The items were averaged to create a score from 1 to 6; higher scores indicated less academic readiness. Prior research on the TOCA has indicated that the Academic Readiness subscale is internally consistent (Cronbach’s α= .97; Werthamer-Larsson et al., 1991), and the scores are predictive of subsequent externalizing behavior problems, such as adjudication for a violent crime in adolescence and meeting the criteria for a diagnosis of antisocial personality disorder at age 19 (Petras, Chilcoat, Leaf, Ialongo, & Kellam, 2004; Schaeffer et al., 2003).

Kaufman Test of Educational Achievement (Grade 12, Reading and Math)

The Kaufman Test of Educational Achievement (KTEA; Kaufman & Kaufman, 1985) was developed to assess students’ school achievement in Grades 1–12. The brief test administered in the current study assessed academic ability in the areas of reading and math. The measure is widely used, and the scores have strong internal consistency (split-half reliability coefficients for each age-group ranged from .85 to .95) and test-retest reliability (coefficients ranged from .83 to .97; Worthington, 1987). Prior research has shown the KTEA scores to be correlated with other commonly used achievement tests (e.g., Wide Range Achievement Test, Peabody Individual Achievement Test, Metropolitan Achievement Test, Stanford Achievement Test, Comprehensive Test of Basic Skills, and the Kaufman Assessment Battery for Children; Worthington, 1987).

Teacher-rated academic performance (Grades 6–12)

The Teacher Report of Classroom Behavior Checklist (Ialongo et al., 2001) was used in Grades 6–12 to assess both classroom behavior and academic performance. This measure is an adaptation of the TOCA–R (described above). Teachers in Grades 6–12 responded to a question on the Teacher Report of Classroom Behavior Checklist regarding the youth’s academic performance in class on the following 5-point scale: excellent, good, fair, barely passing, or failing. An average of the ratings over the 6 years was computed and used as a composite academic performance indicator; higher scores indicated better performance.

Special education service use (Grades 1–12)

The school district provided official records of special education use. These records included students who had an individualized education program. In some cases (less than 5% of the sample), data from the district records were missing on this variable. In such cases, the teacher-reported special education service use was analyzed.

High school graduation

Data were obtained from the district to determine whether the participant had graduated from high school or had passed the General Educational Development test (approximately 4% of the sample). In some cases (less than 5% of the sample), data from the school records were missing on this variable. In such cases, the self-report data from the student’s age 19 interview were used.

College attendance

At the age 19 interview, the youths were asked whether they had attended college (e.g., 4-year college, junior college).

Overview of Statistical Analyses

We used mixed-model regression analysis (i.e., random effects regression; Gibbons, Hedeker, Watemaux, & Davis, 1988) to evaluate the impact of the two classroom-level interventions on the educational outcomes. The first-grade classroom was included as a random effect in the analyses (which accounted for the clustering of students within the 27 participating first-grade classrooms), and intervention condition was modeled as a fixed effect (Gibbons et al., 1988). Dummy variables were created to allow for planned contrasts between the CC intervention and control condition and the FSP intervention and control condition. Consistent with prior research, the analyses were conducted with the full sample and were stratified by gender. All of the analyses examined the impact of the interventions after adjusting for the baseline (the fall of first grade) level of academic readiness as rated by teachers. Cohen’s d (1992) effect size estimates were computed for continuous outcomes, and odds ratios were computed for dichotomous outcomes.

Preliminary Analyses

Equivalence of the intervention conditions at baseline

Chisquare analyses and analyses of variance revealed that the intervention conditions were equivalent with respect to child age, gender, ethnicity, free lunch status, achievement levels, and parenting practices at baseline (i.e., fall of first grade; Ialongo et al., 1999). A significant difference (p < .05) was found between the CC intervention and controls in terms of teacher ratings of early academic readiness; therefore, this variable was included as a covariate in the regression analyses.

Attrition analyses

Of the 653 children with consent to participate in the evaluation in the fall of first grade, 597 (i.e., 91.3%) completed the fall and spring of first-grade assessments and remained in their assigned intervention condition over the first-grade year. A total of 574 students (84.7%) completed assessments during the spring of 12th grade. At age 19, 541 (79.8%) consented to participate. The response rate has been consistently high through the age 19 follow-up, as illustrated by the 80% or greater follow-up rate in the annual assessments. A 20% loss to follow-up is considered a borderline situation, given that inferences could be affected by this amount of missing data. This would be particularly true if there were systematic loss to follow-up (Brown, 1992); however, we have not found evidence of such systematic loss through the age 19 assessments. As noted above, there were no significant differences between the intervention conditions in terms of rates of attrition at the 12th-grade follow-up. Furthermore, there were no differences in the sociodemographic characteristics (ethnicity, gender, age, or free lunch status) in terms of rates of attrition at 12th grade or at the age 19 interview across the intervention conditions.

Level of participation/implementation in the CC and FSP interventions

Each of the nine CC intervention classrooms was assigned a score from 0 to 100 that represented the percentage of the teacher’s implementation of the intervention as designed. Scores were based on the three sources of implementation data identified previously: (a) measures of setting up the classroom, (b) independent classroom observation sessions, and (c) reviews of classroom visit records. CC implementation scores ranged from 30% to 78% (Mdn = 64.37%, M = 59.9%, SD = 17.03%). All but two of the nine CC intervention teachers implemented more than 50% of the intervention protocol. Parents or caregivers in the FSP intervention attended an average of 4.02 (range 0–7, Mdn = 5.0, SD = 2.38) of the 7 core parenting sessions offered in the fall of first grade, or 57.4% of the available sessions. Just under 13% (12.7 %) of the parents or caregivers failed to attend any of the core workshops, whereas just over one third (35.3%) of the parents attended at least 6 of the 7 sessions. On average, parents completed 39.15 (SD = 16.54) of the 64 activities (i.e., 60.9%) of weekly take-home Read Aloud and Fun Math activities. Approximately one third (35.7%) of the families in the FSP condition completed 75% or more of the activities, whereas only 2.3% of the families failed to complete all of the activities.

Results

Effects of the CC Intervention

The mixed-model regression analyses indicated a significant effect of the CC intervention on KTEA reading performance in the overall sample (i.e., for boys and girls, B = 1.828, p < .01; for boys, B = 2.43, p < .05; see means for the full sample and by gender in Table 2 and regression results in Table 3). A similar significant CC intervention effect was found for KTEA math performance both in the overall sample (B = 3.57, p < .01) and for the boys (B = 4.27, p < .01), whereas a marginally significant effect was observed for girls (B = 2.81, p = .072). We observed a marginally significant CC intervention effect in the overall sample (B = 0.168, p = .081) for teacher-rated academic performance in Grades 6–12 (averaged), such that the children in the CC condition tended to have their academic performance rated more favorably than did control children. We also observed a significant effect of the CC intervention on special education service use in the overall sample (B =−0.705, odds ratio [OR] = .494, p < .05) and for the boys in the CC condition (B =−0.934, OR = .393, p < .01; see percentages in Table 2 and regression results in Table 4). That is, the odds of the children in the CC receiving special education at any point between Grades 1 and 12 were reduced nearly 50% for the overall sample as compared to the children in the control condition, whereas the odds of special education service use were reduced nearly 60% for the boys in the CC condition. No significant effects of the CC intervention were observed on special education service use among the girls. With regard to high school graduation, there was a significant positive effect of the CC intervention in the overall sample (B = 0.532, OR = 1.702, p < .05) or for the girls (B = 0.750, OR = 2.118, p < .05). Finally, with regard to college attendance, there was a significant effect of the CC intervention in the overall sample (B = 0.798, OR = 2.222, p < .05) and among the boys in the CC condition (B = 0.982, OR = 2.670, p < .05). A marginally significant intervention effect was observed among the girls in the CC condition (B = 0.690, OR = 1.993, p = .052).

Table 2. Adjusted Descriptive Analyses on Educational Outcomes Among the Participants by Intervention Condition and Gender.

Educational outcome	Overall		Boys		Girls
	M	SE	M	SE	M	SE
Grade 12 KTEA–Reading
CC	42.624	0.392	43.100	0.566	42.178	0.536
FSP	42.299	0.422	42.287	0.669	42.313	0.639
Control	40.900	0.416	40.879	0.608	40.895	0.554
Grade 12 KTEA–Math
CC	43.671	0.616	43.705	0.813	43.520	0.945
FSP	42.739	0.639	42.653	0.911	42.789	1.131
Control	40.102	0.655	39.331	0.877	40.920	0.977
Academic performance
CC	3.013	0.054	2.770	0.072	3.269	0.077
FSP	2.993	0.052	2.827	0.071	3.174	0.084
Control	2.842	0.055	2.639	0.074	3.095	0.077
Special education service, %
CC	22.4	2.8	24.6	3.9	19.5	4.0
FSP	29.3	3.1	34.2	4.4	24.0	4.0
Control	34.8	2.9	43.2	4.3	26.2	3.8
High school graduation, %
CC	61.9	3.3	53.3	4.5	73.6	4.9
FSP	57.3	4.6	56.1	4.6	58.4	6.2
Control	51.3	3.4	44.8	5.0	58.6	4.6
College attendance, %
CC	31.5	2.9	26.0	3.6	40.3	4.9
FSP	28.3	3.8	22.4	4.1	34.5	6.3
Control	19.4	3.0	12.8	3.9	26.4	4.7

Note. All descriptive analyses adjusted for teacher-rated fall of first-grade academic readiness. Overall indicates the pooled sample for both boys and girls. KTEA–Reading = Kaufman Test of Educational Achievement Brief Reading Comprehension, administered in Grade 12; CC = classroom-centered intervention; FSP = Family–School Partnership intervention; KTEA–Math = Kaufman Test of Educational Achievement Math Calculation, administered in Grade 12; academic performance = teacher-rated academic performance, averaged across Grades 6–12; special education service = special education service use at any time in Grades 1–12; SE = standard error of the mean.

Table 3. Summary of Regression Analyses Examining Intervention Impact on Continuous Educational Outcomes Through Age 19.

Continuous outcome	B	SE	p	Cohen’s d
Grade 12 KTEA–Reading
Overall
CC	1.828**	0.617	.009	0.319
FSP	1.160†	0.624	.081	0.252
Boys
CC	2.433*	0.915	.017	0.394
FSP	1.296	0.981	.204	0.232
Girls
CC	1.281	0.927	.185	0.253
FSP	1.039	0.924	.276	0.254
Grade 12 KTEA–Math
Overall
CC	3.569**	0.915	.001	0.420
FSP	2.001*	0.848	.030	0.308
Boys
CC	4.273**	1.328	.005	0.540
FSP	2.530†	1.251	.059	0.391
Girls
CC	2.809†	1.465	.072	0.290
FSP	1.508	1.581	.354	0.192
Academic performance
Overall
CC	0.168†	0.090	.081	0.225
FSP	0.116	0.071	.119	0.200
Boys
CC	0.118	0.110	.295	0.174
FSP	0.148	0.096	.141	0.251
Girls
CC	0.200	0.119	.113	0.240
FSP	0.067	0.122	.589	0.101

Note. The control group is dummy coded as the base group in all analyses. All mixed-model regression analyses included teacher-rated fall of first grade academic readiness as a covariate. Overall indicates the pooled sample for both boys and girls. CC = classroom-centered intervention; FSP = Family–School Partnership intervention; B = unstandardized regression coefficient; SE = standard error of the estimate; KTEA–Reading = Kaufman Test of Educational Achievement Brief Reading Comprehension, administered in Grade 12; KTEA–Math = Kaufman Test of Educational Achievement Math Calculation, administered in Grade 12; academic performance = teacher-rated academic performance, averaged across Grades 6–12.

^†p < .10.

^*p < .05.

^**p < .01.

Table 4. Summary of Regression Analyses Examining Intervention Impact on Categorical Educational Outcomes Through Age 19.

Categorical outcome	B	SE	Wald χ2	p	Odds ratio
Special education service
Overall
CC	−0.705	0.140	50.05	.013	0.494*
FSP	−0.180	0.197	46.78	.444	0.835
Boys
CC	−0.934	0.141	38.10	.009	0.393**
FSP	−0.300	0.231	31.16	.337	0.741
Girls
CC	−0.389	0.256	25.89	.302	0.678
FSP	−0.024	0.334	32.49	.944	0.976
High school graduation
Overall
CC	0.532	0.439	27.67	.039	1.702*
FSP	0.202	0.345	21.25	.473	1.224
Boys
CC	0.405	0.377	11.59	.108	1.499
FSP	0.435	0.427	8.19	.115	1.545
Girls
CC	0.750	0.760	18.68	.037	2.118*
FSP	−0.044	0.359	16.47	.906	0.957
College attendance
Overall
CC	0.798	0.750	39.83	.018	2.222*
FSP	0.434	0.467	17.94	.153	1.543
Boys
CC	0.982	1.290	10.39	.042	2.670*
FSP	0.571	0.638	8.95	.113	1.770
Girls
CC	0.690	0.709	18.71	.052	1.993†
FSP	0.397	0.677	14.77	.384	1.487

Note. The control group is dummy coded as the base group in all analyses. All mixed-model regression analyses included teacher-rated fall of first-grade academic readiness as a covariate. Overall indicates the pooled sample for both boys and girls. Special education service = special education service use at any point in Grades 1–12; CC = classroom-centered intervention; FSP = Family–School Partnership intervention; B = unstandardized regression coefficient; SE = standard error of the estimate.

^†p < .10.

^*p < .05.

^**p < .01.

Effects of the FSP Intervention

The mixed-model regression analyses indicated that there was a marginally significant effect of the FSP intervention on KTEA reading performance in the overall sample (B = 1.160, p = .081) but no significant effects for boys or for girls in the stratified analyses (see means for the full sample and by gender in Table 2 and regression results in Table 3). There was a significant effect on KTEA math performance in the overall sample (B = 2.001, p < .05) and a marginally significant effect for the boys (B = 2.530, p = .059). There were no significant effects of the FSP intervention on teacher-rated academic performance in Grades 6–12. Furthermore, there were no statistically significant effects of the FSP intervention on special education service use, high school graduation, or college attendance (see percentages in Table 2 and regression results in Table 4).

Discussion

In the current study, we used a randomized controlled trial design to examine the effects of two preventive interventions implemented in first grade on academic outcomes through age 19 in a sample of urban children. When we controlled for academic readiness in the fall of first grade, the longitudinal analyses indicated that the CC intervention, which comprised the Good Behavior Game and an enhanced academic curriculum, was associated with significant improvements in reading and general academic achievement, high school graduation, and college attendance, as well as with reductions in special education service use. The effect sizes of the CC intervention (as indicated by the Cohen’s d and odds ratios) are well within the practical significance range according to Lipsey (1998) and are in the small-to-moderate range according to Cohen (1992). The effect sizes varied somewhat by gender, such that the effects tended to be greatest in the overall sample and for boys, whereas the only significant effect among girls was on high school graduation.

Although the overall effects of the CC intervention were significant for several outcomes, the stratified analyses indicated that many of the intervention effect sizes were larger for boys than for girls. It is possible that the targets of the CC intervention (e.g., impulsive and disruptive behavior, academic focus) were more relevant to long-term academic performance for boys than for girls. Alternatively, as noted in a previous study on the outcomes in elementary school, girls tended to display more shy behavior than did boys, whereas boys were at greater risk for displaying externalizing behavior problems (Ialongo et al., 1999). Therefore, it seems plausible that poorly achieving girls receive less attention from teachers than do poorly achieving boys. Researchers should examine these gender differences in greater detail to discern whether elements of the program could be modified or enhanced to address proximal factors associated with girls’ long-term academic performance.

Despite research highlighting the significance of parenting factors on multiple aspects of children’s development (Collins, Maccoby, Steinberg, Hetherington, & Bornstein, 2000; Patterson et al., 1989), there were relatively few significant academic outcomes associated with the FSP intervention. In particular, the FSP intervention was associated with statistically significant improvements in the overall sample’s math performance and with marginally significant improvements in boys’ math performance and the overall sample’s reading performance. All other nonsignificant effects were, however, in the expected direction. Similar trends for the FSP intervention have been noted for other outcomes, such as criteria for conduct disorder, need for mental health services, teacher-rated problem behaviors, social participation/shy behavior (Ialongo et al., 1999, 2001), and illegal drug use (Furr-Holden et al., 2004). This finding suggests that further enhancement may be needed to optimize the overall impact of the FSP intervention (Patrikakou & Weissberg, 2007).

Another possible explanation for the stronger impact of the CC program than the FSP program on the education outcomes may be the differential amount of training and coaching the CC teachers and FSP parents received. Whereas the CC intervention teachers received 60 hr of training and direct supervision in the use of the programs, parents in the FSP program were offered a total of only nine 90-min workshop sessions. However, it was not feasible, from the standpoint of logistics or cost, to provide all first-grade parents with as much training as the CC teachers received. It is also possible that a universal family intervention may not yield an impact sufficient to justify the resources expended, and thus it may be more efficient to target family supports to the children at greatest risk (e.g., those not responding adequately to the universal CC intervention).

We did not test the combined effect of the CC and FSP interventions within the current trial. However, it is possible that the combination of the CC intervention and a parenting program that uses language similar to that of the CC intervention and focuses specifically on the skills fostered through the CC program could build on and help the children generalize the skills developed through the classroom-based program to other settings (e.g., home, neighborhood). This combination in turn might result in greater effects of the integration of CC and a family program than would either program in isolation. Or, as noted above, the family component could be employed for children who do not respond adequately to the CC universal model (see Sugai & Horner, 2006, for a description of multitiered preventive interventions).

The current study focused on the intent-to-treat impact of the CC and FSP interventions; however, future research should also take into consideration implementation level and parent participation as factors influencing program outcomes. For example, the level of parent participation in the FSP intervention varied across students. This is a common concern in parent-focused preventive interventions and services (McKay et al., 2004). In fact, prior research by Jo and Muthén (2002) indicated that the intervention effects of the FSP intervention on subsequent teacher ratings of the children’s academic readiness were weaker among the children whose parents had the lowest participation in the FSP intervention. Further work should identify effective strategies for optimizing parent involvement and engagement in similar preventive interventions (McKay et al., 2004).

Although the randomized controlled longitudinal design bolsters a potentially causal interpretation of the intervention effects, further exploration of the specific mechanisms mediating the change process is required in future studies (Kellam et al., 1998). As described previously, prior research on this sample indicates that the CC intervention is associated with reductions in the child’s level of disruptive behavior in elementary and middle school (Ialongo et al., 1999, 2001). The behavior management component of the CC intervention (i.e., Good Behavior Game) may have provided children with a greater opportunity to learn (Scott & Barrett, 2004) by reducing behavior problems, whereas the academic curriculum component of the CC program may have enhanced the children’s early academic skills (McIntosh et al., 2006). In contrast, the increased reliance on special education services among the children in the control condition may have influenced their long-term outcomes (e.g., increased dropout, school failure) as a result of labeling or other processes (Osterholm, Nash, & Kritsonis, 2007). An important area for future research is identification of the specific pathways or mechanisms through which the CC and FSP interventions influence the academic outcomes. Furthermore, the impact of the intervention may vary as a function of the quality of subsequent educational services or the academic curriculum, as well as the presence of other child, family, or community risk factors. Thus, potential moderators of intervention outcomes warrant further exploration.

Given the combined focus on behavior and academics in the CC intervention, it is unclear whether the Good Behavior Game or the academic components of the CC program account for the observed effects on educational outcomes. A previous randomized trial by Kellam and colleagues, which included a different sample of Baltimore City elementary school students, found that the Good Behavior Game intervention alone had significant effects on disruptive behavior problems but did not impact educational outcomes (Kellam et al., 2008; Kellam, Rebok, Ialongo, & Mayer, 1994). The findings of the current study, which tested a combination of the Good Behavior Game and the enhanced academic curriculum through the CC condition, suggest there may be a synergistic effect on education outcomes when the Good Behavior Game is combined with an enhanced educational program. The design of the current study, however, precludes us from determining the independent contributions of the Good Behavior Game and the enhanced academic curriculum on the observed outcomes.

Strengths of the current study include the focus on longitudinal outcomes (first grade through age 19) and the use of multiple indicators of academic performance. A relatively consistent pattern of findings emerged across the different outcomes, and this suggests that the intervention effects were not limited to select outcomes or one source of information (e.g., self-report). It is interesting that the only outcome for which a significant intervention effect did not emerge was the teacher-reported outcome of academic achievement. However, one effect approached statistical significance, and all the trends were in the expected direction. Additional research could help us understand the factors influencing teachers’ ratings of children’s academic performance.

Conclusions and Implications

The findings of the current study provide further evidence of the enduring effects of the yearlong CC intervention in first grade across a range of educational outcomes. The strong beneficial effects of the CC intervention on academic achievement, as measured by higher scores on standardized achievement tests, reduced utilization of special education services, higher rates of high school graduation, and higher rates of attending college, will likely translate into increased employment opportunities for these youths and concomitant reductions in the risk for mental and behavioral health problems (Garnier, Stein, & Jacobs, 1997; Prevatt & Kelly, 2003; Vernez, Krop, & Rydell, 1999). These findings are particularly noteworthy, given that they occurred within a relatively high risk population of urban, low-income, primarily African American children, for whom the risk of academic problems, special education service use, and school dropout is significantly increased (Institute of Education Sciences, 2007; Miller-Cribbs, Cronen, & Davis, 2002). These effects are also impressive, given that the CC intervention was of relatively low intensity and was administered over the course of a single academic year.

There is growing interest in economic analysis of school-based prevention programs in an effort to determine the fiscal impact of implementing the programs. Such work is particularly important given the current fiscal climate and increasing emphasis on high-quality implementation of evidence-based programs, which can require considerable resources in terms of time, money, and training (Gottfredson & Gottfredson, 2002). Applying a benefit–cost model to the effects of the Good Behavior Game on (only) the reduced use of tobacco observed in the first-generation trials (Kellam & Anthony, 1998), Aos, Lieb, Mayfield, Miller, and Pennucci (2004) concluded that the Good Behavior Game was associated with benefits of $25.92 for each dollar of program cost. This put the Good Behavior Game among the top five programs reviewed with respect to benefits per dollar of cost (Aos et al., 2004). The benefit–cost ratio is likely considerably greater when the broader range of behavioral, mental health, and educational outcomes is considered. The savings are likely even greater when the reduced reliance on special education, mental health, and juvenile justice services resulting from the 1-year intervention is considered. A cost–benefit analysis of the CC intervention on increased high school graduation is currently under way, and preliminary findings suggest that the combined classroom-based model is a worthwhile investment for schools and communities.

Taken together, the current findings highlight the significant impact of the relatively modest 1-year CC universal preventive intervention for promoting a range of positive educational outcomes among economically disadvantaged, urban African American youths. The findings suggest that early intervention with such youths can have a positive impact 12 years later. As noted above, this is the only randomized controlled study of which we are aware that has a follow-up of a sufficient length to attest to the positive consequences of targeting early academic achievement and behavior. Thus, the current findings bolster the available data indicating the effectiveness of the CC model on a broad range of long-term educational outcomes, and they represent an important and novel extension of our prior work on the behavioral outcomes of these universal preventive interventions.