Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2009 Nov 12.
Published in final edited form as: J Clin Child Adolesc Psychol. 2003 Sep;32(3):396–407. doi: 10.1207/S15374424JCCP3203_08

Identifying At-Risk Children at School Entry: The Usefulness of Multibehavioral Problem Profiles

Kelly S Flanagan 1, Karen L Bierman 1, Chi-Ming Kam 1, Conduct Problems Prevention Research Group
PMCID: PMC2776638  NIHMSID: NIHMS146868  PMID: 12881028

Abstract

Found that 1st-grade teacher ratings of aggressive, hyperactive–inattentive, and low levels of prosocial behaviors made unique contributions to the prediction of school outcomes (measured 2 years later) for 755 children. Person-oriented analyses compared the predictive utility of 5 screening strategies based on child problem profiles to identify children at risk for school problems. A broad screening strategy, in which children with elevations in any 1 of the 3 behavior problem dimensions were identified as “at-risk,” showed lower specificity but superior sensitivity, odds ratios, and overall accuracy in the prediction of school outcomes than the other screening strategies that were more narrowly focused or were based on a total problem score. Results are discussed in terms of implications for the screening and design of preventive interventions.

Aggressive behaviors tend to persist over time and portend an insidious developmental trajectory, indicating increased risk for negative social, academic, and behavioral outcomes (Coie, Terry, Lenox, Lochman, & Hyman, 1995; Loeber, 1990). Aggressive children often show concurrent behavior problems, including hyperactivity–inattention and prosocial skill deficits, which impair their academic performance and social adjustment (Pope & Bierman, l999). The effective and reliable identification of at-risk youth is critical to the success of prevention and early intervention programs designed to promote competencies and thereby derail negative transactional processes in their early stages (Conduct Problems Prevention Research Group [CPPRG], l992). Despite an extensive empirical literature examining the developmental trajectories of aggressive children, relatively few studies have addressed optimal strategies for identifying at-risk children for prevention or early intervention. For example, to date no current study has examined explicitly the implications of the co-occurrence of aggression, hyperactivity–inattention, and prosocial skill deficits for the identification of at-risk children at school entry.

Research suggests that the developmental outcomes of children who exhibit concurrent problems, such as aggressive and hyperactive–inattentive behaviors combined, are often more negative than the outcomes of children who show only one behavior problem (Andersson, Magnusson, & Wennberg, 1997; Soussignan et al., 1992). For example, aggressive–hyperactive youth are more likely than aggressive-only or hyperactive-only children to exhibit reading delays, poor cognitive ability, poor peer relations, adolescent antisocial behaviors, and later alcohol problems (Andersson et al., 1997; Pope & Bierman, 1999). Similarly, among clinically diagnosed samples of children with attention deficit hyperactivity disorder, those with comorbid aggression are more likely to experience peer rejection, cognitive impairments, social information processing deficits, academic impairment, emotional difficulties, and delinquent behavior (Hinshaw, 1987; Wilson & Marcotte, 1996).

Concurrent social skills may also affect the developmental trajectories and outcomes associated with aggressive behavior. For example, prosocial skill deficits (e.g., low levels of friendly and cooperative behavior, heightened negative reactivity) increase vulnerability to peer rejection and victimization among aggressive children (Bierman, Smoot, & Aumiller, 1993). Children who are aggressive and socially withdrawn are characterized by deficits in prosocial skills and are more likely than aggressive-only or withdrawn-only children to experience poor peer relations, poor academic performance, sustained behavior problems, substance abuse, and elevated levels of depressive symptoms (Boivin, Poulin, & Vitaro, 1994; Moskowitz & Schwartzman, 1989). These findings suggest that the efficacy of early screening strategies might be increased by considering multiproblem profiles exhibited by young children, including aggressive, hyperactive, and prosocial deficits, rather than focusing on single behaviors alone.

Aggression, hyperactivity–inattention, and prosocial skill deficits appear intertwined developmentally, and children with elevations in any two of these areas often show problems in the third. For instance, aggressive–hyperactive children are more likely than aggressive-only or hyperactive-only children to show prosocial skill deficits (Pope & Bierman, 1999). Similarly, hyperactive–inattentive children with prosocial deficits show higher rates of aggression than those with higher levels of social competence(Greene et al., 1996).

This research is informative, but its implications for the early identification of at-risk children are limited for two reasons. First, previous research has not determined whether all three dimensions must be represented in a screening system or whether, given their overlap, they provide redundant information for screening. Second, the effectiveness of different screening strategies has not been compared. For example, one implication of the available research might be that early screening will be most effective if it is highly selective, focusing only on those children with two or three comorbid problems who appear to be at highest risk for later problems. A contrasting implication might be that, given the negative predictability of each of these problems and the ways that they transact developmentally, an early screening system might be more effective if it cast the net widely, including children with any one of the three behavior problems. Without a direct evaluation, it is not clear which screening strategy is optimal.

To answer these questions, a person-oriented approach is needed that considers all three child behavior problems simultaneously. Person-oriented approaches examine the influence of behavior problems in their intrapersonal context, in contrast to variable-oriented approaches that explore the general predictability of behavioral dimensions in a sample (Bergman & Magnusson, 1997). Determining the most effective screening strategy requires person-oriented study and cannot be extrapolated directly from variable-oriented developmental research.

One goal of this study was to examine the prevalence of child problem profiles at school entry based on three behavior problems (e.g., aggression, hyperactivity–inattention, and prosocial skill deficits) assessed by first-grade teacher ratings. Based on previous research, it was not clear how much overlap would exist across the three dimensions and how many distinct problem profiles might emerge. Hence, the prevalence of different problem profiles was examined. A second goal was to compare the predictive utility of several screening strategies using the teacher ratings of these three problems at school entry. School adjustment outcomes assessed in third grade (2 years after screening) included behavior (teacher ratings of aggressive-disruptive problems), academic performance (language arts grades), and social adjustment (peer sociometric ratings). These outcome measures provided a broad assessment of child adjustment across important domains of school functioning from multiple reports, thus reducing the possibility that estimates of predictive validity would be inflated by common method variance linking first-grade behavior problems and third-grade outcomes. Analyses compared the utility of screening strategies when teacher ratings of the three problems were used to identify at-risk children on the basis of (a) a single dimension of elevated levels of aggressive behavior alone, (b) a narrower screen involving combined dimensions (e.g., elevated levels of aggressive behavior as well as concurrent problems of hyperactivity–inattention or prosocial skill deficits), and (c) a broader screen involving multiple dimensions (e.g., elevated levels on any one of the three behavior problems: aggression, hyperactivity–inattention, or prosocial skill deficits).

Although the study focused on the differential utility of these three screening strategies, two additional screening models were included for baseline comparative purposes. In one of these models, a total externalizing problem score was used to identify at-risk children (e.g., a score created by summing teacher ratings of aggression, hyperactivity–inattention, and prosocial deficits) to determine whether teacher ratings of the three narrowband dimensions of student problems (as utilized in this study) provided more predictive power than teacher ratings of a single, broadband externalizing score. In addition, a model was included to determine how the screening strategies based on a single rating source (teachers) might compare with a more expensive screening strategy that made use of two different raters (e.g., teacher ratings of aggression in combination with peer ratings of rejection). Previous literature suggests that multiple-informant screens (teachers plus peers) identify at-risk children with more accuracy than single-informant screens (teachers only; Bierman & Wargo, 1995; Tremblay, LeBlanc, & Schwartzman, 1988); hence, a dual-informant screen was included to provide a baseline against which to compare the accuracy of the teacher-rating screening strategies employed in this study.

Methods

Participants

Participants were 755 children who comprised the normative and high-risk control samples of a larger longitudinal investigation of the development and prevention of conduct disorders (see CPPRG, 1992). The sample was selected from four diverse geographic sites (Durham, North Carolina; Nashville, Tennessee; Seattle, Washington; and rural central Pennsylvania). At each site, kindergarten teachers rated the aggressive–disruptive behavior problems of each of the children in their classes, using the 10-item Authority Acceptance scale of the Teacher Observation of Classroom Adaptation–Revised (Werthamer-Larsson, Kellam, & Wheeler, 1991). Approximately 100 children at each site were randomly selected for the normative sample, which represented the population at each site in terms of race, sex, and level of behavior problems. To select the high-risk sample, parents of children who scored in the top 35% at each site on teacher ratings of behavior problems were contacted by telephone or in person and asked to rate the frequency of child behavior problems at home on a 24-item scale (for more detail on the screening measures and sample selection procedures, see Lochman & the CPPRG, 1995). Children’s summed scores on the two screen measures (teacher and parent ratings of behavior problems) were averaged; children who fell in the top 10% to 15% at each site on this combined screen were recruited as high-risk participants. Although the Fast Track program includes preventive intervention, the participants in this study were drawn from the normative and high-risk control comparison samples who attended nonintervention schools and did not receive any aspect of the prevention program. The sample included 436 (57.7%) boys and 319 (42.3%) girls, with a mean age of 6 years, 5 months. The ethnic composition was 46% (n = 349) African American, 50% (n = 376) European American, and 4% (n = 30) other minority groups. Based on Hollingshead, the sample was low to middle socioeconomic status.

To test the hypotheses of this study, problem profiles were created for the 720 children (95% of the original sample) who had scores for all three teacher-rated behavior problems in first grade. The prediction of risk relied on the availability of third-grade measures. School records and teacher ratings were available for 89% of the original sample, whereas peer-rated sociometric outcomes were available for 65% of the sample (only those children still attending “core” schools where sociometric data were collected). Chi-square analyses revealed that the children without outcome data 2 years later were not disproportionately represented in any single behavioral classification profile at Grade 1.

Measures

Behavioral predictors

Predictors included first-grade teacher ratings of aggression, hyperactivity–inattention, and prosocial skill deficits. To obtain a “pure” measure of children’s aggression (e.g., one that did not include items describing impulsive or hyperactive–disruptive items), the Teacher Checklist (Dodge & Coie, 1987) was used. This six-item scale included three statements describing proactive aggression (e.g., “this child gets other kids to gang up on a peer that he or she does not like”) and three statements describing reactive aggression (e.g., “when this child has been teased or threatened, he or she gets angry easily and strikes back”), each rated on a 5-point scale ranging from 0 (never) to 4 (almost always true). Ratings were summed across items for a total aggression score (α = .87).

Teachers also completed the Attention Deficit Hyperactivity Disorder Checklist (DuPaul, 1990), which included 14 items, each rated on a 4-point scale ranging from 1 (not at all) to 4 (very well). The 8 hyperactive items (e.g., “interrupts or intrudes,” “fidgets and squirms in seat”) and 6 inattentive items (e.g., “is easily distracted,” “difficulty sustaining attention”) were summed to create a single score representing hyperactivity–inattention (α = .96).

Finally, teachers rated children on the nine-item Social Health Profile (CPPRG, 1997), which included positive behaviors (e.g., friendly, helpful) and appropriate social regulation (e.g., controls temper, appropriate expression of needs/feelings), each rated using a 6-point scale ranging from 0 (almost never) to 5 (almost always). The total summed score representing prosocial skills was internally consistent (α = .92), with established validity (CPPRG, 1997).

Predictive profiles

A central goal of the study was to examine the predictive risk associated with different problem profiles in the first grade. To construct individual behavior problem profiles, the three behavior problem scores were dichotomized using a cutoff point of 1 SD above the mean of the normative population (2.11 for aggression, 1.95 for hyperactivity–inattention, 3.05 for prosocial skill deficits). Scores above the cutoff point were considered indexes of the specific behavior problem, whereas scores below the cutoff points were considered “nonproblematic.” Dichotomized indicators were used to construct the eight problem profiles that resulted from the possible combinations of the behavior problems.

School outcomes

Three dimensions of children’s school adjustment (behavioral adjustment, academic performance, social adjustment) were assessed in third grade. The Authority Acceptance scale of the Teacher Observation of Classroom Adaptation–Revised (Werthamer-Larsson et al., 1991) assessed child behavioral adjustment with 10 aggressive–disruptive items (e.g., yells at others, teases others, breaks things) rated with a 6-point Likert scale ranging from 0 (almost never) to 5 (almost always). Sociometric interviews were conducted individually with each child in the classroom to assess social adjustment. Children were asked to identify classmates they liked best (“like most” [LM] nominations) and classmates they liked least (“liked least” [LL] nominations). Unlimited nominations were accepted. The computation of social preference scores and determination of children’s rejected status followed the Coie, Dodge, and Coppotelli (1982) procedure.1 Language-arts grades assessed academic performance. Grades were collected from school records and rated on a 13-point scale (1 = F, 4 = D, 7 = C, 10 = B, 13 = A).

Procedures

Interviewers administered the Teacher Observation of Classroom Adaptation–Revised and the Social Health Profile to teachers during interviews in the spring of the first- and third-grade years; teachers were paid for their time. Sociometric interviews were conducted individually with children during the spring; the interviewer read through a roster of classmates to ensure the child’s familiarity and then asked for nominations. Research assistants collected grade information directly from school records.

Results

Preliminary Analyses

Preliminary analyses were undertaken with the first-grade behavior problems to examine their intercorrelations and verify their independent predictive validity for third-grade school adjustment. As shown in Table 1, Pearson correlations revealed moderately high levels of association among the three behavior problems. In addition, each of these early behavior problems predicted later school behavior problems, poor academic performance, and low social preference. Although all predictive correlations were significant, first-grade aggression was a stronger predictor of later aggressive–disruptive problems than hyperactivity–inattention (determined with t tests of the difference between dependent rs; Cohen & Cohen, 1983; t = 2.22, p < .05). Conversely, first-grade hyperactivity–inattention and prosocial skill deficits were stronger predictors of both social preference (t = 3.16, p < .01 and t = 2.01, p < .05) and academic performance (t = 2.68, p < .01 and t = 2.71, p < .01) than first-grade aggression.

Table 1.

Intercorrelations Among Behavioral Predictors and Between Behavioral Predictors and School Outcomes

Hyperactivity–
Inattention		 Prosocial Skill
Deficits		 Aggression
First-Grade Behavioral Predictors
  Hyperactivity–inattention — .67 .66
  Prosocial skill deficits — .67
  Aggression —
Third-Grade School Outcomes
  Behavioral problems .48 .53 .54
  Academic performance −.39 −.39 −.31
  Social preference −.41 −.37 −.30
Open in a new tab

Note: All correlations are significant at p < .01.

To examine the combined and unique predictive power of the three first-grade behaviors, multiple regression analyses were conducted. The models accounted for significant amounts of variance in each third-grade outcome, 35% for aggressive–disruptive behavior problems, 18% for academic performance, and 19% for social preference (see Table 2). All three first-grade behavior problems made unique contributions to the prediction of behavioral adjustment, and hyperactivity–inattention and prosocial skill deficits (but not aggression) made unique contributions to the prediction of academic performance and social preference.

Table 2.

Multiple Regression Analyses for Variables Predicting School Outcomes

First-Grade Predictors
Third-Grade Outcomes Hyperactivity–Inattention Prosocial
Skill Deficits		 Aggression F df R2
Behavior problems .15* .27** .28** 114.91 3, 636 .35
Academic performance −.23** −.22** −.01 46.62 3, 637 .18
Social preference −.35** −.16** −.02 36.57 3, 464 .19
Open in a new tab

Note: Values in the table under each behavioral predictor represent β.

*

p < .05.

**

p < .01.

Person-Oriented Analyses

Prevalence of various problem profiles

A central focus of this study was to determine how first-grade problem profiles might enhance the identification of children at risk for later school maladjustment. Eight different problem profile groups emerged (see Table 3 for group sizes, sex representation, and behavior problem group means). A chi-square test revealed more boys than girls in four of the seven problem profile groups (e.g., the Hyperactive, Aggressive, Aggressive + Low Prosocial, and All Problems group; overall χ2 = 54.04, p < .01).

Table 3.

Means and Standard Deviations for Multibehavioral Problem Profile Groups

Hyperactivity–Inattention Prosocial Skill
Deficits		 Aggression
Profile Groups Total N Boys Girls M SD M SD M SD
Aggressive Profiles
  Aggressive 47 33 14 1.37 .41 2.50 .46 2.73 .51
  Aggressive + Hyperactive 39 26 13 2.37 .30 2.48 .53 2.85 .58
  Aggressive + Low prosocial 32 27 5 1.41 .41 3.50 .42 2.83 .44
  All Problems 77 61 16 2.57 .30 3.76 .42 3.08 .54
Nonaggressive Profiles
  Hyperactive–inattentive 62 43 19 2.33 .25 2.35 .48 1.17 .59
  Low prosocial 33 21 12 1.29 .44 3.43 .25 1.23 .55
  Low prosocial + Hyperactive 23 13 10 2.41 .32 3.44 .37 1.54 .47
No-problem comparison 407 188 219 .70 .54 1.57 .86 .56 .58
Open in a new tab

Overall, 195 children displayed elevated levels of aggression in first grade (this high rate reflects the oversampling of children with behavior problems for this study). Relatively few of these aggressive children (24%) showed elevated aggression without accompanying behavioral difficulties. Many of the aggressive children (39%) showed concurrent elevations in hyperactivity–inattention and prosocial skill deficits, or concurrent elevations in either hyperactivity–inattention (20%) or prosocial skill deficits (16%). Of the 201 children who exhibited elevated levels of hyperactivity–inattention in first grade, only 30% had no concurrent difficulties. The most common profile, shown by 40% of the hyperactive children, included elevated levels of both aggression and prosocial skill deficits, whereas another 20% showed concurrent aggression and 10% showed concurrent deficits in prosocial skills. Similarly, 165 children exhibited prosocial skill deficits, but only 20% of these exhibited this problem alone. Most (47%) exhibited concurrent aggression and hyperactivity; another 19% exhibited concurrent aggression, and 23% exhibited concurrent hyperactivity–inattention.

These findings illustrate the interdependency between aggression, hyperactivity–inattention, and prosocial skill deficits, such that children with any one of these problems appeared at increased risk for the other problems. Consider, for example, the probabilities. In this sample, the probability of being aggressive was .27. However, given elevations in hyperactivity–inattention or prosocial skill deficits, the conditional probabilities of aggressiveness were .58 and .66, respectively (Z = 9.81 and 9.60, p < .05). Given both hyperactivity–inattention and prosocial skill deficits, the conditional probability of aggression was .77 (Z = 11.26, p < .05). Clearly, the presence of additional problems involving dysregulation significantly increased concurrent risk for the display of aggression. The next set of analyses examined the impact of a child’s first-grade problem profile on his or her third-grade school adjustment.

Third-grade adjustment of children with different problem profiles

The later adjustment of children showing different first-grade problem profiles was examined by conducting 8 (group) × 2 (sex) analyses of variance on each of the third-grade outcomes. Significant effects for group emerged for all three outcomes, with no significant effects for sex or for the Sex × Group interaction (see Table 4). Post hoc analyses (Tukey–Kramer) revealed that all problem profile groups were more aggressive–disruptive in third grade than the no-problem group. Additionally, the pervasive profile group (all three problems) was significantly more aggressive–disruptive than the hyperactive-only group. All of the problem profile groups also had lower language arts grades in third grade than the no-problem comparison group. Six of the eight problem profile groups had lower levels of social preference than the comparison group; however, the aggressive-only and low prosocial–aggressive profile groups did not differ from the no-problem comparison group on social preference in third grade.

Table 4.

Analyses of Variance for School Outcomes

Profile Groups Behavioral Adjustment
F(7, 632) = 32.42		 Academic Performance
F(7, 633) = 13.54		 Social Adjustment
F(7, 460) = 13.66
Aggressive profiles
  Aggressive 2.09bc 7.16b −.38ab
  Aggressive + Hyperactive 2.06bc 7.10b −.70b
  Aggressive + Low Prosocial 2.03bc 8.12b −.02ab
  All problems 2.31c 7.53b −.70b
Nonaggressive profiles
  Hyperactive–inattentive 1.74b 7.78b −.85b
  Low Prosocial 2.03bc 6.35b −.89b
  Low Prosocial + Hyperactive 2.22bc 6.38b −1.19b
No-problem comparison 1.02a 9.46a .07a
Open in a new tab

Note: All F values are significant at p < .01. Values in the table represent the group means for standardized scores. Different superscripts indicate significant mean differences at p < .05, using Tukey–Kramer post hoc analyses.

These findings suggest that children who showed any of the behavior problem profiles were likely, as a group, to demonstrate lower levels of later school maladjustment than their nonproblem peers. It could thus be argued that prevention programming should target children with any of these problem profiles. However, group means alone do not translate directly into prescriptions for screening because they do not provide information about individual variability in outcomes among children within groups. Hence, a second set of comparative analyses was conducted to examine the potential effectiveness of various profile-based screening strategies.

Screening Models

The next set of analyses was conducted with the normative sample alone (n = 380), which provided representative base rates. (We excluded the high-risk sample in these analyses to avoid any inflation of the screening results due to elevated base rates of behavior problems and negative outcomes.) To determine the predictive accuracy of various screening approaches, we dichotomized the school outcome. Negative third-grade outcomes were operationalized as (a) a standardized score on the Authority Acceptance scale of 1 SD or more above the sample mean, (b) third-grade language-arts grades that were below a C average, and (c) rejected status based on peer sociometric nominations.

Logistic regression analyses provided several indicators of the predictive utility of different screening models. Sensitivity refers to the proportion of children with a negative outcome who were considered to be at-risk, whereas specificity refers to the proportion of children without negative outcomes who were not considered to be at-risk. The odds ratio measures the odds of experiencing the negative outcome among at-risk children over the odds of exhibiting the negative outcome among low-risk children. Thus, odds ratios address the question, “to what degree are children identified as at-risk more likely than children identified as low-risk to experience negative outcomes?” Additionally, Yule’s Q, a special case of gamma applicable to 2 × 2 tables, was used as a measure of association between the independent variable (being at-risk or not) and the dependent variable (having a negative outcome or not); it represents a standardized odds ratio constrained between the range −1.0and1.0, obtained by dividing the incidence of similar or dissimilar cases by the total number of cases. This form of gamma provides a measurement of the proportional reduction in error (PRE) when information obtained from the screening model is used to predict negative outcomes relative to not having that information. Wald statistics were used to determine significance of the regression equations (see Table 5). Finally, the overall correct classification rate of each model was also calculated by combining the proportion of true positive and true negative classifications.

Table 5.

Odds Ratio Analyses Using Different Screening Models

Outcomes/Screening Models χ2 Sensitivity Specificity Odds
Ratio		 γ Correct
Classification (%)
Behavioral outcome
  Elevated aggression n = 56 (17%) 45.58*** .54 .88 8.82 .80 84
  Multiproblem aggression n = 40 (12%) 26.85*** .37 .91 6.18 .72 85
  Dysregulated n = 96 (29%) 61.10*** .80 .79 15.06 .88 79
  Aggressive-rejected n = 15 (5%) 18.77*** .20 .97 8.03 .78 88
  Summed-problem n = 56 (17%) 39.75*** .51 .88 7.74 .77 84
Academic outcome
  Elevated aggression n = 63 (18%) 15.77*** .39 .85 3.58 .56 78
  Multiproblem aggression n = 46 (13%) 8.41** .27 .89 2.85 .48 80
  Dysregulated n = 101 (30%) 31.27*** .63 .76 5.49 .69 74
  Aggressive–rejected n = 16 (5%) .35 .07 .95 1.48 .19 83
  Summed-problem n = 64 (19%) 12.21*** .37 .84 3.12 .51 77
Social outcome
  Elevated aggression n = 40 (17%) 9.06** .34 .86 3.28 .53 79
  Multiproblem aggression n = 28 (12%) 7.77** .26 .91 3.37 .54 81
  Dysregulated n = 65 (27%) 45.92*** .74 .81 12.22 .85 80
  Aggressive–rejected n = 10 (4%) 10.09** .14 .97 6.47 .73 85
  Summed-problem n = 39 (16%) 16.84*** .40 .88 4.77 .65 81
Open in a new tab

Note: Numbers in parentheses indicate the percentage of the total sample that was identified as at-risk by the screening model for that outcome.

**

p < .01.

***

p < .001.

We first compared three different screening models based on teacher ratings of aggression, hyperactivity–inattention, and prosocial skill deficits. The elevated aggression screening model identified any child with elevated aggression as at-risk for later negative outcomes, regardless of his or her scores on the other problem behaviors. The narrower multiproblem aggression model identified aggressive children who also had elevated scores on either hyperactivity–inattention or prosocial deficits as at-risk. Finally, the broader dysregulated screening model identified children with elevations on any one of the three problems (e.g., aggression, hyperactivity, or prosocial deficits) as being at-risk for negative outcomes.

These three screening models using teacher-rated narrowband problem profiles were also compared with two alternative screening methods, including (a) a summed problem screening model, which utilized a broadband teacher rating of child behavior problem (e.g. a summed score for aggression, hyperactive, and prosocial deficits), with a score of 1 SD above the mean of the normative sample indicating risk; and (b) a model based on a combination of teacher and peer ratings, the aggressive–rejected screening model, in which children who were both aggressive (as reported by the teacher) and rejected (as nominated by peers) were identified as at-risk. Analyses were also conducted separately by sex, to determine whether the predictive accuracy of the screening models differed by sex.2

Behavioral outcome

As shown in Table 5, the dysregulated model identified more children as at-risk than any other model and showed the greatest sensitivity, with 88% of children experiencing the negative behavioral outcome correctly identified as being at-risk. However, each of the other screening models had greater specificity than the dysregulated model, correctly identifying more of the children who did not experience the outcome as not at-risk. The aggressive–rejected model identified the fewest children as at-risk with a specificity of 97%, rarely classifying children as at-risk who did not show disruptive problems 2 years later. However, by classifying only 15 children as at-risk (relative to the 96 children identified as at-risk by the dysregulated model), the aggressive–rejected model missed many other children who demonstrated problem behaviors in third grade (failing to identify 80% of the children who had a negative outcome). By casting a wider net, the dysregulated model accurately identified a higher proportion of at-risk children than any of the other screening models, reflected in the greatest PRE (.88) and the greatest odds ratio, with at-risk children being 15 times more likely to experience the negative behavioral outcome than low-risk children. However, the overall correct classification rate for the dysregulated model (79%) was slightly lower than the other screening models (84% to 88%), due to the larger number of children identified as at-risk by the dysregulated model (higher false positive rate).

The analyses conducted separately by sex showed similar results, with the dysregulated model demonstrating the highest predictive accuracy for both boys and girls.3 The PRE was higher for girls (.87 vs. .68), suggesting the dysregulated screening model may be particularly useful for identifying girls who are at risk for later aggressive–disruptive problems.

Academic outcome

For the academic outcome, the dysregulated screening model again identified more at-risk children with a sensitivity that was substantially greater than any of the other models, correctly identifying 63% of the children with negative outcomes. The specificity and overall correct classification of the dysregulated model was lower than the other models, as fewer of the non-risk children were correctly identified (76% compared to 85% for the elevated aggression, 84% for summed problem, and 89% for the multiproblem aggression models). The aggressive–rejected model correctly identified so few of the children who developed academic problems that the results were no greater than chance. Overall, the broad net of the dysregulated model appeared most accurate, demonstrating the greatest association between risk status and later academic outcome and the greatest PRE when screening the total sample. The odds ratio illustrates this superiority, with children identified as at-risk by the dysregulated model being more than five times more likely than children identified as low-risk to have the negative academic outcome, in comparison to the elevated aggression, multiproblem aggression, and summed problem models in which children identified as at-risk were only three times as likely to have the negative academic outcome as children identified as low-risk.

Similar results emerged when the analyses were conducted separately by sex. The dysregulated screening model had the greatest predictive accuracy in identifying both boys and girls who were at-risk for later academic problems. Again, the PRE for the dysregulated screening model was higher for girls than for boys (.72 vs. .51). Although the gamma demonstrated only a 51% PRE when using the dysregulated screening model for identifying boys at risk for academic problems, it was much higher than the other screening models.

Social outcome

In terms of model sensitivity, the dysregulated model again identified as at-risk a greater proportion of the children who were rejected in third grade than any of the other models (74% compared to 14% to 40%). The specificity of the dysregulated model was lower than the other models, as it did not identify accurately as many of the children who were not rejected in third grade (81% compared to 86% to 97%). Similar to the results in the other adjustment domains, the overall measures of screening accuracy favored the broad net cast by the dysregulated model, as this model demonstrated the greatest odds ratio (at-risk children being 12 times more likely than low-risk children to be rejected) and the greatest PRE (.85) compared to the other screening models. It should be noted, however, that the gamma for the aggressive–rejected screening model was greater than those of the elevated aggression, the multiproblem aggression, and the summed problem models but lower (.73) than the gamma for the dysregulated screening model with a slightly higher correct classification rate (85% vs. 80%).

Sex differences emerged in the prediction of peer rejection. The dysregulated screening model was most accurate in identifying at-risk girls (gamma = .83; odds ratio = 11.11), substantially more accurate than the aggressive–rejected model. In contrast, for boys, the dysregulated and the aggressive–rejected models showed similar levels of high overall accuracy (gammas = .67 and .71; odds ratios = 5.00 and 5.90, respectively). The dysregulated model had higher sensitivity, correctly identifying as high-risk 78% of children who became rejected (compared to 29% for the aggressive–rejected model). However, the specificity of the aggressive–rejected model was superior (94% vs. 59%). Thus, for boys, the two models produced similar levels of overall accuracy but notably different rates of identification (131 vs. 31 labeled at-risk) and patterns of error.

Discussion

Consistent with previous research, aggression, hyperactivity–inattention, and prosocial skill deficits, assessed at school entry, each predicted later school difficulties in behavioral, academic, and social adjustment domains. In addition, these three behavior problems were all significantly intercorrelated, and children who exhibited high levels of one of these problems were also likely to display elevated levels of one or more of the others (76% of the aggressive children, 70% of the hyperactive–inattentive children, and 80% of the children with prosocial skill deficits did so). Regression analyses revealed that each first-grade behavior problem made unique contributions to the prediction of aggressive–disruptive problems, and hyperactivity–inattention and prosocial skill deficits also made unique contributions to the prediction of academic and social adjustment.

The person-oriented analyses complemented the regression analyses and directly assessed the accuracy of different screening strategies. The presence of any one of the three behavior problems increased risk for negative school outcomes. However, somewhat surprisingly, evidence of cumulative risk was fairly rare, with only a single case emerging in which children with pervasive problem profiles were at higher risk (for aggressive–disruptive behaviors) than children with single-problem profiles. A broad-based screening model that identified children at risk when they showed elevated levels of any one of these behavior problems (aggression, hyperactivity, or prosocial skill deficits) revealed the greatest sensitivity (i.e., proportion of children with negative school outcomes correctly identified as being at-risk), the greatest PRE, and the greatest odds ratio in the prediction of behavioral, academic, and social problems in third grade. Although the overall correct classification rate for this dysregulated model was slightly lower than more highly restrictive screening models, it was far more effective at identifying at-risk children, with considerably better sensitivity. From this perspective, the broader screening strategy had greater predictive accuracy than a narrower model using information from two sources (e.g., teacher ratings of aggression and peer ratings of rejection), or a model using teacher ratings summed to represent a total problem score.

Cost Benefit and Pragmatic Issues in Screening and Intervention

Although the broad-based dysregulated screening model demonstrated the greatest utility overall, by identifying a broad range of children as at-risk this model also yielded the lowest specificity (i.e., identifying children who did not develop a negative outcome as at-risk), producing slightly lower correct classification rates than other models. The correct classification rates of the other models reflect their higher specificity and the lower base rate of negative versus positive outcomes within the sample. However, these models also tended to “miss” children who developed negative outcomes. When deciding on the best screening strategy, the costs and benefits of different screening models need to be considered (Bennett et al., 1999). For example, a screening approach that correctly identifies a large number of at-risk children, such as the dysregulated model used in this study, has increased coverage and is less likely to miss children who will experience a negative outcome than a more narrow screening strategy. However, the provision of preventive interventions based on this screening approach is more expensive than one based on a narrower screening strategy, as it includes more children who will not develop the problem. In contrast, a screening approach that is more targeted in identifying children at risk for later aggression, such as the elevated aggression or aggressive–rejected screening models, identifies fewer children for intervention, resulting in lower costs and reduced numbers of “false positives.” At the same time, targeted approaches increase the risk of withholding intervention to a significant number of children who will eventually develop problems (“false negatives”). The success of selective preventive interventions depends on the accurate identification of at-risk children. The results of this study suggest a dramatic tradeoff between a broad-based screening strategy that maximizes the correct identification of at-risk children and the potential cost of treating children for whom preventive intervention is not absolutely necessary. To fully evaluate the benefit-to-cost relations of a broad versus a narrow approach to screening, the costs associated with the screening and intervention need to be compared with the cost of failing to identify and provide preventive intervention to at-risk children (Glascoe, Foster, & Wolraich, 1997).

One practical issue affecting the utility of a particular screening strategy is the expense involved in the administration of the screen itself. One advantage of using teacher ratings as a screen is that they are less expensive and easier to administer than are peer ratings, parent ratings, or behavioral observations. This study adds to the literature on the identification of at-risk children by demonstrating that a screening model based solely on teacher ratings of multiple behaviors can be as useful as a model that utilizes information from teachers and peers, at least in first grade (Ialongo, Vaden-Kernan, & Kellam, 1998). When teachers provide information on adaptational skills (e.g., hyperactivity–inattention and prosocial skill deficits) as well as aggression at school entry, they may be providing information about child vulnerabilities and developmental status that is similar to the information provided by peer rejection. In fact, several investigators have suggested that peer rejection adds predictive variance because it is a “marker” of child competency deficits (e.g., underlying hyperactivity–inattention or prosocial skill deficits; Pope & Bierman, 1999). When teacher ratings include child regulatory deficits as well as aggression, they may be equivalent or better predictors of later problems than screening strategies that use teacher ratings combined with peer rejection nominations (Vitaro, Tremblay, Gagnon, & Pelletier, 1994). In general, screening strategies that can rely on teacher ratings and do not require peer ratings are more practical and cost-efficient. Peer ratings can be cumbersome and difficult to obtain, and some schools and review boards are particularly reluctant to include negative nominations in sociometric interviews (Asher & Dodge, 1986).

Sex Differences

In this study, the more broad-based model provided the most accurate screen for both boys and girls for the behavioral and academic adjustment outcomes, with even more utility when used with girls than with boys. This finding is consistent with research suggesting that at-risk girls may be underdetected when overt aggressive behaviors are used as the primary criteria for risk (Hill, Lochman, Coie, Greenberg, & CPPRG, 2002; Zoccolillo, Tremblay, & Vitaro, 1996). The results from this study suggest that using a broad base of behavior problems (hyperactivity–inattention and prosocial skill deficits, as well as aggression) may increase the ability to correctly identify girls who are at risk for school maladjustment.

In the screening of high-risk boys, the aggressive–rejected screening model had slightly greater predictive utility than the dysregulated screening model. This finding is consistent with a large literature that has focused primarily on boys and documented the negative predictive utility of aggressive–rejected status among boys (e.g., Coie, Christopoulos, Terry, Dodge, & Lochman, 1989). However, it must be noted that the slight superiority of the aggressive–rejected model relative to the dysregulated model was due primarily to its outstanding specificity; in terms of sensitivity, the aggressive–rejected screening model only identified 29% of the children with a negative social outcome, whereas the dysregulated screening model correctly identified 78%.

Developmental Considerations

The “equifinality” reflected in the emergence of similar negative outcomes associated with the three different first-grade behavior problems and the failure to find much evidence for additive risk associated with multiproblem profiles (compared to single-problem profiles) are somewhat surprising findings and may reflect the developmental period at which screening took place. School entry presents children with new contextual demands for well-regulated and goal-directed activity, including sustained behavioral inhibition, compliance with rules, effective interpersonal communication, and the initiation of new relationships with teachers and peers (Kellam, Rebok, Ialongo, & Mayer, 1994). Interpersonal reinforcement schedules are typically sparse, and academic demands are substantially higher than most children have experienced in preschool or kindergarten. In this context, aggression, hyperactivity–inattention, or prosocial skill deficits may impair the child’s capacity to respond in effective ways, resulting in similar patterns of social and academic difficulty. For example, whether a child behaves with hostility, intrusive impulsivity, or awkward insensitivity, the impact on social adjustment may be similar, as each of these behavioral styles impede peer acceptance (Bierman et al., l993).

Further study may reveal some common core regulatory difficulties among children who show different behavioral manifestations and problem profiles at school entry. For example, deficits in executive functioning, or the capacity to inhibit prepotent responses, show anticipatory planning, and modify ongoing behaviors in response to performance feedback may reduce goal-directed behavior and be reflected behaviorally in either impulsive overactivity, reactive aggression, or social incompetence. Similarly, deficits in emotion regulation, which involves the capacity to respond adaptively to emotionally arousing situations, may act as a core deficit, impairing both behavioral control and social adaptation (Barkley, 1997; Eisenberg et al., 1996). Future research might examine common core deficits related to aggression, hyperactivity–inattention, and prosocial skill deficits. Furthermore, environmental variables, such as teaching practices and classroom context, may also affect children’s functioning and should be considered in future research and intervention development.

Implications for Preventive Intervention Design

The prevalence of multiproblem profiles among at-risk children argues against the use of single-component prevention programs aimed at specific behaviors and argues for a broader prevention focus or multicomponent approach. For example, an intervention aimed at decreasing aggression through anger management training may not be as effective as a multicomponent program that also addresses comorbid hyperactivity–inattention or prosocial skill deficits. Interventions that incorporate social skills training and self-regulation skills, as well as behavioral management to decrease aggression, may have a stronger impact, given the common co-occurrence of these behavior problems among children at risk in first grade (CPPRG, l992).

Limitations and Future Directions

A few limitations of this study should be noted. First, the effectiveness of different screening systems may change developmentally or within different populations. Consider the predictive accuracy of peer ratings. Whereas continuity in peer rejection is evident for one third of the children rejected in kindergarten (Vitaro et al., 1994), stability increases to nearly half of rejected children in the later grade school years (Coie & Dodge, 1983), suggesting increasing predictive power for peer rejection at later ages.

Second, analyses of predictive accuracy required the dichotomization of continuous teacher ratings—a practice that constrains variation and may result in a loss of information about individual differences. Farrington and Loeber (2000) have argued for the benefits of using dichotomous variables when assessing risk and demonstrated that dichotomization does not necessarily cause a decrease in the strength of association between variables and outcomes. Nonetheless, the choice of an alternative cutoff point may impact results by affecting the number of children identified as having particular behavior problems (Hill et al., 2002). Indeed, this study included high-risk schools and children, and results could vary in other community samples where the base rates of problem behaviors and negative outcomes are lower.

Finally, this study focused on screen measures readily available within the school context (e.g., teacher and peer ratings) and targeted school adjustment outcomes. Other screening approaches have included information from home contexts, including measures of familial, ecological, and child characteristics (e.g., family adversity, parental psychopathology, and IQ) and have targeted outcomes that include community-based as well as school-based functioning (Vitaro et al., 1994). In general, multiple gating and source screens using information from school and home settings may increase predictive accuracy when the outcome involves more broadly defined maladjustment, rather than school-based adaptation, which was the focus of this study (August, Realmutto, Crosby, & MacDonald, 1995; Lochman & CPPRG, 1995). For example, using Fast Track data, Hill et al. (2002) demonstrated increased accuracy in screening for later community-based delinquent activity when parent and teacher ratings are both used. Our results suggest that multiproblem teacher ratings, an easily administered screen in the school context, may be quite sensitive to the prediction of risk for later school adjustment problems. Additional indexes of child functioning and multiple informants may be necessary when child adjustment outside of the school setting is the targeted outcome.

In summary, this study illustrates the value of teacher-rated multiproblem profiles in first grade as a relatively inexpensive and easy-to-administer method for identifying children at risk for later academic, social, and behavioral adjustment problems at school. Compared with other models, the findings demonstrated that a broad-based screening strategy that included children with elevated rates of any of the three targeted problems had relatively less specificity but provided the greatest sensitivity, odds ratio, and PRE.

Acknowledgments

Support for this project came from National Institute of Mental Health (NIMH) Grants R18MH48083, R18MH50951, R18MH50952, and R18MH50953. Additional support was provided by the National Institute of Drug Abuse and the Center for Substance Abuse Prevention (through memorandums of agreement with the NIMH). This work was also supported in part by the Department of Education grant S184430002 and NIMH grants K05MH00797 and K05MH01027.

We are grateful for the close collaboration of the Durham Public Schools, the Metropolitan Nashville Public Schools, the Bellefonte Area Schools, the Tyrone Area Schools, the Mifflin County Schools, the Highline Public Schools, and the Seattle Public Schools. We greatly appreciate the hard work and dedication of the many staff members who implemented the project, collected the evaluation data, and assisted with data management and analyses. Appreciation is also expressed to the parents and students who supported this project.

Footnotes

Publisher's Disclaimer: Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf

This article may be used for research, teaching and private study purposes. Any substantial or systematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

1

The total number of positive LM nominations and LL nominations received by each child from his or her peers were calculated and standardized within classroom. The standard LL score was subtracted from the LM score to generate a social preference score, which was also standardized within classroom. Rejected status was given to children who received a social preference score less than −1.0, a LL score greater than 0, and a LM score less than 0 (Coie et al., 1982).

2

Analyses conducted separately by sex utilized the complete sample (high-risk plus normative) to provide a sufficient sample size to compare the various screening models. Analyses were conducted with cutoffs based both on total sample norms and on sex-specific norms. The results were similar, hence only results using cutoffs based on total sample norms are presented.

3

A complete description of these analyses is available on request to the corresponding author.

References

  1. Andersson T, Magnusson D, Wennberg P. Early aggressiveness and hyperactivity as indicators of adult alcohol problems and criminality: A prospective longitudinal study of male subjects. Studies on Crime and Crime Prevention. 1997;6:7–20. [Google Scholar]
  2. Asher SR, Dodge KA. Identifying children who are rejected by their peers. Developmental Psychology. 1986;22:444–449. [Google Scholar]
  3. August GJ, Realmutto GM, Crosby RD, MacDonald AW. Community-based multiple-gate screening of children at risk for conduct disorder. Journal of Abnormal Child Psychology. 1995;23:521–544. doi: 10.1007/BF01447212. [DOI] [PubMed] [Google Scholar]
  4. Barkley RA. Behavioral inhibition, sustained attention, and executive functions: Constructing a unifying theory of ADHD. Psychological Bulletin. 1997;121:65–94. doi: 10.1037/0033-2909.121.1.65. [DOI] [PubMed] [Google Scholar]
  5. Bennett KJ, Lipman EL, Brown S, Racine YR, Boyle MH, Offord DR. Predicting conduct problems: Can high-risk children be identified in kindergarten and grade 1? Journal of Consulting and Clinical Psychology. 1999;67:470–480. doi: 10.1037//0022-006x.67.4.470. [DOI] [PubMed] [Google Scholar]
  6. Bergman LR, Magnusson D. A person-oriented approach in research on developmental psychology. Development and Psychopathology. 1997;9:291–319. doi: 10.1017/s095457949700206x. [DOI] [PubMed] [Google Scholar]
  7. Bierman KL, Smoot DL, Aumiller K. Characteristics of aggressive-rejected, aggressive (nonrejected), and rejected (nonaggressive) boys. Child Development. 1993;64:139–151. [PubMed] [Google Scholar]
  8. Bierman KL, Wargo JB. Predicting the longitudinal course associated with aggressive-rejected, aggressive (nonrejected), and rejected (nonaggressive) status. Development and Psychopathology. 1995;7:669–682. [Google Scholar]
  9. Boivin M, Poulin F, Vitaro F. Depressed mood and peer rejection in childhood. Development and Psychopathology. 1994;6:483–498. [Google Scholar]
  10. Cohen J, Cohen P. Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale, NJ: New Jersey: Lawrence Erlbaum Associates, Inc.; 1983. [Google Scholar]
  11. Coie JD, Christopoulos C, Terry R, Dodge KA, Lochman JE. Types of aggressive relationships, peer rejection, and developmental consequences. In: Schneider BH, Attili G, et al., editors. Social competence in developmental perspective, NATO Advanced Science Institutes series. Series D: Behavioural and social sciences. Vol. 51. Dordrecht, The Netherlands: Kluwer Academic; 1989. pp. 223–237. [Google Scholar]
  12. Coie JD, Dodge KA. Continuities and changes in children’s social status: A five-year longitudinal study. Merrill–Palmer Quarterly. 1983;29:261–282. [Google Scholar]
  13. Coie JD, Dodge KA, Coppotelli HA. Dimensions and types of social status: A cross-age perspective. Developmental Psychology. 1982;15:1427–1434. [Google Scholar]
  14. Coie JD, Terry R, Lenox KF, Lochman J, Hyman C. Childhood peer rejection and aggression as predictors of stable patterns of adolescent disorder. Development and Psychopathology. 1995;7:697–713. [Google Scholar]
  15. Conduct Problems Prevention Research Group. A developmental and clinical model for the prevention of conduct disorder: The Fast Track program. Development and Psychopathology. 1992;4:509–527. [Google Scholar]
  16. Conduct Problems Prevention Research Group. Social Health Profile and TOCA–R: Technical report. 1997 Unpublished manuscript. Retrieved from the World Wide Web www.fasttrackproject.org.
  17. Dodge KA, Coie JD. Social-information-processing factors in reactive and proactive aggression in children’s peer groups. Journal of Personality and Social Psychology. 1987;53:1146–1158. doi: 10.1037//0022-3514.53.6.1146. [DOI] [PubMed] [Google Scholar]
  18. DuPaul GJ. The ADHD Rating Scale: Normative data, reliability and validity. Worchester: The University of Massachusetts Medical Center; 1990. Unpublished manuscript. [Google Scholar]
  19. Eisenberg N, Fabes RA, Guthrie IK, Murphy BC, Maszk P, Homgren R, Suh K. The relations of regulation and emotionality to problem behavior in elementary school children. Development and Psychopathology. 1996;8:141–162. [Google Scholar]
  20. Farrington DP, Loeber R. Some benefits of dichotomization in psychiatric and criminological research. Criminal Behaviour and Mental Health. 2000;10:100–122. [Google Scholar]
  21. Glascoe FP, Foster EM, Wolraich ML. An economic analysis of developmental detection methods. American Academy of Pediatrics. 1997;99:830–837. doi: 10.1542/peds.99.6.830. [DOI] [PubMed] [Google Scholar]
  22. Greene RW, Biederman J, Faraone SV, Ouellette CA, et al. Toward a new psychometric definition of social disability in children with attention–deficit hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry. 1996;35:571–578. doi: 10.1097/00004583-199605000-00011. [DOI] [PubMed] [Google Scholar]
  23. Hill LG, Lochman JE, Coie JD, Greenberg MT. Conduct Problems Prevention Research Group. Effectiveness of early screening for externalizing problems: Issues of sensitivity and specificity in predicting to fourth and fifth grade outcomes in a high-risk population. 2002. Manuscript submitted for publication. [Google Scholar]
  24. Hinshaw SP. On the distinction between attentional deficits/hyperactivity and conduct problems/aggression in child psychopathology. Psychological Bulletin. 1987;101:443–463. [PubMed] [Google Scholar]
  25. Ialongo NS, Vaden-Kernan N, Kellam S. Early peer rejection and aggression: Longitudinal relations with adolescent behavior. Journal of Developmental and Physical Disabilities. 1998;10:199–213. [Google Scholar]
  26. Kellam SG, Rebok GW, Ialongo N, Mayer LS. The course and malleability of aggressive behavior from early first grade into middle school: Results of a developmental epidemiologically-based preventive trial. Journal of Child Psychology and Psychiatry. 1994;35:259–281. doi: 10.1111/j.1469-7610.1994.tb01161.x. [DOI] [PubMed] [Google Scholar]
  27. Conduct Problems Prevention Research Group. Lochman JE. Screening of child behavior problems for prevention programs at school entry. Journal of Consulting and Clinical Psychology. 1995;63:549–559. doi: 10.1037//0022-006x.63.4.549. [DOI] [PubMed] [Google Scholar]
  28. Loeber R. Development and risk factors of juvenile antisocial behavior and delinquency. Clinical Psychology Review. 1990;10:1–41. [Google Scholar]
  29. Moskowitz DS, Schwartzman AE. Painting group portraits: Studying life outcomes for aggressive and withdrawn children. Journal of Personality. 1989;57:723–746. doi: 10.1111/j.1467-6494.1989.tb00492.x. [DOI] [PubMed] [Google Scholar]
  30. Pope AW, Bierman KL. Predicting adolescent peer problems and antisocial activities: The relative roles of aggression and dysregulation. Developmental Psychology. 1999;35:335–346. doi: 10.1037//0012-1649.35.2.335. [DOI] [PubMed] [Google Scholar]
  31. Soussignan R, Tremblay RE, Schaal B, Laurent D, Larivee S, Gagnon C, et al. Behavioural and cognitive characteristics of conduct disordered-hyperactive boys from age 6 to 11: A multiple informant perspective. Journal of Child Psychology and Psychiatry. 1992;33:1333–1346. doi: 10.1111/j.1469-7610.1992.tb00953.x. [DOI] [PubMed] [Google Scholar]
  32. Tremblay RE, LeBlanc M, Schwartzman AE. The predictive power of first-grade peer and teacher ratings of behavior: Sex differences in antisocial behavior and personality at adolescence. Journal of Abnormal Child Psychology. 1988;16:571–583. doi: 10.1007/BF00914267. [DOI] [PubMed] [Google Scholar]
  33. Vitaro F, Tremblay RE, Gagnon C, Pelletier D. Predictive accuracy of behavioral and sociometric assessments of high-risk kindergarten children. Journal of Clinical Child Psychology. 1994;23:272–282. [Google Scholar]
  34. Werthamer-Larsson L, Kellam S, Wheeler L. Effects of first-grade classroom environment on shy behavior, aggressive behavior, and concentration problems. American Journal of Community Psychology. 1991;19:585–602. doi: 10.1007/BF00937993. [DOI] [PubMed] [Google Scholar]
  35. Wilson JM, Marcotte AC. Psychosocial adjustment and educational outcome in adolescents with a childhood diagnosis of Attention Deficit Disorder. Journal of the American Academy of Child and Adolescent Psychiatry. 1996;35:579–587. doi: 10.1097/00004583-199605000-00012. [DOI] [PubMed] [Google Scholar]
  36. Zoccolillo M, Tremblay R, Vitaro F. DSM–III–R and DSM–III criteria for conduct disorder in preadolescent girls: Specific but insensitive. Journal of the American Academy of Child and Adolescent Psychiatry. 1996;35:461–470. doi: 10.1097/00004583-199604000-00012. [DOI] [PubMed] [Google Scholar]

RESOURCES