Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2007 Mar 19.
Published in final edited form as: Psychol Sch. 2007 Mar 16;44(4):333–350. doi: 10.1002/pits.20226

A Descriptive Study of School Discipline Referrals in First Grade

Julie C Rusby 1,, Ted K Taylor 1, E Michael Foster 2
PMCID: PMC1828691  NIHMSID: NIHMS13423  PMID: 17372623

Abstract

School discipline referrals (SDRs) may be useful in the early detection and monitoring of disruptive behavior problems to inform prevention efforts in the school setting, yet little is known about the nature and validity of SDRs in the early grades. For this descriptive study, SDR data were collected on a sample of first grade students who were at risk for developing disruptive behavior problems (n = 186) and a universal sample (n = 531) from 20 schools. Most SDRs were given for physical aggression and the predominant consequence was time out. As expected, boys and at-risk students were more likely to receive an SDR and to have more SDRs than were girls and the universal sample. A large difference between schools regarding the delivery of SDRs was found. A zero-inflated Poisson model clustered by school tested the prediction of school-level variables. Students in schools that had a systematic way of tracking SDRs were more likely to receive one. Also, schools with more low-income students and larger class sizes gave fewer SDRs. SDRs predicted teacher ratings, and to a lesser extent, parent ratings of disruptive behavior at the end of first grade. Practitioners and researchers must examine school-level influences whenever first grade discipline referrals are used to measure problem behavior for the purpose of planning and evaluating interventions.

A Descriptive Study of School Discipline Referrals in First Grade

Schools commonly collect information on office referrals for student discipline problems, yet do not always do so in a systematic way that offers useful information for understanding and ameliorating individual student and school-wide disruptive behavior problems. School discipline referrals (SDRs) are typically used as an indicator of how individual students are behaving and how well a school as a whole is doing in managing student behavior. Practitioners in the school setting may utilize discipline referrals to help determine when and how to intervene with a student. School personnel also can utilize discipline referrals to examine trends for discipline problems (e.g., location or time of day) to gain a better understanding of the problems in their school, work towards reducing them, and to evaluate school-wide behavior intervention efforts (Irvin, Tobin, Sprague, Sugai, & Vincent, 2004). There is a large variation regarding the extent to which schools and teachers deliver discipline referrals, complicating the interpretation and utility of school discipline referral information. This descriptive study was performed to inform early intervention and prevention efforts by investigating the salience and nature of discipline referrals for students in first grade, the variance between schools and school-level predictors of this variance, and the extent to which discipline referrals predict teacher- and parent-reported disruptive behavior.

First grade is a critical time for having efficient, yet adequate ways for measuring disruptive behavior problems in the school setting. A breadth of research has highlighted the importance of early detection of behavioral difficulties and early intervention to prevent the escalation of such problems. Children who enter school displaying disruptive behavior, such as oppositional and aggressive behavior, are at elevated risk for continued social and academic difficulties throughout elementary school (Moffit, 1993; Patterson, Reid, and Dishion, 1992; Shores et al., 1993; Wehby, Symons, & Shores, 1995). These early behavior problems, along with failure to develop positive peer relationships, are associated with the development of later social adjustment problems such as school dropout, delinquency, teenage pregnancy, substance abuse, violence, and criminal activities in adulthood (Gabel & Shindledecker, 1991; Kupersmidt, Coie, & Dodge, 1990; Loeber et al., 2000; Parker & Asher, 1987). Although there is clear evidence supporting the importance of early prevention efforts for disruptive behavior problems, schools often fail to identify students in need of services early enough (Walker et al., 1996). The early identification of challenging behavior in schools is clearly an important step in preventing the persistence and intensification of these disruptive behavior problems.

The collection of information on student disruptive behavior problems in schools by systematically recording and monitoring discipline referrals has potential for being useful in such prevention efforts. This information can be collected efficiently and most schools already keep some record of discipline problems. However, little research on the utility and validity of SDRs in elementary schools has been conducted. A review of recent research on early intervention with children at risk for behavioral disorders demonstrated that there has been little consistency in the ways in which behavioral outcomes were measured and only 3 out of 16 studies in early school settings utilized official records to measure behavioral progress (Hester, Baltodano, Gable, Tonelson, & Hendrickson, 2003). Moreover, the delivery of discipline referrals is not consistent across schools or across teachers within a school, thus further complicating our understanding of their utility and validity. The limited research available on discipline referrals for elementary school students is summarized below.

Prevalence of Discipline Referrals

Only a few studies provide descriptive data about discipline referrals in elementary schools. These studies offer information about the variability of the prevalence of discipline referrals within and across schools. Wright and Dusek (1998) found that the percentage of students receiving discipline referrals was quite stable within a school, but differed greatly between two elementary schools in the same school district. A wide variance of discipline referral delivery also was found across 16 elementary schools and 15 middle schools (Sugai, Sprague, Horner, & Walker, 2000). The percentage of students who had received one or more discipline referrals in elementary school ranged from 10% to 39% and from 15% to 66% in the middle schools. Further, the average percentage of students who received one or more discipline referrals in middle school was more that two times greater than in elementary school (48% and 21% respectively). Within an elementary school substantially higher percentages of discipline referrals were found in fifth grade compared to lower grades and were the lowest in first grade (Putnam, Luiselli, Handler, & Jefferson, 2003). Moreover, elementary teachers within a school were found to vary widely regarding the number of discipline referrals they gave out during the school year. Although 70% of the school staff delivered 1–5 discipline referrals over the course of the school year, about 6% delivered more than 25 discipline referrals. These studies have demonstrated that the amount of discipline referrals delivered tends to increase as students get older. Results also showed that the prevalence of discipline referrals greatly varies between different schools. Differences in reporting and tracking of discipline problems may in part contribute to these grade level and school differences of SDRs.

School-level Predictors of Discipline Referrals

These descriptive studies do not provide information about school-level factors that may impact the prevalence of discipline referrals, such as class size or spending per student, or whether prevalence is mainly a function of how well a school has documented discipline referrals. Understanding potential school-level factors is important in issues of utility, such as how useful discipline referrals are in monitoring student behavior problems across schools in a district or state. Additionally, research evaluating the effectiveness of school prevention programs using discipline referral data as an outcome variable would need to control for known predictors of school-level variance.

Investigations to explain the school-level variance of discipline referrals are needed, yet studies that include school-level factors are lacking. Nelson, Gonzalez, Epstein, and Benner (2003) reviewed studies that examined factors that predicted the delivery of school discipline referrals. Of the 13 studies included in the review, only 2 analyzed school-level predictors. One of the studies found that a higher rate of discipline referrals per student was given in larger-sized schools than in smaller schools, and in schools with a higher percentage of students receiving free and reduced lunches (Winbinger, Katsiyannis, & Archwamety, 2000). The second study found that smaller schools used a lower percentage of suspensions and expulsions as consequences for discipline referrals compared to larger schools (Rose, 1988). These studies provide initial indication that larger schools in communities with greater economic disadvantage tend to have greater discipline problems and that larger schools tend to deliver more severe consequences for these discipline problems. However, these studies do not provide information about how well discipline problems were monitored across the different schools. Until there is more research investigating predictors of school-level variation, including evidence for systematic monitoring of discipline problems in schools, the validity of SDR data are in question.

Concurrent and Predictive Associations of School Discipline Referrals and Problem Behavior

There is some evidence for the concurrent and predictive validity of school discipline referral data. In a study of 72 students in three elementary schools in which SDRs were systematically collected, students with two or more SDRs had significantly higher ratings on the problem behavior scale of the Social Skills Rating System (Walker, Cheney, Stage, & Blum, 2005). This study also provided support for using multiple screening methods to identify students in need of behavioral support such as the Systematic Screening for Behavior Disorders (SSBD; Walker & Severson, 1992) along with SDR data.

The predictive validity of SDRs has been supported in several studies, most of which were conducted in middle and high schools. Discipline referrals for misbehavior in middle school have been found to be negatively related to academic achievement (Bryant, Schulenberg, Bachman, O’Malley, & Johnston, 2000; Morrison, Anthony, Sotrino, & Dillon, 2001; Murdock, Anderman, & Hodge, 2000) and predictive of high school failure (Tobin & Sugai, 1999). There also is evidence of the stability of school discipline referrals for misbehavior from middle school to high school (Bryant et al., 2000; Murdock et al., 2000; Tobin & Sugai, 1999) and in elementary school (Wright & Dusek, 1998). A few studies also have demonstrated the predictive validity of school discipline referrals of male students to future problem behavior outside the school setting. Farrington (1989) found that discipline referrals of 8- to 10-year-old boys predicted self-reports of violence and convictions of violent crime in adolescents and adults. School discipline referrals of boys in elementary and middle school also have predicted later arrest rates (Walker, Stieber, Ramsey, & O’Neill, 1991, 1993). Further research is clearly needed—particularly studies that take place in elementary school and include girls in the sample.

School Discipline Referrals as an Outcome of School-wide Interventions

There also is some support that discipline referral data are sensitive to change in the context of school-wide interventions. A number of studies in elementary and middle schools have used discipline referrals to test the effects of school-wide interventions. For example, significantly greater reductions in discipline referrals were found in elementary schools that participated in the Positive Action Program than in comparison control schools (Flay, Allred, & Ordway, 2001). The program effects were higher for schools with higher numbers of low income students. Other small-scale studies in elementary schools have also resulted in school-wide reductions in discipline referrals (Putnam et al., 2003; Twemlow et al., 2001). Moreover, school discipline referrals have been used as a formative assessment technique for assisting schools in making data-based decisions as part of a whole-school positive behavior support intervention. McCurdy, Mannella, & Elridge (2003) present a case study of one urban elementary school utilizing school discipline referrals as monthly feedback to the school. After 1 year of implementing the Positive Behavior Support (PBS) program, discipline referrals were cut almost in half and significant reductions were found in referrals for fighting. PBS or similar Effective Behavior Support (EBS) interventions have also resulted in reduced discipline referrals for middle school students (Metzler, Biglan, Rusby, & Sprague, 2001; Taylor-Greene et al., 1997).

In summary, there is evidence for the associations among individual student school discipline records with academic achievement and continuing problem behavior. The available evidence also suggests that SDRs can be useful for detecting and monitoring problem behavior in students and for evaluating school-wide intervention efforts. However, the rate of discipline referrals delivered per student can differ widely between schools with the same prevalence of problem behavior. Little is known about the factors that impact the between-school variance of discipline referrals for misbehavior.

The purpose of this present study is to add to this literature by (a) offering more in-depth descriptive information of first grade school discipline referrals, (b) analyzing the extent of school-level variance and predictors of that variance, and (c) examining the degree to which school discipline referrals predict teacher and parent ratings of student behavior at the end of first grade. Although fewer first grade students receive discipline referrals compared to older students, first grade SDR data may be a good indicator for the need to intervene early with these students to prevent the development of increasing problem behavior.

Method

Participating Schools

Initially 20 elementary schools from 12 school districts were recruited for participation in this study. The present study is part of a larger project, Success through the Incredible Years (Taylor et al., 2006), a randomized-control evaluation of the Incredible Years parent, teacher, and child interventions (Webster-Stratton, 2000) aimed at reducing disruptive behavior and increasing social skills. Participating schools were located in three clusters of 13 small to mid-sized communities in Oregon with populations of 1,478 to 137,893. Each cluster of schools began participation in 3 successive years. The percentage of students eligible for free or reduced lunches in these schools varied from 24% to 72%, with a median of 49% eligible students. The present study involves the teachers, students, and parents who were participating in the Success through the Incredible Years study from all 20 schools when students were in first grade.

Participating Teachers

First grade teachers (n = 55) from the participating schools were recruited to participate in the study. In the first cluster 19 teachers from six schools participated in the study, in the second cluster 13 teachers from six schools participated, and in the third cluster 23 teachers from eight schools participated. All of the participating first grade teachers were female. Most of the teachers were Caucasian and 4% were of Hispanic or Latino ethnicity. The minimum education level of participating teachers was a BA or BS degree, and 35% had additional graduate-level courses and 57% had a graduate degree. Participating teachers varied in experience; 29% had taught for 5 or fewer years, 25% for 6–15 years, 22% for 16–22 years, and 24% for more than 22 years.

Participating Students and Parents

Participants were 717 students and their “primary” parents. Two samples of students were recruited into the Success through the Incredible Years study in two stages. A sample of students who were rated as being “at-risk” for disruptive behavior problems by their kindergarten teachers (n = 186) was recruited from participating schools prior to first grade. All participating schools had at least two first-grade classrooms. The students in the “at-risk” sample were randomly assigned into participating first-grade classrooms (these classrooms were later randomly assigned into intervention or control conditions for the intervention evaluation study). A “universal” sample of students (n = 531) were recruited from the same participating first grade classrooms during the end of first grade. Details of recruitment procedures and description of the two samples follows.

The at-risk sample

This sample of students was selected based on elevated levels of teacher-rated disruptive behavior. Kindergarten teachers from the participating schools completed the Child and Adolescent Disruptive Behavior Inventory Screener (CADBI Screener; Taylor, Burns, Rusby, & Foster, in press) on all children in their classrooms. The CADBI Screener is a brief questionnaire consisting of 25 items from the oppositional to peers, oppositional to adults, and the hyperactivity/impulsivity scales from the CADBI (Burns, Taylor, & Rusby, 2001). The CADBI Screener has good reliability (alpha = .97) and was negatively associated (r = −.71, p < .001) with peer-preferred social skills from the Walker-McConnell Scale of Social Competence and School Adjustment (1995). The scales for the CADBI Screener were also validated using confirmatory analyses on a sample of 824 kindergarten children and replicated on 534 children (Taylor et al., in press). Students who were rated above the 65th percentile on the behavioral screener were considered for participation. Essentially the students scoring in the top third on disruptive behavior problems were selected to reflect an at-risk sample, rather than an already identified sample (this procedure purposefully erred on the side of over-selecting rather than under-selecting students who over time may develop disruptive behavior problems). It has been demonstrated that even being in the top half of the sample on disruptive behavior places children at increased risk longitudinally (Patterson et al., 1992).

Parents were given the opportunity to have their child excluded from the screening process and 7% of parents did so. Students who exhibited severe developmental disabilities or autism were considered ineligible to participate in the study (2% were considered ineligible). Of the 1,438 kindergarten children who were screened, 321 met criteria for “at risk” and were invited to participate in the study. Families were asked if they were willing to allow their child to be randomly assigned to a first grade classroom and to participate in a parenting group if offered. A total of 203 families (63%) from this sample agreed to participate in the study. The 17 students who moved away from the area of a participating school prior to first grade were not retained in the study. Of the 186 participating at-risk students, 59% were boys and 11% were of Hispanic ethnicity. Most of the non-Hispanic students were Caucasian, 6% were American Indian, 2% were African-American, and 3% reported to be of another race or did not report their race. The median range of family incomes for this sample was between $30,000 and $34,069, and 25% of the families had an annual income below $20,000.

The universal sample

This sample of families was recruited when students were in first grade. The students were selected from the classrooms that were already participating in the study (these were the classmates of the at-risk sample described previously). Given the timing of recruitment for this sample not all students could be recruited before the end of the school year, particularly for the larger sample in the third cluster. To avoid introducing bias, a subsample was randomly selected to be recruited first, before the first grade school year ended. Seventy-four percent of this sample was recruited in time to collect first grade teacher ratings and the late recruitment did not impact the collection of archival records or parent ratings.

Students were ineligible for participation if they had a sibling already in the study, or had moderate or profound developmental disability or autism. Children with parents who spoke either English or Spanish were eligible for participation, as some project personnel were fluent in Spanish and written materials were available in Spanish. Forty-three students did not meet the eligibility criteria and 809 students were eligible. Of those families, 531 (66%) agreed to participate in the study. Nearly half were boys (46%) and 18% were of Hispanic ethnicity. Most of the non-Hispanic students were Caucasian, 2% were American Indian, 1% were Asian, .5% were African-American, and 8% were of other, mixed, or unreported race. The median range of family incomes for this universal sample was between $30,000 and $34,069, and 22% of the families had an income below $20,000.

Participating Parents

In both the universal sample and at-risk sample the “primary” parents were asked to participate in the study and provide information about their child’s behavior. The primary parent was defined as the adult who primarily provided care for the child. In most cases the primary parent was the mother, or in single-father cases, the father. In both samples 91% of participating primary parents were female. The family income, mothers’ level of education, percent of single-parent families, and percent of nonminority families for both the at-risk and universal groups are reported in Table 1. Mean differences were tested using analysis of variance with a correction for unequal cell sizes. Parent income was significantly lower for the families with participating at-risk children. Differences on the other family demographics between these groups were not significant.

Table 1.

Family Demographics for Students in the At-Risk and Universal Samples.

Family Variable Mean SD Mean SD F-Ratio
At-Risk Sample Universal Sample
Family Income1 7.01 3.77 7.73 3.74 3.91*
Mother’s Education2 4.50 1.32 4.55 1.50 0.10
Single Parent (percent) .06 .23 .04 .19 0.57
Caucasian (percent) .91 .29 .86 .34 2.33
1

Scale: 1 = < 4,999, 2 = 5,000–9,999, 3 = 10,000–14,999, 4 = 15,000–19,999, 5 = 20,000–24,999, 6 = 25,000–29,999, 7 = 30,000–34,999, 8 = 35,000–39,999, 9 = 40,000–44,999, 10 = 45,000–49,999, 11 = 50,000–59,999, 12 = 60,000–69,999, 13 ≥ 70,000.

2

Scale: 1 ≤ 8th grade, 2 = 9–11 grade, 3 = High school diplomat or GED, 4 = some college, 5 = college graduate, 6 = post college coursework.

*

p < .05

Procedures

Assessment procedures were the same for the at-risk and universal samples. At the end of first grade teachers and parents were asked to fill out questionnaires on participating children’s behavior. Archival records of discipline referrals and school-level information were collected after the end of the first grade year.

Teacher questionnaires

Teachers completed questionnaires on the behavior of all participating students in their classroom. These questionnaires were either picked up at the school or mailed to project offices in prepaid return envelopes. Teachers were paid $10 per questionnaire completed. Questionaires were filled out by first grade teachers for 94% of the at-risk students and 89% of the students from the universal sample.

Parent questionnaires

Questionnaires on family demographics, parenting skills, and child behavior were mailed to the primary parent’s home. After completion the questionnaires were mailed to project offices in prepaid return envelopes. Parents were paid $30 for completing the full questionnaire. Families in the third cluster of schools were offered a shorter questionnaire and were paid $15 for completion. Both the full and shorter questionnaires had the same student behavior ratings used in the present study. Parents were also offered gift certificates for completing the questionnaire within two weeks. Follow-up phone calls and letters were conducted as needed to obtain questionnaires. The questionnaires were completed by parents on 84% of the at-risk sample of students and on 82% of the universal sample of students.

Archival records

Archival records on discipline referrals were collected from participating schools after the end of students’ first grade year. Parents of all participating students gave permission to collect school records. Project research assistants gathered information from students’ academic, Special Education, and behavioral records. School-level information was collected from Oregon Department of Education after the end of the first grade school year.

Measures

School discipline referrals

The date, student action or behavior, and the consequence for each school discipline referral were recorded. Some schools used systematic databases for keeping discipline referral records, such as School-Wide Information System (SWIS; May et al., 2000, www.SWIS.org) and Pentamation (SUNGARD© Pentamation Inc., 2005, www.pentamation.org). SWIS is designed for school staff to enter discipline referral information into a secure Web-based data system. Reports can be generated from SWIS that offer graphs and charts of individual student and school-wide discipline referrals and consequences, as well as when and where discipline problems occur. SWIS is intended to assist schools in data-based decision making for intervention planning. Pentamation Web-based software allows schools to manage daily information about their students, including disciplinary problems. The software allows school personnel to enter a description of the incident, date, witnesses, victims, and location, as well as consequences and guidance offered. In those schools using either of these tracking systems, the discipline referral information for participating students was provided via printouts from the database. Data collectors kept records of whether or not schools used a systematic data base for tracking school discipline referrals.

After the collection of discipline referral data from school records, each student behavior listed was coded into one of 14 behavioral categories and each consequence listed in the records was coded into 1 of 12 categories. A manual with examples of the behavior and consequence categories was created (Table 2 includes examples for each category). Reliability checks of the coding were conducted on 20% of randomly selected records of the students with discipline referrals. An expert rater (the supervisor) decided the accurate code in cases where the two coders did not agree. On the average, 82% agreement was obtained on the behavior and 91% on consequence coding.

Table 2.

Behavior for which Students Received Discipline Referrals and Consequences Delivered

Percent of SDRs Behaviors and consequences Examples
Student behavior resulting in SDR
46% Physical aggression Hitting, kicking, pinching, scratching, pushing, grabbing, shaking, biting, spitting on, knocking down, or hurting another student. Aggressive or violent interactions. Twisting one’s arm or pulling one’s hair. Throwing things at someone. Fighting.
12% Disruptive, breaks rules in common areas. Common areas = playground, hallway, cafeteria, or bathroom. Playing where not supposed to. Poor line behavior. Screaming in halls. Minor misbehavior.
10% Defiance, insubordination Wouldn’t go to time out. Doesn’t follow rules. Refuses to follow directions. Disrespectful or defiant towards teacher. Yells at teacher.
10% Potentially dangerous behavior Behavior that could hurt others or self (wrestling, throwing wood chips, kicking sawdust, rough play, play fighting). Dangerous behavior on play equipment. Safety violation.
7% Disruptive, breaks rules in classroom Disrupts classroom work (e.g., being loud, repeated belching, turning lights on and off). Minor problems in class.
5% Verbal fighting, teasing Name calling. Bossy with others. Arguing. Harassing, taunting, teasing others. Threatening someone. Provoking others to hurt someone.
2% Vandalism, misuse of equipment Banging on the door. Writing on school wall. Breaking or damaging students’ or school property. Breaking equipment rules.
2% Other rule breaking Rude behaviors. Spitting (not at a person). Unsnapped pants. “Minor problem” (with no location specified). Misbehavior in records is undecipherable or uninterruptible.
1% Stealing, taking others possessions Hiding other student’s things. Taking ball away from others. Grabbing glasses or hat off student. Theft.
1% Uses obscenity Uses bad language. Says “f” word. Inappropriate language.
1% Bus problem Any behavior problem on bus.
.4% Leaves without permission, skipping Leaves class or playground without a pass. Leaves school grounds.
.2%a Weapon possession, use Has or uses knife, gun, or other weapon.
.2%a Physical aggression to school staff member Physical aggression to teacher, aid, bus driver, principal, counselor, or school staff.
1% Reason for SDR missing in record
Consequence delivered for SDR
55% Time out Served time. Spent recess in “chill out”. 10 minutes, 15 minutes, etc. “Benched”.
8% Removal of privilege or item Item taken to office. Loss of privilege. Banned from an area or play structure. Suspended from riding bus. Assigned seat. Sit at “manners” table at lunch.
7% Held in from recess In for morning, lunch, or afternoon recess.
5% Meeting/conference with student See Mr/s. __________ at lunch. Individualized instruction. Talked to student. Problem solved with Mr/s. __________
5% Sent to office, principal Taken to office. Sent to principal. Sent to office to problem solve.
2% Contacted parent Letter home. Called home. Parent contact.
2% Warning given Warning. Probation. Citation.
2% Other consequence Writing assignment. Extra adult monitoring (“shadowing”). Make up study hall (for skipping). Any other consequence.
.5% Apology, restitution Wrote note of apology. Helped pay for damage.
.3% In-school detention, removal from class Sent to detention room.
.2% a Behavior plan developed Put on point system. Peer or self-monitoring. Set behavior goals. Check-in system.
.2% a Work chore, community service Helped clean up, fix, or improve area or equipment. Volunteer for school/community.
13% Consequence missing in records

Note. Total SDRs = 507.

a

Represents one incident.

School-level factors

Information about the participating schools was collected from statistics provided by the Oregon Department of Education (ODE) Web site (Oregon Department of Education, 2004). The ODE data included percentage of students eligible for free and reduced lunch, school spending per student, and average class size. The data used in the analysis represented all project schools during the year participating students were in first grade.

Teacher reports of student behavior

Teachers of participating students completed the Child and Adolescent Disruptive Behavior Inventory (CADBI version 2.3; Burns, Taylor, & Rusby, 2001). The CADBI 2.3 is revised from the CADBI TRS1, a validated measure of teacher reported children’s problem behavior (Burns, Walsh, Owen, & Snell, 1997). The instrument was designed to obtain teacher ratings of symptoms of oppositional defiant disorder, conduct disorder, and attention deficit hyperactivity disorder as defined by the DSM-IV-TR (American Psychiatric Association, 2000). The CADBI 2.3 has parallel items to assess oppositional behaviors toward adults and toward peers, with the same eight DSM-IV-TR symptoms for ODD as the original version (e.g., “loses temper with adults” and “loses temper with peers”). Items were also added to assess behaviors associated with conduct disorder (as defined in the DSM-IV-TR) such as starting fighting, bullying, lying, and taking things without permission, as well as behaviors found to be associated with conduct disorder, such as association with negative peers (Patterson et al., 1992), relational aggression (Crick, 1995), callous/unemotional responses (Frick et al., 1994), and mild covert behavior (Loeber & Hay, 1994). An 8-point scale (“never in the past month” to “10 or more times per day”) was used. A scale depicting teacher-reported oppositional and conduct problems was created by averaging 28 items (8 items on oppositional to adult, 8 items on oppositional to peer, and 12 items on conduct problems). The reliability (Cronbach’s Alpha) for this scale was .966.

Parent reports of student behavior and family income

Similarly, parents of participating students completed the CADBI 2.3 questionnaire. A scale of parent-reported oppositional and conduct problems was created in the same manner as the teacher scale. Cronbach’s Alpha reliability for the parent scale was .945. In addition, parent reported income was used for the measure of student SES, using a categorical scale for income ranges (see Table 1 for the scale).

Analytical Procedures

First, descriptive information was summarized, such as the percentage of first grade boys and girls with discipline referrals from both the at-risk and universal samples, as well as what types of student behaviors warranted discipline referrals and what type of consequences were given. For analyses using gender and risk status, girls were coded as 0 and boys as 1, and the at-risk group was coded as 1 and the universal group as 2. Planned comparison tests of significant differences between gender and risk groups, as well as the gender by group interaction, were conducted.

A Poisson regression model and a zero-inflated Poisson regression model were next analyzed in which the counts of discipline referrals were clustered by school using MPlus (Muthén & Muthén, 2004).1 The Poisson regression model is appropriate for analyzing variables that are a count and the zero-inflated model is appropriate for data with many observations at zero. (As noted above, 82% of participating students had no discipline referrals in first grade.) As implemented in MPlus, the zero-inflated Poisson model predicts the value of the dependent variable (the count of discipline referrals) as well as the probability of being unable to assume any value except zero using a logistic regression estimated with maximum likelihood with robust standard errors to account for the multilevel structure of the data. The individual-level predictors included in these models were gender, risk group, and student SES. The following school-level predictors were also included in the models: (a) whether the school systematically tracked SDRs, (b) school SES based on the percentage of students eligible for free and reduced lunches, (c) average class size, and (d) school spending per student.

Last, to analyze the extent to which discipline referrals at first grade predicted teacher and parent reports of problem behavior at the end of first grade, hierarchical regression analyses were conducted. The number of SDRs was recoded into the following categorical levels: none, one, some = 2 to 9 SDRs (50th to 85th percentile), and high = 10 or more SDRs (above the 85th percentile), and entered into the model as a predictor variable along with gender and risk group. Gender and risk status were first entered into the model, and then the level of SDRs was entered. Separate analyses were conducted for teacher and parent reports of student problem behavior.

Results

Descriptive Information of First Grade Discipline Referrals

Number of referrals by gender and by risk status

To examine mean differences in SDRs between girls and boys and between the at-risk sample of students and the universal sample, a two by two ANOVA was computed with gender and risk status. As expected boys had more SDRs than girls (F = 30.89, p < .001) and students in the at-risk sample had more SDRs than students in the universal sample (F = 18.19, p < .001). The interaction between gender and risk status also was significant (F = 14.38, p < .001). The average number of SDRs for boys in the at-risk sample was 2.99, four times greater than for boys in the universal sample (mean = .74). For girls in the at-risk sample the average number of SDRs was .37, about 1½ times greater than for girls in the universal sample (mean = .24).

The percentages of boys and girls from both samples who received any SDR, only one SDR, 2–9 SDRs, and more than 10 SDRs were also calculated (see Figure 1). Forty-two percent of at-risk boys had at least one SDR and 20% of boys from the universal sample had at least one SDR. Only 15% of girls in the at-risk sample had at least one SDR and 8% of girls in the universal sample had at least one SDR. The figure illustrates that the boys in the universal sample look quite similar to the girls in the at-risk sample—for each of these groups, 8% had only one SDR in first grade and 8% had 2–9 SDRs. No girls had more than 9 discipline referrals in first grade. Ten percent of at-risk boys had 10 or more SDRs in first grade and the greatest number of SDRs that any boy had was 26.

Figure 1.

Figure 1

Percentage of first grade students with school discipline referrals.

Percentage of students with SDRs by school

There were also differences in the means of discipline referrals by school (F = 11.06, p < .001). To illustrate the school differences the percentage of students with one or more discipline referral for each school was calculated (see Figure 2). One school is not included in this analysis as a result of too much missing data for first grade SDRs. These data are missing because the school destroyed all but one of the students’ records of SDRs before we could collect the information from them. This school had a practice of destroying discipline referral data of young students before the next school year occurred (due to concerns of negative effects that could occur by negative labeling of students with discipline referrals). School personnel had forgotten that they promised to keep the archival data in their records until research project staff recorded them. Of the 19 schools depicted, 3 had no participating students with discipline referrals in first grade. The highest percentage was a school in which 61% had at least one discipline referral. The median was a school with 15% of participating students with at least one discipline referral in first grade.

Figure 2.

Figure 2

Percentage of students in each school with at least one SDR in first grade.

Type of behavior for which students receive SDRs and type of consequences delivered

The most prevalent behaviors that received discipline referrals for first grade students, representing 85% of the SDRs, were physical aggression, disruption or rule breaking in common school areas and in the classroom, defiance or subordination, and potentially dangerous behavior (Table 2). The system for coding SDRs also included lying or cheating, tardiness, unexcused absence, dress code violation, tobacco possession or use, alcohol possession or use, and drug possession or use—no SDR reports included these behaviors from first grade students. The most prevalent consequences, representing 80%, were time out, held in from recess, removal of privilege or item, meeting with student, and sent to the office or principal. The system for coding consequences also included conference with parent, staying after school or after school detention, sent home or out of school suspension, and expulsion—none of these were recorded in the records reviewed.

School-level Predictors of Discipline Referrals

For the Poisson and the zero-inflated Poisson models, as expected the individual-level variables gender and risk status significantly contributed to the prediction of SDR counts in first grade. For the school-level variables, spending per student was not a significant predictor in either model and was removed from the models. The results of the zero-inflated Poisson model are presented in Table 3.

Table 3.

Zero-Inflated Poisson Model with School-level Predictors of Discipline Referrals

Variable Coefficienta S.E. z-score
Predicting SDR counts
 Gender 0.77 0.36 2.18*
 Risk status −0.62 0.11 −5.70***
 Student SES −0.03 0.02 −1.40
 Systematic tracking of SDRs 0.41 0.38 1.09
 % eligible for free/reduced lunch −6.67 1.49 −4.49***
 Average class size −0.09 0.05 −1.72*
Predicting Probability of not receiving SDR
 Gender −1.22 0.36 −3.40**
 Risk status 1.01 0.39 2.55**
 Student SES 0.10 0.04 2.52**
 Systematic tracking of SDRs −1.12 0.40 −2.81**
 % eligible for free/reduced lunch 4.71 2.46 1.92*
 Average class size 0.18 0.07 2.50**
a

Regression coefficients are on a log scale.

*

p ≤ .05.

**

p ≤ .01.

***

p ≤ .001.

The zero-inflated Poisson model predicted the count of SDRs and also the probability of not receiving an SDR. Boys and at-risk students had higher numbers of SDRs and had a greater probability of having at least one SDR in first grade. Student SES, on the other hand, did not significantly predict the number of SDRs a student received, but did predict the probability of receiving at least one SDR. Students of lower SES were more likely to receive an SDR. First grade students were more likely to receive an SDR in schools that had a systematic tracking system; however, systematic tracking did not predict the number of SDRs. Odds ratios also were calculated to examine the strength of the influence of systematic tracking of SDRs. The odds were 3.45 times higher for a student to have a discipline referral if he or she were in a school that had a systematic method for tracking school discipline referrals (likelihood ratio = 16.433, p < .001). Students in schools with a higher of percentage of students on free or reduced lunch (i.e., lower school SES) had fewer discipline referrals. To measure magnitude of this relationship, the estimated poisson coefficient was exponentiated, divided by the standard deviation, and then subtracted by one (Wooldridge, 2002). The resulting magnitude was 99%, representing the percentage increase in SDRs resulting in one standard deviation increase in school SES. School SES also significantly predicted the probability of getting an SDR. Average class size significantly predicted the SDR counts (magnitude of 56%) and the probability of receiving an SDR (magnitude of 42%). Students in schools with larger class sizes had fewer SDRs and a lower probability of getting an SDR.

Prediction of Teacher- and Parent-Rated Behavior Problems

Results for the hierarchical regression to demonstrate the extent to which the level of school discipline referrals predict teacher- and parent-reported student problem behavior are in Table 4. First, gender and risk status were entered into the model, each significantly contributing to the prediction of problem behavior as rated by first grade teachers. These student characteristics explained 16% of the variance. The level of SDRs was entered next, significantly adding to the prediction and explaining an additional 8% of the variance of teacher-reported problem behavior. On the other hand, gender did not predict parent-reported problem behavior and risk status significantly contributed to only a small portion (3%) of the variance. The level of SDRs significantly added to the prediction (explaining an additional 3% of the variance) of parent-reported problem behavior at the end of first grade. In contrast to the prediction of discipline referrals, none of the school-level variables added to the prediction of teacher- or parent-reported problem behavior.

Table 4.

Hierarchical Regression Predicting Teacher and Parent Reports of Student Problem Behavior

Variables β t R2 ΔR2
Predicting teacher report of oppositional and conduct behavior
 Gender .137 −9.19***
 Risk status −.358 3.52*** .16
 Level of SDRs .282 6.92*** .24 .08
Predicting parent report of oppositional and conduct behavior
 Gender .038 0.92
 Risk status −.160 −3.89*** .03
 Level of SDRs .178 3.96*** .06 .03
***

p ≤ .001

Discussion

Summary of Results

The results of this descriptive study demonstrate that in first grade boys are more likely to receive a discipline referral than girls and they also receive greater numbers of discipline referrals than girls. This is not surprising given that the most frequent misbehavior for which discipline referrals were delivered was physical aggression, a behavior that is more predominately displayed by boys (e.g., Pellegrini & Blatchford, 2000; Serbin, Marchessault, McAffer, Peters, & Schwartzman, 1993). Discipline referrals were next mostly given for disruptive and defiant behaviors. The most prevalent consequence for students who received discipline referrals was “time out.”

Results also indicate that students who were identified by their kindergarten teacher as presenting more challenging behaviors (the at-risk sample) were more likely to receive a discipline referral as well as greater numbers of discipline referrals than their peers who did not present such challenging behaviors in kindergarten. The families of first grade children in the at-risk sample had significantly lower SES, yet student SES did not predict the number of SDRs a student received, only the probability that a student would receive at least one SDR. Presentation of problem behavior in kindergarten appears to be more salient than family income in predicting the prevalence of problem behavior at school in first grade.

Moreover, receiving greater levels of discipline referrals in first grade predicted both teacher- and parent-reported problem behavior at the end of the school year. A stronger prediction was found for teacher ratings of problem behavior, which was expected given that discipline referrals were delivered in the school setting. Although not examined in this study, it would be interesting to investigate the longitudinal trends throughout elementary school for discipline referrals and problem behavior reported by teachers and parents.

This descriptive study also attempted to define school-level factors that explained the between-school variance of SDR delivery. When examining school-level influences, results demonstrated that first grade students are more likely to receive a discipline referral if they were in a school with a systematic method for tracking them. It is possible that schools that systematically track or monitor discipline referrals also tend to monitor their students behavior better, and thus are more likely to be aware of misbehavior when it happens. It also is possible that the delivery of SDRs was more accurately maintained in the records of schools with systematic tracking, resulting in higher numbers of SDRs.

Results also indicated that schools with a higher percentage of students receiving free or reduced lunches (an indicator of lower SES) and with larger class sizes had fewer discipline referrals. These findings were in contrast to findings by Winbinger et al. (2000) in which larger-sized schools and schools with higher percentages of students receiving free and reduced lunches had more discipline referrals. The between-school variance was not explained by the average amount of money the school spent per student in the year. An analysis examining how schools are actually spending their money (e.g., higher teacher salaries, more behavior support staff, expensive curriculums, better computers, etc.) is needed to better understand how school expenditures may influence school discipline problems.

This study also examined potential school-wide predictors of teacher- and parent-reported problem behaviors. Results indicate that none of the school-level indicators predicted either the teacher or parent ratings. These ratings do not appear to be as highly influenced by school-level demographics, behavior management procedures, or policies as school discipline referrals appear to be.

Limitations

A limitation of this study is that adequate information about individual and school-wide interventions taking place in the schools was not collected. An important school-level influential factor to investigate would be the level and type of individual and school-wide behavioral supports being provided to students. Although school-wide intervention programs have resulted in decreases in discipline referrals in elementary schools (Flay et al., 2001; Putnam et al., 2003; Twemlow et al., 2001), the systematic tracking of discipline referrals is often part of the school-wide intervention process (e.g., Sprague et al., 2001). In the present study, higher levels of discipline referrals were found in schools that systematically tracked them. Schools beginning a similar school-wide program may see a surge in discipline referrals if they are new to tracking them in a systematic way, but then see decreases in SDRs as the school-intervention program becomes fully in place.

Another limitation is that student records were lacking in reports of positive supports offered or incentives for positive behavior. Although the data collectors looked for records of positive referrals, incentives, and consequences in the school records, very few were found. Only one incidence of developing a support plan involving a point system was found as a consequence for a discipline referral in the first grade student records. The data collectors noticed that some schools were offering incentives and points systems to students, yet did not seem to keep systematic records of them. Working with schools to set up a method for collecting such information at the start of the school year would have been helpful. This limitation also illustrates a shortcoming in current school practices as well. Providing more attention to incentives and positive consequences may be beneficial given the evidence that school environments can increase positive behavior by teaching and reinforcing it (Walker, Colvin, & Ramsey, 1995). For example, Kellam, Ling, Merisca, Brown, and Ialongo (1998) found that setting clear behavioral expectations and rewarding appropriate behavior reduced aggressive behavior in first grade classrooms, and this reduction lasted into middle school.

Results of this study are limited to English and Spanish speaking families as most of the study participants were Caucasian or Hispanic. There were not enough numbers of families of different minorities to analyze differences by race. Thus it is inappropriate to generalize these findings to other ethnic populations.

Future Research

It is clear that more studies are needed to explain the sizeable between-school variance of the delivery of school discipline referrals. Other potential explanations might be related to the schools’ varying philosophies on the best way to handle misbehavior of young children. For example, in one school misbehavior may be handled by individual teachers in their classrooms and discipline referrals are only to be used for very extreme behaviors that the teacher cannot handle. In another school, the policy may be to deliver a discipline referral for every incidence of problem behavior regardless of the intensity. Another influential factor may be the support teachers receive in dealing with challenging behavior. Teachers who feel supported and confident that the administration will appropriately follow through on discipline referrals may more likely give a discipline referral when a problem occurs than a teacher who is lacking in support and follow through from the administration.

Implications for Practice

This study provides important information about the nature of discipline referrals for first grade students that has implications for prevention work in the schools. First of all, discipline problems for students most often were given for physically aggressive behavior. Physical aggression is most often found in boys and the present study findings show that boys get more discipline referrals than girls. In practice, these findings indicate that SDRs may be a good way to detect and monitor boys’ problem behavior, but an alternative mechanism may be needed for detecting and monitoring problem behavior, such as social aggression, in girls.

The results also indicate that caution must be taken when using discipline referrals as an indicator of risk or need of intervention for a given student since schools were found to vary widely regarding the percentage of first grade students who received discipline referrals and the numbers of discipline referrals that were delivered. This finding of between-school variation has important implications for both practitioners and researchers. First, students whose behavior is being monitored for diagnostic or intervention purposes and who move to a different school may experience great increases or decreases in their numbers of discipline referrals due to the different schools’ methods for dealing with problem behavior. Moreover, the substantial between-school variance makes it difficult to validly compare discipline referrals of individual students attending different schools. It would therefore be difficult to obtain valid national norms of school discipline referrals for individual students. Researchers also need to take school-level variance into account, particularly when examining intervention effects between different schools with discipline referrals as an outcome.

Given the importance of early detection of risk for behavioral problems, it is imperative that practitioners and researchers have a clear understanding of the nature of the way in which problem behaviors are measured in young children. School discipline referrals are typically used to measure problem behavior in the school setting; however, there appears to be systematic bias on the delivery of discipline referrals between schools. Further research on the school-level factors that influence the delivery of discipline referrals is needed. Systematic tracking of school discipline referrals is recommended for problem solving and planning of behavior management and support at the school level (Sugai et al., 2000). It is also recommended that practitioners and researchers collect multiple measures of problem behavior for identifying individual students in need of intervention, selecting appropriate interventions, and for evaluating intervention effects.

Footnotes

This research was supported by grants #RO1-DA12202 and #RO1-DA12986 from the National Institute on Drug Abuse.

1

Model fit of these two models was compared using Akaike Information Criterion (Akaike, 1973) corrected for sample size (AICc; Hurvich & Tsai, 1989) to determine which was the best fitting model. The zero-inflated Poisson model demonstrated substantially improved fit over the Poisson model (ΔAICc = 250.2). AICc values greater than 4 indicate empirical support for a better fitting model (Burnham & Anderson, 2002).

References

  1. Akaike H. Information theory and an extension of the maximum likelihood principle. In: Petrox BN, Caski F, editors. Second international symposium on information theory. Budapest; Akademiai Kiado: 1973. pp. 267–281. [Google Scholar]
  2. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4th ed., TR. Washington, DC: Author; 2000. [Google Scholar]
  3. Bryant AL, Schulenberg J, Bachman JG, O’Malley PM, Johnston LD. Understanding the links between school misbehavior, academic achievement, and cigarette use: A national panel study on adolescents. Prevention Science. 2000;1:71–88. doi: 10.1023/a:1010038130788. [DOI] [PubMed] [Google Scholar]
  4. Burns GL, Taylor TK, Rusby J. Child and Adolescent Disruptive Behavior Inventory-Version 2.3. Pullman, WA: Author; 2001. [Google Scholar]
  5. Burns GL, Walsh JA, Owen SM, Snell J. Internal validity of attention deficit hyperactivity disorder, oppositional defiant disorder, and overt conduct disorder symptoms in young children: Implications from teacher ratings for a dimensional approach to symptom validity. Journal of Clinical Child Psychology. 1997;26:266–275. doi: 10.1207/s15374424jccp2603_5. [DOI] [PubMed] [Google Scholar]
  6. Crick NR. Relational aggression: The role of intent attributions, feelings of distress, and provocation type. Development and Psychopathology. 1995;7:313–322. [Google Scholar]
  7. Farrington DP. Early predictors of adolescent aggression and adult violence. Violence and Victims. 1989;4:79–100. [PubMed] [Google Scholar]
  8. Flay BR, Allred CG, Ordway N. Effects of the Positive Action program on achievement and discipline: Two matched-control comparisons. Prevention Science. 2001;2:71–89. doi: 10.1023/a:1011591613728. [DOI] [PubMed] [Google Scholar]
  9. Frick PJ, O’Brien BS, Wootton JM, McBurnett K. Psychopathy and conduct problems in children. Journal of Abnormal Psychology. 1994;103:700–707. doi: 10.1037//0021-843x.103.4.700. [DOI] [PubMed] [Google Scholar]
  10. Gabel S, Shindledecker R. Aggressive behavior in youth: Characteristics, outcome, and psychiatric diagnoses. Journal of the American Academy of Child and Adolescent Psychiatry. 1991;30:982–988. doi: 10.1097/00004583-199111000-00017. [DOI] [PubMed] [Google Scholar]
  11. Hester PP, Baltodano HM, Gable RA, Tonelson SW, Hendrickson JM. Early intervention with children at risk of emotional/behavioral disorders: A critical examination of research methodology and practices. Education and Treatment of Children. 2003;26:362–381. [Google Scholar]
  12. Hurvich CM, Tsai C. Regression and time series model selection in small samples. Biometrika. 1989;76:297–307. [Google Scholar]
  13. Irvin LK, Tobin TJ, Sprague JR, Sugai G, Vincent CG. Validity of office discipline referral measures as indices of school-wide behavioral status and effects of school-wide behavioral interventions. Journal of Positive Behavior Interventions. 2004;6:131–147. [Google Scholar]
  14. Kellam SG, Ling X, Merisca R, Brown CH, Ialongo N. The effect of the level of aggression in the first grade classroom on the course and malleability of aggressive behavior into middle school. Development and Psychopathology. 1998;10(2):165–185. doi: 10.1017/s0954579498001564. [DOI] [PubMed] [Google Scholar]
  15. Kupersmidt JB, Coie JD, Dodge KA. The role of poor peer relationships in the development of disorder. In: Asher SR, Coie JD, editors. Peer rejection in childhood. New York: Cambridge University Press; 1990. pp. 274–305. [Google Scholar]
  16. Loeber R, Green SM, Lahey BB, Frick PJ, McBurnett K. Findings on disruptive behavior disorders from the first decade of the developmental trends study. Clinical Child and Family Psychology Review. 2000;3:37–60. doi: 10.1023/a:1009567419190. [DOI] [PubMed] [Google Scholar]
  17. Loeber R, Hay DF. Developmental approaches to aggression and conduct problems. In: Rutter M, Hay DF, editors. Development through life: A handbook for clinicians. Oxford: Blackwell; 1994. pp. 488–516. [Google Scholar]
  18. May S, Ard W, Todd A, Horner R, Glasgow A, Sugai G, et al. School-wide Information System (SWIS ©) University of Oregon: Educational and Community Supports; 2000. [Google Scholar]
  19. McCurdy BL, Mannella MC, Eldridge N. Positive behavior support in urban schools: Can we prevent the escalation of antisocial behavior? Journal of Positive Behavior Interventions. 2003;5:158–170. [Google Scholar]
  20. Metzler CW, Biglan A, Rusby JC, Sprague JR. Evaluation of a comprehensive behavior management program to improve school-wide positive behavior support. Education and Treatment of Children. 2001;24:448–479. [Google Scholar]
  21. Moffitt TE. Adolescence-limited and life-course-persistent antisocial behavior: A developmental taxonomy. Psychological Review. 1993;100:674–701. [PubMed] [Google Scholar]
  22. Morrison GM, Anthony S, Storino M, Dillon C. An examination of the disciplinary histories and the individual and educational characteristics of students who participate in an in-school suspension program. Education and Treatment of Children. 2001;24:276–293. [Google Scholar]
  23. Murdock TB, Anderman LH, Hodge SA. Middle-grade predictors of students’ motivation and behavior in high school. Journal of Adolescent Research. 2000;15:327–351. [Google Scholar]
  24. Muthén LK, Muthén BO. MPlus User’s Guide. Los Angeles: Muthén & Muthén; 2004. [Google Scholar]
  25. Nelson JR, Gonzalez JE, Epstein MH, Benner GJ. Administrative discipline contacts: A review of the literature. Behavioral Disorders. 2003;28:249–281. [Google Scholar]
  26. Oregon Department of Education. Statewide Report Card—2003–4—An Annual Report to the Legislature on Oregon Public Schools. 2004 Retrieved from http://www.ode.state.or.us/data/annreportcard\rptcard2004.pdf.
  27. Parker JG, Asher SR. Peer relations and later personal adjustment: Are low-accepted children at risk? Psychological Bulletin. 1987;102:357–939. doi: 10.1037//0033-2909.102.3.357. [DOI] [PubMed] [Google Scholar]
  28. Patterson GR, Reid JB, Dishion TJ. Antisocial boys: Asocial interactional approach. Vol. 4. Eugene, OR: Castalia Publishing; 1992. [Google Scholar]
  29. Pellegrini AD, Blatchford P. The child at school: Interactions with peers and teachers. London: Arnold; 2000. [Google Scholar]
  30. Putnam RF, Luiselli JK, Handler MW, Jefferson GL. Evaluating student discipline practices in a public school through behavioral assessment of office referrals. Behavior Modification. 2003;27:505–523. doi: 10.1177/0145445503255569. [DOI] [PubMed] [Google Scholar]
  31. Rose TL. Current disciplinary practices with handicapped students: suspensions and expulsions. Exceptional Children. 1988;55:230–239. doi: 10.1177/001440298805500305. [DOI] [PubMed] [Google Scholar]
  32. Serbin LA, Marchessault K, McAffer V, Peters P, Schwartzman AE. Patterns of social behavior on the playground in 9- to 11-year-old girls and boys: Relation to teacher perceptions and to peer ratings of aggression, withdrawal, and likability. In: Hart CH, editor. Children on playgrounds: Research perspectives and applications. Albany, NY: State University of New York Press; 1993. pp. 162–183. [Google Scholar]
  33. Shores RE, Jack SL, Gunter PL, Ellis DN, DeBriere T, Wehby J. Classroom interactions of children with severe behavior disorders. Journal of Emotional and Behavioral Disorders. 1993;1:27–39. [Google Scholar]
  34. Sprague J, Walker H, Golly A, White K, Myers DR, Shannon T. Translating research into effective practice: The effects of a universal staff and student intervention on indicators of discipline and school safety. Education and Treatment of Children. 2001;24:495–511. [Google Scholar]
  35. Sugai G, Sprague JR, Horner RH, Walker HM. Preventing school violence: The use of office discipline referrals to assess and monitor school-wide discipline interventions. Journal of Emotional and Behavioral Disorders. 2000;8:94–101. [Google Scholar]
  36. SUNGARD© Pentamation, Inc. PLUS SERIES student information management systems software. 2005 Retrieved from http://www.pentamation.com/
  37. Taylor TK, Burns GL, Rusby JC, Foster EM. Oppositional defiant disorder toward adults and oppositional defiant disorder toward peers: Initial evidence for two separate constructs. Psychological Assessment. doi: 10.1037/1040-3590.18.4.439. (in press) [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Taylor TK, Rusby JC, Metzler CW, Foster EM, Jones D, Blair J, Johnson-Shelton D, Biglan A. Success Through the Incredible Years: A randomized controlled trial prevention study. 2006 Manuscript in preparation. [Google Scholar]
  39. Taylor-Greene S, Brown D, Nelson L, Longton J, Gassman T, Cohen J, et al. School-wide behavioral support: Starting the year off right. Journal of Behavioral Education. 1997;7:99–112. [Google Scholar]
  40. Tobin TJ, Sugai GM. Using sixth-grade school records to predict school violence, chronic discipline problems, and high school outcomes. Journal of Emotional and Behavioral Disorders. 1999;7:40–53. [Google Scholar]
  41. Twemlow SW, Fonagy P, Sacco FC, Gies ML, Evans R, Ewbank R. Creating a peaceful school learning environment: A controlled study of an elementary school intervention to reduce violence. American Journal of Psychiatry. 2001;158:808–810. doi: 10.1176/appi.ajp.158.5.808. [DOI] [PubMed] [Google Scholar]
  42. Walker B, Cheney D, Stage S, Blum C. Schoolwide screening and positive behavior supports: Identifying and supporting students at risk for school failure. Journal of Positive Behavior Interventions. 2005;7:194–204. [Google Scholar]
  43. Walker HM, Colvin G, Ramsey E. Antisocial behavior in the school: Strategies and best practices. Pacific Grove, CA.: Brooks/Cole Publishing; 1995. [Google Scholar]
  44. Walker HM, Horner RH, Sugai G, Bullis M. Integrated approaches to preventing antisocial behavior patterns among school-age children and youth. Journal of Emotional and Behavioral Disorders. 1996;4:194–209. [Google Scholar]
  45. Walker HM, McConnell SR. Walker-McConnell scale of social competence and school adjustment: Elementary version. San Diego: Singular; 1995. [Google Scholar]
  46. Walker HM, Severson HH. Systematic screening for behavior disorders. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement, Educational Resources Information Center; 1992. [Google Scholar]
  47. Walker HM, Stieber S, Ramsey E, O’Neill RE. Longitudinal prediction of the school achievement, adjustment, and delinquency of antisocial versus at-risk boys. Remedial and Special Education. 1991;12:43–51. [Google Scholar]
  48. Walker HM, Stieber S, Ramsey E, O’Neill RE. Fifth grade school adjustment and later arrest rate: A longitudinal study of middle school antisocial boys. Journal of Child and Family Studies. 1993;2:295–315. [Google Scholar]
  49. Webster-Stratton C. The Incredible Years Training Series. Office of Juvenile Justice and Delinquency Prevention Bulletin, June (pp. 1–24) Washington, DC: Office of Justice Programs, U.S. Department of Justice; 2000. [Google Scholar]
  50. Wehby JH, Symons FJ, Shores RE. A descriptive analysis of aggressive behavior in classrooms for children with emotional and behavioral disorders. Behavioral Disorders. 1995;20:87–105. [Google Scholar]
  51. Winbinger B, Katsiyannis A, Archwamety T. Disciplinary practices in Nebraska’s public schools. Journal of Child and Family Studies. 2000;9:389–399. [Google Scholar]
  52. Wooldridge JM. Econometric analysis of cross section and panel data. Cambridge, MA: MIT Press; 2002. [Google Scholar]
  53. Wright JA, Dusek JB. Compiling school base rates for disruptive behaviors from student disciplinary referral data. School Psychology Review. 1998;27:138–147. [Google Scholar]

RESOURCES