Effects of School-Wide Positive Behavioral Interventions and Supports on Child Behavior Problems

Abstract

OBJECTIVE:

School-Wide Positive Behavioral Interventions and Supports (SWPBIS) is a universal prevention strategy currently implemented in >16 000 schools across the United States. SWPBIS intends to reduce students’ behavior problems by altering staff behaviors and developing systems and supports to meet children’s behavioral needs. The current study reports intervention effects on child behaviors and adjustment from an effectiveness trial of SWPBIS.

METHODS:

The sample of 12 344 elementary school children was 52.9% male, 45.1% African American, and 46.1% Caucasian. Approximately 49% received free or reduced-priced meals, and 12.9% received special education services at baseline. The trial used a group randomized controlled effectiveness design implemented in 37 elementary schools. Multilevel analyses were conducted on teachers’ ratings of children’s behavior problems, concentration problems, social-emotional functioning, prosocial behavior, office discipline referrals, and suspensions at 5 time points over the course of 4 school years.

RESULTS:

The multilevel results indicated significant effects of SWPBIS on children’s behavior problems, concentration problems, social-emotional functioning, and prosocial behavior. Children in SWPBIS schools also were 33% less likely to receive an office discipline referral than those in the comparison schools. The effects tended to be strongest among children who were first exposed to SWPBIS in kindergarten.

CONCLUSIONS:

These findings provide support for the hypothesized reduction in behavior problems and improvements in prosocial behavior and effective emotion regulation after training in SWPBIS. The SWPBIS framework appears to be a promising approach for reducing problems and promoting adjustment among elementary school children.

What’s Known on This Subject:

School-Wide Positive Behavioral Interventions and Supports (SWPBIS) is a widely used universal prevention strategy currently implemented in >16 000 schools across the United States. Previous research has shown positive effects on school climate and school-level discipline problems.

What This Study Adds:

This study reports multilevel results on data from a 4-year randomized controlled effectiveness trial of SWPBIS in 37 elementary schools. Results indicate significant impacts on children’s aggressive behavior problems, concentration problems, office discipline referrals, emotion regulation, and prosocial behavior.

Conduct and disruptive behavior problems pose a significant concern for children’s development. An onset of disruptive and aggressive behavior problems in elementary school is associated with an increased risk for academic problems, placement in special education programs, school dropout, substance abuse problems, and antisocial behavior.^1,2 There is growing interest in school-wide prevention models, such as School-Wide Positive Behavioral Interventions and Supports (SWPBIS),^3,4 as an approach for preventing an early-onset of behavior problems and promoting positive adjustment.

SWPBIS is a noncurricular universal prevention strategy that aims to alter the school’s organizational context to implement enhanced procedures and systems to guide data-based decisions related to student behavior problems and academics. It applies behavioral, social learning, and organizational principles to an entire student body consistently across all school contexts.⁵ Schools establish a set of positively stated, school-wide expectations for student behavior, which are taught to all students and staff. It aims to prevent disruptive behavior and enhance organizational climate by implementing a 3-tiered prevention framework⁶ in which 2 levels of selective and indicated programs are implemented to complement the universal school-wide components (for a review, see Sugai and Horner, Taylor-Greene and Kartub, and Horner et al^3,4,7,8). However, most of the >16 000 schools currently implementing SWPBIS have focused on the universal components (see pbis.org).

Two randomized controlled trials (RCTs) were recently conducted of SWPBIS in elementary schools and indicated positive outcomes for students and staff. Specifically, a 1-year waitlist RCT indicated significant improvements in school climate and achievement.⁹ Previous studies reporting data from the current sample of 37 elementary schools enrolled in a 4-year RCT indicated significant improvements in the schools’ organizational health, school-level discipline data, and the implementation of classroom-based supports.^10–12 To date, there has been no research using longitudinal RCT designs that has examined the effect on teacher ratings of behavior problems and social-emotional adjustment.

We used data from the 4-year RCT to examine the hypothesis that children in schools implementing SWPBIS would have better teacher-rated emotion regulation and prosocial behaviors and fewer concentration problems and disruptive behaviors. We also hypothesized that children in SWPBIS schools would be less likely to be referred to the principal’s office or suspended. Given the group RCT design (ie, students nested within schools), we used a multilevel modeling approach¹³ and adjusted for select covariates at the school (eg, enrollment) and child levels (eg, gender).¹⁴ These findings will enhance our understanding of the effects achieved through the commonly used SWPBIS approach by determining its impact on a range of outcomes.

Method

Design

Data came from a group randomized controlled effectiveness trial^13,15 of the universal SWPBIS model that aimed to determine the impact of SWPBIS on discipline problems and the school environment. Only public elementary schools were eligible for inclusion, and all schools approached about participation agreed to enroll. An open-cohort design was used, such that new students could enroll at each data collection; however, resources were not available to follow students who left the participating schools. Schools were matched on select baseline demographics (eg, school enrollment), of which 21 schools were randomized to the intervention condition and 16 to the comparison condition. The comparison schools refrained from implementing SWPBIS for 4 years.

Training

The 21 schools assigned to receive SWPBIS training each formed SWPBIS teams, comprising 5 to 6 members (eg, teachers, administrators), who attended an initial 2-day summer training co-led by one of the developers of SWPBIS. To maintain consistently high levels of fidelity, the SWPBIS teams attended annual 2-day booster training events. Consistent with the effectiveness trial design,¹⁵ all initial training and booster training events were coordinated and led by the PBIS Maryland State Leadership Team and were also attended by other SWPBIS teams from across the state.¹⁶ All SWPBIS schools received at least monthly on-site support and technical assistance from a trained behavior support coach (eg, school psychologist) for the duration of the trial; these staff were trained by the state and supervised by the district. Professional development and technical assistance were provided to the behavior support coaches through state-coordinated training events conducted 4 times each year (see Barrett et al¹⁷ for additional information).

Fidelity

Annual assessments of SWPBIS implementation were conducted in all 37 schools by trained assessors who were unaware of the schools’ implementation status using the validated School-wide Evaluation Tool (SET)¹⁸ and staff self-reports^11,12; both indicated that all SWPBIS schools reached and maintained high-fidelity implementation (eg, 80% on the School-wide Evaluation Tool)¹⁸ by the end of the trial, and no schools in the comparison condition consistently reached high fidelity; 66% of the SWPBIS schools met the 80% high-fidelity threshold¹⁸ within the first year of the trial (see means in Fig 1 and Bradshaw et al^11,12,19 for additional information on the methods and fidelity in the intervention and control conditions).

FIGURE 1.

Mean School-wide Evaluation Tool (SET) fidelity scores at SWPBIS and comparison schools at baseline and years 1 through 4. Analysis of the SET data suggested a significant effect (ie, intervention condition × time interaction) for the overall SET score; Wilks’ Λ = .38, F(4,32) = 13.36, P = .001, η2 = .63, d = 3.22.

Sample

The sample included 37 elementary schools, the size of which was determined through a power analysis. Five data points (fall and spring year 1, spring years 2-4) were collected over the 4 school years (2002–2007) on 12 334 children who were in kindergarten, first, and second grade when the study started. See the CONSORT diagram in Fig 2 and school demographics in Table 1.

FIGURE 2.

CONSORT diagram for the SWPBIS RCT.

TABLE 1.

Student and School Demographic Characteristics

Characteristics	N (%) or M (SD)
Student characteristics (N = 12 344), N (%)
Gender
Male	6482 (52.9)
Female	5782 (47.1)
Race/ethnicity
American Indian/Alaskan Native	76 (0.6)
Asian/Pacific Islander	516 (4.3)
African American	5462 (45.1)
White	5588 (46.1)
Hispanic	473 (3.9)
Grade cohort
Kindergarten (up to third grade)	4156 (33.7)
First (up to fourth grade)	4141 (33.5)
Second (up to fifth grade)	4047 (33.0)
Received special education services	1540 (12.9)
Received free or reduced priced meals	5850 (49.4)
School characteristics (N = 37 schools), mean (SD)
Student mobility	23.6 (8.2)
School enrollment	486.4 (157.8)
Faculty turnover rate	16.1 (7.6)
Student/teacher ratio	11.3 (3.3)

Measures

The Teacher Observation of Classroom Adaptation—Checklist (TOCA-C)²⁰ was completed for each child by their primary classroom teacher. The TOCA-C is a checklist version of the TOCA (TOCA-R),²¹ which has been used in several previous RCTs of school-based prevention programs.^22–24 The TOCA-C measures each child’s level of “aggressive and disruptive behaviors” (fights; 9 items, α = .92), “concentration problems” (pays attention; 7 items, α = .96), “prosocial behaviors” (shows empathy; 5 items, α = .92), and “emotion regulation” (stops and calms down when angry or upset; 4-items, α = .89). Teachers responded to each question using a 6-point Likert scale (1 = never to 6 = almost always), which were averaged to create the 4 subscales, such that higher scores on disruptive behaviors and concentration problems indicated greater problems, whereas higher scores on the prosocial behaviors and emotion regulation scales reflected better adjustment. These scales exhibit strong internal consistency, have a consistent factor structure over time,²⁰ relate to external criteria,²⁵ and are sensitive to relatively modest intervention effects.²⁶ The TOCA-C also included questions regarding each child’s receipt of an office disciplinary referral (ODR) and an out-of-school suspension during that school year using a yes/no dichotomous format. Teacher reports of these events have been shown to be a valid indicator of the child’s discipline problems.²⁷ These items were collapsed over the 5 time points to create an aggregated dichotomous indicator for ODRs and suspensions (1 = ever, 0 = never received) over the course of the trial.

Procedure

Survey packets were mailed to the school and distributed to the teachers by an administrator or secretary. The packets contained a checklist in reference to each child in their classroom, and each survey had a unique identifier for each child, thereby allowing child-specific data to be tracked over the 4 years. Teachers completed a survey for each child in the class 5 times over the course of 4 years (ie, fall baseline, spring of 4 years). The project was approved by the researchers’ institutional review board; passive parental consent was used for child participants.

Analyses

Longitudinal 3-level hierarchical linear models were estimated by using hierarchal linear modeling (HLM6)²⁸ to examine the effect of SWPBIS on children’s changes in behaviors (disruptive behaviors, prosocial behaviors, concentration problems, emotion regulation) over the course of the intervention (Figs 3, 4, 5, and 6). At level 1, the within-individual time scores were entered into the model. At level 2, child characteristics (ie, special education status, race [black vs nonblack children], cohort, free or reduced meals status, gender) were included. Given the group RCT design, intervention status (SWPBIS vs control) was modeled as a school-level characteristic (level 3). Additionally, multilevel logistic regression analyses were conducted in HLM to examine the effect of SWPBIS (modeled as a school-level variable) on receiving an ODR and a suspension.

FIGURE 3.

Impact of SWPBIS on disruptive behaviors.

FIGURE 4.

Impact of SWPBIS on concentration problems.

FIGURE 5.

Impact of SWPBIS on emotion regulation.

FIGURE 6.

Impact of SWPBIS on positive behaviors.

On the basis of previous research with this data,^11,12 the following school-level variables were included as covariates: student mobility, enrollment, student/teacher ratio, and faculty turnover rate. We grand-mean centered all predictor variables at levels 2 and 3, except intervention status.²⁹ Model fit indices (Akaike information criterion [AIC] and Bayesian information criterion [BIC]) are reported in the tables and are interpreted such that smaller values indicate better fit.²⁸

Although the participation rate was consistently high, we examined the missing data patterns but did not find evidence that missingness was problematic.^30,31 For example, baseline scores on concentration problems were unrelated to subsequent missingness on this measure (adjusted odds ratio [AOR] = 1.00, 95% confidence interval = 0.96–1.04). Baseline scores on disruptive behavior were significantly associated with an increased odds of subsequent missingness (AOR = 1.05, 95% confidence interval = 1.03–1.07), yet this difference was small and likely has limited clinical significance. Neither gender nor intervention status had a significant effect on subsequent missingness on teacher ratings of behavior problems. Therefore, our analyses assumed data were missing at random, which assumes that the reason for missingness is not related to the missing value itself or is deemed random after controlling for the variables that are observed.^32,33 HLM adjusts parameter estimates for attrition using full-information maximum-likelihood estimation, a widely recognized and appropriate means of handling missing data³⁴ under the assumption that data are missing at random.³⁵ Specifically, individuals can have missing data across any of the time points and still be included in the analyses; therefore, HLM is robust to missing data within repeated measures.^35,36

Results

The sample of 12 344 children was 52.9% male, 45.1% African American, and 46.1% Caucasian (see Table 1). Approximately 49% received free or reduced-price meals, and 12.9% received special education services. The multilevel analyses indicated a significant positive intervention effect on disruptive behaviors (γ = –0.02, t = –2.23, P < .05, effect size (ES) = 0.12), such that children in SWPBIS schools had lower levels of aggressive and disruptive behaviors compared with those in the control schools (see Table 2). A similar effect was observed on concentration problems (γ = –0.03, t = –2.08, P < .05, ES = 0.08; see Table 3). We explored for possible cross-level interaction effects between SWPBIS and grade cohort, gender, and special education status; however, none were significant.

TABLE 2.

HLM Results for 3-Level Model Examining the Effect of SWPBIS on Disruptive Behaviors

Disruptive Behaviora	Coefficient	SE	t Ratio	P Value
Intercept
Intercept	1.5811	0.0268	59.04	<.001
School-level variables
Mobility	0.0027	0.0028	0.94	NS
Student/teacher ratio	−0.0035	0.0058	−0.60	NS
Faculty turnover	0.0027	0.0031	0.87	NS
Enrollment	0.0031	0.0141	0.22	NS
Student-level variables
Special education status	0.1561	0.0245	6.37	<.001
Ethnicity (black)	0.2381	0.0329	7.25	<.001
Grade cohort	−0.0944	0.0425	−2.22	NS
FARMS	0.1177	0.0181	6.52	<.001
Gender	0.2567	0.0147	17.42	<.001
Slope (Growth)
Intercept	0.0237	0.0099	2.39	<.05
School-level variables
SWPBIS intervention	−0.0202	0.0091	−2.23	<.05
Mobility	0.0006	0.0007	0.85	NS
Student/teacher ratio	0.0037	0.0015	2.49	<.05
Faculty turnover	0.0011	0.0012	0.90	NS
Enrollment	−0.0080	0.0041	−1.97	NS
Student-level variables
Special education status	−0.0012	0.0069	−0.18	NS
Ethnicity (black)	0.0292	0.0054	5.37	<.001
Grade cohort	0.0297	0.0137	2.17	<.05
FARMS	0.0191	0.0051	3.76	<.001
Gender	0.0221	0.0057	3.91	<.01
Post hoc cross-level interactions
Grade cohort × SWPBIS	−0.0205	0.0151	−1.35	NS
Gender × SWPBIS	0.0099	0.0104	0.96	NS
Special education status × SWPIS	−0.0002	0.0118	−0.020	NS
Random Effect	Variance Component	χ2		P Value
Level 1	0.2918
Level 2	0.3026	45753.2		<.001
Level 3 time/slope	0.0023	220.8		<.001

Akaike information criterion = 97001.1; Bayesian information criterion = 96916.7. N = 12 344; SWPBIS was coded 1 (intervention) or 0 (comparison). FARMS indicates free or reduced-price meals status (1 = received FARMS, 0 = did not receive). Gender was coded 1 (male), 0 (female). Ethnicity was coded 1(black), 0 (all others). Grade cohort indicates the grade the youth was in when the trial began, coded 1 (kindergarten), 0 (grades 1 or 2). Special education indicates child receives special education services (coded 1 for yes, 0 for no).

^aResults presented are from the model that did not include the post hoc interactions. The intraclass correlation for disruptive behavior was 0.05.

TABLE 3.

HLM Results for 3-Level Model Examining the Effect of SWPBIS on Concentration Problems

Concentration Problemsa	Coefficient	SE	t Ratio	P Value
Intercept
Intercept	2.2616	0.0347	65.22	<.001
School-level variables
Mobility	−0.0005	0.0030	−0.16	NS
Student/teacher ratio	0.0190	0.0107	1.77	NS
Faculty turnover	−0.0021	0.0031	−0.67	NS
Enrollment	0.0141	0.0182	0.77	NS
Student-level variables
Special education status	0.7057	0.0422	16.72	<.001
Ethnicity (black)	0.2221	0.0430	5.17	<.001
Grade cohort	−0.1818	0.0559	−3.26	<.01
FARMS	0.3048	0.0266	11.47	<.001
Gender	0.4262	0.0178	23.96	<.001
Slope (growth)
Intercept	−0.0036	0.0126	−0.29	NS
School-level variables
SWPBIS intervention	−0.0254	0.0122	−2.08	<.05
Mobility	0.0022	0.0009	2.51	<.05
Student/teacher ratio	−0.0066	0.0029	−2.28	<.05
Faculty turnover	0.0026	0.0010	2.48	<.05
Enrollment	−0.0107	0.0047	−2.28	<.05
Student-level variables
Special education status	−0.0204	0.0093	−2.18	<.05
Ethnicity (black)	0.0352	0.0117	3.01	<.001
Grade cohort	0.0449	0.0195	2.30	NS
FARMS	0.0197	0.0076	2.60	NS
Gender	0.0367	0.0078	4.69	<.01
Post hoc cross-level interactions
Grade cohort × SWPBIS	−0.0396	0.0300	−1.32	NS
Gender × SWPBIS	0.0153	0.0122	1.26	NS
Special education status × SWPIS	0.0199	0.0165	1.21	NS
Random Effect	Variance Component	χ2		P Value
Level 1	0.5597
Level 2	0.6692	51851.3		<.001
Level 3 time/slope	0.0017	104.6		<.001

Akaike information criterion = 100306.4; Bayesian information criterion = 100208.0. SWPBIS was coded 1 (intervention), 0 (comparison). FARMS indicates free or reduced-price meals status. Gender was coded 1 (male), 0 (female). Ethnicity was coded 1 (black), 0 (all others). Grade cohort indicates the grade the youth was in when the trial began, coded 1 (kindergarten), 0 (grades 1 or 2).

^aResults presented are from the model that did not include the post hoc interactions. The interclass correlation for concentration problems was 0.02.

The multilevel analyses also indicated a significant intervention effect on prosocial behavior (γ = 0.03, t = 2.11, P < .05, ES = –0.17), such that children in the intervention had higher levels of positive behaviors compared with those in the control. A similar effect was observed on emotion regulation (γ = 0.03, t = 2.30, P < .05, ES = –0.11), such that children in the intervention schools had better emotion regulation than those in the control. Again we examined for possible interaction effects of intervention status and grade cohort, gender, and special education. The results indicated that children who were in kindergarten when the trial began fared better in SWPBIS schools than in comparison schools on both prosocial behavior (γ = 0.08, t = 2.77, P < .01) and emotion regulation (γ = 0.05, t = 2.38, P < .05). No other interactions were significant (see bottom of Tables 4 and 5 for interaction terms only). We explored the fit of quadratic and cubic growth in the HLM analyses for these continuous outcomes; however, neither resulted in a significant improvement in model fit.

TABLE 4.

HLM Results for 3-Level Model Examining the Effect of SWPBIS on Prosocial Behaviors

Positive Behaviorsa	Coefficient	SE	t Ratio	P Value
Intercept
Intercept	5.1609	0.0376	137.32	<.001
School-level variables
Mobility	−0.0036	0.0033	−1.10	NS
Student/teacher ratio	0.0021	0.0105	0.20	NS
Faculty turnover	−0.0010	0.0037	−0.27	NS
Enrollment	−0.0303	0.0174	−1.74	NS
Student-level variables
Special education status	−0.3323	0.0343	−9.70	<.001
Ethnicity (black)	−0.2090	0.0335	−6.25	<.001
Grade cohort	0.0807	0.0541	1.49	NS
FARMS	−0.2069	0.0252	−8.21	<.001
Gender	−0.2609	0.0179	−14.57	<.001
Slope (growth)
Intercept	−0.0651	0.0166	−3.93	<.01
School-level variables
SWPBIS intervention	0.0335	0.0159	2.11	<.05
Mobility	−0.0006	0.0011	−0.59	NS
Student/teacher ratio	−0.0048	0.0030	−1.59	NS
Faculty turnover	−0.0029	0.0013	−2.19	<.05
Enrollment	0.0143	0.0055	2.59	<.05
Student-level variables
Special education status	−0.0079	0.0108	−0.73	NS
Ethnicity (black)	−0.0133	0.0082	−1.63	NS
Grade cohort	−0.0056	0.0210	−0.27	NS
FARMS	−0.0064	0.0081	−0.79	NS
Gender	−0.0200	0.0077	−2.61	<.05
Post hoc cross-level interactions
Grade cohort × SWPBIS	0.0846	0.0306	2.77	<.01
Gender × SWPBIS	−0.0109	0.0132	−0.83	NS
Special education status × SWPIS	−0.0004	0.0161	−0.02	NS
Random Effect	Variance Component	χ2		P Value
Level 1	0.5328
Level 2	0.3877	36099.8		<.001
Level 3 time/slope	0.0029	401.3		<.001

Akaike information criterion = 94385.8; Bayesian information criterion = 94323.4. SWPBIS was coded 1 (intervention), 0 (comparison). FARMS indicates free or reduced-price meals status. Gender was coded 1 (male), 0 (female). Ethnicity was coded 1(black), 0 (all others). Grade cohort indicates the grade the youth was in when the trial began, coded 1 (kindergarten), 0 (grades 1 or 2).

^aResults presented are from the model that did not include the post hoc interactions. The interclass correlation for prosocial behavior was 0.05.

TABLE 5.

HLM Results for 3-Level Model Examining the Effect of SWPBIS on Emotion Regulation

Emotion Regulationa	Coefficient	SE	t Ratio	P Value
Intercept
Intercept	4.9780	0.0438	113.65	<.001
School-level variables
Mobility	−0.0015	0.0037	−0.41	NS
Student/teacher ratio	−0.0083	0.0105	−0.79	NS
Faculty turnover	−0.0001	0.0046	−0.02	NS
Enrollment	−0.0016	0.0216	−0.07	NS
Student-level variables
Special education status	−0.3605	0.0361	−9.98	<.001
Ethnicity (black)	−0.3300	0.0507	−6.51	<.001
Grade cohort	0.0286	0.0623	0.46	NS
FARMS	−0.2086	0.0355	−5.88	<.001
Gender	−0.4221	0.0233	−18.14	<.001
Slope (growth)
Intercept	−0.0089	0.0164	−0.54	NS
School-level variables
SWPBIS intervention	0.0277	0.0120	2.30	<.05
Mobility	−0.0023	0.0012	−2.02	NS
Student/teacher ratio	−0.0023	0.0022	−1.06	NS
Faculty turnover	−0.0031	0.0014	−2.23	<0.05
Enrollment	0.0145	0.0073	1.98	NS
Student-level variables
Special education status	0.0217	0.0109	2.00	<.05
Ethnicity (black)	−0.0397	0.0091	−4.37	<.001
Grade cohort	−0.0143	0.0213	−0.67	NS
FARMS	−0.0128	0.0098	−1.31	NS
Gender	−0.0307	0.0089	−3.43	<.01
Post hoc cross-level interactions
Grade cohort × SWPBIS	0.0543	0.0229	2.38	<.05
Gender × SWPBIS	−0.0157	0.0135	−1.16	NS
Special education status × SWPIS	−0.0079	0.0150	−0.52	NS
Random Effect	Variance Component	χ2		P Value
Level 1	0.6244
Level 2	0.5908	43611.1		<.001
Level 3 time/slope	0.0026	185.3		<.001

Akaike information criterion = 102279.3; Bayesian information criterion = 102206.9. SWPBIS was coded 1 (intervention), 0 (comparison). FARMS indicates free or reduced meals status. Gender was coded 1 (male), 0 (female). Black was coded 1(Black), 0 (all others). Grade cohort indicates the grade the youth was in when the trial began, coded 1 (Kindergarten), 0 (grades 1 or 2).

^aResults presented are from the model that did not include the post hoc interactions. The interclass correlation for emotion regulation skills was 0.06.

The final set of multilevel analyses indicated that children in SWPBIS schools were 33% less likely to receive an ODR than those in the comparison schools (AOR = 0.67, [0.57–0.79], P < .001). We also found a significant interaction between gender and intervention status (AOR = 1.27 [1.04–1.56], P < .05) in which girls in SWPBIS schools were less likely to receive an ODR than girls in comparison schools, but there was no difference for boys. With regard to suspensions, there were no significant differences between SWPBIS and comparison schools and no significant interactions (see Table 6).

TABLE 6.

HLM Results Examining the Effect of SWPBIS on Office Discipline Referrals and Suspensions

Predictor Variables	Received Office Discipline Referral		Received Out-of-School Suspension
	AOR	CI	AOR	CI
Student-level variables
Grade cohort	0.67***	(0.57–0.79)	0.73**	(0.59–0.91)
Gender	3.58***	(3.21–4.00)	4.69***	(4.04–5.44)
Ethnicity (black)	2.09***	(1.80–2.42)	2.56***	(2.18–3.01)
Special ed	1.59***	(1.39–1.81)	2.09***	(1.82–2.41)
FARMS	1.68***	(1.52–1.86)	1.61***	(1.38–1.88)
School-level variables
SWPBIS intervention	0.66**	(0.49–0.89)	0.97	(0.69–1.36)
Student mobility	1.00	(0.98–1.02)	1.02	(0.99–1.04)
Student/teacher ratio	0.99	(0.97–1.03)	1.02	(0.99–1.05)
Enrollment	0.87***	(0.82–0.93)	1.04	(0.92–1.17)
Faculty turnover	1.02	(0.99–1.04)	1.02	(0.99–1.05)
Post hoc cross-level interactions
Grade cohort × SWPBIS	.78	(0.57–1.08)	0.72	(0.47–1.10)
Gender × SWPBIS	1.27*	(1.04–1.56)	1.29	(0.97–1.73)
Special ed status × SWPBIS	0.96	(0.75–1.24)	1.34	(0.99–1.79)

SWPBIS was coded 1 (intervention), 0 (comparison). FARMS indicates free or reduced-price meals status. Gender was coded 1 (male), 0 (female). Ethnicity was coded 1(Black), 0 (all others). Grade cohort indicates the grade the youth was in when the trial began, coded 1 (kindergarten), 0 (grades 1 or 2). CI = 95% confidence interval. Ed, education. *P < .05, **P < .01, ***P < .001.

Discussion

This study used data from a 4-year randomized controlled effectiveness trial to examine the hypothesis that children in schools implementing SWPBIS would experience better adjustment and fewer problem behaviors relative to their peers in comparison schools. As hypothesized, the multilevel, longitudinal analyses indicated that relative to the children in comparison schools, those in SWPBIS schools displayed lower levels of disruptive behavior problems and concentration problems, and better emotion regulation and more prosocial behavior. We also observed significant intervention effects on children’s odds of receiving an ODR. However, no significant effects were observed on suspensions. We explored for potential interactions with select demographic variables and generally found that the effects were strongest among those children who began the trial when they were in kindergarten. Interestingly, no other demographic characteristics explored were significant effect modifiers.

It was not surprising that one of the strongest effects was on ODRs, given a core component of the SWPBIS model is the establishment of a system to track, monitor, and use ODR data.⁴ The heightened attention to this particular source of data within SWPBIS likely contributed to the reduction in the odds of ODRs observed among children in the trained schools. Although not specifically examined within this study, it is theorized that the reduction in ODR use is mediated by reductions in children’s behavior problems. Future studies should explore the extent to which the reductions in teacher-rated behavior problems lead to the reductions in ODRs. Although the current study focused on the overall impact on receipt of ODRs, it is possible that the pattern of findings may vary by the reason for the ODR (eg, bullying, disrespect); however, data regarding these outcomes are not available for schools in both conditions. It is also possible that the more proximal impacts observed in the trial translate into longer-term effects on the need for school-based services, such as special education, and academic outcomes.

Previous studies have reported a school-level impact of SWPBIS on suspension rates^9,12; therefore, we hypothesized that there would be a significant effect on students’ receipt of a suspension; however, such an effect did not reach significance. It is possible that the intervention effects would increase with longer implementation of the model. More targeted strategies, such as those used in the full 3-tiered SWPBIS model, may be necessary to reduce suspensions. Furthermore, we only explored main effects of the universal SWPBIS model and select interactions related to demographics, which indicated that the effects tended to be strongest for children who were in kindergarten when they were first exposed to SWPBIS. This suggests that the earlier the exposure to SWPBIS, the greater the potential impact of the model. From a developmental perspective, it is possible that younger children’s behaviors are more malleable and responsive to adults’ expectations and positive reinforcement for good behavior. Additional work is needed to better understand the impact of SWPBIS when implemented in middle and high schools. Future studies also should examine whether the intervention effects are greatest for children with a particular baseline risk profile.³⁷ Although not a focus of the current study, race and special education status were significantly associated with each outcome across all models. Given the literature on disproportionality in discipline,³⁸ future research will examine the effects on disproportionality in special education service use and discipline problems.

The data were obtained through teacher reports over the course of the trial because archival data (eg, ODRs) or pretraining teacher-report data are not available for analysis. Although we recognize that a teacher report of need is not equivalent to an assessment made by a clinician or a diagnostic assessment, teachers are the most common source of children’s mental health and special education referrals,³⁹ thus their assessments are important in the context of school-based interventions and have been shown to predict mental health problems.^2,20 Furthermore, the SWPBIS and control schools did not have common measurement systems for ODRs that would allow a functional comparison of these data elements; however, teacher reports of ODRs have been validated.²⁷ Future research should examine other effects of SWPBIS, such as the impact on achievement and attendance.

The effect sizes were relatively modest; however, small effect sizes are common in longitudinal universal prevention studies^15,40; we anticipate that the effects will be stronger for higher-risk students.²⁴ Also bear in mind that the RCT was an effectiveness trial in which all the training, implementation, and support activities were led by the state and local school districts, not by the researchers. These training events also included schools not participating in the RCT, and thus we anticipate that the findings would generalize to other schools in the state who participated in these events (see Stuart et al⁴¹ for information on generalizability).

Conclusions

This is the first RCT to demonstrate impacts of SWPBIS on trajectories of children’s behavior problems and adjustment over multiple school years. These findings suggest that there are proximal effects of SWPBIS on a range of behavior problems, such as ODRs, concentration difficulties, and aggressive or disruptive behavior, as well as improvements in prosocial behaviors and emotion regulation. The effects of SWPBIS on prosocial behavior and emotional regulation are relatively unique in the literature. The finding that these effect sizes were as strong as or stronger than disruptive behavior is also noteworthy. Demonstrating the impact of SWPBIS on a range of early-onset behavior and social-emotional problems has important public health significance, especially in light of the wide dissemination of SWPBIS.²⁹

These findings provide support for the hypothesized reduction in behavior problems and improvements in adaptive skills through SWPBIS. Although these effects are promising, there are some children who are not responding adequately to the universal model,⁴ and thus additional work is needed to identify these children so that their needs can be better met within the school. The tiered prevention model also provides an infrastructure for the delivery of more intensive services and programs for children with greater needs.^4,42 Furthermore, SWPBIS holds promise for improving the organizational context to support higher-quality implementation of selective and indicated preventive interventions for nonresponders.¹¹

Glossary

AOR

adjusted odds ratio

ES

effect size

HLM

hierarchal linear modeling

ODR

office disciplinary referral

RCT

randomized controlled trials

SWPBIS

School-Wide Positive Behavioral Interventions and Supports

TOCA-C

Teacher Observation of Classroom Adaptation, Checklist