Skip to main content
. 2018 Jan 9;14(1):i–216. doi: 10.4073/csr.2018.1

Author

Type of Study Sample

Mean 35

Age

(SD)

School‐based Programme

Universal/ Indicated 36

% FSM

Cluster 37

Extracted data for

effect size calculations

Effect size calculation

Measure of exclusion

1) Allen et al. (1997)

Journal article

695

15.8

(1.13)

Teen outreach

Unclear

Unknown

No

Suspension

Baseline T1: 58 cases (17%); N=342 C1: 81 cases (23.8%); N=353

Post treatment (immediately after treatment) T2: 42 cases (13%); N=324 C2: 93 cases (28.7%); N=323

Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal .75. See methods section for further details

Presence/absence

2) Arter, (2005)

Journal article

52

Secondary school

Positive Alternative Learning Support (PALS)

Indicated 40% FSM

No

Suspensions No baseline reported Post Treatment (presumably after treatment) T2: M=.675; SD=.194; N=23 C2: M= 675; SD=.227; N=17

SMD was calculated using equations 3 and 4 in the methods section.

Nº of days

3) Barnes et al. (2003)

Journal article

45

16

(1.3)

Stress reduction

Universal

Unknown

No

Suspension

Baseline T1: M=0.8 days; SD=1.8; N=23 C1: M=0.0 days; SD=.0; N=18

During intervention T2(during): M= 0.5 days; SD=1.2; N=23

C2(during): M=1.2 days; SD=3.0 N=18

The principal investigator provided N size for T2. SMD was calculated as the difference between time 1 and time 2, accounting for the covariation between pre‐ and post measures (equations 8 and 9 in the methods section).

Nº of days

4) Berlanga (2004)

PhD

Thesis

80

Eighth

grade

Grades, Attendance and Behaviour (GAB)

Indicated

Unknown

No

In School Suspension

Baseline

T1: M=1.15; SD=1.29; N=32 C1: M=.61; SD=1.14; N=31

Post Intervention (immediately after treatment) T2: M=1.12; SD=1.21; N=32 C2: M=1.03; SD=1.88; N=31

Suspension

Baseline T1: M=.31; SD=.53; N=32 C1: M=.35; SD=.83; N=31

Post Intervention(immediately after treatment) T2: M=.34; SD=.90; N=32 C2: M=.58; SD=1.11; N=31

Removal/Expulsion

Baseline T1: M=.06; SD=.24; N=32 C1: M=.06; SD=.24; N=31 Post Intervention (immediately after treatment) T2: M=.03; SD=.17; N=32 C2: M=.16; SD=.45; N=31

Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal. 75. See methods section for further details

Nº of events

5) Bradshaw et al. (2012)

Journal article

12,334

Elementary school

School‐Wide Positive Behavioural Interventions and Support (SWPBIS)

Universal

49% FSM

Yes

Out‐of‐school suspension Follow‐up (four years)

Student level

Control: 21 schools (N=5124) Treatment: 16 schools (N=6614)OR= .73; 95% CI .59 and .91

The study reports results using multi‐level analysis. In this case, we have not applied any correction of standard errors. We assume that MLM accounted for clusters and subsequently corrected the bias (see p. e1140). See methods section for further details

Nº of events

6) Bragdon (2010)

PhD Thesis

68

 

Teach Team Project

Indicated

49% FSM

No

Suspension

Baseline T1:M=.06; SD=.34; N= 34 C1:M=.96; SD=2.42; N=34 Post treatment (during)

T2:M=.03; SD=.17; N= 34 C2:M=1.10; SD=2.61; N=34

Follow‐up (three months later)

T3:M=.09; SD=.51; N=34 C3:M=1.07; SD=3.19; N=34

Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal. 75. See methods section for further details.

Nº of days

7) Brett (1993)

PhD Thesis

126

12‐14 years

Efficacy, DC

Universal

Unknown

Yes

School Suspension

Clusters: 3 control (n=66) 3 Experimental (n=60)

Post‐treatment (1 Month) T2: M=.53; SD= 1.02; N=40 C2: M=.63; SD=1.21; N=57;

Based on Hedges (2007) and Spier et al. (2013), effect sizes were computed using dT2 , assuming equal cluster sample size, ρ=.05. See methods section for further details

Nº of events

8) Burcham (2002)

PhD Thesis

71

Middle school

Social problem solving skills training

Indicated

38% FSM

No

In‐School Suspension‐Baseline T1:M=8.62; SD=6.44; N=37 C1:M=7.88; SD=4.47; N=32 p=.58

‐Immediately after treatment T2: f=.18; p= 0.67; N=69

‐18 months after treatment T3: f=.04; p=0.84; N=38

Out‐of‐school Suspension Baseline T1: M=3.22; SD=3.71; N=37 C1: M=2.56; SD=3.40; N=32 p=.45 Immediately after treatment T2: f=1.09; p=.30; N=69 18 months after treatment T3: f= 1.83; p=.18; N=38

Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal. 75. See methods section for further details

Nº of events

9) Collier (2002)

PhD Thesis

60

5‐14 years

Pro‐social skills training

Indicated

Unknown

No

School Suspension (+)

Baseline T1: M=1.93; SD=.4498; N=26 C1: M=1.86; SD=.5074; N=25

Post treatment (presumably after treatment) T2:M=1.15; SD=.6748; N=26 C2:M=2.16; SD=.3742; N=25

Study was identified as an outlier value. It was winsorised as suggested by Wilson & Lipsey 2001.

Effect size was calculated as the difference between baseline and post treatment. We corrected final calculation by adding the value of pre/post correlation, assumed to be equal .75.

Nº of days

10) Cook et al. (2014)

Technical report

106

 

BAM (skills‐training) and MATCH (tutoring)

Indicated

26% FSM

No

Baseline data is reported but incomplete

Out‐of‐school suspension (ITT) b=‐.642; SE=.501;

(unclear the number of months/weeks of post treatment measured)

Data was entered into CMA by using the option Log OR and its SE. No further corrections.

Even if the evaluation was testing two different interventions, data was reported in a composite measure. The original author took that option because they recognised contamination between groups (spill‐over).

Nº of events

11) Cornell et al. (2012)

Journal article

201

 

Threat assessment

Indicated

Unknown

Yes

Long‐term suspension

Post treatment (presumably after treatment)

T2: 25 (25%); N=100

C2: 49 (49%); N=101

No corrections

Nº of events

12) Crowder (2001)

PhD Thesis

109

 

Gang Resistance, Education and Training (GREAT)

Unclear

Unknown

No

Out of School Suspension No baseline measure reported Post intervention (presumably after treatment) T2: M=.1329; SD=.4629; N=53 C2: M=.1429; SD=.1610; N=56

In School Suspension No baseline measure reported Post intervention (presumably immediately after treatment) T2: M=.3584; SD=.7464; N=53 C2: M=.4464; SD=.8464; N=56

SMD was calculated using equations 3 and 4 in the methods section. No further corrections.

Nº of events

13) Dynarski et al. (2003:2004)

Technical report

968

Elementary school

21st Century Community Learning

Unclear

Unknown

No

Post treatment 12 months (2003)

T2: 7.1% (38) N=537

C2: 5.2% (16) N=317

Follow up 24 months

T3: 60 (6.2%) N=537

C3: 43 (4.4%) N=317

Data was entered into CMA by using a 2x2 table. No further corrections.

Nº students

14) Edmunds et al. (2012)

Journal article

1607

15.3

Early College High School Academic skills enhancing

Unclear

50.6%

No?

% Suspended at least once

T2: 6.4%; (57) N=885

C2: 13.3%; (86) N=644

The principal investigator provided measures for effect size calculation (via mail communication).

Nº of events

15) Farrell et al. (2001)

Journal article

626

11.7

(0.6)

Responding in Peaceful and Positive Ways (RIPP)

Universal

Unknown

Yes

In‐school suspension Post intervention (immediately after treatment) OR=5.0 (95%CI 1.5; 17.1) 6 months OR=1.4 (95% CI .7; 2.8) 12 months OR=1.4 (95%CI .6; 3.0) Out‐of‐School Suspension Post‐intervention (immediately after treatment) OR=0.9 (95%CI .5; 1.8) 6 months OR=1.1 (95% CI .6; 2.0 12 months OR=0.9 (95% CI .6; 1.4)

Although the study is based on clustered data, we have not applied any correction of standard errors. The author mentions the use of GEE to calculate robust estimates of standard errors (see Farrell et al., 2001, p. 455).

See methods section for further details.

Nº of events

16) Feindler et al. (1984)

Journal article

36

13.8

(.68)

Anger control training

Indicated

Unknown

No

School Expulsion

Baseline

T1: M=1.45; SD=.71; N=18

C1: M=1.40; SD=.44; N=18

Five‐weeks follow‐up

T2: M=.77; SD=.29; N=18

C2: M=.1.2; SD=.46; N=18

Effect size was calculated as the difference between baseline and post treatment. We corrected final calculation by adding the value of pre/post correlation assumed to be equal .75. See methods section for further details.

Nº of events

17) Harding (2011)

PhD Thesis

48

Eighth grade

Over to you

Indicated

Unknown

No

Six‐months follow‐up

Fixed term exclusion

T2: 5 N=20

C2: 5 N=23

Permanent exclusion

T2: 1 N=20

C2: 1 N=23

Data was entered into CMA by using a 2x2 table. No further corrections.

Nº of events

18) Hawkins et al. (1988)

Journal article

160

Seventh grade

Proactive Classroom Management

Indicated

Unknown

No

Times Suspended

Post‐treatment (presumably after treatment)

T2: M=.48; SD=1.3; N=67 C2: M=.89; SD=2.1; N=75

SMD was calculated using equations 3 and 4 in the methods section. No further corrections.

Nº of events

19) Hirsch et al, (2011)

Technical report

535

15.9

After School Matters

Indicated

86%

No

School Suspension

Baseline

T1: M=1.27; SD=.63; N=259

C1: M=1.29; SD=.64; N=178

Post treatment (immediately after treatment)

T2: M=1.36; SD=.72; N=259

C2: M=1.40; SD=.75; N=178

Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal .75. See methods section for further details.

Nº of events

20) Hostetler & Fisher (1997)

Journal article

317

Third grade

Project CARE (Skill for parents and children)

Indicated

Unknown

No

Suspension

Baseline T1: M=0.13; SD=0.56; N=151 C1: M=0.07; SD=0.35; N=140

Post treatment (a few months after treatment, no clear specification) T2: M=0.20; SD=0.53; N=155 C2: M=0.25; SD=0.89 N=141 One year follow‐up T3: M=0.26; SD=0.80; N=90 C3: M=0.15; SD=0.66, N=86 Two year follow‐up T4: M=0.27; SD=0.74; N=30 C4: M=0.09; SD=0.29; N=34

Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal .75. See methods section for further details.

Nº of events

21) Ialongo et al. (2001)

Journal article

678

6.20

(.34)

Two interventions

i) Classroom‐centred (CC)

ii) Family‐school partnership (FSP)

Universal

62.3% FSM

Yes

Suspension

Five year follow‐up

Classroom‐centred vs. control OR=.73; (95%CI=.56; .95) ** treatment group less likely to be suspended Family‐school partnership OR=.59 (95%CI .35; .97) Boys: OR=1.13 (95%CI .61; 2.09) Girls: OR=.38 (95%CI .17; .86) ** treatment group less likely to be suspended

Since data was dichotomous and nested in clusters, we corrected standard errors of effect sizes. The design effect was corrected by using the formula suggested by Higgins & Green (2011) expressed by the equation [1+(M‐1) x1]. See methods section for further details.

Presence/absence

22) Johnson (1983)

PhD Thesis

60

Seventh and eighth grade

ATTEND (Counselling and monitoring)

Indicated

Unknown

No

Suspension Baseline T1: M=.76; SD=.85; N=30 C1: M=.83; SD=.87; N=30 Post intervention (after treatment) T2: M=.36; SD=.55; N=30 C2: M=1.5; SD=1.25 N=30

Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal .75.

Nº of events

23) Lewis et al. (2013)

Journal article

624

Elementary school

Positive Action

Universal

Grade 3

84% FSM

Yes

Suspension

Baseline (2001)

T1: M=40.95; SD=48.13; N=3648

C1: M=65.25; SD=56.15; N=3800

Post treatment (2004)

T2: M=55.17; SD=64.84; N=3407

C2: M=77.63; SD=66.8; N=3687

Follow‐up (2005)

T3: M=68.08; SD=80.02; N=3367

C3: M=88.96; SD=76.56; N=3539

The principal investigator provided data for calculations. Based on Hedges (2007) and Spier et al. (2013), effect sizes were computed using dT2 , assuming equal cluster sample size, ρ=.05.. See methods section for further details.

Nº of events

24) Mack (2001)

PhD Thesis

20

Fourth to sixth grade

ICAN Kids! Behavioural group counselling

Indicated

95% (for school, no stated for sample)

No

Out‐of‐school Suspension

Baseline T1: M=1.5000; SD=.9718; N=10 C1: M=1.9000; SD=.8756; N=10 3 weeks (during) Tduring: M=.8000; SD=.6325; N=10

Cduring: M=.9000; SD=.7379; N=10

Post‐treatment T2: M=.3000; SD=.4830; N=10 C2: M=.4000; SD=.5164; N=10 + 3 weeks T3: M=.0000; SD=.0000; N=10 C3: M=1.0000; SD=.6667; N=10

Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation assumed to be equal .75. See methods section for further details.

Nº of events

25) Obsuth et al. (2016)

Journal article

738

13.9

Engage in Education (Skills training)

Indicated

32%

Yes

Exclusion

Official Records

Baseline

T1: OR=2.784; SE=.300; p=.001

Post‐treatment (1 month)

T2: OR=1.444; SE=.389; p=.344

The study reports results using multi‐level analysis. In this case, we have not applied any correction of standard errors. We assume that MLM accounted for clusters and subsequently corrected bias (see p. 11). The study offered measures of impact based on self‐reporting, teachers’ reports and official records. We extracted from OR based on official records (most of our studies report official records of suspension).

Presence/absence

26) Okonofua et al. (2016)

Journal article

1682

Middle school

Empathic Discipline

Universal

Unknown

Yes

Suspension

Post treatment (unclear number of months/weeks)

T2: OR=.42; z= ‐3.33; p=.001; N=1449

31 clusters

Since data was dichotomous and nested in clusters, we corrected standard errors of effect sizes. The design effect was corrected by using the formula suggested by Higgins & Green (2011) expressed by the equation [1+(M‐1) x1]. See methods section for further details. Published data did not provide confidence intervals or SE. We tried to contact authors but it was not possible. We calculate an approximate SE=.013.

Presence/absence

27) Panayiotopoulos & Kerfoot (2004)

Journal article

124

10

Home and School Support Project (HASSP)

Indicated

Unknown

No

Exclusion T1: M=9.50; SD=14.81; N=62 C1: M=5.11; SD=7.56; N= 62 Post treatment (After three months) T2: M=4.95; SD=13.11; N=61 C2: M=5.51; SD=11.94; N=62

Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal .75.

Nº days

28) Peck (2006)

PhD Thesis

1050

Fifth to eighth grade

Student Targeted with Opportunities for Prevention (STOP)

Unclear

Unknown

No

Suspension

Post‐treatment (unclear number of weeks/months after treatment)

T2: 22; N=315

C2: 22; N=321

Data was entered into CMA by using a 2x2 table. No further corrections.

Nº of events

29) Reese et al. (1981)

Journal article

98

Seventh to ninth grade

Preparation through Responsive Education Programs (PREP)

Indicated

Unknown

Matched peers

Suspension

During school year

T2 vs C2: X2 (1)= 6.58, p<.02

Data was entered into CMA by using X2 originally reported. No further corrections.

Nº of days

30) Russell (2007)

PhD Thesis

61

11.5

(.46)

Coping Power (Skills training for reducing aggression)

Indicated?

Unknown

No

Suspension

Post treatment T2(School A): M=.15; SD=.38; N=13; C2(School B): M=.31; SD=.60; N=16; T2(School B: M=.29; SD=.61; N=14 C2(School B):M=.00; SD=.00;N=10

SMD was calculated using equations 3 and 4 in the methods section. No further corrections.

Nº of events

31) Shetguiri et al. (2011)

Journal article

108

14

Violence and drug use reduction

Indicated

100%

No

Suspended or Expelled

Baseline T1: 8 (21%) N=40 C1: 10 (22%) N=46 Eight months follow‐up T2: 6 (14%) N=40 C2: 4 (8%) N=46

Effect size was calculated as the difference between baseline and post treatment. We corrected final calculations by adding the value of pre/post correlation, assumed to be equal .75.

Presence/absence

32) Smith (2004)

PhD Thesis

40

 

The Personal Responsibility Group (Emotional Intelligence skills)

Indicated

Unknown

No

In‐school Suspension

Post‐treatment

T2: f=11.085; p greater than or equal to .002; Out‐of‐school Suspension

Post treatment T2: f= 10.088; p greater than or equal to .003

SMD was calculated based on f‐test. No further corrections.

Nº of events

33) Snyder et al. (2010)

Journal article

544

Elementary school children

Positive Action

Universal

55%

Yes

Suspension (% of students suspended) 2002 T1: M=1.12; SD=1.10; N=5000 C1: M=.98; SD=1.11; N=5000 2006 T2: M=.67; SD=.64; N=5000 C2: M=1.72; SD=1.55; N=5000 2007 T3: M=.84; SD=.61; N=5000 C3: M=2.53; SD=2.80; N=5000

The principal investigator provided sample sizes for calculations. Based on Hedges (2007) and Spier et al. (2013), effect sizes were computed using dT2 , assuming equal cluster sample size. Since the report presented the value of ρ, we used this value in calculations.

Nº of students

34) Sprague et al. (2016)

Unpublished paper

13,498

Middle school children

School‐Wide Positive Behavioural Interventions and Support (SWPBIS)

Universal

Unknown

Yes

Expulsion

Baseline

T1: M=.002; SD=.004; N=6492

C1: M=.003;SD=.004; N=7006

Post treatment

T2: M=.002; SD=.004; N=6492

C2: M=.003; SD=.005; N=7006

Follow‐up (1 year later)

T3: M=.003; SD=.006; N=6492

C3: M=.003; SD=.004; N=7006

In School Suspension

Baseline

T1: M=.071; SD=.094; N=6492

C1: M=.135;SD=.189; N=7006

Post treatment

T2: M=.064; SD=.087; N=6492

C2: M=.097; SD=.133; N=7006

Follow‐up (1 year later)

T3: M=.058; SD=.060; N=6492

C3: M=.095; SD=.145; N=7006

Out‐of‐School Suspension

Baseline

T1: M=.082; SD=.063; N=6492

C1: M=.078; SD=.065; N=7006

Post treatment

T2: M=.076; SD=.077; N=6492

C2: M=.061; SD=.042; N=7006

Follow‐up (1 year later)

T3: M=.073; SD=.064; N=6492

C3: M=.075; SD=.051; N=7006

 

Principal investigator provided data for calculations. Based on Hedges (2007) and Spier et al. (2013), effect sizes were computed using dT2 , assuming equal cluster sample size, ρ=.05. See methods section for further details.

Nº of events

35) Tilghman (1988)

PhD Thesis

100

12.5

Counsellor Peers

Indicated?

Unknown

No

Suspension T2: 11 (N=46) C2: 26 (N=45) Significance test

Data was entered into CMA by using a 2x2 table. No further corrections.

Nº of students

36) Ward & Gersten (2013)

Journal article

33 schools

≈ 25,000 students record

Elementary school children

Safe and Civil Schools

Universal

90%

Yes

Post treatment (end of intervention) OR=.83; SE=.05 Follow‐up (1 year later)

(cumulative impact) OR=.77; SE=.04

Since data was nested in clusters, we corrected standard errors of effect sizes. The design effect was corrected by using the formula suggested by Higgins & Green (2011) expressed by the equation [1+(M‐1)x1].

Nº of days

37) Wyman et al. (2010)

Journal article

226

K ‐ 3rd

Rochester Resilience Programme

Indicated 90%

Yes

Suspension events

Post‐treatment (immediately after intervention)

TC2: Exp b=−0.57; SE=0.23; z=−2.48; p=0.013

Controlling for suspension T1

59 classrooms, 4 students per classroom

Since data was nested in clusters, we corrected standard errors of effect sizes. The design effect was corrected by using the formula suggested by Higgins & Green (2011), expressed by the equation [1+(M‐1) x1]. See methods section for further details

Nº of events

Abbreviations : T1(treatment group baseline measure); C1 (control group baseline measure); T2 (treatment group post treatment measure); C2 (control group post treatment measure); T3(treatment group follow up measure); C3(control group follow up measure); FSM (free school meals); M (mean); SD (standard deviation); N (sample size); OR (Odds ratio); 95% CI (95% confidence interval); SE (standard error); f (ANCOVA coefficient); p (p‐value); b (beta coefficient); X2 (chi‐squared).

35

When the mean age was not available in the original study, their grade in school has been reported. Their school grade gives the reader a general idea of the age of the students.

36

Universal intervention strategies are those oriented to reach the entire population of students, without regard to individual risk factors. Indicated programmes were defined as those targeting students displaying behavioural problems, punished at school or presenting a specific risk to their educational development.

37

Due to the nature of the settings (schools) some studies reported clustered data. We corrected SE errors when it was needed. See methods section for further details.